For those of you who haven’t heard of Canstar, they’re Australia’s biggest financial comparison site, researching and rating 30,000 products from over 300 brands across 30+ finance categories. As you can imagine, getting a prestigious Star Rating from Canstar is a pretty big deal.

We wanted to see how the team at Canstar implement user testing in their organisation and how UX design has played an integral role. So we caught up with Matt, who is a Digital Team Lead of an incredibly versatile team, from developers to digital specialists.

We asked Matt why, and how often his team runs user tests, and his answer might surprise you.

“At Canstar, it’s not purely to elevate ideas, it’s also to explore. We conduct regular user testing with our customers, aiming for monthly sessions that focus on a specific section of the website and explore a limited subset of scenarios that represent our most common use cases. Our intention is to both validate proposed changes as well as identify new opportunities for improvements.”

So what Matt’s saying is yes, they user test to validate, but they don’t limit their value from user testing. They use it to discover what could be a new project and look for problems that they can fix. Matt and his team user test regularly, which helps Canstar create a better and more fluent user experience.

“If there’s a good user experience then you’ll likely see good conversion gains.”

Canstar is focused on the customer experience and helping consumers find the right information at the right time so that they can confidently find the right home loan, insurance policy or other financial services for them.

“Conducting regular user testing is one way we can be sure that these values are reflected across our digital assets. Our experience to date has also demonstrated that this approach can deliver considerable commercial benefits, and this is something that we recognise is hugely valuable.”

User testing is something we all value, but along with regular user tests, Matt and his team are incredibly data driven. “We’ve found that digital analytics can be hugely beneficial in identifying potentially issues on our website and helping point us in the right direction when exploring new ways to improve the customer experience” Don’t underestimate the importance of your quantitative data. This can help point you in the right direction, and in some cases solve problems.

Remember to look for problems and not solutions. You can create something great, but it will be truly magnificent if it serves a purpose.


Top tips from Matt on running your user test

Be a good host
Be a good host, offer them water and keep conversations really chatty to begin with to make them comfortable. I follow Steve Kruggs method, to have a bit of a spiel you mention before and iterate that we’re testing the website and not them. I repeat it over and over.

Best practices for observation 
We’ve tried a number of different things. Each observer writes down their observations on post it notes, puts it up on a wall and each observers writes down as many notes as possible and we try to categorise into observation, error, quote. Once everyone has their notes we go around and share them. The research scribes and writes them all in a Google Doc – once we get that tally it’s almost a score of the design.

Go looking
You don’t realise how important this is until you’re experiencing it, but look for problems instead of solutions. You can always come up with a great design, but it means nothing if it’s not actually a problem. You should really celebrate when you discover problems.