Before joining the world of digital marketing, I was a product design engineer. Most of my clients at Distilled are now CRO projects and I’ve found my background to be surprisingly useful. There’s a lot of overlap between designing physical products and designing websites that convert well. I’d like to share some of the research methods that I use for CRO that I learned while designing physical products. I like to use a framework of Learn, Look, Ask, and Try.
I first came across this while at university. It’s the design research methods used by
IDEO. They released this as a pack of playing cards. In their own words:
“IDEO Method Cards show 51 of the methods we use to inspire great design and keep people at the center of our design process.”
There are 51 cards each with a research method that belongs to one of the above categories. The cards have since been made into an app. I find it useful when trying to come up with new ways to get customer insights. You can download the app
here.
Design methods in the learn section are about analyzing the information you’ve collected to identify patterns and insights. One you might not have heard of is error analysis.
Error analysis
In simple terms you can think of error analysis as going around your site and saying “what happens if I do that?” In product design you might hear this called failure mode effects analysis (FMEA).
“Failure modes” means the ways, or modes, in which something might fail. Failures are any errors or defects, especially ones that affect the customer.
”
Effects analysis” refers to studying the consequences of those failures.
While a lot of FMEA is overkill for designing a website (hopefully nobody is going to die if they click the wrong button) I think the principles can be used to help proactively find faults. Every website is different so you’ll need to think of your own scenarios but here are some to get you started:
- What if I use my email instead of username to login?
- What if I press the back button in the checkout funnel?
- What if I need a refund?
- What if I want to get the product delivered to my work address?
- What If I order the wrong product?
You can see that these potential errors can be a mix of usability and customer service. The point is to be proactive and anticipate what could go wrong. You can then fix true errors (things that are just broken) or put processes in place to ensure mistakes can be fixed easily when they do go wrong.
Extra tip
Google analytics has a great report that can help you find some of the most common errors or problems. Look at the reverse goal path report and pick a goal (for example people reaching the thank you page).
The report will then show you the most common routes that people take on their way to that page. The image below shows this report on one of my client’s sites.
I’ve had to blank a lot out for privacy reasons, but the point can still be seen: Out of the top 10 routes to the thank-you page,
- 4 included people visiting the terms and conditions page, and
- 2 included people visiting the FAQ page.
What are they looking for? If we can find out, we can make that information clearer and hopefully stop them having to go to those pages. Which brings me to my next section:
The “Ask” section is pretty simple; it’s about asking people to explicitly tell you what they do or do not like.
For the terms and conditions example above, the solution is an easy one—live surveys. I’m sure everyone has heard of
Quaraloo by now so I’m not going into detail on this. The solution is to ask people that leave the funnel via the T+C page what information they’re looking for using Quaraloo. Once we find out the reason, we can add that information to the pages leading up to the conversion and hopefully reduce anxiety and distractions leading up to the purchase.
The real research method I want to talk about for this section is extreme user interviews.
Extreme interviews
Any golfers reading? If so, you’ll recognise the image below as the famous Big Bertha.
Big Bertha is one of the bestselling drivers, so how did Callaway come up with the design? They did it by focusing on a particular demographic. While competitors were all focused on asking
golf players what they wanted, Callaway focused on a different set of users. They surveyed non-golfers. More specifically they wanted to know why people who loved sports, could afford to play and already belonged to country clubs chose not to play. In other words, why do people whom on paper should like and play golf choose not to?
By interviewing lots of people who fit that criterion, they were able to find the answer. People don’t want to feel embarrassed. If you’ve ever tried golf you’ll relate to how frustrating and embarrassing the first couple of years are, especially if you’re used to being good at other sports. This is where Callaway gained their insight and competitive advantage.
It turns out that consistently making contact with the face of a small driver is hard, really hard. If you fail, the ball can end up anywhere. Callaway decided to focus on the need of these people (people that should play but don’t) by designing a driver that had a massive club head and huge face. The result was it was much easier for beginners to hit the ball and avoid embarrassment of constantly losing the ball. To this day Big Bertha is one of the most successful drivers on the market.
So how do we use this for CRO? Extreme interviews? Instead of just surveying the people that do buy from you, or that are familiar with your brand, survey two groups of users.
- Experts: Repeat buyers or people that are familiar with your brand
- Novices: People that have never been to your website but that at least understand what your product or service does.
The hard thing about this is recruiting these people. You can’t just ask members of the public; you need to ask people that are on your site. A tool I like to use for this is
Ethn.io.
Ethn.io lets you recruit users for user testing and pre-qualify them. For example, in the above you can create a popup that looks something like this:
You can then ask users a qualifying question and group them appropriately.
Fly on the wall
Your customers are liars. Harsh, but true. Even in the extreme interviews technique you might not get the right insights that you’re looking for. If you ask people why they don’t convert, they might not be able to tell you. There are two reasons for this:
- The reasons are subconscious so they actually don’t know the answer.
- They don’t want to tell you for fear of embarrassment.
That’s why using the techniques in the look category is a great idea.
Here’s a scenario. Imagine for a second you gave someone this:
If you were to ask someone how they would open this package, you’d likely get a sensible answer like “I’d use scissors”, but as soon as you leave the room and they don’t think you’re looking, you’re likely to see something like the scene below:
If you’ve ever tried to open one of those blister packs, you’ll know how annoying they can be to get into. They’ve led to the
many injuries and the emergence of the term “wrap rage“.
Anyway, the point I’m trying to make is you can’t always trust your users to tell you all of the reasons they don’t do the things you want them to.
In the offline world, designers use a technique called “fly on the wall”. This involves watching people in the environment or using the product that you’re trying to design. Sometimes the subject is aware that you’re watching them (shadowing), other times (ideally) they’re not (fly on the wall). Watch this scene from Madmen and you’ll get the picture:
So how do we do this in the digital world? We can’t sit over everyone’s shoulders as they use your website, but there are some tools that come close. Both of which I’m sure you’ve heard of.
Shadowing: Usertesting.com
I like usertesting.com but my one complaint is the sample of users is unlikely to be representative of your customers. Even if you select the options like low level of internet experience—the fact they are on a UX testing website already puts them above the average internet user. They’re also not your customers. One way around this is to combine Ethnio with uerstesting.com. Recruit real users then set them tasks using usertesting.com or just watch them live using some kind of screen sharing tool.
Fly-on-the-wall: Clicktale
Clicktale might be above many peoples’ budget, but if you can afford it I’d recommend it. Clicktale records anonymous sessions on your website. Obviously it would take a long time to watch all sessions, but you can segment by things like location, browser, or even sessions that had errors. Using these tools you can dig into why certain segments of traffic may not be converting as well.
Another tool that I’ve heard of, but not personally tried, is
http://www.uxcam.com/ It’s like Clicktale but with a specialization in mobile usability testing. I spoke to the founder at one of our meetups about a month ago and was impressed by the features. It’s also still in beta so it’s free at the moment if you want to give it a try.
Empathy tools
As a product designer, your most powerful tool is empathy. If you can empathize with your customers and understand what they really want, you’ll create great products. The best way to empathize with your customers is to do what they do—try it.
In the offline world, product designers go to great lengths to understand users. For example, when designers are creating a product where the primary audience is the elderly, they might use empathy tools like those shown below:
The image above is a picture of me with coins strapped to the back of my knuckles. This can be used to simulate the limited dexterity that can come as a result of arthritis.
The designers in the image above strapped up their legs using a kind of split. Again, this one done to simulate reduced mobility when trying to climb stairs.
Finally and probably the one that could most easily be applied to the online world is a pair of empathy glasses. These can come in various types depending on the condition you want to simulate. If you want to get an idea of how various eye conditions can affect vision, look at the side-by-side comparison of healthy eyes to advanced stages of cataracts and glaucoma below. You can see the tools here:
Cataract simulator, Glaucoma simulator.
Cataract simulation
Glaucoma simulation
So if you’re not designing a website for the elderly, can you still use this technique? Absolutely. I like to use “scenario empathy.” This works by enforcing criteria on your users under test conditions, the most popular being a mix of:
- Time: You have six minutes to book a flight to London. GO!
- Money: Find the best product X for under $50.
- Product criteria: Find me a hotel that has a spa and is dog-friendly.
- Technology restrictions: You’re on a mobile with a slow edge connection.
You can then mix any of the above to create powerful scenario simulation. If you can make someone that’s short on time, has a low budget, really specific requirements and a slow connection happy, chances are the rest of your customers will also be happy.
That’s it.
In summary, there’s a lot of research techniques out there that can give you excellent insights about why your customers aren’t converting. Try them out; don’t just stick to the same techniques that you see on CRO blogs all the time.
For more ideas, take a look at the presentation I gave a few weeks ago at our
meetup. Also, for those in the UK, I’ll be presenting on a similar topic at Measurefest this week.
