Problem Validation and Customer Research: Challenges and Lessons Learned
Doing your first user research might be scary. You don’t know where to start and what to expect, without even mentioning that it has to be conducted remotely these days. In this article, I would like to share my experience, challenges I faced, and lessons I learned. I was fortunate enough to have a coach guiding me through the labyrinth of user research and sharing her wisdom (can’t thank her enough for that, honestly). So, let’s dive into it!
Finding a Problem
Always start with a problem. Ultimately, your research aim would be to validate the problem you think exists and then find solutions. If you start with a solution and then try to fit a problem, you won’t be able to come up with a good solution or you would spend your time solving a problem that doesn’t exist.
To find the ‘right’ problem, you can look at trends, analyse data and read blogs. Depending on the industry you work for, there might be numerous magazines, analytical journey or you can even attend panel discussions (a lot of them are free). You can also identify a problem by observing customers and talking to people to identify their pain points.
In my case, I was looking at the lack of confidence with money in adults in the UK, which I thought was due to the lack of educational resources teaching money management. I looked at the Money Charity reports and other non-profit organisations working with young adults.
Once you established the problem exists, it’s time to validate it!
Assumptions and Hypotheses
You’ve done your preliminary research and you’ve got a feeling of what you want to investigate more.
Firstly, you need to write down as many assumptions as possible. It could be a good idea to facilitate a brainstorming workshop. There’re a number of remote tools to help: Whiteboard in Teams or Miro. You can always use sticky notes, but someone will need to write up everything you come up with.
An assumption is a statement saying what you think is happening. For example, in my case, three main assumptions I had: ‘children should start to learn about money from the age of 8’, ‘teachers play a part in educating children about money’, ‘education should be engaging’.
It is not enough to write all of them down. You will need to rate your assumptions on how much you know they’re true and how critical they’re to validating your problem. You will want to concentrate on those that are unknown and critical, because your time is precious, and you want to use it efficiently.
Once your assumptions are done, you use them to form hypotheses. A hypothesis tends to follow the following pattern:
If [ASSUMPTION] then [CONDITION] because [PREDICTION]
These are the ones you will prove or reject during your research.
For example, some of my hypotheses were:
H1: If children learn about basic finances from the age of 8 then they will be able to manage their money better when they’re older because they have a foundation of financial literacy.
H2: If the school (teachers) teach children about money then children are more likely to pick up better skills because they ere in the environment of learning.
H3: If financial education is engaging then children will learn better because many children find money and numeric subjects boring.
Research Approach
After you created your hypotheses, you need to decide your research approach. What would be the best way to validate your hypotheses?
If your hypotheses require a vast amount of data to be collected and analysed, you might need to do a quantitative study (survey). This will allow you to reach a lot of people that meet your criteria (provided you’ve got a specific target audience in mind). There are a number of tools available, including paid services such as Attest and UserZoom that tend to deliver results within 2–3 days and free services such as Google Forms and SurveyMonkey. Be careful though, that Survey Monkey free functionality is very limited and Google Forms might be blocked by some companies’ security settings (something I encountered myself). There is also OnePulse which is an opinion platform and a very quick way to test an idea; it’s a paid service and limited to 3 questions, but delivers results within minutes.
If your hypotheses require talking to people about their experience or conducting an experiment, opt for a qualitative study (interviews, focus groups, etc). I personally didn’t want to do focus groups as I wanted people to share their own experiences without having peer pressure, but focus groups can be useful as well.
For my problem validation, I opted for a combined approach: I launched a survey followed by interview sessions, because some of my hypotheses wouldn’t work with a survey: I wanted to see how parents would approach teaching about money and how children would react. I found very insightful interviewing parents and children together as well as introducing a small activity to interview sessions (I would ask a parent to explain the importance of money to their child using whichever method they wanted). I don’t interact with children on a daily basis, so i wouldn’t know otherwise what it meat explaining a complicated topic like money to a child.
Research planning
So, you have decided on your research approach, now it’s time to draft the questions! Don’t skip this stage, always draft your questions before you load them to the platform of your choice. And ask somebody else to look at them and tell you if they don’t make sense. I’ve seen a few surveys asking participants to input their names, while surveys are expected to be anonymous. If for whichever reason, you need to collect names, make it optional and include data protection statement.
A few more tips my wonderful coach gave me:
- Write down all the things you want to find out — this will form the basis of your questions.
- Write at least 20–25 questions including qualifying questions (especially important if you’re looking for participants meeting certain criteria; e.g. in my case, I wanted to survey parents based in the UK with children under 18 years old).
- Provide as many answer options as possible. I found many people put two options and ‘other’ in every question, which poses a couple of problems: it would be hard to analyse after when everybody writes their own ‘other’ answer (obvious one) and you don’t give enough options to the respondents (you’ve got to think of users and their circumstances all the time!).
- Don’t ask leading questions (prompting participants to answer in a particular way).
I don’t think I need to mention that you have to go for a sufficient sample: generally, around 100 responses would be sufficient, but could be more or less depending on your target population. For example, if your survey is worldwide, maybe add a question on geographical region and get enough responses from each to allow for a meaningful comparison.
You will have to go through the same process for interviews, the only difference being that your interview questions would require an open ending. Create interview structure and always start with an informal chat to make participants comfortable. I noticed that I didn’t had to ask all the questions I wrote down, answers just naturally came up as part of conversation. Well, and don’t forget to obtain a consent form: after all, you get access to people data and you need to treat it with care.
I did struggle with getting participants for interviews, as people are generally reluctant to talk about money; making network connections work helped a lot! Never underestimate your network.
Last but not least, don’t forget to take notes or have a note taker with you and/ or record (provided you’re granted the participant’s permission to do this).
Research Findings
You’ve done all the hard work, now the fun part! Not saying it’s not hard, but I’ve always found data analysis fascinating, which is nerdy me coming from the analytics background, convinced that data can be fun.
You can download survey results in data format and then… Well, you’ve got so many tools at your disposal nowadays, starting from humble Excel to fancy tools like Tableau and PowerBI. Visualising data can be a powerful tool as it helps to create insights. Otherwise, summarising data and running correlations can be as useful. You might want to look at your data first to see if you need to rearrange it or combine various response options because they look sort of similar.
Whichever way you choose to approach it, remember: you would need to analyse the results against your hypotheses and demonstrate that you can prove or reject those. In addition, putting some dramatic insights together would help your solutions pitch, should you proceed that route (ok, this is a topic for a different blog, but you should always think of your pitch audience in this case, which would normally be company managers or investors).
Now, it’s a bit less straightforward with the data from interviews. I found affinity maps doing the job for me, but there’re other tools at your disposal including text analysing software.
You would need to listen to recordings or read through your notes. Write down as many things as come to your mind while doing this and group them to identify themes. You can go old school with post-it notes on a whiteboard, or you can use Miro (save the trees!).
Based on themes, you can write statements that you can use to prove or reject your hypotheses.
‘Money was a taboo topic when I was growing up’
‘I was unprepared for dealing with money: I don’t want the same for my children’
‘They (children) like playing phone games with their friends’
As you can see, this exercise is already suggesting some directions you may go with your product proposition. Paired with some data from your quantitative research, this forms a good foundation for coming up with creative solutions.
And by the way, it’s ok if you haven’t managed to find all the answers, you can always do further research if you find it might be critical to validation of your problem. Good luck!
I work in financial services with focus on analysis, reporting as well as internal communications and I’m also a freelance graphic designer with passion for UX and background in data analytics and economics. I hope you enjoyed reading this and feel free to check out my graphic design work on Instagram and my website or connect with me on LinkedIn.
