← Back to  Writings

Is two years too long for a user research project?

Apr 2, 2024

Insights gained from assessing the viability of a two-year user research project, along with key considerations for smooth execution.

Note: This conclusion is based on my discussions with Anindya Fitriyanti and Rayi Harjani of Somia CX, who led the two-year research project. For further details on the project, you can visit the project’s site.

Two years seems like a long time for user research. Being a designer and researcher myself, I always lean towards shorter research duration. A shorter timeframe keeps the discovery loop tight, leading to faster, more actionable insights. Meaning, you could have multiple smaller studies with built-in validation rounds given the two years.

So, when my colleague Anin mentioned their two-year project, I was curious! I wanted to understand the context where a two-year project makes sense and how they managed it. This is my key learnings from three separate interviews with the team.

If you’d like to quickly read the learnings, feel free to jump to the conclusion at the end of this writing.

The team is walking on a path towards a big green hill and the end of the road is hidden behind the hill.
It’s like going through a long journey where you cannot see where the end lies…

The project

The client, Bill and Melinda Gates Foundation (BMGF), wanted to facilitate access improvements to digital financial services for unbanked and underbanked demographics. So they hired researchers in multiple countries to look into this. One of the countries is Indonesia, and Somia CX is the one appointed for the research in the country.

The goal of the research is to gain understanding and insights surrounding the adoption of digital financial services among the unbanked and underbanked. The insights are then to be disseminated to the players in the industry, regulators, and the general public so that improvements in the sector can be accelerated.

To make sure that general insights can be driven from all participating countries, coordination and insights sharing with other participating countries are required. Due to differences in circumstances, each country’s research team is given the freedom to determine the research method as they see fit.

To ensure that the insights are valid and useful for the target audience, a target of 200 research participants and a timeline of two years with five milestones were set for each country. The five milestones are:

  1. preparation and desk research,
  2. field research phase 1 followed by an interim report,
  3. field research phase 2 followed by an interim report,
  4. field research phase 3, then
  5. overall synthesis and dissemination.

Now that we know what the project is about, let’s try to use this project as a proxy to answer my main question:

Does a 2 year long user research project make sense?

The simple answer is: yes.

Based on the team’s experience, the two-year timeline allows you to:

So yes, but with some considerations…

As you can see, a long-term project has its merit. But there are things that you need to consider to make sure that you get the optimal result.

Keep in mind that these are my takeaways from the three discussion sessions I have with Anin and Rayi. These takeaways have been shared with the team and you can see their comments in quotes.

The team members are:

Rigorousness and result validity

As mentioned above, 2 years project makes sense when you need a high level of result validity. So you might need to first answer this question: “How much rigor and validity level do you need?”

To assess this question, you need to understand who the target audience of your research is and how big is the risk of action taken from the research insight. In principle, the higher the risk is the higher confidence is needed. Keep in mind that the size of the risk is determined not by the researcher, but by the one who will utilize the research result.

Pictures of the team doing research in different situations and contexts.
The team talks with 200 people with different backgrounds, settings, and contexts to make sure their findings are valid across the nation.

For this project, a high level of rigor and validity are needed. This is because the insights from this project are meant to be used by a wide range of audiences, from businesses to the government. As this might affect business or regulatory decisions, it is important to minimize the risk of having invalid insights.

For illustration, these are the things that the team needs to address to achieve the necessary rigorousness:

Some of the above might not necessarily be the things that you need in your user research project or can be done in moderation. So it is very important to ask the question upfront.

Uka’s comment: “I personally feel like 2 years for research [is] too damn long. [With that time,] we should have moved to solution space. And along the way, we witnessed policy changes during the research.”

It must inspire actions

Two years of research will take up a huge amount of cost. To justify this cost, the research must be ensured to inspire its audience to act. Non-action will not be justifiable. So ask this question: “What action do we want to inspire from this project?”

Having a clear and action-oriented goal from the beginning will also help the team focus on the impact to be made. Even on a short project, lacking clear and action-oriented goals increases the risk of:

The effect above will be amplified even further on a two-year project. And it happened with the research team.

From their first two research, the team successfully identified a lot of problems that happened in the field. They carefully navigate through the findings to make sure that they answer the main objective asked at the beginning of the project. They then shared their findings with the industry players hoping to inspire action. But they found it failed to inspire action. This is because knowing the problem is only half part of what is needed, the other half is to find the right approach to tackle the problem.

As we know, finding the right approach is a huge task to tackle as it requires multiple trials and feedback loops. The team noticed that the players had already tried to tackle the problems, albeit, each tackle with different solutions and on various problem spaces. So there are already a lot of learnings accumulated in the field, but each parties have no access to the learnings the others have.

Since the research team understood that the research must inspire action, the team then decided to add the research goal from just capturing what happened in the field to also capture tried solutions and their effectiveness.

By having a clear goal of making the research actionable, the team can be sensitive and slightly adapt their last phase. The changes allow the findings to be more tangible and actionable for their audience.

If you’re interested, you can reach out to the team to gain access to the project’s actionable insights.

Proactively share the findings, don’t wait until the end

When I asked what their concern was during the project, Rayi, expressed that they “…were worried that the result may no longer be relevant by the end of year two since the beginning. Especially because the digital financial service industry changes at a significantly fast pace.”

So the team decided to disseminate their findings during the project, in between the field research phases. This enables them to make sure that they deliver their research insights when it is still relevant.

The feedback they gained from these dissemination processes also helped them understand what kind of insights their audience was already aware of and what not. This allowed them to adjust their next research phase to make sure that the insights were actionable at the end of the project.

The team shares the research's findings on multiple different occasions.
The team did multiple dissemination efforts throughout the project to make sure their insights were shared when they were relevant.

Rayi’s comment: “I felt relieved that at the last dissemination event, [the industry players] said that they encounter new insights that they did not know before and can act upon. … it’s a nice payoff for the extra work we did to make sure the research is actionable.”

Spare some time for uncertainties

A two-year project will naturally have things that go in an unexpected direction. This is one of the reasons why I like smaller projects as it would be easier to manage these uncertainties. These uncertainties can come from different parts of your project, and some are external such as:

If the team cannot be flexible, these uncertainties may cause them to rush, missed opportunities, and ultimately add extra stress. The three are not the best recipe to make sure the result is optimal.

Having extra time provides the team with the flexibility to adjust to multiple different situations that the team cannot foresee at the beginning. This ensures the goal of the research can be achieved optimally.

A trick that the team used to gain flexibility was allocating an empty one day in a week. During a retrospection time, the project lead mentioned to me that “…in practice, each field research phase can actually be completed in three months, but I would add an additional month on each phase just to accommodate for uncertainties. Otherwise, it’s going to be too stressful.”

Maintaining the team’s morale

Both researchers mentioned that working on the same research question for a lengthy period can be boring. Especially on the chance that there are no new insights to be gathered at the later phase.

As a researcher’s strength lies in their curiosity, boredom can easily dull their sensitivity to a subject. Which in turn will compromise the result of the research. This is the reason why managing the team’s morale is a very important thing to consider.

On this project, the team can be considered lucky as both have a passion for the topic. That passion helped drive them forward without facing significant burnout. However, they were aware that they could not rely on passion alone, so they tried to manage this by allocating some time for fun activities. Here are some things that the team claims “helped keep us [the team] sane”:

The team can be seen doing some sightseeing, looking at some local crafts, and enjoying local cuisine.
The team spends a lot of time moving around and talking in different contexts and areas; partially as an effort to be more immersed with the researched context but as well to keep things interesting for themselves.

Uka’s comment: “In retrospect, [I] would set up the project differently. Anin & Rayi are out of town for 6 months. [During the time they] did not meet their family in places they were unfamiliar with. Next, if there’s a similar project, I will make it so that they return back home more often just to calibrate and keep the sanity.”

Chin’s comment: “I’d also add that we may allocate more researchers so that the team can have alternative mindsets but always maintaining at least one core researcher per round of field work. [That way, the] researcher(s) can take a break plus fresh ideas are being injected.”

Answer the riskiest question first

After the discussions and hearing their experience, my personal view on this is that the project’s phases can be better adjusted to lessen the risk of the research being invalid. To do this, the team can change the focus of each field research phase to answer the riskiest question at the given phase.

Let’s take a step back first: the project’s goal was to capture the onboarding process and how it’s received across the country. This means, the researcher needs to make sure they have participants that represent the whole country’s population.

To accomplish this, they divide the country based on the National Development Planning Agency’s (Bappenas) development program. They then split the phases into:

The team then split the phases into:

In hindsight, the solution looks good because it covers the whole nation. But if we’re talking about research validity, it has a glaring issue for me. I think the riskiest thing that the team should check first is if all regions behave the same way.

From discussion with the team, I found that the team is lucky enough that all region shows the same behavioral pattern. So their research insights end up being fully sound and solid.

But what if there are glaring differences in behavior?

If that was the case, the team would not be aware of this in phase 1 as they would only have discovered one type of behavior. The soonest the differences in behavior can be discovered will be in the middle of phase 2. This will make it hard for them to check if the causes of differences exist and why they have different implications in the previous region. Which will be a huge problem if country-wide rigor and validity are important. Worse when significant differences are only found in the last phase.

I think it would be better if they changed the approach to checking the riskiest assumption. Here’s what I think should be done:

With the approach given above; the riskiest question is answered right away. The team would know the similarities and differences in each region. The ideas that come up would already take into account the similarities and differences of each region as well as be thoroughly researched.

Granted, the solution I propose is less budget-friendly as that means triple the travel funds. But considering the reason for a two-year project is to have higher validity and reduce risk, this is the better option to pick. Keep in mind that knowing the problem at the end of the project may put the result at a significant risk and can be costly as well.

Chin’s comment: “I agree that in an ideal situation, with the multiple regions per phase as a better way to have a broader view. But if I am being practical, I will still go with one region first then be flexible with it, but not considering the time. So in short, I will still go with one region per phase and see what Round 2 unfolds, then make a call for Round 3.”

In conclusion

So, depends. The two-year project does have merit albeit with very important things to consider.

In a TLDR; fashion, here are the things that you need to consider when faced with the prospect of a long-term research project:

Now let me ask you:

Please feel free to share your thoughts below.

Huge thanks and credits to Somia CX especially Anindya Fitriyanti and Rayi Harjani for sharing their personal experience and thoughts with me. Special thanks to Ketut Sulistyawati, who enables my initiative to write this and share her thoughts on this piece.