Gathering Requirements for Analytics Projects: Lessons from Experience

When embarking on an analytics project or any data-driven initiative, gathering requirements effectively is crucial. Misunderstanding or overlooking what the client truly needs can lead to wasted time, effort, and resources. In this blog, we’ll delve into the importance of gathering requirements effectively and share some valuable insights on bridging the gap between business needs and technical solutions.

Real-Time Data Requests: Uncovering the Actual Need

One common scenario in analytics projects is when clients initially request “real-time data.” However, upon further discussion, it often turns out that they only need data updated a few times per week. This highlights the importance of thoroughly understanding what the client is truly asking for.

Gathering requirements is not just a preliminary step; it is a foundational one. It sets the tone for the entire project. As Ben Robichon points out, businesses might have a general idea of what they want, like “generative AI” or “machine learning,” but they might not fully grasp the technical implications or capabilities of these technologies.

Asking the Right Questions

Businesses often struggle to articulate their needs. They might have read about the latest tech trends and think they need a particular solution without understanding its feasibility or appropriateness for their specific situation. This is where we, as data professionals, need to step in and ask the right questions. Rather than making assumptions, we should go back and ask clients what they specifically want and why they want it.

Robichon emphasizes that effective requirement gathering can be a career-defining skill. It is essential for the success of any project and can set you apart in consulting or within your organization.

Bridging the Gap Between Business and Technical Teams

There is often a significant gap between the technical side and the business side of a project. These two groups have different skills, goals, and perspectives. Business stakeholders might base their requirements on what they’ve read in articles, which may not align with technical realities. This disconnect can lead to misunderstandings and unrealistic expectations.

To bridge this gap, it’s crucial to have open and ongoing communication. We need to translate technical-sounding requests into the actual needs of the business. For instance, a business might request a machine learning model when a simpler decision tree might suffice. Asking “why” repeatedly helps uncover the true motivations and requirements behind these requests.

The Importance of Understanding the “Why”

Understanding the underlying business needs is vital. For example, if a content manager asks for the average conversion rate for videos, digging deeper to understand why they need this information might reveal that they want to know if video viewers are more likely to make purchases. This insight allows us to provide a more appropriate and effective solution.

There are modern solutions, like Estuary, that can make real-time data more accessible. However, it’s still essential to understand the actual business needs to determine whether such solutions are necessary.

Defining the Problem

Often, businesses may not have a clear idea of what they are asking for because they are constantly dealing with problems and issues. Part of our job is to help them define their requirements more concretely. Being able to speak both the technical and business languages makes us valuable assets in this process.

Identifying Stakeholders

After understanding the business needs, the next step is to identify the key stakeholders and understand their perspectives. This ensures that the project meets their needs and expectations.

The “Why” and “Who” Behind Projects

It’s not enough to execute a request blindly. Understanding the purpose and intended users of a project is crucial. For instance, the speaker shared an anecdote about building a dashboard at Facebook without fully understanding the context, which led to an embarrassing situation in a meeting. Knowing who the end users are and what they will do with the product helps us see the bigger picture and align our work with the overall business goals.

Making Work Visible

As engineers, much of our work involves “prep work” that enables the “real” work others want to see. This prep work often goes unnoticed. Therefore, it’s important to make our work visible and understandable to others. Creating visual representations, such as dashboards or diagrams, helps communicate our work and its value.

Collaborative Requirement Gathering

Finally, gathering requirements should be a collaborative process rather than a formal, one-way document exchange. Open communication, iteration, and building trust and understanding between stakeholders lead to better outcomes. This collaborative approach fosters empathy and ensures that we are addressing the real business needs.

In conclusion, gathering requirements effectively is a cornerstone of successful analytics projects. By asking the right questions, understanding the business needs, and bridging the gap between technical and business perspectives, we can deliver solutions that provide immediate and lasting value.