When you ask engineering leaders (be it CTOs or VPs of Engineering or Directors of Engineering) in companies and startups, they will generally give fairly similar answers to these questions. Company processes at different sizes may experience a bit of a struggle, which is why it's important to know some of the strategies for creating your data engineering roadmaps.
What is a data engineering roadmap?
The data engineering roadmap is a list of features that the engineering team aims to build. It exists so that other stakeholders in the company (CEO, CFO, or Head of Product) can see what the direction is. This helps them feel confident that the correct features are focused on. This allows visibility into the value they are producing and it helps prevent new features from being requested in a reactive haphazard way.
Regardless of its formal format, it's some sort of list that is easy to understand. It's recommended to leave implementation details and background information and business reasoning off of it altogether. Remember who is reading it (see below).
There is a surprising amount of agreement among engineering leaders about the general format for a data engineering roadmap. It is clearly defined and prioritized for timely projects, and less defined for long-term projects.
Components of the Data Engineering Roadmap
Current
These are aspects of the data engineering roadmap you are currently working on. This bucket should not change. At the end of a sprint features that are done are removed and whatever has the highest priority in the next bucket is added.
Next
This bucket is usually somewhere between two weeks to three months out. The stakeholders grab features from the future bucket and put them IN PRIORITY ORDER in this bucket.
Future
Focus on features that are paramount for the future of the data engineering roadmap. The features should have a strong likelihood of implementation. It is up to your organization to recommend what is most critical for success. Not to mention, make an effort to implement feedback from key stakeholders. These features will be prioritized work within the next three to 12 months. If set longer, priorities may be changed too frequently that they get discarded.
Vision
The vision or "mission" or "general direction" of the engineering team should be part of the roadmap. It's a sentence or two that describes what you are working towards. It may change, but more often it settles in as the company progresses toward product market fit. The vision can be referenced when stakeholders want something that doesn't fit it. Either the vision needs to change or they are recommending something that may divert away from the core product that engineering should be focusing on.
Who creates the Data Engineering Roadmap?
Engineers are responsible for keeping this updated, but there may be stakeholders or other company leaders who have input as well. This may include the CTO, CEO, Head of Product, and Head of Marketing, but also may vary. Remember, the fewer the stakeholders, the better productivity for the entire organization. The last thing is to have department leaders lose focus on their specific responsibilities within the company.
How often should we update the roadmap?
Some recommend having a constant conversation between the stakeholders about features and company direction to keep everyone in sync. Others recommend having a meeting after every sprint to update the priorities and others say every quarter is enough. Find a system that works for your team. If you are in charge of creating the data engineering roadmap, you will probably polish it continually as a living document.
A Data Engineering Roadmap Example
This is for a company that is trying to be your one-stop-shop app for hiking maps.
Current:
Allow hikers to rate hikes.
integrate Colorado public hike dataset with our data.
Next:
Allow hikers to friend each other.
Allow hikers to recommend hikes to each other.
Allow hikers to see notifications of when one of their friends completes a hike.
Future:
Allow hikers to recommend hikes to the system. (crowdsource)
Add a way for shoe manufacturers to recommend shoes through our platform to hikers depending on which hikes they are going on.
Parse hike data from Europe.
Vision:
Allow hikers to find and see maps of hikes that suit their physique and tastes in any country.
This example highlights how the vagueness amplifies as you progress into the future. Next is in priority order. The vision doesn't actually take into account the shoe recommendations, but it seems like a legit business vertical to monetize on and it is still very much within the realm of hiking.
This is an intuitive approach to a data engineering roadmap. Remember to modify when needed to integrate the best system that works for your company or situation.
A roadmap should never be set in stone. Rather, it must be a livable document to reflect the current priorities of the engineering team's most important client—the company.
Your time matters and your systems should work. Invest in a remote access system built from the ground up for industrial control networks, uniquely secured with moving target defense, with no compromises on security.
Get your quote at https://dispel.io
