From Cron to Modern Data Stack (MDS): Dataflow Automation and Its Current State
The concept that makes the technological miracles of today possible are defined by data. Enormous amounts of data are collected...
The client is an OKR (Objective and Key Results) software startup that improves employee engagement and project success by aligning people, strategy, and results. The company was looking to re-architect their data platform to meet the requirements of a new segment of large enterprise customers. The client planned to expand their platform capabilities to compete with other vendors offering continuous performance management.
Their technical goals included improved backend performance as well as the ability to provide additional insights via client dashboards and advanced analytics. To achieve their goals, they developed a multi-year roadmap that detailed their goals for building a new data platform in order to enable robust analytics and establish the foundation for data science and machine learning initiatives. Blue Orange was brought in for a short-term consulting engagement to provide an assessment of the current roadmap and guidance around architectural improvements to reduce cost and time to market.
The client had planned a future architecture through an AWS centric data platform. They also required a Machine Learning Architecture to help their Data Science Team in building, iterating, and deploying models quickly with little to no Devops support. In addition to providing NLP based recommendations to their clients, the company assumed that these projects would be zero maintenance and would require no support post-completion within the aggresive timelines.
This planned future architecture would work well to support the product roadmap; however, Blue Orange’s assessment identified potential risks to the timeline:
Based on the potential risks the Blue Orange Audit Team identified with the timeline, they were able to create recommendations that would address the data flows, service time, and governance structure. To accelerate the development roadmap, Blue Orange proposed adopting a buy (not build) strategy, using SaaS products.
The solution was designed around the following considerations:
In order to implement these considerations, the Blue Orange Audit Team created a listing of suggested data analytics and cloud tools that would be the best fit for their company goals. These tools included:
The modified architecture proposed by the Blue Orange Audit Team was aimed at reducing the time to market while taking into consideration team capabilities and proposing a buy (not build) strategy using SaaS products. We recommended sprinting to market on the backs of best-in-class SaaS products to allow the features and capability requirements to be validated quickly.
These capabilities and the increased speed to market would provide a distinct competitive advantage for the client, while reducing configuration and maintenance overhead for the company. After validation, the client’s Data Science Team can focus on optimizing the performance and cost of these solutions to support scaling over the coming years.
To learn more about Blue Orange’s Machine Learning and Data Transformation capabilities, please contact our team. If you are interested in reading more about our custom client solutions, you can view our full Case Studies listing here.