4 Keys to ML Project Success

Executive Summary 

In order to capture the desired value from your investment in Machine Learning and Artificial Intelligence, companies need to take a business-centric focus to their technical solutions. Companies find working with data, especially data at high velocities, difficult to manage because they failed to do the proper planning and development before embarking on an AI initiative. There are 4 keys areas of focus to reduce the risk of wasting time and resources on AI and Machine Learning.  

• Data Infrastructure
• Business Objectives
• Machine Learning Solutioning
• Business Evaluation Feedback

Having the proper infrastructure and business goals in place will allow you to measure your success and find the places for greater ROI. To learn more about Blue Orange Digital go to blueorange.digital.

Introduction

What do terms like Data Science, Machine Learning, and Artificial Intelligence mean to you? If the first answer that comes to your mind is buzz words with abstruse meaning, you have come to the right article. If you walk through the graveyard of poorly executed initiatives you will find these buzzwords on the tombstones. Here at Blue Orange Digital, our goal is to provide you with the blue(orange)print to successfully implement these transformative solutions by providing a different perspective. We take a business-centric focus to your technical problems, working closely with your team as partners, to develop solutions that meet your unique business needs. 

Do you have the right data infrastructure to gain the most value out of investing in Data Science? 

The first step to gaining insights and seeing the value of any data science project is to have the proper upstream data infrastructure to support your mission. Is it possible to perform machine learning and AI on csv files? Maybe, but you are going to waste a lot of time (and money) trying to work around a suboptimal pipeline. There are three capabilities that are the pillars of success before you should consider using ML/AI:

• Modern, sophisticated data pipeline
  • Moving, cleaning and organizing data into flexible, cost-efficient storage is the key to unlocking the potential of your data for machine learning initiatives
  • For more information on data pipelines, please view our article on cloud data storage.
• Business Intelligence/Analytics
  • What is the story you want to tell with your data? Your business intelligence suite should be used to define the key performance indicators (KPIs) you hope to achieve. 
  • For a better understanding, please view our article on applying analytics.
• Decision Support Tools (Dashboards, visualization, etc.)
  • It should be easy for your leadership to clearly analyze and interpret the metrics of your data.
  • For a better understanding, please view our article on the best visualization tools.

What is the problem we are trying to solve?

This is the most important step in any digital transformation project and should be answered before you consider moving forward. In our experience, business problems are often framed in the ML solution that is needed to solve them. Have you ever walked into a meeting and someone proclaims, “we need NLP!” When the technology is talked about as the solution, you are off to a bad start. The business problem should be an outcome you hope to achieve with clear metrics that you want to measure against, including how success is defined. An example of a good business problem would be, we want to reduce the number of man-hours needed to process incoming documents. Determining the right use cases and business problems will be heavily influenced by the data and analytics you are already capturing.  

Is Machine Learning the Right Solution? 

With a well-defined business problem, you can now start working through your possible solutions. There is no reason to over-engineer a solution, sometimes a simple regression model or a rules-based algorithm works best. If the problem is more sophisticated than a simple solution can solve, then you can move on to more complex methods.  Since you already have an infrastructure in place, we know you have the right data to properly train a machine learning model. Now it’s time to develop the technical solution. (*note all the needed steps before you reach “we need NLP!”)

There are a couple of different stages to technical solutions to make sure you are answering your business objective. These are universally agreed upon best practices of machine learning architecture and definitions are lifted from the “AWS Well-Architected Framework – Machine Learning Lens.” If you wish to have a deeper understanding of these subjects, we recommend this article as a great read.  

1. Feature Engineering- Feature engineering is a process to select and transform variables when creating a predictive model using machine learning or statistical modeling. Feature engineering typically includes feature creation, feature transformation, feature extraction, and feature selection.
2. Model Training - In this phase, you select a machine learning algorithm that is appropriate for your problem and then train the ML model. As part of that training, you provide the algorithm with the training data to learn from and set the model parameters to optimize the training process.
3. Model Evaluation - After the model has been trained, evaluate it to determine if its performance and accuracy will enable you to achieve your business goals. You might want to generate multiple models using different methods and evaluate the effectiveness of each model.

Business Feedback Loop

By now you are receiving outputs from your solutions that need to be measured against the metrics you chose while framing your business problem. This is where iteration comes into play. Are you getting the metrics you expected? No? Then you need to loop back and rethink your solution. The key here is that the business metrics are driving your decisions, that should be the first step of your review. Next, you may want to ask questions like, are the metrics realistic? Do your current analytics and decision support tools provide the right information? If your goal was to decrease the man-hours needed to perform a task, do we have the right tools to measure that data? What is the appropriate amount of time to validate our findings? Using your data through this business-focused lens you’ll be able to clearly determine the success of your digital transformation initiatives. 

About the Author: Matthew Paris, Vice President of Data Science at Blue Orange Digital