From Cron to Modern Data Stack (MDS): Dataflow Automation and Its Current State
The concept that makes the technological miracles of today possible are defined by data. Enormous amounts of data are collected...
Without the answer to that question, your marketing team is stuck with guesswork and hope. If you are launching a brand new product or company, you may have limited data. However, established companies can leave guessing behind.
Our client, who has revenues of approximately $33.5 billion in the energy industry, has a global brand and leverages many different marketing channels. They wanted to know which levers to pull and which to ignore to enhance their reputation.
Vertical: Marketing Optimization
Model: Natural Language Processing
When we started working together, they already had a few good ideas regarding the most significant drivers of their online reputation. However, they didn’t know which drivers of their brand and reputation were the most effective. That’s when we recommended harnessing the power of a natural language analysis engine. This technology lets us collect and analyze data from millions of sources online to optimize marketing results. Moreover unifying all this data into one source let us analyze data from all customer touchpoints: emails, call center interactions, webchats, transaction logs, earned media, paid media, and social media were all in scope for this study.
The sales and marketing implications are non-negligible: unified data provides a single access point to ever-increasing amounts of customer information. Marketing automation tools, CRM systems, and campaign monitoring platforms all become connected at the data layer. When user-generated information (web & mobile activity, sensor data, etc.) is added to the mix, we can say that unified data provides the full 360-degree customer view. When data is unified and analyzed together with the transactional data, more reliable Customer Lifetime Value estimates are obtained.
The benefits of Machine Learning segmentation include the potential to scale on-demand, removing the human bias, and dealing with an unlimited number and size of segments. Automated, ML-based segmentation is a proven way to cut down manual processing costs and remove the need for labor-intensive tasks. Such capabilities would be unimaginable in a traditional approach, based on siloed data warehouses and human processing. But when the data is unified and predictive technology is employed, there are no limits to how you can let your customer data do the work for you.
This project delivered results through a series of phases:
This project included ingesting, correlating and aggregating all publicly available media sources on the open web related to the company or corresponding reputation drivers.
Obtaining a large volume of data was the next critical step on the market. In this case, we gathered data from the following sources: the public Web, Twitter, Facebook, and LinkedIn. We looked for every data point that referenced the company, its products, and reputation drivers. After all data streams have been unified it is now possible to clean and enrich your data. With all data centralized, the ingestion of new company data and outside third-party data can now be utilized to improve the quality and accuracy of insights.
In this phase, we brought order to the data. All data was scraped, ingested, and processed using a series of NLP techniques. The analysis engine used several capabilities to bring order to the data including:
For advanced processing analysis, data was persisted in a graph database for advanced in-pipeline analytics. The underlying data volume was very large, requiring in-memory processing for ongoing analysis.
The accuracy of the prediction becomes a guarantee for well-invested resources and reduces the time potentially wasted with unsuccessful leads. When data is unified and analyzed together with the transactional data, more reliable CLTV estimates are obtained.
We delivered a series of custom dashboards focused on topic modeling, time-series, anomaly detection and aggregation summaries in an interactive application. Due to the data volume, we used both large precomputation and near real-time aggregation indexing jobs to provide interactive dashboards.
In order to carry out a successful project, we created a foundation for success. Using AWS, we built a custom data platform. This platform was built to scale. It has the ability to ingest gigabytes of new data daily. That means we can deliver real-time marketing optimization insights rather than relying on yesterday’s data.
We have seen how unified data and predictive technology enable cheaper segmentation efforts, more accurate behavior modeling, and quicker inference for targeted marketing campaigns.
The same big data used for segmentation and behavior modeling can be repurposed to create personalized messages that customers are more likely to react to. Moreover, quick access and real-time inference are now possible and that opens a whole world of possibilities since timing is a crucial aspect in customer interaction all throughout the purchase consideration cycle.
The possibility to extract relevant ads and provide speedy responses to customer interactions can shorten lead-to-cash cycles. Employing unified data and predictive technology also translates to less time and fewer resources for building personalized interactions with your customers. Such a strategic advantage is crucial for a successful marketing strategy.