Tag: Data Architecture

StreamFlake: Real-Time CDC Pipeline with Kafka and Snowflake

Building a Real-Time CDC Pipeline with Kafka and Snowflake (Part 1) 🚀 In a modern enterprise, there are several compelling reasons why you would/should want to replicate changes from one place to another, and even more for doing it constantly, reliably, efficiently, and fast. In this article, we will show how we replicated not one… Continue reading StreamFlake: Real-Time CDC Pipeline with Kafka and Snowflake

Databricks + Fabric: Better Together

  Organizations continuously seek robust, flexible, and scalable solutions to handle their diverse data needs in today’s rapidly evolving data landscape. Databricks and Microsoft Fabric have emerged as leading platforms, each with strengths and target personas. While they share some similar technical features, their unique approaches to data management make them highly complementary services. This… Continue reading Databricks + Fabric: Better Together

Migrate from Stored Procedures to dbt

We all love shortcuts. That’s why many organizations use stored procedures in their data warehousing processes. They’re an excellent solution for packaging and scheduling complex transformations through logical conditions. Stored procedures have become a core building block of teams’ workflows. But do they have any cons?  Data workers indicate a noticeable increase in data warehouse… Continue reading Migrate from Stored Procedures to dbt

OpenLineage and Airflow Simplify Data Lineage

The GDPR (General Data Protection Regulation), asks organizations to implement data lineage for a clear understanding of the data used within the systems. Paying attention to data lineage not only helps organizations comprehend complex issues within their data operations but also helps explain and justify errors which your users might stumble upon. Primarily, the function of data… Continue reading OpenLineage and Airflow Simplify Data Lineage

Forget About Managing Data Warehouse Infrastructure: Use Amazon Redshift Serverless

Data analytics use is on the rise and organizations are constantly searching for ways to remove the hurdles that limit access for team members with minimal expertise. Data warehouses are necessary systems used to report and analyze data, but they require quite a lot of learning. Not everyone has the time and capability to learn how… Continue reading Forget About Managing Data Warehouse Infrastructure: Use Amazon Redshift Serverless

Replicate Data and Utilize Change Data Capture (CDC) Easily with Datastream

As billions of internet users exchange information with each other and as platforms diversify, data grows exponentially together with the complexity to manage, organize, and analyze it. Companies are faced with dull data architectures that require better solutions, and the ideal alternative that exists is called change streaming. Change streaming relates to the direct display of data… Continue reading Replicate Data and Utilize Change Data Capture (CDC) Easily with Datastream

Your strategic Data Partner

Get started