Paper Contents
Abstract
Change Data Capture (CDC), a contemporary data engineering technique, effectively tracks changes made to Online Transaction Processing (OLTP) systems and sends them to analytical platforms downstream. Instead of reloading entire datasets, which can be resource-intensive and slow in standard ETL procedures, CDC focuses on recording only the inserts, updates, and deletions. This leads to better performance and almost instantaneous analytics. In this survey, we look at how to implement CDC using SQL Server, Azure Data Factory (ADF), and Databricks. We will discuss current methods such as log-based CDC in SQL Server, orchestration patterns using ADF, and the advanced processing capabilities offered by Databricks Change Data Feed (CDF) and Delta Lake. This study discusses the rationale behind this approach, the techniques employed, possible roadblocks, and best practices for creating scalable CDC pipelines.
Copyright
Copyright © 2025 Mr. Bhairav D. Attarde. This is an open access article distributed under the Creative Commons Attribution License.
