SDRF: A unified CDC abstraction for cloud-native SaaS data replication
2026
The proliferation of Software-as-a-Service (SaaS) applications has created significant data integration complexity. Connecting N SaaS sources (e.g., Salesforce) to M analytical targets (e.g., data lakes, data warehouses) traditionally requires N×M bespoke integrations. This paper presents the SaaS Data Replication Format (SDRF), a JSON-based Change Data Capture (CDC) envelope that decouples SaaS source connectors from downstream targets through a common intermediate format on object storage. Unlike database CDC formats that rely on transaction logs, SaaS systems expose only timestamp-based modification fields and provider-specific deletion indicators, requiring a different approach. This paper makes two primary contributions and two supporting design points. The primary contributions are: (i) a (x=epoch_ms, y=event_count) coordinate mapping that translates timestamp-based SaaS changes onto a monotonically-increasing total order, for which we provide informal correctness arguments for monotonicity, restart determinism, and bounded staleness; and (ii) a table-level checkpoint-isolation protocol with a linked checkpoint chain, enabling per-table parallel extraction with independent resumability. The supporting design points are: the Link Manifest — a manifest-based integration contract that decouples source and target implementations — and a compare-and-detect schema-evolution procedure. A production deployment of SDRF powers Salesforce data ingestion into Apache Iceberg on S3, and we report measurements from that deployment: a controlled 10 M-row Salesforce ingestion completes in 6 min 20 s.
Research areas