Sync Modes
Sync modes shape how data moves from your sources into the systems where you analyze and act on it. The right mode helps you balance freshness, load on upstream systems, and how closely your destination mirrors the source—whether you prefer a clean snapshot each time or a steady stream of what changed.
What sync modes are for
At a high level, a sync mode combines two ideas:
- How we read — Sometimes you want everything on every run; other times you only need what’s new or updated since the last successful sync.
- How we write — Whether the destination should be replaced wholesale or updated in place as new data arrives.
You can tune these behaviors per stream within a connection, so different datasets in the same pipeline can follow different rules when that makes sense for your team.
Common patterns
Connect supports the familiar patterns from modern data integration: full refreshes for simplicity and incremental paths for scale.
Full refresh, overwrite-style delivery
The idea: Each run pulls a complete picture of the selected data from the source, and the destination reflects that picture end to end—useful when you want your warehouse or downstream app to always match “what’s true right now” in the vendor system you are mirroring.
Why teams like it: Conceptually simple. No guessing about what changed; you’re always looking at a current slice of the world. A natural fit for smaller datasets or when a full reread is acceptable relative to how often you sync.
Incremental reads with merge-style delivery
The idea: After an initial baseline, later runs focus on records that were added or changed, then fold those updates into what’s already stored—so you’re not re-pulling the entire history on every schedule tick.
Why teams like it: Lighter on source APIs and databases, friendlier to frequent syncs, and a better match for large or chatty datasets—as long as your source exposes a reliable way to know what’s “new since last time” (timestamps, monotonic IDs, or similar).
Things to keep in mind: Incremental models need a clear signal for updates and a stable way to match rows on the destination side. Edge cases like hard deletes in the source often call for a deliberate strategy — for example, occasional full refreshes or other reconciliation.
Choosing a direction
- Favor full refresh when the dataset is modest, you want the destination to be an obvious mirror of source state, or operational simplicity matters more than minimizing API volume.
- Favor incremental when volume, rate limits, or sync frequency make “read everything, every time” costly—and when your source can express change in a trustworthy way.
Schedules and recovery — How often you run syncs and how interrupted runs behave ties into sync schedules. Connect is designed for predictable, recoverable runs without leaving destinations in awkward half-states.
Exact behavior may vary by connector. If you need something specific for a go-live, your Propexo team can walk you through the details for your stack.