Skip to content

sequinstream/sequin

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sequin

Sequin is the fastest change data capture (CDC) platform for Postgres. Sequin makes it easy to stream Postgres to streaming platforms, queues, search indexes, and more (e.g. Kafka, GCP Pub/Sub, another Postgres, etc.). You can backfill existing rows and stream new changes in real-time.

Sequin is a standalone Docker container that you can deploy next to your Postgres database. Alternatively, you can use our hosted offering or we'll manage Sequin for you in your cloud.

Sequin is open source. To help us make this project great, tell us what you're building in our Discord server or Slack community.

Getting started

See our quickstart to get up and running in your dev environment in about 5 minutes.

Alternatively, you can try Sequin for free on Sequin Cloud. Follow the instructions in the app to start streaming your data in a couple minutes.

Performance

Sequin delivers industry-leading performance for change data capture (CDC), sustaining 50k operations per second (40 MB/s) with 55ms average latency.

Tool / Deployment Sustained throughput Avg latency 99p latency
Sequin >50 k ops / s 55 ms 253 ms
Debezium · MSK 6 k ops / s 258 ms 499 ms
Debezium · Server 23 k ops / s 210 ms  440 ms 
Fivetran - 5+ minutes -
Airbyte - 1+ hours -

Read more about our benchmarking methodology and performance in our docs.

Use cases

Sequin works great for change data capture use cases like:

  • Streaming events: Stream database changes as events for other services to consume using Kafka, SQS, Pub/Sub and more.
  • Audit logging: Track, enrich, and record every change in your database for compliance or feature development.
  • Maintaining a cache: Keep application caches in sync with your database by streaming changes to Redis compliant caches.
  • Refreshing search indexes: End-to-end tooling to warm search indexes like Typesense and Elasticsearch and keep them in sync with sub-second latency.

Sinks

We're adding new destinations almost every week.

Icon Sink Docs Description
Azure EventHubs Azure EventHubs Reference Publish messages to Azure EventHubs
GCP Pub/Sub GCP Pub/Sub Reference Publish messages to Google Cloud Pub/Sub topics
Elasticsearch Elasticsearch Reference Index database changes with Elasticsearch
HTTP Pull Sequin Stream (HTTP GET) Reference Consume changes directly from Sequin with exactly-once processing
Kafka Kafka Reference Stream changes to Apache Kafka topics
Kinesis Kinesis Reference Send messages to Amazon Kinesis streams queues
Meilisearch Meilisearch Reference Index database changes with Meilisearch
NATS NATS Reference Stream changes to NATS subjects
RabbitMQ RabbitMQ Reference Publish messages to RabbitMQ exchanges
Redis Stream Redis Stream Reference XADD to Redis Streams
Redis String Redis String Reference SET to Redis keys
S2 S2 Reference Send messages to S2 streams
SQS SQS Reference Send messages to Amazon SQS queues
SNS SNS Reference Send messages to Amazon SNS
Typesense Typesense Reference Index database changes with Typesense
Webhook Webhooks (HTTP POST) Reference Send changes to any HTTP endpoint

Killer features

  • Bring your database: Sequin is not an extension. It works with any Postgres database version 14+.
  • Never miss a change: Sequin ensures 100% delivery of database changes to sinks with strict ordering.
  • Filters: Write custom filters to exclude or include changes to sink.
  • Transforms: Transform message payloads by writing functions in low-latency Elixir.
  • Routing: Route messages to specific topics, endpoints, or indexes.
  • Exactly-once processing: Sequin ensures exactly-once processing of database changes using idempotency keys.
  • Backfills: Backfill sinks at any point in time. Run partial backfills to only replay specific rows.
  • Infrastructure as code: Manage Sequin as code using a YAML file paired with the Sequin CLI. Or use our Management API.
  • Rich web console: Your team can configure and monitor sinks from a full feature web console.
  • Observability: Sequin ships a Prometheus metrics endpoint.

How Sequin works

Sequin connects to any Postgres database. Specify the schemas and tables you want to stream, as well as optional filters and transformations. Route changes to sinks like Kafka, Google Pub/Sub, Redis, etc.

When you setup a sink, you can opt to backfill data from the source table to the sink.

After setup, Sequin will stream new changes to the sink as they occur in real-time. If there are any issues with delivery, Sequin will automatically retry delivery with exponential backoff.

Sequin comes with a web console/UI for configuration and monitoring. You can also configure Sequin as code using YAML config files and our Management API.

Why Sequin

We all know Postgres is great for storing and querying data. But what about when you need to stream changes to other systems?

Existing tools like Debezium require significant infrastructure to operate (Kafka), endless guess-and-check configuration, and limited tooling. Other ETL tools (e.g. Fivetran) run in batches and are expensive.

This leads many teams to build brittle, homegrown solutions that are difficult to maintain and lack essential features.

Fast, simple, reliable Postgres CDC

Sequin provides fastest, simple Postgres CDC. It's tuned to Postgres to stream changes in real-time, provides strict guarantees, and doesn't require complex infrastructure to operate.

Contributing

See CONTRIBUTING.md for detailed instructions on how to contribute to Sequin.