Real-Time Event Handling Between Your Platform and SaaS Apps: Why ETL Is Not the Right Tool

A pattern shows up repeatedly in conversations with product engineering teams scaling integrations: the team starts looking for "open source ETL tools that support real-time event handling between our platform and external SaaS apps," and the search itself is the symptom of a category mismatch. ETL was never the right tool for that problem. ETL is for moving data from operational systems into analytical systems. Real-time event handling between your product and the SaaS systems your customers use is integration infrastructure, and the architectural primitives are different.

This post pushes back on the framing of the question, then walks through what the right architecture actually looks like, why it matters, and how Ampersand approaches it. The goal is not to argue that ETL tools are bad. They are good at what they were built for. The goal is to argue that they are the wrong layer for the problem most product teams are actually trying to solve, and that the existence of open source ETL tools should not be confused with the existence of open source real-time integration infrastructure.

The Category Confusion Worth Naming

ETL stands for Extract, Transform, Load. The original architectural intent was to pull data out of transactional systems, reshape it, and load it into a data warehouse for analytical use. ELT, the modern variant, defers transformation to the warehouse but inherits the same shape: data flows from operational sources into analytical destinations on a periodic schedule. The dominant ETL and ELT tools (Airbyte, Fivetran, Stitch, Meltano, dbt for the transformation layer, and the various cloud-native equivalents) optimize for throughput, schema management, and warehouse loading semantics.

Real-time event handling between your product and external SaaS apps is a fundamentally different problem. You are not loading a warehouse. You are reacting to changes in a customer's CRM, ERP, ticketing system, or messaging platform inside your product, in time for the user to feel the response. You need sub-second latency, not hourly batches. You need bi-directional writes back into the customer's systems, not one-way data movement. You need per-tenant runtime isolation, not shared sync infrastructure. You need compliance coverage that treats your customer's data as sensitive operational data, not as warehouse rows.

When teams try to bend an ETL tool to do this, they discover the bend hurts and can actually completely snap your product integrations. They run sync intervals tighter and tighter to approximate real-time, and they hit rate limits. They add custom logic on top of the ETL output to fan out writes, and the custom logic becomes its own integration layer. They handle multi-tenancy by spinning up many instances of the ETL pipeline, and the operational footprint balloons. After enough of this, they realize what they actually need is native product integrations built for real-time, multi-tenant, bi-directional event handling. ETL was never going to be that.

We covered this category boundary in detail in what is integration infrastructure. The short version is that real-time integration between products and SaaS systems is its own architectural layer, with primitives like webhooks, Change Data Capture streams, platform events, granular write APIs, and per-tenant orchestration. ETL is something else.

Ampersand Is Not an ETL Platform

It is worth saying this directly. Ampersand is not an ETL or ELT platform. It is integration infrastructure for product engineering teams that need to ship native, bi-directional, real-time native product integrations between their SaaS product and the systems of record their customers use. The architecture is purpose-built for the problem of multi-tenant, customer-facing integration, with primitives that ETL tools do not have and would not benefit from adding.

What that means concretely is this. Ampersand handles bi-directional read and write with hundreds of CRM, ERP, GTM, HRIS, accounting, life sciences, and healthcare systems. It supports custom objects and dynamic field mapping. It manages authentication with automatic token refresh. It executes scheduled reads, backfills, and bulk write optimization, and it exposes on-demand read and write API endpoints for the cases that need synchronous interaction. It includes Subscribe-style real-time event handling, where changes in the customer's system are pushed to your application as webhooks within seconds. It treats integration configuration as version-controlled YAML that ships through your CI/CD. It exposes dashboards with logs, alerting, error handling, and quota management. It is SOC 2 Type II audited, ISO certified, and GDPR compliant, with the integration runtime fully in scope.

None of this looks like ETL because none of this is ETL. The mental model that maps closely is "Stripe for integrations": a platform that abstracts the runtime of a hard problem so your engineering team can build the product on top.

What Real-Time Event Handling Between Products and SaaS Apps Actually Looks Like

Let's get specific about the architecture that real-time event handling actually requires. There are three primary integration patterns at play.

The first is the read pattern, where your product needs to know what is currently in the customer's system. This can be served by scheduled reads (pull the latest state every N minutes), backfills (pull all historical state on first connection), or on-demand reads (pull specific records when the user needs them). For real-time use cases, scheduled reads at sufficient frequency are sometimes enough, but more often you need a streaming pattern.

The second is the streaming pattern, where your product receives events as they happen. Different SaaS systems expose this differently. Salesforce has Change Data Capture and Platform Events. HubSpot has webhooks and a Pub/Sub layer. Microsoft Dynamics 365 has Dataverse Web API webhooks. Zendesk has triggers and webhooks. Gmail has push notifications via Pub/Sub topics. Each of these has its own subscription model, retention policies, gap recovery semantics, and authentication patterns. A real-time integration platform abstracts these differences into a unified consumer model.

The third is the write pattern, where your product changes state in the customer's system. This is the pattern ETL tools handle worst, because they were not designed for it. Real-time write paths need idempotency keys, retry semantics that distinguish transient from terminal failures, audit logs of every operation, and the ability to surface errors back to the user in a way they can act on. Bulk write optimization matters when your product needs to push many records at once. Fine-grained write APIs matter when your product needs to update one field on one record in response to a user action.

A real-time integration architecture that handles all three of these patterns, across many SaaS systems, multi-tenant, with the observability and compliance posture an enterprise customer expects, is integration infrastructure. ETL covers a small part of the read pattern and almost none of the rest.

The Rate-Limit Math That Forces the Issue

There is a numerical reality that pushes teams off the "ETL with tight intervals" path eventually. SaaS APIs enforce rate limits, and those rate limits are designed assuming consumers are doing periodic reads, not high-frequency polling.

Salesforce gives most orgs 15,000 to 100,000 API calls per day depending on edition and licensing. HubSpot gives 150,000 to 250,000 daily calls depending on plan, with burst limits per second. Zendesk allows 700 to 2500 requests per minute depending on plan. Microsoft Dynamics 365 has its own limits split across user-bound and app-bound entitlements. Each of these limits assumes the customer's own workflows, internal integrations, and existing third-party tools are also consuming the quota.

A product polling every customer's Salesforce every two minutes will burn through the quota quickly, especially during peak hours. Customers will eventually notice that your product is the reason their other integrations are failing, and the conversation that follows is not enjoyable. We covered this dynamic in our piece on why conversational AI platforms need deep integration infrastructure to scale.

The arithmetic gets worse as you scale. 100 customers polling Salesforce every two minutes is 72,000 API calls per customer per day, multiplied by 100 customers is 7.2 million API calls per day across your customer base. Streaming integrations through Change Data Capture, Platform Events, or webhooks shift these reads onto a different quota class with different cost economics, and they reduce the load on the customer's API budget to nearly zero. The economic case for real-time event handling is not just user experience. It is also that polling does not scale.

Industry Context: Why Real-Time Integration Demand Is Spiking

A few category-level shifts have made real-time integration infrastructure non-optional for many product categories.

The first is the rise of AI agents that act inside customer systems. An AI sales agent that drafts emails, updates Salesforce, books meetings in calendars, and responds to changes in lead status cannot work on hourly batch sync. We wrote about this category shift in how AI agents break every integration pattern that worked for traditional SaaS.

The second is the consolidation of customer-facing integrations as a competitive moat. Products that integrate "fine" no longer differentiate. Products that integrate so deeply they feel native do. Real-time event handling is part of how depth shows up to the user.

The third is the expansion of integration scope into operational workflows. Older integrations were read-heavy, used for analytics or display. Modern integrations are bi-directional, used for action: closing tickets, updating opportunities, sending messages, scheduling events. Bi-directional write at real-time speed is the architectural shift, and ETL tools were not built for it.

The fourth is the move toward usage-based integration pricing, which is itself a recognition that the per-connector model favored by older platforms does not match the realities of modern integration depth. We covered the economics in why usage-based integration pricing beats per-connector models for SaaS teams scaling integrations.

How Ampersand Solves Real-Time Event Handling

Ampersand handles the real-time event handling problem by treating integration as infrastructure with declarative configuration. The Subscribe action lets you declare which objects in which SaaS systems you want to react to, which fields you care about, and where the events should be delivered. The platform handles the streaming connection (Change Data Capture for Salesforce, webhooks for HubSpot, equivalent primitives for each system), the replay ID persistence, the gap recovery, the rate-aware fallbacks, the per-tenant isolation, and the webhook delivery to your application.

For write paths, the platform exposes on-demand read and write API endpoints with idempotency, retry semantics, and audit logging. Bulk operations are optimized at the platform level, so you can push many records to the customer's system without worrying about chunking and retry logic.

For read paths that do not need streaming, scheduled reads and backfills cover the common cases with declarative configuration and automatic retry on failures.

The architecture spans hundreds of systems of record across CRM, ERP, GTM, HRIS, accounting, life sciences, and healthcare, with the same primitives applied consistently. This means your product gets one set of integration semantics, not one per SaaS system you support.

The platform is multi-tenant by design, with per-customer credential isolation, per-customer sync state, and per-customer observability. The customer-facing connection flow handles OAuth, scope management, and error presentation, so your engineering team is not in the loop for routine connections.

The compliance posture covers the entire integration runtime: SOC 2 Type II, ISO certified, GDPR compliant, with audit logs and subprocessor disclosure available under NDA.

The 11x team described what this kind of infrastructure does to product latency: "Using Ampersand, we cut our AI phone agent's response time from 60 seconds to 5." That sixty-second-to-five-second improvement is the gap between "polling, no streaming" and "real-time event handling on integration infrastructure." Engineering at Hatch, the Yelp company, framed the impact as a focus shift: "Ampersand lets our team focus on building product instead of maintaining integrations. We went from months of maintenance headaches to just not thinking about it." That maintenance burden is the runtime that ETL tools were not designed to provide and that managed integration infrastructure absorbs.

For teams that want to dig into the architecture, the Ampersand documentation walks through the primitives. The how it works overview shows the platform in product context. For teams thinking specifically about agent-ready integration infrastructure, our piece on why CRM platforms need agent-ready integration infrastructure is the right starting point.

Comparison: Approaches to Real-Time Event Handling

The pattern is consistent. ETL tools, even with tight intervals, do not deliver real-time event handling. Self-built streaming infrastructure can, at high cost. Managed integration infrastructure built for the problem delivers it as a configuration.

The Ampersand Approach: The Right Layer for the Right Problem

The reason teams looking for "open source ETL tools that support real-time event handling" should pause is not that the search is wrong. It is that the framing is wrong. ETL is not the layer where this problem gets solved well. Integration infrastructure is.

Ampersand exists to be that integration infrastructure layer. It handles the real-time event handling, the bi-directional writes, the multi-tenant runtime, the customer-facing connection flow, the observability, and the compliance posture as one coherent platform. Engineering teams that move from polling-and-glue-code to managed integration infrastructure see the latency, reliability, and engineering velocity improvements show up immediately.

If your product needs real-time event handling between your platform and the SaaS apps your customers use, the right architectural question is not "which open source ETL tool can I bend to do this." It is "what integration infrastructure am I going to build on, and what does the runtime cost of operating it look like over the next two years." That second question gets you to a different and better answer.

FAQ: Real-Time Event Handling and Integration Infrastructure

Why isn't ETL the right tool for this?

ETL was designed for moving data from operational systems into analytical destinations on a schedule. Real-time event handling between your product and SaaS apps is a different problem with different primitives: streaming subscriptions, fine-grained writes, per-tenant runtime, bi-directional flow, and compliance posture covering customer data flow. The architectural mismatch shows up as rate limit issues, latency problems, and growing operational burden as you scale.

Could I bend Airbyte or Meltano to do real-time integration?

Some teams try. The pattern is to run sync intervals as tight as the source system allows and to pair the ETL output with custom code that fans out writes back to source systems. This works in narrow cases. In production at scale, it accumulates complexity, hits rate limits, and produces a system that has the operational characteristics of a custom build with the constraints of an ETL tool. The teams that go this route generally migrate to dedicated integration infrastructure within a year or two.

What about Kafka Connect?

Kafka Connect is a flexible streaming framework, and many integration platforms use it under the hood. For a product team to use Kafka Connect directly as the basis for embedded multi-tenant integration is a substantial engineering project. You inherit the operational complexity of running Kafka, plus the work of building per-tenant orchestration, customer-facing connection flows, observability, and compliance. It is feasible. It is also where teams discover that they have built half an integration platform without intending to.

How does Ampersand handle the difference between systems with native streaming and systems without?

The Subscribe action abstracts the difference. For systems with native streaming (Salesforce CDC, HubSpot webhooks, Microsoft Dynamics 365 webhooks, Gmail Pub/Sub), the platform consumes the streaming source. For systems without native streaming, the platform falls back to scheduled reads at the highest frequency the API allows, with delta detection logic that approximates streaming as closely as the source supports. Your application code does not change based on which mode is in play.

How is the customer-facing connection flow structured?

The platform exposes a customer-facing connection UI that handles OAuth, scope selection, error states, and the customer's IT admin review. You can embed this flow inside your product or use the hosted version, depending on your product design. The point is that your engineering team is not in the loop for routine connections. The customer can self-serve, and the connection lifecycle is managed by the platform.

What are the compliance properties?

Ampersand is SOC 2 Type II audited and ISO certified, with the integration runtime in scope. The platform is GDPR compliant and supports data residency commitments. Audit logs cover every read and write to connected systems. Subprocessors are disclosed. The compliance posture is inherited as soon as you build on the platform, which is the major architectural difference from a self-hosted open source stack.

How does pricing work compared to per-connector models?

Ampersand's pricing is usage-based rather than per-connector. We covered the rationale in why usage-based integration pricing beats per-connector models for SaaS teams scaling integrations. The short version is that per-connector pricing punishes the products that need depth, which is exactly the opposite of what a serious product engineering team is trying to optimize for.

Conclusion: Integration Is Not ETL, and the Right Layer Matters

The category of real-time event handling between your product and external SaaS apps is native product integration infrastructure, not ETL. The tools that have optimized for ETL workflows do not extend cleanly into the real-time, multi-tenant, bi-directional integration space, and the teams that try to bend them eventually rebuild what they were trying to avoid.

Ampersand was built for the native product integration infrastructure problem specifically. It handles real-time event handling through Subscribe actions, bi-directional writes with on-demand and scheduled APIs, multi-tenant runtime with per-customer isolation, customer-facing connection flows, observability, and compliance posture in one coherent platform. The engineering teams that move from cobbled-together stacks to managed integration infrastructure see the difference in latency, reliability, and shipped product.

If you are building a SaaS product that depends on reacting to events in your customers' CRM, ERP, ticketing, or messaging systems in real time, the right architectural question is what integration infrastructure you are going to build on. More on the platform and the architecture is at withampersand.com, and the Ampersand documentation covers the technical patterns in depth.