Most data moves through Ampersand using webhooks which are ideal for most builders: lightweight, flexible, and quick to set up. Your webhook receives real-time updates and processes the data before loading it into your own database.
As data volumes grow, you may want more robust infrastructure. Ampersand also streams large volumes of data through Amazon Kinesis for high-throughput pipelines.
However there are cases where you need a persistent data storage to hold onto data until you’re ready to process it. Ampersand now delivers your customers’ bulk data to a new destination: directly to your own Amazon S3 bucket.
| Destination | Processing | Best for |
|---|---|---|
| Webhooks | Real-time | Reacting and processing real-time events |
| Kinesis | Real-time streaming | Streaming high volumes at scale |
| Amazon S3New | Async, batched | Storing bulk data to process later |
Set it up
# reference the destination by name specVersion: 1.0.0 integrations: - name: syncToLake provider: hubspot read: objects: - objectName: contacts destination: myDataLake
The destination name in amp.yaml matches the S3 destination you created in the dashboard.
Move bulk data to storage you control, then process it on your own schedule.