All docs / Connect
Pipelines
Collect lots of small events and write them to file storage as compressed batches.
Open in the consoleA pipeline accepts a high volume of small JSON records (page views, clicks, logs) and writes them to a bucket as gzipped JSON-lines files once a size or time limit is reached. You get cheap, durable raw data to analyse later.
Create a pipeline
- 1Create a bucket under File storage for the output.
- 2Create a pipeline under Pipelines, choose the bucket and the batch limits (for example 64 MB or 60 seconds).
- 3Create an ingest token on the pipeline's page if you will send from outside your apps.
Send records
From an app, bind the pipeline and send arrays of records; no token is needed:
jsonc
"pipelines": [{ "binding": "EVENTS", "pipeline": "pageviews" }]