Amazon Kinesis
Kinesis
You need to ingest, process, and analyze massive volumes of real-time streaming data with low latency.
Reach for it when
- Ingesting high-volume application clickstreams or IoT sensor telemetry feeds continuously.
- Processing real-time log events and routing them to multiple analytics targets like OpenSearch and S3.
- Building real-time dashboards that update instantly as new events arrive from the stream.
Do not reach for it when
- Passing simple messages between decoupled microservices where processing order is not critical — use SQS instead.
- Orchestrating long-running workflows or processing events with complex conditional branching — use Step Functions instead.
- Broadcasting notifications or alerts to multiple human endpoints like SMS or emails — use SNS instead.
Alternatives, and how to choose
| Service | Pick it instead when |
|---|---|
| SQS | Choose it when you need simple message queueing and buffering between decoupled components. |
| MSK (Managed Kafka) | Choose it when migrating existing open-source Apache Kafka applications to AWS. |
How you pay
- The model
- Pay per shard hour allocated (for provisioned mode) or per GB ingested (for on-demand mode).
- The line item that surprises people
- Leaving unused shards provisioned during low-traffic periods generates continuous hourly charges.
What trips people up
- Each shard has hard limits: 1MB/sec write and 2MB/sec read; exceeding these limits throws a ProvisionedThroughputExceededException.
- Data in the stream is ephemeral; the default retention period is 24 hours, after which unprocessed data is lost.
- Failing to distribute partition keys evenly across shards leads to hot shards, causing ingest bottlenecks.
Verify the live service
This page is a concept reference. Cost models are qualitative; confirm the current offering, Region and pricing before deploying.