AWS Kinesis
2 min readFeb 22, 2021
This is part of a blog series giving a high level overview of the different services examined on the AWS Solution Architect Associate exam, to view the whole series click here.
Kinesis Summary
- Kinesis is a fully managed platform that can receive streaming data and process/analyse that data in real time.
- Kinesis can ingest a range of different data such as video, audio, logs, analytics etc.
- It is very scalable and can handle high amounts of streaming data from hundreds of thousands of sources all with very low latency.
- 4 types: video streams, data streams, firehose, analytics
Kinesis Video Stream
- Fully managed service that makes it easy to securely stream live video from devices to the cloud.
- Enables playback, analytics and machine learning on video data that has been ingested.
- Can capture data from millions of sources.
- Automatically stores data and encrypts it at rest.
- You can specify a custom retention period on data.
Kinesis Data Stream
- Service for collecting and processing large amounts of real time data at scale.
- Can capture gigabytes of data per second from hundreds of thousands of sources.
- Collected data is available in milliseconds to be analysed and create analytics and dashboards.
- Store data for a minimum of 24 hours
- Takes data from a producer e.g. EC2 and passes it through a SHARD to the consumer
Shards
- A shard is a sequence of data records in a stream with a fixed unit of capacity.
- A shard can support 5 transactions per second for reads with a max 2mb read rate per second.
- Only data streams have shards
- The total capacity of a stream is the sum of the capacities of its shards
Kinesis Firehose
- Fully managed service that allows you to reliably load real time streaming data into data stores like S3, RedShift, Elasticsearch etc.
- Can compress, transform and batch data to minimise the amount of storage.
- Encrypts your data streams before loading.
- Pay for the volume of data that transmits through the service.
Kinesis Analytics
- Kinesis Analytics allows you to analyse streaming data in real time to gain actionable insights.
- Allows you to process and analyse data using standard SQL.
- Can gain realtime dashboards and create real time metrics.
- Can use with Data Streams or Firehose as the streaming source.