Amazon Kinesis, a platform for streaming data on AWS, offers powerful services that make it easier to build data processing applications, load massive volumes of streaming data, and analyze it in real time.
Almost everyone uses streaming data to gain real-time insight into customer activity and immediate business trends. Streaming data, which is generated continuously from thousands of data sources, includes a wide variety of data such as log files from your mobile or web applications, e-commerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services, and telemetry from connected devices.
This solution uses Amazon Kinesis Streams to load streaming data into Amazon Kinesis Analytics. An Amazon Kinesis Analytics application filters and processes the data, and puts it into Amazon Kinesis Firehose. Then, an Amazon Kinesis Firehose delivery stream delivers the analyzed data to various data stores for search, storage, or further analytics.
You can choose to put your analyzed data into an Amazon S3 bucket, an Amazon Redshift cluster, an Amazon Elasticsearch Service domain, or an Amazon Kinesis stream. If you choose to persist raw data, AWS Lambda decodes the data and puts it into Amazon Kinesis Firehose which delivers it to Amazon S3.
You’ll get real-time analytics with Amazon Kinesis leveraging an Amazon AWS reference implementation to automatically provision and configure the AWS services necessary to start consuming and analyzing streaming data in minutes. You’ll be able to specify up to three external destinations for analyzed data.
By default, the Streaming Analytics Pipeline allows users to specify a single external destination for their analyzed data. Finally, Amazon Kinesis Data Analytics allows you to specify a schema to classify your streaming data before it executes SQL queries against your input Kinesis stream.