processing REAL TIME EVENTS WITH AWS KINESIS FIREHOSE and AWS EMR

State of business prior to engagement
WalkMe collects millions of click events every day to optimize the guidance and engagement platform and make it more useful for its customers. As the event rate grew, it became more challenging to efficiently collect, process and analyze the events in scalable and robust way.

Problem statement
Reliably collecting, processing, transforming and analyzing real time events could be quite challenging task even for larger organizations. Issues such as high rate of events, reliability of data and flexibility in processing could be big obstacles achieving these goals.  

Technical solution
By utilizing AWS Kinesis Firehose as well as Kinesis Streams and Spark on AWS Elastic MapReduce, WalkMe can now effortlessly process millions of daily events.

Amazon Kinesis integration with Apache Spark is via Spark Streaming. Spark Streaming is an extension of the core Spark framework that enables scalable, high-throughput, fault-tolerant stream processing of data streams such as Amazon Kinesis Streams. Spark Streaming provides a high-level abstraction called a Discretized Stream or DStream, which represents a continuous sequence of RDDs.

magic-logo-280x284.png

WalkMe Inc, founded in 2011, launched the WalkMe guidance and engagement platform in April 2012 with the vision to transform the world’s online user experience into one that was simple, effortless and efficient.

Today, WalkMe’s platform is used by over 800 enterprise service providers and SaaS vendors, including many Fortune 500 companies.