Skip to content

LeoPlatform/bus

Repository files navigation

bus

Leo Platform Quick Start Guide: https://github.com/LeoPlatform/Leo

Better Data Flow

LEO EventBus is an open source project connecting several AWS services (Kinesis, DynamoDB, S3 and Lambda) together to provide a more robust Data Pipeline. LEO is deployed in your own Amazon cloud. Essentially, we created Apache Kafka for your own serverless AWS cloud.

Benefits

  • Multiple stream events (or Topics) on one Kinesis Stream
  • Durability of stream events
  • Replay of stream events
  • Unlimited Subscribers to any stream event
  • Lower AWS cost through reusing Kinesis, gziping data, auto archiving.

Kafka-like comparison

Kafka LEO
Communication Stream Kinesis
Service Catalog (# of Subscribers, where in log) DynamoDB
Notification & Broker Lambda
Data Store S3, DynamoDB

LEO Architecture

screen

Use Cases

Use LEO to solve these types of business problems

  • Custom integrations
  • Business Intelligence and Data Warehousing
  • Big Data and Data Science projects
  • Microservice async communication
  • Machine Learning and AI

All of these projects require access to core business events which can be easily streamed to LEO. Once in LEO, the same data event can be leveraged for all data projects. When your data flows, innovation follows.

Setup

Go to cloudformation pasted image at 2017_11_10 08_34 pm

click "Create Stack" pasted image at 2017_11_10 08_34 pm 1

Specify an Amazon S3 template

Region us-west-2: https://s3-us-west-2.amazonaws.com/leo-cli-publishbucket-mzhr7agmqo7u/leo-bus/2.0.9/cloudformation.json

Region us-east-1: https://s3.amazonaws.com/leo-cli-publishbucket-abb4i613j9y9/leo-bus/2.0.9/cloudformation.json

Click "next" pasted image at 2017_11_10 08_37 pm

Specify a name (eg. "DevBus"), click next pasted image at 2017_11_10 08_38 pm

click next pasted image at 2017_11_10 08_38 pm 1

acknowledge that it might create IAM resources by clicking the checkbox. Click Create pasted image at 2017_11_10 08_39 pm

Wait for the status to change from create_in_progress to update_complete pasted image at 2017_11_10 08_40 pm

Use one of our SDKs in order to interact with this stream NodeJs: https://github.com/LeoPlatform/Nodejs

Using Kinesis Directly

echo -e "{"id":"cli-bot","event":"cli-queue","payload":{"hello":"world","number":2345}}\n{"id":"cli-bot","event":"cli-queue","payload":{"hello":"world4","number":67894}}" | gzip -f | base64 -w 0 | xargs aws kinesis put-record --region us-west-2 --stream-name <---BusStreamName---> --explicit-hash-key="0" --partition-key="cli-queue" --data

Support

Want to hire an expert, or need technical support? Reach out to the Leo team: https://leoinsights.com/contact


V3