Demo : Twitter streaming and sentiment analysis using Kafka, OCS, MongoDB & OpenShift (Kubernetes)

You know tech tools are cool, but unless you have defined use-case it kinda hard to put things into perspective and understand how different tools can interact with each other, help solve a problem or explore new use cases.

So to educate and motivate our technical buyers, sellers and customers, i created a fancy use case of ingesting live twitter tweets and applying sentiment analysis to it. For this demo i used the following tools

• Twitter API : Realtime streaming data source
• Red Hat AMQ Streams : Apache Kafka cluster to store real time streaming data coming into the system
• MongoDB : Storing tweets for long term persistence from Kafka into a schema-less NoSQL database
• Red Hat OpenShift Container Storage : Used for providing RWO (in this project), RWX, Object Storage persistence storage for Kafka and MongoDB apps running on OpenShift
• Red Hat OpenShift Container Platform : Enterprise grade k8s distribution for container apps
• Aylien : Sentiment analysis solution backend
• Python : Backend API app to trigger data sourcing from twitter, move data from Kafka to MongoDB, server data to frontend app
• Frontend : basic HTLM, CSS, Javascript based frontend to plot some graphs

This slide deck should give you a glimpse of how the demo would look like (demo youtube/github link below)

And here is the actual demo recording that you can go through, where i have explained how these component work together and making this a viable solution if you have a real-world use case along the same lines

If you are interested in running this demo by yourself, you can find the code in my repo, Github project link : https://github.com/ksingh7/twitter_streaming_app_on_openshift_OCS

Happy Analysing Live Tweets