An open-source streaming platform used for versatile pipelines, data integration, goal-oriented applications, and streaming applications. It’s one of the most popular projects of Apache Software Foundation, which was released in January 2011. It aims at providing low latency real-time data feeds. Apache Kafka is based on the commit log, and it allows users to subscribe to it and publish data to any number of systems or real-time applications.
- Platform: Apache Kafka
- Author: LinkedIn
- Released: January 2011
- Type: Stream processing
- License: Apache License v2
- Support: Cross-platform
- Language: Multi-language
- GitHub: apache/Kafka
- Perform computations at an in-memory state
- Graph processing
- Machine Learning
- Complex Event Processing
- Event-driven analytics
- Real-time search index building
- Continuous ETL in e-commerce
- Top users are banks, manufacturers, insurance, and telecom.
- It integrates with many event sources and event sinks—for example, Postgres, JMS, Elasticsearch, AWS S3, and more.
- The Kafka cluster can handle failures with the masters and databases.
- By using ingest pipelines, it can replicate the events.
- Kafka is very fast and guarantees zero downtime and zero data loss.