This is the core Raft implementation specified by KIP-595: https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum. We have created a separate "...

Apache Kafka came to the rescue by inspiring us to leverage the time-ordered event log paradigm

:books: Tech blogs & talks by companies that run Kafka in production - dttung2905/kafka-in-production

Learn how Apache Kafka can empower teams with its lower entry barrier for real-time data pipelines.

A deep dive into a system that serves as the heart of many companies’ architecture

Trello has been using RabbitMQ for the last three years. Prior to RabbitMQ, we were relying on a Redis Pub-Sub implementation.…

Apache Kafka: A Distributed Streaming Platform.

Nowadays, every application tries to personalize its feed based on your interests. You may have seen whenever you search for something on…

Learn the main concepts needed to use the transaction API in Apache Kafka effectively.

In part one, we learned about how at Nordstrom we’ve been exploring and implementing event-sourcing as an architectural pattern. In part…

Apache Kafka: A Distributed Streaming Platform.

Apache Kafka: A Distributed Streaming Platform.

In today’s data processing architectures, Apache Kafka is often used at the ingress stage. Usually, this step is used to enrich and filter…

Apache Kafka: A Distributed Streaming Platform.

Kafka Streams provides an extremely simple deployment model for scaling out your streaming applications. You deploy your application many…

Microservices appear to be the major topic of discussion at this year’s O’Reilly Software Architecture conference in New York, with

Visualizing Kafka’s most misunderstood configuration setting

In my previous post I discussed some of the features of different stream processing platforms, but I did not touch on how one goes about choosing one.  In the book Kafka: The Definitive Guide, the …

Blockchain technology and Apache Kafka share characteristics which suggest a natural affinity. For instance, both share the concept of an ‘immutable append only log’. In the case of a Kafka partition:

Apache Kafka: A Distributed Streaming Platform.

Real-time change replication with Kafka and Debezium

World's largest Open Source foundation’s 300+ freely-available, enterprise-grade Apache projects power some of the most visible and widely used...

Sometimes a database is like a collection of wax tablets that you can stack and sort through to update, and these days, sometimes it is more like a river

We’ve made big changes this year to our Open Source Awards. We want to celebrate the communities of contributors that make open source run…

if you're considering microservices

Is the reactive, immutable, functional style of microservices enabled by Kafka and its Streams API the right fit for your application?

Following  CTO Paul Dix’s original release announcement for InfluxDB 2.0 and a new release of InfluxDB Cloud 2.0 to public beta, I thought the community would be interested in learning about how...

One of the main problems we are encountering these days are the amount of disk space used by Apache Kafka topics. Our messages are…

I’ve been working with Kafka Streams for a few months and I love it! Here’s the great intro if you’re not familiar with the framework. In the sections below I assume that you understand the basic...

Watch the talk of ‘From Kafka to BigQuery’, presented at Kafka Summit London (April 2018)

Learn the difference between a Table and a Stream in Kafka and when to use them.

KCache is a library that provides an ordered key-value store (OKVS) abstraction for a compacted topic in Kafka. As an OKVS, it can be used to treat Kafka as a multi-model database, allowing Kafka t…

In 2018, an extremely fast message bus enables us to build microservices to handle data at velocities and volumes previously unimaginable.

Improved Java client compatibility policy in Apache Kafka 0.10.2, which makes upgrades to newer Kafka client versions easier.

An in-depth analysis of the various ways to join streams and tables in Apache Kafka's Streams API.

reuse decompression buffers in consumer Fetcher switch lz4 input stream to operate directly on ByteBuffers avoids performance impact of catching exceptions when reaching the end of legacy record ba...

Strange Loop is a conference for software developers covering programming langs, databases, distributed systems, security, machine learning, creativity, and more!

Don’t just capture your changes, analyse and enrich them too!

Discover how Kafka Streams addresses scalability, fault tolerance, and recovery while providing strong delivery and ordering guarantees.