There is some functional overlap between Solace and Kafka, but if you look at the underlying architecture and operational characteristics of the two tools you’ll realize they solve different problems in very different – and frequently complementary – ways. Kafka excels at log data aggregation and streaming to data repositories. Solace complements Kafka by adding robust bi-directional event distribution between diverse applications and IoT devices running in hybrid clouds and around the world. You can learn more about the differences between Solace and Kafka here.

Kafka Connect-compliant Connectors

Kafka Connect is an API/tool for scalably and reliably streaming data between Apache Kafka and other systems. It makes it simple to quickly define connectors that move large collections of data into and out of Kafka. Solace provides two free, open source connectors based on the Kafka Connect specification — one Source and one Sink — that make it easy to integrate Kafka and Solace so information can flow seamlessly between the environments.

Source Connector

The Solace Source Kafka Connector for PubSub+ Connector uses the Kafka Connect API to consume Solace queue or topic data events and stream them to a Kafka topic.
The Solace Source Connector uses Solace’s high performance Java API to move Solace data events to the Kafka Broker. Solace PubSub+ supports popular messaging APIs and protocols like AMQP, JMS, MQTT, REST, WebSocket and more so the Connector can move any message to a Topic (Keyed or not Keyed) on the Kafka Broker.

Sink Connector

The Solace Sink Kafka Connector for PubSub+ uses the Kafka Connect API to consume Kafka topic records and stream the data events to the Solace PubSub+ event broker as a Topic and/or Queue data event. Therefore, from the single Solace Sink Connector any Kafka Topic (Key or not Keyed) Sink Record is instantly available for consumption by any consumer that uses one of the Solace supported open standards languages or transport protocols.

Use Cases

Connectivity Across Hybrid Cloud

If you’re using Kafka to aggregate log data and perform offline analytics on it, but want to use a real-time analytics service running in the cloud to promote products based on sentiment analysis or real-time weather conditions, Solace PubSub+ can take the event stream from Kafka and route a filtered set of information to the analytics engine. PubSub+ lets you link applications and datastores across multiple cloud and on-premises environments without unwieldy adapters or bridges. PubSub+ keeps bandwidth consumption low by using fine-grained topic filtering to deliver exactly and only the events required.

High-Volume Web and Mobile Connectivity

Solace PubSub+ supports MQTT connectivity at massive scale, able to establish reliable, secure, real-time communications with tens of millions of devices or vehicles so you can collect data and hand it off to Kafka analytics system. Since Solace can also stream events via other popular open protocols and APIs, you can stream events from all of your applications, running in all kinds of cloud and on-prem environments, to Kafka via the same Solace Source Connector.

IoT Command and Control

In addition to supporting the inbound aggregation of events from millions of connected devices, PubSub+ supports bi-directional messaging and the unique addressing of millions of devices through fine-grained filtering. For example, with Solace and Kafka working together you could send a tornado warning alert to a specific vehicle, or all vehicles in or approaching the affected area. The Solace Sink Connector allows PubSub+ to send record events placed in a single Kafka Topic to whatever vehicles satisfy a given condition or topic, whether that’s as general as being in the tri-county area or as specific as a VIN#, i.e., a single vehicle.

Connector Setup Walkthrough

In this video Heinz Scaffner demonstrates how to install and configure a Solace Sink Kakfa Connector or Solace Source Kafka Connector.

Demo

In this video Heinz Schaffner shows how Kafka and Solace can work together as part of a connected vehicle use case.