Reference · built on requestConnectorStreams, queues & webhooks

Redpanda

Events from Redpanda topics through the Kafka API, the HTTP Proxy, the Admin API, and the Schema Registry API.

Request accessPricing ↗

About

Redpanda is a streaming platform from Redpanda Data that accepts Apache Kafka clients built for Kafka protocol version 0.11 or later. Alongside the Kafka API it ships three HTTP interfaces. The HTTP Proxy, enabled by default on port 8082, exposes a subset of Kafka operations as REST calls: list topics and brokers, produce events, subscribe with consumer groups, fetch events, and commit offsets. The Admin API manages brokers and partition leadership. The Schema Registry API manages schemas within a cluster.

Through Fibric, an operator reads events from the topics you name, either with a Kafka client or through an HTTP Proxy consumer instance. It proposes a produce or an offset commit for your approval, and the connector performs it once.

This is a reference listing. It documents what Fibric would read from Redpanda and what it could propose, based on the vendor's published interfaces. Fibric builds it under a managed deployment when you request it; selecting it here installs nothing.

Inputs

  • Events from topics, read with a Kafka client such as franz-go, librdkafka, kafka-python, or KafkaJS
  • Events fetched over HTTP from a consumer instance subscribed to a topic through the HTTP Proxy
  • The topic and broker lists from the HTTP Proxy
  • Consumer group offsets read from the HTTP Proxy
  • Broker and partition leadership state from the Admin API, which mostly requires a superuser
  • Schemas registered in the cluster through the Schema Registry API

Proposed actions

  • Target capability: propose producing an event to a topic through the HTTP Proxy with Content-Type application/vnd.kafka.json.v2+json
  • Target capability: propose committing offsets for this connector's consumer group through the HTTP Proxy
  • Target capability: propose creating a topic with a Kafka client, since the HTTP Proxy does not create topics

Proposed actions are target capabilities. Every action runs propose-first and needs a validated deployment and the appropriate permissions.

What you can build

  • Report a subscriber that fell behind

    The operator reads consumer group offsets from the HTTP Proxy and compares them with the end of each partition. When a group stops advancing, it names the group and topic.

    With Data Freshness, Integration Health

  • Re-read events after a downstream fix

    A consumer of a topic dropped events during an outage. The operator proposes a consumer instance that starts at an earlier offset and re-reads the range. You approve. The receipt records the offsets read.

    With Webhook Replay

  • Feed device telemetry to an operator

    Sensor readings published to a topic reach an operator through an HTTP Proxy consumer. The operator flags sensors that go quiet or drift out of range.

    With Sensor Health

Requirements

  • A Redpanda cluster whose Kafka API and HTTP Proxy listeners are reachable from the connector
  • A SCRAM user with one mechanism, since a user cannot hold SCRAM-SHA-256 and SCRAM-SHA-512 at once
  • Authorization for the user on the topics it reads and the consumer group it commits to
  • For Admin API reads, admin_api_require_auth enabled and a superuser, as most Admin API endpoints require one
Authentication
SASL/SCRAM credentials (SCRAM-SHA-256 or SCRAM-SHA-512) for the Kafka API, sent as HTTP Basic Authentication on the HTTP Proxy, or mTLS with a principal mapped from the certificate's Distinguished Name, or an OIDC bearer token with an enterprise license.

Limits

  • HTTP Proxy consumers expire after five minutes of inactivity. The connector keeps fetching or recreates the consumer under the same name
  • The HTTP Proxy does not support topic or ACL create, update, or delete. Those go through the Kafka API
  • The Kafka request rate quota, request_percentage, is not supported
  • Clusters that allow only SASL/PLAIN are not supported. OIDC authentication requires an enterprise license

Access and pricing

Reference listing. Fibric builds the connector under a managed deployment when you request it. Your quote covers the build, capabilities, usage, and support.

Request Redpanda ↗

Questions and answers

Which Kafka clients does Redpanda document?
Redpanda lists Apache Kafka Java Client, librdkafka, franz-go, confluent-kafka-go, Sarama, kafka-python, confluent-kafka-python, kafka-rust, KafkaJS, and confluent-kafka-javascript, and states that other clients built for Kafka protocol 0.11 or later, such as those based on librdkafka, work as well.
What can the HTTP Proxy do and not do?
It lists topics and brokers, produces events, creates consumers and consumer groups, subscribes them to topics, fetches events, and commits offsets. It does not create, update, or delete topics or ACLs. Consumers expire after five minutes of inactivity.
How does the connector authenticate to the HTTP Proxy?
With the SCRAM username and password in a Basic Authentication header, or with an OIDC token in a Bearer Authorization header when OIDC is enabled, which needs an enterprise license.
Ask about Redpanda

Ask about the capabilities and requirements in this listing.

For project-specific requirements, contact Fibric.

This connector listing is developed, published, and supported by Fibric. It describes integration with Redpanda through published interfaces. Third-party names and logos identify the systems an integration connects to; they are the property of their respective owners, who are not affiliated with Fibric and do not sponsor or endorse this listing. Trademark policy