Kafka → Hooks

Hooks is an HTTP relay, so pushing from Kafka means running a sink that POSTs each record to an endpoint’s receive URL. This guide uses Confluent’s HTTP Sink connector (kafka-connect-http), which works on Confluent Cloud, MSK Connect and self-managed Kafka Connect.

1. Create the endpoint

1. In the console, open Endpoints → New endpoint.
2. Set Provider to None (generic webhook); Kafka records carry no GitHub/Stripe style signature header.
3. Choose Verification: shared secret and keep the default header X-Webhook-Token (or pick your own).
4. Copy the receive URL — it looks like https://hooks.events/in/ep_abc123 — and the shared secret.

The receive URL is the only address a producer needs. The shared-secret header stops random traffic; the sink sends it on every request.

HMAC mode is not usable here: the X-Hooks-Signature header covers a timestamp plus the raw body, which a static HTTP sink cannot compute. Use shared secret, or none only for a throwaway endpoint.

2. Point an HTTP Sink at it

Confluent Cloud / managed connector

Add a HTTP Sink connector with the topic you want to forward and this configuration:

{
  "connector.class": "io.confluent.connect.http.HttpSinkConnector",
  "topics": "hooks-events",
  "http.api.url": "https://hooks.events/in/ep_abc123",
  "request.method": "POST",
  "headers": "X-Webhook-Token: <shared_secret>",
  "value.converter": "org.apache.kafka.connect.json.JsonConverter",
  "value.converter.schemas.enable": "false",
  "key.converter": "org.apache.kafka.connect.storage.StringConverter"
}

• http.api.url is the Hooks receive URL. The connector appends nothing; each record becomes one POST.
• headers is a comma-separated Name: value list, so the shared secret travels as a request header.
• value.converter.schemas.enable=false sends the bare JSON value, not a {"schema":…,"payload":…} envelope.

Self-managed Kafka Connect

confluent-hub install confluentinc/kafka-connect-http:latest

Create hooks-sink.json with the same properties and start the connector:

curl -X POST http://localhost:8083/connectors \
  -H 'content-type: application/json' \
  -d @hooks-sink.json

3. Shape the event Hooks should store

Without extraction rules Hooks stores the record as a generic.received event and uses the payload hash for deduplication. Two optional settings make the event nicer:

Console field	Example	Effect
Event type path	$.type	Uses a field from the record value as events.event_type.
Delivery id path	$.id	Uses the producer’s id for dedupe instead of the payload hash.
Dedupe key	{{event.subject.id}}	Collapses repeats inside the dedupe window.

Set them under Endpoint → Edit → Source extraction. A typical record:

{ "id": "order-1042", "type": "order.created", "amount": 4999 }

then produces an event of type order.created with delivery id order-1042.

Records whose value is not JSON (plain text, Avro without a JSON converter) are stored as text, so configure a JSON converter or transform the value before the sink.

4. Verify

Produce one record and watch both sides:

echo '{"id":"order-1042","type":"order.created","amount":4999}' \
  | kafka-console-producer --topic hooks-events --bootstrap-server localhost:9092

1. Events shows the record as order.created with the x-webhook-token request header redacted.
2. Deliveries shows one delivery per matching destination; open the attempt to see the target response.
3. The connector task log shows 202 for the POST. Hooks answers as soon as the event is persisted; downstream fan-out is asynchronous.

Troubleshooting

Symptom	Cause	Fix
Connector logs 401	Shared secret missing or wrong header name.	Match the connector’s headers name to the endpoint’s verification header, and rotate the secret in both places.
404	Receive URL typo or the endpoint was archived.	Copy the current URL from Endpoints → the endpoint.
Connector returns 200 but Events stays empty	The record did not leave the sink (topic mismatch, converter error).	Check the connector task status and the topic name.
413	Record exceeds the plan’s payload limit.	Split the record or raise the plan; see Rate limits for payload accounting.
Duplicate events	The producer retried the POST.	Set Delivery id path so Hooks deduplicates instead of storing a second payload hash.
429	The endpoint’s ingest rate limit was exceeded.	Lower the sink’s max.poll.records/task count or raise the endpoint’s limit.

Related

• Kafka REST — the reverse direction: deliver Hooks events to a Kafka topic over REST.
• Receive events — headers, verification modes and response behavior.
• Retries — how outbound deliveries are retried.