Telemetry Egress

The primary data plane: raw kernel/OS events become OCSF records and land in Kafka. It spans the Kernel Drivers → Endpoint Agent → Gateway → Kafka.

Sequence

Stages

1. Collect. sources/ turn driver and OS signals into a uniform EDREvent (service/src/event.rs). Kernel-callback and minifilter events arrive from the drivers; ETW, the Event Log, WFP tamper watch, and lifecycle events are produced in-agent.
2. Normalize. pipeline/normalizer canonicalizes fields (paths, identifiers).
3. Deduplicate. pipeline/dedup drops repeats via a bloom filter (utils/bloom_filter) — cheap suppression of high-volume duplicates before the expensive stages.
4. Enrich. pipeline/enricher attaches process-tree ancestry (utils/process_tree) so an event carries its parent chain.
5. Map to OCSF. pipeline/mapper emits the flat OcsfEvent with the right class_uid (see the UID table in Proto Contracts).
6. Batch & ship. pipeline/batcher groups events into an OcsfBatchRequest; channel::telemetry sends it over the OcsfEventService gRPC (lazy-connected, keep-alive, mTLS). The gateway schema-encodes each event and produces to ocsf-events.

Failure handling

If the gRPC send fails, channel::ndjson spools the batch to a local NDJSON file and replays it once the link recovers — telemetry is not lost across a gateway outage. mTLS certs hot-swap on renewal without dropping the pipeline (see Enrollment).