Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

System Description

AnyOne EDR is an endpoint detection and response platform by MarkAny GaneshaIT, deployed as two halves: endpoint agents on Windows hosts and a backend server in the cloud. Each agent collects kernel-level telemetry, normalizes it to OCSF, and streams it up to the backend over mTLS gRPC. The backend ingests and stores that telemetry, detects malicious patterns, and drives remediation, whether operator-dispatched from the SOC dashboard or automated. Response actions travel back down to the agent as cryptographically signed commands.

This is the whole-system technical reference: the endpoint agent and every backend service in scope. For installing, configuring, and operating just the agent, see the Installation Guide.

Document Purpose

This reference explains the architecture and internals of the AnyOne EDR system: what each component is, how it is built, and how the components act together. It is written for engineers building, extending, or debugging the platform, not for operators installing the agent.

It is organized two ways:

  • Structure: what each part is. One page per in-scope repository, covering its boundary, responsibilities, and internal components. Diagrams are component views (C4-ish). Start at System Overview.
  • Behavior: how the parts act together. One page per cross-service flow. Diagrams are sequence views. Start at Telemetry Egress.

Document Scope

In scope are six repositories, one structure page each: AnyOne-Endpoint, AnyOne-Kernel, AnyGateway, AnyOne-StaticAPI, AnyOne-SOC-Backend, AnyProto.

Out of scope are deployment infrastructure (AnyOne-Infra) and upstream detection and correlation (ocsf-events to alerts). The SOC backend consumes detection results, not raw telemetry.

Trust-bundle minting (offline anchor key) and the IOC filter build are standalone offline ops steps, not online services. The backend and agent only consume their outputs.

System Overview

The AnyOne EDR system at a glance: a context view of the whole platform and the outside actors it touches, plus a repository map linking each in-scope repo to its own structure page.

System context

ℹ️ Arrow colors:

  • Blue : data plane (telemetry flow)
  • Orange : control plane (commands, enrollment, and remediation actions).

The two planes

Traffic between endpoint and backend splits into two planes, and that split is the core of the security model:

  • Data plane (up). The agent ships OCSF telemetry to the gateway over one gRPC service and holds a second, long-lived bidirectional gRPC stream for control and liveness. File verdicts take a separate HTTPS path to the Static Analysis API. All three are mTLS. Everything the endpoint sends is a report. It issues no commands upward.
  • Control plane (down). Response actions (kill process, quarantine file, isolate host) originate at the SOC backend, which Ed25519-signs each command, and flow down through Kafka and the gateway to the agent. The gateway is the only endpoint-facing service that touches Kafka, and it never parses command bodies. It forwards them as opaque, backend-signed bytes (see Command Trust). Trust bundles ride the same downward path.

Repository map

RepoLanguageRoleRuns onPage
AnyOne-EndpointRustAnyOneAgent service for telemetry, the OCSF pipeline, static-analysis orchestration, remediation, and enrollment. Primary repo.EndpointEndpoint Agent
AnyOne-KernelCAnyOneKcallback (kernel-callback and WFP telemetry) and AnyOneMinifilter (file-system minifilter).EndpointKernel Drivers
AnyGatewayRustgRPC gateway that terminates agent mTLS and bridges telemetry and enrollments up to Kafka, and commands and trust-bundles back down.BackendGateway
AnyOne-StaticAPIRustStatic file-analysis service with a gateway and worker, backed by Redis and Postgres.BackendStatic Analysis API
AnyOne-SOC-BackendRustOrchestrator behind the SOC dashboard that ingests alerts, enrollments, and results, dispatches signed commands, issues enrollment, and distributes trust bundles.BackendSOC Backend
AnyProtoprotoCentral protobuf contracts (anyproto crate), the wire ABI between agent and gateway, plus the OCSF telemetry schema.(compiled into consumers)Proto Contracts
AnyOne-Infrayaml/dockerDeployment config. Out of scope.n/an/a

Endpoint Agent

Repo: AnyOne-Endpoint (Rust workspace). The per-host agent that runs as the AnyOneAgent Windows service (anyone-agent.exe). It collects kernel and OS telemetry, runs the OCSF pipeline, orchestrates static analysis, executes remediation, and manages its own enrollment identity.

Workspace crates

The workspace has five members. Only the drivers and the service run on the endpoint long-term.

CrateBinaryRole
serviceanyone-agent.exeThe agent itself, covering everything below.
clianyone-agent-cli.exeInteractive shell that talks to the running agent over a named pipe.
notifieranyone-agent-notifier.exeHelper spawned into the active desktop session to show toast notifications.
updateranyone-updater.exeBoot-time service that verifies a staged update and rolls back on failure.
shared(lib)Types shared across crates: events, the IPC contract, and constants.

Note: package names for the client/notifier crates (anyedr-cli, anyedr-notifier) differ from their binary names.

Component view

ℹ️ Arrow colors:

  • Blue : data plane (telemetry flow)
  • Orange : control plane (commands, enrollment, and remediation actions).

Source tree

The service crate is a single binary. Its source is organized by subsystem.

service/src/
├── main.rs
├── service.rs
├── bootstrap.rs
├── config.rs
├── context.rs
├── event.rs
├── stats.rs
├── constants.rs
├── driver.rs
├── sources/
│   ├── etw/
│   ├── eventlog.rs
│   ├── kcallback/
│   ├── minifilter/
│   ├── lifecycle.rs
│   └── tamperwatch.rs
├── pipeline/
│   ├── normalizer.rs
│   ├── dedup.rs
│   ├── enricher.rs
│   ├── mapper.rs
│   └── batcher.rs
├── channel/
│   ├── remotectl.rs
│   ├── telemetry.rs
│   ├── static_analysis.rs
│   ├── localctl.rs
│   ├── notification.rs
│   ├── ndjson.rs
│   └── handlers/
├── static_api/
│   ├── filter.rs
│   └── worker.rs
├── commands/
│   ├── dispatcher.rs
│   ├── verifier.rs
│   ├── registry.rs
│   ├── trust_bundle.rs
│   ├── kill_process.rs
│   ├── file_quarantine.rs
│   ├── isolate_host.rs
│   ├── update.rs
│   ├── update_package.rs
│   ├── handler.rs
│   ├── context.rs
│   ├── outcome.rs
│   └── noop.rs
├── remediation/
│   ├── executor.rs
│   ├── kill.rs
│   ├── quarantine.rs
│   ├── isolation.rs
│   ├── verdict.rs
│   ├── audit.rs
│   └── types.rs
├── enrollment/
│   ├── identity.rs
│   ├── keystore.rs
│   ├── certmanager.rs
│   ├── stepca.rs
│   ├── tls.rs
│   ├── fingerprint.rs
│   ├── metadata.rs
│   └── startup.rs
└── utils/
    ├── process_tree.rs
    ├── bloom_filter.rs
    ├── hash.rs
    ├── pipe.rs
    ├── path.rs
    ├── install.rs
    ├── executable.rs
    └── user_resolver.rs

External communication

The diagram shows the wires. This table describes what actually crosses each one, the external processes the agent talks to, what moves, and over which protocol. Internal module-to-module calls are omitted.

FromToDataProtocol
AnyOneKcallback (kernel)sources/ kcallbackKernel events: process, thread, image load, registry, object, and network activityIOCTL and shared ring buffer
AnyOneMinifilter (kernel)sources/ minifilterFile-system activity eventsMinifilter communication port
Event Tracing for Windowssources/ etwOperating-system trace events from ETW providersETW real-time session
Windows Event Logssources/ eventlogWindows event-log recordsEvent Log subscription
channel/ telemetryGatewayBatched OCSF telemetrygRPC, mTLS
channel/ remotectlGatewayUp: enrollment, heartbeats, and command results. Down: signed commands and trust bundlesgRPC bidirectional stream, mTLS
channel/ staticanalysisStatic Analysis APIFiles to scan and hashes to look up, with verdicts returnedHTTPS, mTLS
enrollment/ certmanagerstep-caCertificate bootstrap (CSR signing) and renewalHTTPS, pinned root
AnyOne CLIchannel/ localctlLocal operator commands and status queriesNamed pipe
channel/ notificationAnyOne NotifierToast notification requestsChild-process handoff

Module layout (service)

What each module and subsystem is responsible for.

Entry / host:

ModuleResponsibility
main.rsDevelopment-only entry point for running the agent directly from the command line. Slated for removal, since production installs run the agent as a Windows service with its entry point in service.rs.
service.rsThe production entry point. Registers with the Service Control Manager, handles start and stop requests, and brings up the subsystems: configuration, shared state, the control channel, the pipeline, and the sources.
bootstrap.rsOne-time startup setup: process hardening, acquiring the privileges the agent needs, and initialising logging.
config.rsA global configuration singleton populated from environment variables.
context.rsShared runtime state handed to every subsystem.
event.rsThe internal event model and the set of telemetry sources.
stats.rsRuntime counters used for metrics and diagnostics.
constants.rsFixed values shared across the service.
driver.rsTransport to the AnyOneKcallback kernel driver. Opens the device and moves telemetry and commands across the kernel boundary.

Subsystems:

ModuleResponsibility
sources/Collectors that gather telemetry from the kernel drivers and the operating system.
pipeline/Transforms raw events into batched OCSF telemetry through normalize, deduplicate, enrich, map, and batch stages.
channel/All external transport: the telemetry and control channels to the gateway, the static-analysis client, the local CLI server, and the notifier handoff.
static_api/Orchestrates static-analysis API calls, using a local filter to decide which files actually need a scan.
commands/Receives commands from the server, verifies their signatures, and dispatches them to the matching handler.
remediation/Executes response actions (kill process, quarantine file, isolate host) and records an audit trail.
enrollment/Manages the agent’s cryptographic identity: enrollment, secure key storage, and certificate renewal.
utils/Shared helper routines used across the service.

Configuration

The agent takes all of its configuration from environment variables, read once at startup into a global configuration singleton. There is no configuration file the agent parses itself.

How those variables are set depends on the install path:

  • Normal (dashboard) install. The installer provisions the credentials and writes the configuration for you, so there is nothing to set by hand.
  • Manual install. You write the variables yourself as the service’s registry Environment value, which the Service Control Manager injects into the process when it starts.

The mandatory variables are:

VariablePurpose
GRPC_ENDPOINTGateway address for the telemetry and control channels.
KERNEL_DRIVER_PATHLocation of the AnyOneKcallback driver file.
MINIFILTER_DRIVER_PATHLocation of the AnyOneMinifilter driver file.
CA_URL, CA_FINGERPRINT, CA_PROVISIONERstep-ca connection and root pinning for enrollment.
ENROLLMENT_CERT_SUBJECTThe device id, which must match the enrollment certificate.
ENROLLMENT_X5C_CERT_PATH, ENROLLMENT_X5C_KEY_IMPORT_PATHThe delivered device credential.
COMMAND_ANCHOR_PUBLIC_KEY_BASE64Anchor public key used to verify signed commands.

For the full procedure and every available setting, see Manual Installation → Configuring the Agent in the Installation Guide.

Library inventory

Key third-party libraries the agent depends on, for traceability when reading the code.

LibraryUsed for
tokioAsynchronous runtime for all concurrent tasks.
tonic, prostgRPC transport and Protocol Buffers encoding.
anyprotoShared AnyOne protobuf contracts for telemetry and control.
reqwestHTTP client for the Static Analysis API.
windows, windows-serviceWindows system APIs and Windows service integration.
ed25519-dalekVerifying signed server commands.
p256, pkcs8, rcgen, x509-parserEnrollment certificates and key handling.
xorfLocal membership filter for the static-analysis pre-check.
arc-swapHot-swapping the rotating identity and filter without a restart.
sha2, hex, base64Hashing and encoding.
serde, serde_json, postcardSerialization.
quick-xmlParsing Windows event-log XML.
dashmap, once_cellShared concurrent state.
clapCommand-line parsing.
flexi_logger, logLogging.
zipExtracting update packages.

Kernel Drivers

Repo: AnyOne-Kernel (C, WDK). Two kernel drivers feed the Endpoint Agent with the events user mode cannot see on its own. Both build from the shared solution AnyEDR-Drivers.sln.

  • AnyOneKcallback (built from the AnyKDriver/ project as aokclbk.sys) is a control device hosting the OS notify callbacks and the network filtering callouts. It streams events to the agent over a shared-memory ring buffer and takes control commands (isolation, ring setup) over device I/O controls.
  • AnyOneMinifilter (built from the AnyMDriver/ project as aomnflt.sys) is a Filter Manager minifilter that observes file activity. It sends events to the agent over a filter communication port, which also carries a small runtime command channel.

Why two drivers? They use three different kernel extension mechanisms: OS notify callbacks (process, thread, image, registry, handle events), the Windows Filtering Platform for network traffic, and a Filter Manager minifilter for file activity. The notify callbacks and WFP pair naturally in one ordinary kernel driver (AnyOneKcallback), but the minifilter has a distinct registration model and lifecycle, so it is simpler to develop as its own driver and repository (AnyOneMinifilter) for now. Merging the two into a single driver is planned.

AnyOneKcallback

A software-only driver loaded as a kernel service. It captures process, thread, image, registry, object, and network activity, and it also enforces host network isolation on command.

Component view

Blue is event and telemetry data flowing toward the agent. Orange is control, setup, and commands (callback registration, IOCTL control, isolation). Dashed arrows are internal dependencies and the ring signal.

Module layout

FileResponsibility
Driver.cEntry point. Creates the control device and a separate raw device for the network filter, wires every subsystem together, restores isolation state on load, and tears everything down on unload.
Callbacks.cThe OS kernel notify callbacks for process, thread, image, registry, and object activity. Builds telemetry events and also strips dangerous handle rights to protected processes for self-protection.
WfpCallouts.cRegisters the network filtering session, callouts, and filters. Emits network connect, accept, and close telemetry and enforces isolation blocking.
WfpSecurity.cStamps a SYSTEM-only access control list onto the isolation filter objects and the persisted-isolation registry key.
Isolation.cThe network isolation state machine: the isolation flag, the lifeline allowlist, the block decision, and persistence across reboot and Safe Mode.
EventFilter.cA registry-path allowlist that drops non-persistence-relevant registry events before they reach the ring buffer.
EventQueue.cThe shared-memory ring buffer: initialization, event push, copy-based drain, mapping into the agent, and the user notification event.
IoctlHandler.cThe single dispatcher for every control request the agent sends: ring mapping, statistics, drain, isolation, lifeline, and protected-process registration.
TelemetryCommon.hThe kernel-to-agent binary contract: device names, control codes, and event structs, kept byte-compatible with the Rust agent.
DeviceContext.hThe per-device state: the ring buffer, the registry filter, the callback handles, and the protected-process list.

Ring buffer

The agent maps the buffer once and then reads it directly, waiting on a named event that the driver signals on every push. The producer region is read-only to user mode, and the agent maps a separate read-write region holding only its read cursor, so the writer (kernel) and the reader (agent) never share writable memory. On overflow the driver sets a flag and increments a dropped-events counter instead of blocking. A copy-based drain control is the fallback path.

RegionWriterPurpose
control header (32 B)kernelWrite index, dropped count, signalled flag, layout fields
events (default 16384 × 1160 B, about 17 MB)kernelThe telemetry event slots
consumer (8 B)agentThe read cursor, on its own read-write page

Control surface

The agent drives the driver through device I/O controls in the range 0x801 to 0x823. Beyond ring setup, these also command host network isolation, tying this driver into Remediation.

CategoryCodesPurpose
Ring setup0x810 / 0x811 / 0x817Map and unmap the ring buffer, and hand over the agent's notification event
Event pull0x805Copy-based drain, the fallback to the mapped ring
Statistics0x801 / 0x804 / 0x816 / 0x81AQueue count, statistics, dropped count, and callback status
Isolation0x820 / 0x821 / 0x822 / 0x823Isolate, un-isolate, read status, and set the lifeline allowlist
Anti-tamper0x802Register a protected process (accepted only from the agent image)
Diagnostics0x818 / 0x819Network filter diagnostics and last status

AnyOneMinifilter

A Filter Manager minifilter that watches file activity for ransomware, wiper, and tampering behavior, and defends the agent’s own files.

Component view

Blue is event data flowing toward the agent. Orange is control and setup (operation registration and config commands). Black dashed is an internal dependency, one module using another. The Command Handler is Driver.c’s message-notify callback, which reads and applies g_Config and reports queue stats.

Module layout

FileResponsibility
Driver.cEntry point. Registers the minifilter and its operation callbacks, starts filtering, creates the communication port, handles volume attach and detach, and runs the runtime command channel.
Callbacks.cThe pre and post callbacks for every monitored file operation. Gathers evidence into per-file context and hands finished events to the dispatcher.
PostCreateDecision.cA pure decision function that classifies a file open's intent and decides whether to report it and as which event type.
ProcUtils.cProcess trust logic: whitelisting the agent's own service, LocalSystem checks, and browser-process detection.
Utils.cFile classification helpers (suspicious extensions, dropper locations, mark-of-the-web, decoy files) and the self-defense protection-rule engine.
EventDispatcher.cBuilds the user-mode messages, queues them onto a worker thread, and sends them over the port. Critical alerts fall back to a synchronous send if the queue is full.
Config.hThe runtime configuration (monitoring and self-defense toggles) and the queue and timeout constants.
Driver.hThe master header: the message layout, the command protocol, and the per-file context structs.

Monitored operations

The minifilter registers pre and post callbacks for file open, write, set-information, file-system control, and cleanup. Rather than report every operation, it defers most reporting to cleanup, when the final outcome is known, and it classifies open intent on the create path.

OperationWhat it detects
CreateOpen intent, suspicious location and extension, mark-of-the-web, browser-dropped files, decoy access, and self-defense blocking of the agent's own files
WriteWrite tracking (size, offset, running total, and a per-file write counter)
Set-informationRename (capturing old and new path) and delete intent
File-system controlThe zero-data wiper path (a write-equivalent that bypasses normal writes), plus reparse-point set and delete used for privilege escalation and evasion
CleanupThe deferred emit point, sending the final delete or modify event once the handle closes

External communication

FromToDataProtocol
Windows kernel notify routinesAnyOneKcallbackProcess, thread, image, registry, and object eventsIn-kernel callback registration
WFP filter engineAnyOneKcallbackNetwork connect, accept, and close eventsWFP callouts
AnyOneKcallbackEndpoint agentTelemetry event streamShared-memory ring buffer (read-only map) plus a named signal event
Endpoint agentAnyOneKcallbackRing setup, drain, isolation, lifeline, protected-process registrationDevice I/O control over \\.\AnyOneKcallback
Filter ManagerAnyOneMinifilterFile create, write, set-information, file-system control, and cleanup operationsMinifilter pre/post callbacks
AnyOneMinifilterEndpoint agentFile, volume, and named-pipe eventsFilter communication port \AnyOneMinifilterPort
Endpoint agentAnyOneMinifilterRuntime configuration commands (ping, get, set)Filter communication port \AnyOneMinifilterPort

Build and installation

The drivers build from the shared Visual Studio solution alongside the agent, and in a normal deployment the agent installs and starts both of them on its own startup rather than being registered by hand. The full procedures live in the Installation Guide, so they are not repeated here.

Kernel components

The key Windows and WDK facilities these drivers build on, for traceability.

AreaFacilities
FrameworkKernel-Mode Driver Framework (WDF) control device and I/O queues; Filter Manager (FltMgr) for the minifilter
Notify callbacksPsSetCreateProcessNotifyRoutineEx, PsSetCreateThreadNotifyRoutine, PsSetLoadImageNotifyRoutine, CmRegisterCallbackEx, ObRegisterCallbacks
NetworkWindows Filtering Platform callouts and filters (FwpsCalloutRegister0, FwpmFilterAdd0)
File systemFilter Manager operation callbacks, FltSendMessage, and communication ports
ToolchainWDK with the MSVC v143 toolset, built for Release | x64

Gateway

Repo: AnyGateway (Rust). The gateway sits between the endpoints and the backend. It transports telemetry and enrollments up to Kafka, and forwards commands and enrollment results down to the agents.

gRPC services

Two gRPC services defined in AnyProto contracts.

Proto serviceRust impl (src/service/)RPCRole
OcsfEvent Serviceocsf.rsOcsfBatch (unary)Schema-encodes each OCSF event, produces it to Kafka, and returns accept and reject counts.
Endpoint Serviceendpoint_ channel.rsEndpointChannel (bidirectional stream)The long-lived control channel (see below).

The control stream requires Hello as its first frame (otherwise failed_precondition), verifies the endpoint identity, publishes an enrollment event, and registers the live downstream sender in the connection registry keyed by endpoint_id. It then pumps commands and enrollment results down, and heartbeats and command results up.

Component view

Arrow colors: blue = data plane (telemetry flowing up), orange = control plane (commands, enrollment, and remediation actions flowing down).

External communication

The diagram shows the wires. This table describes what actually crosses each one, the external processes the gateway talks to, what moves, and over which protocol. Internal module-to-module calls are omitted.

FromToDataProtocol
Endpointsservice/ocsfBatched telemetrygRPC, mTLS
Endpointsservice/endpoint_channelControl stream. Up: Hello, heartbeats, and command results. Down: signed commands and trust bundlesgRPC bidirectional stream, mTLS
brokerKafkaUp topics: raw.endpoint, enrollment.endpoint, command.endpoint.responseKafka
KafkabrokerDown topics: command.endpoint, enrollment.endpoint.responseKafka
certmgrstep-caCertificate enrollment and renewalACME over HTTPS
registrySchema RegistryOCSF schema registration and lookupHTTP

Module layout (src)

ModuleResponsibility
server/gRPC server wiring: the TLS authentication and resolver, certificate setup, and shutdown.
service/gRPC service implementations plus the enrollment publisher.
identity/Endpoint SPKI-based identity verification from the peer mTLS certificate, fingerprint checking, and revocation.
connections/Stream each endpoint_id message to its downstream sender.
downlink/The Kafka-to-delivery loop that decides whether each frame is a command or a trust bundle.
delivery/Routes a downstream frame to the matching live stream.
certmgr/Managed certificate lifecycle with step-ca: enroll, renew, and hot-swap.
registry/Schema Registry client and the message encoder.
broker/Kafka producer and consumer.
config/Environment-based configuration for the server, TLS, and Kafka.

Kafka topics

TopicDirectionCarries
raw.endpointupOCSF telemetry (OcsfService)
enrollment.endpointupEnrollment events
command.endpoint. responseupCommandResult acknowledgements
command.endpointdownBackend commands, forwarded as Command frames
enrollment.endpoint. response (optional)downTrust-bundle responses, forwarded as TrustBundle frames

mTLS scheme

Two modes, chosen at startup in priority order (config/server.rs). The gateway refuses to start with neither configured, and partial configuration in either mode is rejected:

  1. Managed mTLS (recommended). step-ca ACME (HTTP-01) when CA_URL, CA_FINGERPRINT, and ACME_PROVISIONER are set. The gateway enrolls itself, auto-renews at a fraction of the certificate lifetime, and hot-swaps the renewed certificate onto live listeners.
  2. Static mTLS. GRPC_TLS_CERT_PATH, GRPC_TLS_KEY_PATH, and GRPC_TLS_CLIENT_CA_PATH, for use when step-ca is unavailable.

Both modes are secure and fail closed. There is no plaintext or self-reported-key path.

Identity verification

Each agent has an endpoint_id, a short fingerprint of its public key (the first 8 bytes of the SHA-256 of the certificate’s SubjectPublicKeyInfo, written as 16 lowercase hex characters). Because it is derived from the key, the same agent always produces the same id, and no other agent can reproduce it without the matching private key.

The gateway computes this fingerprint from the certificate the agent presented during the mTLS handshake. It compares that against the endpoint_id the agent claims in its Hello, rejects the connection on a mismatch, then checks the id against a revocation list and rejects revoked devices (identity/fingerprint.rs, service/endpoint_channel.rs).

Downstream delivery

The gateway sends two kinds of frame down to an agent, and treats both as opaque payloads. It forwards the bytes verbatim and never decodes, inspects, or signs them.

  • Commands. Remediation actions from the backend, such as isolate host, kill process, or quarantine file.
  • Trust bundles. The “enrollment result” shown on the diagram. A signed set of keys the agent uses to verify the authenticity of a command.

The gateway parses a routing header (the endpoint_id) to pick the destination stream, then forwards the payload into a downstream frame unchanged (downlink/runner.rs, service/endpoint_channel.rs). The gateway acts as a dumb relay. Verification happens end-to-end at the agent using the trust bundle sent earlier (see Command Trust).

Configuration

The gateway is configured entirely from environment variables. This table covers the connection security variables behind the mTLS scheme and identity verification above. Required means required within the selected mode. The full set (Kafka tuning, batching, and topic overrides) lives in the AnyGateway README. These are gateway variables, distinct from the agent’s install-time variables in the Installation Guide even where a name is shared.

VariableRequiredPurpose
CA_URLYesManaged. step-ca base URL. With the fingerprint and provisioner set, selects managed mTLS. Takes precedence over the static paths.
CA_FINGERPRINTYesManaged. SHA-256 hex of the step-ca root, the trust anchor for fetching and pinning the root.
ACME_PROVISIONERYesManaged. step-ca ACME provisioner name.
CA_ROOT_PATHNoManaged. Optional shipped root PEM. When omitted, the root is fetched from the CA URL and pinned against the fingerprint.
CERT_RENEW_FRACTIONNoManaged. Fraction of the certificate lifetime at which to renew.
GATEWAY_IDENTITY_DNSNoManaged. DNS name ordered from ACME and placed in the certificate SAN.
GRPC_TLS_CERT_PATHYesStatic. Gateway TLS certificate.
GRPC_TLS_KEY_PATHYesStatic. Gateway TLS private key.
GRPC_TLS_CLIENT_CA_PATHYesStatic. CA used to validate client certificates.
REVOKED_ENDPOINTS_PATHNoRevocation. File of revoked endpoint ids the gateway refuses to serve. Feeds the revocation gate.

Library inventory

Key third-party libraries the gateway depends on, for traceability when reading the code.

LibraryUsed for
tokioAsynchronous runtime for all concurrent tasks.
tonic, tonic-prost, prostgRPC transport and Protocol Buffers encoding.
anyprotoShared AnyOne protobuf contracts.
rdkafkaKafka producer and consumer.
rustls, tokio-rustls, rustls-pemfileTLS termination and certificate loading.
instant-acmeACME client for the managed certificate lifecycle.
rcgen, x509-parserCertificate generation and parsing peer certificates.
reqwest, hyper, hyper-util, http, http-body-utilHTTP client and server plumbing.
sha2Hashing for endpoint identity fingerprints.
dashmap, arc-swapConcurrent connection registry and hot-swapping certificates.
serde, serde_jsonSerialization.
tracing, tracing-subscriberLogging and diagnostics.
metricsRuntime metrics.
futures, async-trait, tokio-streamAsync utilities.

Static Analysis API

Repo: AnyOne-StaticAPI (Rust, Axum + Tokio, edition 2024). A file-analysis service the Endpoint Agent submits hashes and files to for a verdict. The agent answers “not known-bad” locally first (see Static Analysis Flow); this service handles the escalations.

The workspace has six crates; two are binaries.

CrateKindRole
gatewaybinAxum HTTP API — upload, hash, dedup, enqueue, poll. Owns ACME/mTLS. I/O only, zero analysis.
workerbinConsumes jobs off Redis, runs the YARA-X engine, writes verdicts back.
queuelibRedis client (queue, job state, dedup, verdict cache).
databaselibPostgres IOC repo + moka cache.
sharedlibDomain types (VerdictCategory, job payloads).
clientbinExample CLI (currently a stale stub — POSTs JSON, not multipart).

Component view

HTTP API

All routes nest under /api/v1 (gateway/src/routes/mod.rs).

MethodPathBehavior
GET/health{status, uptime_secs}.
POST/scanmultipart/form-data, field file, exactly one. Streams to a temp file, SHA-256 on the fly, caps at 100 MiB, renames to samples_dir/{sha256} (content-addressed). Returns {job_id, sha256, status:"queued"}. Codes: 400 / 413 / 500 / 200.
GET/scan/{id}{id} is a UUID; returns {job_id, sha256, status, verdict}; 404 if unknown.
POST/lookupJSON {sha256}{found, verdict}. Queries the Postgres IOC repo (Malicious on hit, Unknown on miss).

Data stores

Redis (queue/src/lib.rs) — transient job plumbing:

KeyTypePurpose / TTL
sag:jobslistThe queue. Gateway LPUSH, worker BRPOP (5 s block).
sag:job:{id}string (JSON)Job lifecycle state — what the client polls. TTL 24 h.
sag:dedup:{sha256}stringIn-flight lock (SET NX EX). TTL 1 h; cleared on completion.
sag:cache:{sha256}stringVerdict memo. Malicious = no expiry, Benign = 72 h, Unknown = 1 h.

Postgres — the IOC hash repository. Table ioc_hashes(id, hash, adversary, first_seen_utc), queried by hash via IocHashRepository, fronted by an in-memory moka cache (database/src/cache.rs). Migrations run on gateway startup.

YARA-X engine (worker/src/engine/yara.rs) — pure-Rust yara-x, compiles every *.yar under the rules dir at init. A match scores 80; EngineRunner aggregates weighted scores: ≥ 80 → Malicious, else Benign; empty/all-failed → Unknown. VerdictCategory has exactly three variants — Malicious / Benign / Unknown (no Suspicious).

The README/CLAUDE.md describe a multi-engine cascade (filetype → PE → fuzzy → ML → TI); the current worker runs YARA-X only — the cascade is aspirational.

Scan flow

Submit → dedup-acquire → enqueue → worker BRPOP → mmap sample → YARA (spawn_blocking) → write verdict + cache → client polls Queued → Running → Done/Failed. Full walk-through with a sequence diagram is on the Static Analysis Flow page.

Relationship to MARKANYFILTER

The local binary-fuse filter (MARKANYFILTER) lives in the agent, not this repo. It gives a fast negative answer (no false negatives) so the agent only calls this service on a possible hit or an Unknown — see the Static Analysis Flow. The artifact is built offline from this service’s ioc_hashes set (a standalone ops step).

SOC Backend

Repo: AnyOne-SOC-Backend (Rust, axum). The HTTP API and orchestration service behind the SOC dashboard, and the endpoint’s command/enrollment counterpart. A single axum binary (anyone-soc-backend), Postgres-backed (schema soc), with best-effort Kafka consumers and producers. This is where operator intent becomes a signed endpoint command, and where enrollment and trust-bundle distribution are driven.

Detection/correlation — turning raw ocsf-events into output.sigma alerts — happens upstream and is out of scope; this service consumes the results, not raw telemetry.

Component view

Layering

Strict api/ → services/ → repository/, with domain/ pure types and middleware/ extractors (src/api/v1/mod.rs). Boot order (src/main.rs): config → PgPool → migrate → seed admin → spawn Kafka consumers → serve. Axum 0.8, sqlx 0.8, JWT HS256 + argon2, utoipa/Swagger.

Kafka topics

Topic (default)DirPurpose
output.sigmaconsumeSIGMA-detection alerts → Postgres → SSE.
uba.anomaly.detectedconsumeUBA anomalies (Confluent protobuf via Schema Registry).
endpoint.enrollmentsconsumeDevice enrollments — sole writer of soc.endpoints.
commands.responseconsumeCommandResult acks from the gateway.
commands.endpointproduceSigned operator commands.
endpoint.enrollment-responsesproduceTrust-bundle blob keyed by endpoint_id.
rule_updatesproduceRule lifecycle events.

Kafka is optional/best-effort. Note it does not consume ocsf-events — alerts arrive already-detected on output.sigma.

Command dispatch & signing

POST /api/v1/commands (admin) → services/command_dispatch.rs. Builds PROCESS_KILL / HOST_ISOLATION / FILE_QUARANTINE (domain/command.rs), Ed25519-signs the canonical bytes via services/signing.rs LocalSigner, publishes to commands.endpoint keyed by endpoint_id (acks=all, idempotent), and persists soc.command_history. The signing key is loaded in-process from COMMAND_SIGNING_PRIVATE_KEY_BASE64 (id COMMAND_SIGNING_KEY_ID, default k1) and embedded as sig_key_id. Gated by COMMAND_DISPATCH_ENABLED (503 otherwise). Signer is a trait, so a KMS backend can drop in later. See Command Trust.

Trust bundle — hold / validate / distribute (not mint)

The backend ingests a pre-signed bundle from COMMAND_TRUST_BUNDLE_BASE64 (domain/trust_bundle.rs); the anchor private key stays offline and the backend never signs bundles. At startup it validates that the bundle advertises the active COMMAND_SIGNING_KEY_ID with a matching public key, or fails fast. On every enrollment it republishes the blob to endpoint.enrollment-responses keyed by endpoint_id; the gateway forwards it down so endpoints build their command-verifier key set. Monotonic version gives downgrade protection (enforced endpoint-side).

Enrollment

enrollment_consumer is the sole writer of soc.endpoints (integrity check that endpoint_id matches the SHA-256 of the presented public key). One-click issuance POST /api/v1/bootstrap/enrollment/issue (admin) signs a per-device ECDSA P-256 leaf in process (services/enrollment_ca.rs; the CA key is age-encrypted at rest) and returns a one-time claimable link that renders the enrollment .bat. The enrollment CA itself is created offline; mTLS is terminated at the Gateway against step-ca-signed certs. See Enrollment.

Other subsystems

SIGMA→OCSF normalizer + custom-rule CRUD; container/system observability (Prometheus, docker-socket-proxy); G-Bridge CACAO playbook proxy; Neo4j ATT&CK knowledge-graph explorer; SSE alert stream. External deps: Postgres, Kafka (+ optional Schema Registry), G-Bridge, Neo4j, Prometheus.

Offline provisioning (not in this repo)

Trust-bundle minting (offline anchor key) and the IOC filter build remain standalone offline ops steps — this backend only consumes their outputs (a pre-signed bundle; a populated ioc_hashes set).

Proto Contracts

Repo: AnyProto (protobuf, Buf). The central wire ABI, consumed everywhere as the anyproto crate (prost messages + tonic stubs). Two concerns live here: the control channel between agent and gateway, and the OCSF telemetry schema.

Proto tree

Control channel (endpoint/v1/endpoint.proto)

One service, one bidirectional stream — there is no separate enrollment or telemetry service here (enrollment rides in-band on Hello.shared_enrollment_secret; telemetry lives in ocsf.v1).

service EndpointService {
  rpc EndpointChannel(stream EndpointChannelRequest)
                returns (stream EndpointChannelResponse);
}
DirectionEnvelope oneofMessages
endpoint → serverEndpointChannelRequestHello, Heartbeat, CommandResult
server → endpointEndpointChannelResponseCommand, TrustBundle
  • Helloendpoint_id, mac_addr, public_key (bytes), shared_enrollment_secret, HelloMetadata (hostname, os, agent_version, mac_addresses[], machine_uuid). First frame.
  • Commandendpoint_id + bytes raw (opaque, backend-signed).
  • TrustBundleendpoint_id + bytes raw (anchor-signed signing-key bundle).
  • CommandResultcommand_id, ResultStatus (OK/ERROR/SIG_ERROR/TARGET_REJECTED/EXPIRED), detail.

OCSF telemetry (ocsf/v1/)

The schema is not one message per class. There is a single flat OcsfEvent (event.proto) with a class_uid discriminator and many optional object fields; each field is annotated with the class_uid that populates it. The telemetry service is OcsfEventService with unary OcsfBatch(OcsfBatchRequest{agent_id, events}) → OcsfBatchResponse{accepted, rejected, errors} (ocsf.proto).

Class UIDs actually used (verified against event.proto — note the corrections from earlier project docs):

ClassUIDField
FileSystemActivity1001file / file_result
KernelExtensionActivity1002
ModuleActivity1005module
ScheduledJobActivity1006job
ProcessActivity1007process
ScriptActivity1009
RegistryKeyActivity201001reg_key
RegistryValueActivity201002
WindowsServiceActivity201004
ProcessRemediation7003command_uid
NetworkActivity / DnsActivity4001 / 4003
AccountChange / Authentication3001 / 3002
DetectionFinding2004

Corrections vs older docs: Module is 1005 (1006 is ScheduledJob); remediation is 7003; there is no 999999 agent-lifecycle class in the proto.

Streaming/ML messages (EnrichedEvent, WindowResult, AnomalyDetected) and the enrich.v1 IpReputation type also live here but belong to downstream correlation, out of this project’s scope.

Consumption

Consumed as a git dependency tracking main (or pinned to a rust/v… tag):

anyproto = { git = "https://github.com/MarkAny-GaneshaIT/AnyProto", branch = "main" }

Buf codegen (buf.gen.yaml) runs neoeinstein-prost + neoeinstein-tonic + a crate generator; the generated crate is committed at gen/rust/ and CI regenerates it on push. Cargo features endpoint-v1, enrich-v1, ocsf-v1 (+ proto_full) let a consumer compile only the packages it uses.

Telemetry Egress

The primary data plane: raw kernel/OS events become OCSF records and land in Kafka. It spans the Kernel DriversEndpoint AgentGateway → Kafka.

Sequence

Stages

  1. Collect. sources/ turn driver and OS signals into a uniform EDREvent (service/src/event.rs). Kernel-callback and minifilter events arrive from the drivers; ETW, the Event Log, WFP tamper watch, and lifecycle events are produced in-agent.
  2. Normalize. pipeline/normalizer canonicalizes fields (paths, identifiers).
  3. Deduplicate. pipeline/dedup drops repeats via a bloom filter (utils/bloom_filter) — cheap suppression of high-volume duplicates before the expensive stages.
  4. Enrich. pipeline/enricher attaches process-tree ancestry (utils/process_tree) so an event carries its parent chain.
  5. Map to OCSF. pipeline/mapper emits the flat OcsfEvent with the right class_uid (see the UID table in Proto Contracts).
  6. Batch & ship. pipeline/batcher groups events into an OcsfBatchRequest; channel::telemetry sends it over the OcsfEventService gRPC (lazy-connected, keep-alive, mTLS). The gateway schema-encodes each event and produces to ocsf-events.

Failure handling

If the gRPC send fails, channel::ndjson spools the batch to a local NDJSON file and replays it once the link recovers — telemetry is not lost across a gateway outage. mTLS certs hot-swap on renewal without dropping the pipeline (see Enrollment).

Control Channel

A single long-lived bidirectional gRPC stream carries everything that is not bulk telemetry: identity, liveness, commands, and trust bundles. It runs between the agent’s channel::remotectl and the gateway’s EndpointService (EndpointChannel RPC), and bridges to Kafka on the gateway side.

Sequence

Frames

The first frame must be Hello, or the gateway rejects the stream with failed_precondition. After identity is verified (see Enrollment), the gateway registers the live downstream sender keyed by endpoint_id in its connection registry — this is what lets a Kafka command find the right open stream.

DirectionFrameMeaning
upHelloIdentity + host metadata; opens the stream.
upHeartbeatLiveness; the gateway “touches” the connection.
upCommandResultOutcome of a dispatched command → published to commands.response.
downCommandOpaque backend-signed action (see Command Trust).
downTrustBundleAnchor-signed signing-key list; hot-swapped by the agent.

Reconnect

The agent reconnects with exponential backoff on stream loss and re-sends Hello. The gateway drops the stale registry entry when the stream ends, so downstream frames for a disconnected endpoint have nowhere to route until it re-registers.

Enrollment

How a device gains a transport identity the gateway will trust. A per-device enrollment certificate is issued by the SOC Backend, and the agent proves possession over mTLS. Identity is bound to the key, not the hostname.

Sequence

Identity derivation

endpoint_id = SHA-256(SubjectPublicKeyInfo_DER)[..8] — the first 8 bytes of the SHA-256 of the full SPKI DER, as 16 lowercase hex chars. Both sides compute it the same way: the agent from its own keypair (enrollment/fingerprint.rs), the gateway from the presented mTLS leaf cert (identity/mtls_trust.rs). The gateway enforces verify_fingerprint(claimed_id, spki) and rejects a mismatch, then applies a revocation-list gate. The SOC backend independently records the enrollment after an integrity check on the same id.

Credential lifecycle

The SOC backend’s one-click issuance (POST /api/v1/bootstrap/enrollment/issue) signs a per-device ECDSA P-256 leaf in process against an enrollment CA whose key is age-encrypted at rest; the CA itself is created offline. The device receives this as its durable X5C enrollment credential; the private key is stored in the DPAPI-protected keystore (enrollment/keystore.rs). The agent then uses the X5C credential to obtain and renew its short-lived operational mTLS cert from step-ca, and enrollment/certmanager.rs hot-swaps renewals without tearing down the Control Channel or the telemetry link.

In dev, the gateway can instead trust the SPKI self-reported in Hello.public_key when ALLOW_INSECURE_HELLO_TRUST=true — never in production.

Command Trust

Commands are authenticated end-to-end, independent of mTLS transport. The SOC Backend signs; the Gateway forwards opaque bytes; the agent verifies. This survives the gateway’s mTLS termination — a compromised or curious gateway cannot forge a command.

Sequence

The trust chain

  • Anchor (offline). An Ed25519 anchor key signs the trust bundle. Minting happens with a standalone offline ops tool; the anchor private key never touches any online service. Its public key is pinned in the installer and stamped into the enrollment payload (AGENT_COMMAND_ANCHOR_PUBLIC_KEY_BASE64).
  • Trust bundle. An anchor-signed list of currently-valid command-signing public keys with a monotonic u64 version. The SOC backend ingests a pre-signed bundle (COMMAND_TRUST_BUNDLE_BASE64), validates at startup that it advertises the active signing key, and republishes it on each enrollment to endpoint.enrollment-responses. The gateway forwards it down the Control Channel as a TrustBundle frame. The agent accepts a new bundle only if its version increases (anti-downgrade) and its anchor signature verifies, then hot-swaps it (commands/trust_bundle.rs).
  • Command. Ed25519-signed by the SOC backend’s in-process signing key (services/signing.rs LocalSigner, id k1), embedded as sig_key_id. The agent’s commands/verifier.rs checks the signature against the keys in the current bundle before dispatch.

Verification & dispatch

commands/dispatcher.rs runs a command only after verifier accepts it. A bad signature returns CommandResult status SIG_ERROR; other rejections map to TARGET_REJECTED / EXPIRED. Accepted commands proceed to Remediation. The canonical signing form (canonical_bytes) is byte-identical across backend and agent — guarded by a frozen cross-repo parity test — so a signature computed on the backend verifies on the endpoint.

Static Analysis Flow

Tiered file triage. The agent answers “definitely not known-bad” locally in microseconds and only reaches out to the Static Analysis API when it must. The design bias is fail-open: uncertainty never blocks a file.

Sequence

Tiers

  1. Local filter (static_api/filter.rs). MARKANYFILTER is a xorf::BinaryFuse8 over IOC hashes (key = first 8 bytes of the SHA-256, big-endian). A binary fuse filter has no false negatives, so contains() == false means “not a known-bad hash” — answered with zero network. Hot-reloaded every ~30 s via mtime + ArcSwap. Built off-host as an offline ops step (see tier 2).
  2. Confirm a possible hit (/lookup). contains() == true is “probably present” (~1/256 false positive), so the agent confirms against the API’s Postgres IOC repo. Malicious → act; Benign → done; Unknown → escalate. Bounded retry, fails open — an unconfirmed positive never blocks. The MARKANYFILTER artifact itself is built offline from the StaticAPI ioc_hashes set (a standalone ops step, not an online service) and hot-reloaded by the agent.
  3. Full scan (/scan + poll). A local miss’s Unknown (or a lookup Unknown) escalates to a YARA-X scan: submit the file, poll GET /scan/:id until Done/Failed, then read the verdict.

A Malicious verdict drives Remediation; everything else lets the file proceed. See the Static Analysis API page for the server side of /lookup, /scan, and the Redis/Postgres stores.

Remediation

Response actions on the endpoint: kill a process, quarantine a file, isolate the host from the network. An action runs only from a verified signed Command or an autonomous local verdict, and every action is audited.

Sequence

Actions

ActionModuleWhat it does
Killremediation/kill.rsTerminates the target process (and optionally its tree).
Quarantineremediation/quarantine.rsMoves the file into a protected QuarantineStore and neutralizes it.
Isolate / unisolateremediation/isolation.rsDrives the driver’s WFP-based host network isolation.

Isolation is enforced in the kernel: the executor issues NETWORK_ISOLATE / NETWORK_UNISOLATE IOCTLs to AnyOneKcallback, with SET_LIFELINE allow-listing the addresses the agent needs to keep talking to the gateway (so an isolated host can still receive the un-isolate command). See Kernel Drivers.

Gating & audit

remediation/verdict.rs decides whether an action is warranted; remediation/executor.rs performs it; remediation/audit.rs records the outcome. Server-dispatched actions arrive verified via Command Trust and report back a CommandResult up the Control Channel; autonomous actions (e.g. a local Malicious verdict from Static Analysis) run under policy without a round trip.