Introduction
System Description
AnyOne EDR is an endpoint detection and response platform by MarkAny GaneshaIT, deployed as two halves: endpoint agents on Windows hosts and a backend server in the cloud. Each agent collects kernel-level telemetry, normalizes it to OCSF, and streams it up to the backend over mTLS gRPC. The backend ingests and stores that telemetry, detects malicious patterns, and drives remediation, whether operator-dispatched from the SOC dashboard or automated. Response actions travel back down to the agent as cryptographically signed commands.
This is the whole-system technical reference: the endpoint agent and every backend service in scope. For installing, configuring, and operating just the agent, see the Installation Guide.
Document Purpose
This reference explains the architecture and internals of the AnyOne EDR system: what each component is, how it is built, and how the components act together. It is written for engineers building, extending, or debugging the platform, not for operators installing the agent.
It is organized two ways:
- Structure: what each part is. One page per in-scope repository, covering its boundary, responsibilities, and internal components. Diagrams are component views (C4-ish). Start at System Overview.
- Behavior: how the parts act together. One page per cross-service flow. Diagrams are sequence views. Start at Telemetry Egress.
Document Scope
In scope are six repositories, one structure page each: AnyOne-Endpoint, AnyOne-Kernel, AnyGateway, AnyOne-StaticAPI, AnyOne-SOC-Backend, AnyProto.
Out of scope are deployment infrastructure (AnyOne-Infra) and upstream detection and correlation
(ocsf-events to alerts). The SOC backend consumes detection results, not raw telemetry.
Trust-bundle minting (offline anchor key) and the IOC filter build are standalone offline ops steps, not online services. The backend and agent only consume their outputs.
System Overview
The AnyOne EDR system at a glance: a context view of the whole platform and the outside actors it touches, plus a repository map linking each in-scope repo to its own structure page.
System context
ℹ️ Arrow colors:
- Blue : data plane (telemetry flow)
- Orange : control plane (commands, enrollment, and remediation actions).
The two planes
Traffic between endpoint and backend splits into two planes, and that split is the core of the security model:
- Data plane (up). The agent ships OCSF telemetry to the gateway over one gRPC service and holds a second, long-lived bidirectional gRPC stream for control and liveness. File verdicts take a separate HTTPS path to the Static Analysis API. All three are mTLS. Everything the endpoint sends is a report. It issues no commands upward.
- Control plane (down). Response actions (kill process, quarantine file, isolate host) originate at the SOC backend, which Ed25519-signs each command, and flow down through Kafka and the gateway to the agent. The gateway is the only endpoint-facing service that touches Kafka, and it never parses command bodies. It forwards them as opaque, backend-signed bytes (see Command Trust). Trust bundles ride the same downward path.
Repository map
| Repo | Language | Role | Runs on | Page |
|---|---|---|---|---|
| AnyOne-Endpoint | Rust | AnyOneAgent service for telemetry, the OCSF pipeline, static-analysis orchestration, remediation, and enrollment. Primary repo. | Endpoint | Endpoint Agent |
| AnyOne-Kernel | C | AnyOneKcallback (kernel-callback and WFP telemetry) and AnyOneMinifilter (file-system minifilter). | Endpoint | Kernel Drivers |
| AnyGateway | Rust | gRPC gateway that terminates agent mTLS and bridges telemetry and enrollments up to Kafka, and commands and trust-bundles back down. | Backend | Gateway |
| AnyOne-StaticAPI | Rust | Static file-analysis service with a gateway and worker, backed by Redis and Postgres. | Backend | Static Analysis API |
| AnyOne-SOC-Backend | Rust | Orchestrator behind the SOC dashboard that ingests alerts, enrollments, and results, dispatches signed commands, issues enrollment, and distributes trust bundles. | Backend | SOC Backend |
| AnyProto | proto | Central protobuf contracts (anyproto crate), the wire ABI between agent and gateway, plus the OCSF telemetry schema. | (compiled into consumers) | Proto Contracts |
| AnyOne-Infra | yaml/docker | Deployment config. Out of scope. | n/a | n/a |
Endpoint Agent
Repo: AnyOne-Endpoint (Rust workspace). The per-host agent that runs as the AnyOneAgent
Windows service (anyone-agent.exe). It collects kernel and OS telemetry, runs the OCSF pipeline,
orchestrates static analysis, executes remediation, and manages its own enrollment identity.
Workspace crates
The workspace has five members. Only the drivers and the service run on the endpoint long-term.
| Crate | Binary | Role |
|---|---|---|
service | anyone-agent.exe | The agent itself, covering everything below. |
cli | anyone-agent-cli.exe | Interactive shell that talks to the running agent over a named pipe. |
notifier | anyone-agent-notifier.exe | Helper spawned into the active desktop session to show toast notifications. |
updater | anyone-updater.exe | Boot-time service that verifies a staged update and rolls back on failure. |
shared | (lib) | Types shared across crates: events, the IPC contract, and constants. |
Note: package names for the client/notifier crates (
anyedr-cli,anyedr-notifier) differ from their binary names.
Component view
ℹ️ Arrow colors:
- Blue : data plane (telemetry flow)
- Orange : control plane (commands, enrollment, and remediation actions).
Source tree
The service crate is a single binary. Its source is organized by subsystem.
service/src/
├── main.rs
├── service.rs
├── bootstrap.rs
├── config.rs
├── context.rs
├── event.rs
├── stats.rs
├── constants.rs
├── driver.rs
├── sources/
│ ├── etw/
│ ├── eventlog.rs
│ ├── kcallback/
│ ├── minifilter/
│ ├── lifecycle.rs
│ └── tamperwatch.rs
├── pipeline/
│ ├── normalizer.rs
│ ├── dedup.rs
│ ├── enricher.rs
│ ├── mapper.rs
│ └── batcher.rs
├── channel/
│ ├── remotectl.rs
│ ├── telemetry.rs
│ ├── static_analysis.rs
│ ├── localctl.rs
│ ├── notification.rs
│ ├── ndjson.rs
│ └── handlers/
├── static_api/
│ ├── filter.rs
│ └── worker.rs
├── commands/
│ ├── dispatcher.rs
│ ├── verifier.rs
│ ├── registry.rs
│ ├── trust_bundle.rs
│ ├── kill_process.rs
│ ├── file_quarantine.rs
│ ├── isolate_host.rs
│ ├── update.rs
│ ├── update_package.rs
│ ├── handler.rs
│ ├── context.rs
│ ├── outcome.rs
│ └── noop.rs
├── remediation/
│ ├── executor.rs
│ ├── kill.rs
│ ├── quarantine.rs
│ ├── isolation.rs
│ ├── verdict.rs
│ ├── audit.rs
│ └── types.rs
├── enrollment/
│ ├── identity.rs
│ ├── keystore.rs
│ ├── certmanager.rs
│ ├── stepca.rs
│ ├── tls.rs
│ ├── fingerprint.rs
│ ├── metadata.rs
│ └── startup.rs
└── utils/
├── process_tree.rs
├── bloom_filter.rs
├── hash.rs
├── pipe.rs
├── path.rs
├── install.rs
├── executable.rs
└── user_resolver.rs
External communication
The diagram shows the wires. This table describes what actually crosses each one, the external processes the agent talks to, what moves, and over which protocol. Internal module-to-module calls are omitted.
| From | To | Data | Protocol |
|---|---|---|---|
| AnyOneKcallback (kernel) | sources/ kcallback | Kernel events: process, thread, image load, registry, object, and network activity | IOCTL and shared ring buffer |
| AnyOneMinifilter (kernel) | sources/ minifilter | File-system activity events | Minifilter communication port |
| Event Tracing for Windows | sources/ etw | Operating-system trace events from ETW providers | ETW real-time session |
| Windows Event Logs | sources/ eventlog | Windows event-log records | Event Log subscription |
channel/ telemetry | Gateway | Batched OCSF telemetry | gRPC, mTLS |
channel/ remotectl | Gateway | Up: enrollment, heartbeats, and command results. Down: signed commands and trust bundles | gRPC bidirectional stream, mTLS |
channel/ staticanalysis | Static Analysis API | Files to scan and hashes to look up, with verdicts returned | HTTPS, mTLS |
enrollment/ certmanager | step-ca | Certificate bootstrap (CSR signing) and renewal | HTTPS, pinned root |
| AnyOne CLI | channel/ localctl | Local operator commands and status queries | Named pipe |
channel/ notification | AnyOne Notifier | Toast notification requests | Child-process handoff |
Module layout (service)
What each module and subsystem is responsible for.
Entry / host:
| Module | Responsibility |
|---|---|
main.rs | Development-only entry point for running the agent directly from the command line. Slated for removal, since production installs run the agent as a Windows service with its entry point in service.rs. |
service.rs | The production entry point. Registers with the Service Control Manager, handles start and stop requests, and brings up the subsystems: configuration, shared state, the control channel, the pipeline, and the sources. |
bootstrap.rs | One-time startup setup: process hardening, acquiring the privileges the agent needs, and initialising logging. |
config.rs | A global configuration singleton populated from environment variables. |
context.rs | Shared runtime state handed to every subsystem. |
event.rs | The internal event model and the set of telemetry sources. |
stats.rs | Runtime counters used for metrics and diagnostics. |
constants.rs | Fixed values shared across the service. |
driver.rs | Transport to the AnyOneKcallback kernel driver. Opens the device and moves telemetry and commands across the kernel boundary. |
Subsystems:
| Module | Responsibility |
|---|---|
sources/ | Collectors that gather telemetry from the kernel drivers and the operating system. |
pipeline/ | Transforms raw events into batched OCSF telemetry through normalize, deduplicate, enrich, map, and batch stages. |
channel/ | All external transport: the telemetry and control channels to the gateway, the static-analysis client, the local CLI server, and the notifier handoff. |
static_api/ | Orchestrates static-analysis API calls, using a local filter to decide which files actually need a scan. |
commands/ | Receives commands from the server, verifies their signatures, and dispatches them to the matching handler. |
remediation/ | Executes response actions (kill process, quarantine file, isolate host) and records an audit trail. |
enrollment/ | Manages the agent’s cryptographic identity: enrollment, secure key storage, and certificate renewal. |
utils/ | Shared helper routines used across the service. |
Configuration
The agent takes all of its configuration from environment variables, read once at startup into a global configuration singleton. There is no configuration file the agent parses itself.
How those variables are set depends on the install path:
- Normal (dashboard) install. The installer provisions the credentials and writes the configuration for you, so there is nothing to set by hand.
- Manual install. You write the variables yourself as the service’s registry
Environmentvalue, which the Service Control Manager injects into the process when it starts.
The mandatory variables are:
| Variable | Purpose |
|---|---|
GRPC_ENDPOINT | Gateway address for the telemetry and control channels. |
KERNEL_DRIVER_PATH | Location of the AnyOneKcallback driver file. |
MINIFILTER_DRIVER_PATH | Location of the AnyOneMinifilter driver file. |
CA_URL, CA_FINGERPRINT, CA_PROVISIONER | step-ca connection and root pinning for enrollment. |
ENROLLMENT_CERT_SUBJECT | The device id, which must match the enrollment certificate. |
ENROLLMENT_X5C_CERT_PATH, ENROLLMENT_X5C_KEY_IMPORT_PATH | The delivered device credential. |
COMMAND_ANCHOR_PUBLIC_KEY_BASE64 | Anchor public key used to verify signed commands. |
For the full procedure and every available setting, see Manual Installation → Configuring the Agent in the Installation Guide.
Library inventory
Key third-party libraries the agent depends on, for traceability when reading the code.
| Library | Used for |
|---|---|
tokio | Asynchronous runtime for all concurrent tasks. |
tonic, prost | gRPC transport and Protocol Buffers encoding. |
anyproto | Shared AnyOne protobuf contracts for telemetry and control. |
reqwest | HTTP client for the Static Analysis API. |
windows, windows-service | Windows system APIs and Windows service integration. |
ed25519-dalek | Verifying signed server commands. |
p256, pkcs8, rcgen, x509-parser | Enrollment certificates and key handling. |
xorf | Local membership filter for the static-analysis pre-check. |
arc-swap | Hot-swapping the rotating identity and filter without a restart. |
sha2, hex, base64 | Hashing and encoding. |
serde, serde_json, postcard | Serialization. |
quick-xml | Parsing Windows event-log XML. |
dashmap, once_cell | Shared concurrent state. |
clap | Command-line parsing. |
flexi_logger, log | Logging. |
zip | Extracting update packages. |
Kernel Drivers
Repo: AnyOne-Kernel (C, WDK). Two kernel
drivers feed the Endpoint Agent with the events user mode cannot see on its
own. Both build from the shared solution AnyEDR-Drivers.sln.
- AnyOneKcallback (built from the
AnyKDriver/project asaokclbk.sys) is a control device hosting the OS notify callbacks and the network filtering callouts. It streams events to the agent over a shared-memory ring buffer and takes control commands (isolation, ring setup) over device I/O controls. - AnyOneMinifilter (built from the
AnyMDriver/project asaomnflt.sys) is a Filter Manager minifilter that observes file activity. It sends events to the agent over a filter communication port, which also carries a small runtime command channel.
Why two drivers? They use three different kernel extension mechanisms: OS notify callbacks (process, thread, image, registry, handle events), the Windows Filtering Platform for network traffic, and a Filter Manager minifilter for file activity. The notify callbacks and WFP pair naturally in one ordinary kernel driver (AnyOneKcallback), but the minifilter has a distinct registration model and lifecycle, so it is simpler to develop as its own driver and repository (AnyOneMinifilter) for now. Merging the two into a single driver is planned.
AnyOneKcallback
A software-only driver loaded as a kernel service. It captures process, thread, image, registry, object, and network activity, and it also enforces host network isolation on command.
Component view
Blue is event and telemetry data flowing toward the agent. Orange is control, setup, and commands (callback registration, IOCTL control, isolation). Dashed arrows are internal dependencies and the ring signal.
Module layout
| File | Responsibility |
|---|---|
Driver.c | Entry point. Creates the control device and a separate raw device for the network filter, wires every subsystem together, restores isolation state on load, and tears everything down on unload. |
Callbacks.c | The OS kernel notify callbacks for process, thread, image, registry, and object activity. Builds telemetry events and also strips dangerous handle rights to protected processes for self-protection. |
WfpCallouts.c | Registers the network filtering session, callouts, and filters. Emits network connect, accept, and close telemetry and enforces isolation blocking. |
WfpSecurity.c | Stamps a SYSTEM-only access control list onto the isolation filter objects and the persisted-isolation registry key. |
Isolation.c | The network isolation state machine: the isolation flag, the lifeline allowlist, the block decision, and persistence across reboot and Safe Mode. |
EventFilter.c | A registry-path allowlist that drops non-persistence-relevant registry events before they reach the ring buffer. |
EventQueue.c | The shared-memory ring buffer: initialization, event push, copy-based drain, mapping into the agent, and the user notification event. |
IoctlHandler.c | The single dispatcher for every control request the agent sends: ring mapping, statistics, drain, isolation, lifeline, and protected-process registration. |
TelemetryCommon.h | The kernel-to-agent binary contract: device names, control codes, and event structs, kept byte-compatible with the Rust agent. |
DeviceContext.h | The per-device state: the ring buffer, the registry filter, the callback handles, and the protected-process list. |
Ring buffer
The agent maps the buffer once and then reads it directly, waiting on a named event that the driver signals on every push. The producer region is read-only to user mode, and the agent maps a separate read-write region holding only its read cursor, so the writer (kernel) and the reader (agent) never share writable memory. On overflow the driver sets a flag and increments a dropped-events counter instead of blocking. A copy-based drain control is the fallback path.
| Region | Writer | Purpose |
|---|---|---|
| control header (32 B) | kernel | Write index, dropped count, signalled flag, layout fields |
| events (default 16384 × 1160 B, about 17 MB) | kernel | The telemetry event slots |
| consumer (8 B) | agent | The read cursor, on its own read-write page |
Control surface
The agent drives the driver through device I/O controls in the range 0x801 to 0x823. Beyond ring setup, these also command host network isolation, tying this driver into Remediation.
| Category | Codes | Purpose |
|---|---|---|
| Ring setup | 0x810 / 0x811 / 0x817 | Map and unmap the ring buffer, and hand over the agent's notification event |
| Event pull | 0x805 | Copy-based drain, the fallback to the mapped ring |
| Statistics | 0x801 / 0x804 / 0x816 / 0x81A | Queue count, statistics, dropped count, and callback status |
| Isolation | 0x820 / 0x821 / 0x822 / 0x823 | Isolate, un-isolate, read status, and set the lifeline allowlist |
| Anti-tamper | 0x802 | Register a protected process (accepted only from the agent image) |
| Diagnostics | 0x818 / 0x819 | Network filter diagnostics and last status |
AnyOneMinifilter
A Filter Manager minifilter that watches file activity for ransomware, wiper, and tampering behavior, and defends the agent’s own files.
Component view
Blue is event data flowing toward the agent. Orange is control and setup (operation registration and config commands). Black dashed is an internal dependency, one module using another. The Command Handler is Driver.c’s message-notify callback, which reads and applies
g_Configand reports queue stats.
Module layout
| File | Responsibility |
|---|---|
Driver.c | Entry point. Registers the minifilter and its operation callbacks, starts filtering, creates the communication port, handles volume attach and detach, and runs the runtime command channel. |
Callbacks.c | The pre and post callbacks for every monitored file operation. Gathers evidence into per-file context and hands finished events to the dispatcher. |
PostCreateDecision.c | A pure decision function that classifies a file open's intent and decides whether to report it and as which event type. |
ProcUtils.c | Process trust logic: whitelisting the agent's own service, LocalSystem checks, and browser-process detection. |
Utils.c | File classification helpers (suspicious extensions, dropper locations, mark-of-the-web, decoy files) and the self-defense protection-rule engine. |
EventDispatcher.c | Builds the user-mode messages, queues them onto a worker thread, and sends them over the port. Critical alerts fall back to a synchronous send if the queue is full. |
Config.h | The runtime configuration (monitoring and self-defense toggles) and the queue and timeout constants. |
Driver.h | The master header: the message layout, the command protocol, and the per-file context structs. |
Monitored operations
The minifilter registers pre and post callbacks for file open, write, set-information, file-system control, and cleanup. Rather than report every operation, it defers most reporting to cleanup, when the final outcome is known, and it classifies open intent on the create path.
| Operation | What it detects |
|---|---|
| Create | Open intent, suspicious location and extension, mark-of-the-web, browser-dropped files, decoy access, and self-defense blocking of the agent's own files |
| Write | Write tracking (size, offset, running total, and a per-file write counter) |
| Set-information | Rename (capturing old and new path) and delete intent |
| File-system control | The zero-data wiper path (a write-equivalent that bypasses normal writes), plus reparse-point set and delete used for privilege escalation and evasion |
| Cleanup | The deferred emit point, sending the final delete or modify event once the handle closes |
External communication
| From | To | Data | Protocol |
|---|---|---|---|
| Windows kernel notify routines | AnyOneKcallback | Process, thread, image, registry, and object events | In-kernel callback registration |
| WFP filter engine | AnyOneKcallback | Network connect, accept, and close events | WFP callouts |
| AnyOneKcallback | Endpoint agent | Telemetry event stream | Shared-memory ring buffer (read-only map) plus a named signal event |
| Endpoint agent | AnyOneKcallback | Ring setup, drain, isolation, lifeline, protected-process registration | Device I/O control over \\.\AnyOneKcallback |
| Filter Manager | AnyOneMinifilter | File create, write, set-information, file-system control, and cleanup operations | Minifilter pre/post callbacks |
| AnyOneMinifilter | Endpoint agent | File, volume, and named-pipe events | Filter communication port \AnyOneMinifilterPort |
| Endpoint agent | AnyOneMinifilter | Runtime configuration commands (ping, get, set) | Filter communication port \AnyOneMinifilterPort |
Build and installation
The drivers build from the shared Visual Studio solution alongside the agent, and in a normal deployment the agent installs and starts both of them on its own startup rather than being registered by hand. The full procedures live in the Installation Guide, so they are not repeated here.
- Building the agent and both drivers from source: Building from Source.
- Enabling test signing and disabling Secure Boot on a test machine: Environment Setup.
- Installing on an endpoint, either the standard SOC dashboard installer or by manual service registration.
Kernel components
The key Windows and WDK facilities these drivers build on, for traceability.
| Area | Facilities |
|---|---|
| Framework | Kernel-Mode Driver Framework (WDF) control device and I/O queues; Filter Manager (FltMgr) for the minifilter |
| Notify callbacks | PsSetCreateProcessNotifyRoutineEx, PsSetCreateThreadNotifyRoutine, PsSetLoadImageNotifyRoutine, CmRegisterCallbackEx, ObRegisterCallbacks |
| Network | Windows Filtering Platform callouts and filters (FwpsCalloutRegister0, FwpmFilterAdd0) |
| File system | Filter Manager operation callbacks, FltSendMessage, and communication ports |
| Toolchain | WDK with the MSVC v143 toolset, built for Release | x64 |
Gateway
Repo: AnyGateway (Rust). The gateway sits between the endpoints and the backend. It transports
telemetry and enrollments up to Kafka, and forwards commands and enrollment results down to
the agents.
gRPC services
Two gRPC services defined in AnyProto contracts.
| Proto service | Rust impl (src/service/) | RPC | Role |
|---|---|---|---|
OcsfEvent Service | ocsf.rs | OcsfBatch (unary) | Schema-encodes each OCSF event, produces it to Kafka, and returns accept and reject counts. |
Endpoint Service | endpoint_ channel.rs | EndpointChannel (bidirectional stream) | The long-lived control channel (see below). |
The control stream requires Hello as its first frame (otherwise failed_precondition), verifies
the endpoint identity, publishes an enrollment event, and registers the live downstream sender in
the connection registry keyed by endpoint_id. It then pumps commands and enrollment results down,
and heartbeats and command results up.
Component view
Arrow colors: blue = data plane (telemetry flowing up), orange = control plane (commands, enrollment, and remediation actions flowing down).
External communication
The diagram shows the wires. This table describes what actually crosses each one, the external processes the gateway talks to, what moves, and over which protocol. Internal module-to-module calls are omitted.
| From | To | Data | Protocol |
|---|---|---|---|
| Endpoints | service/ocsf | Batched telemetry | gRPC, mTLS |
| Endpoints | service/endpoint_channel | Control stream. Up: Hello, heartbeats, and command results. Down: signed commands and trust bundles | gRPC bidirectional stream, mTLS |
broker | Kafka | Up topics: raw.endpoint, enrollment.endpoint, command.endpoint.response | Kafka |
| Kafka | broker | Down topics: command.endpoint, enrollment.endpoint.response | Kafka |
certmgr | step-ca | Certificate enrollment and renewal | ACME over HTTPS |
registry | Schema Registry | OCSF schema registration and lookup | HTTP |
Module layout (src)
| Module | Responsibility |
|---|---|
server/ | gRPC server wiring: the TLS authentication and resolver, certificate setup, and shutdown. |
service/ | gRPC service implementations plus the enrollment publisher. |
identity/ | Endpoint SPKI-based identity verification from the peer mTLS certificate, fingerprint checking, and revocation. |
connections/ | Stream each endpoint_id message to its downstream sender. |
downlink/ | The Kafka-to-delivery loop that decides whether each frame is a command or a trust bundle. |
delivery/ | Routes a downstream frame to the matching live stream. |
certmgr/ | Managed certificate lifecycle with step-ca: enroll, renew, and hot-swap. |
registry/ | Schema Registry client and the message encoder. |
broker/ | Kafka producer and consumer. |
config/ | Environment-based configuration for the server, TLS, and Kafka. |
Kafka topics
| Topic | Direction | Carries |
|---|---|---|
raw.endpoint | up | OCSF telemetry (OcsfService) |
enrollment.endpoint | up | Enrollment events |
command.endpoint. response | up | CommandResult acknowledgements |
command.endpoint | down | Backend commands, forwarded as Command frames |
enrollment.endpoint. response (optional) | down | Trust-bundle responses, forwarded as TrustBundle frames |
mTLS scheme
Two modes, chosen at startup in priority order (config/server.rs). The gateway refuses to
start with neither configured, and partial configuration in either mode is rejected:
- Managed mTLS (recommended). step-ca ACME (HTTP-01) when
CA_URL,CA_FINGERPRINT, andACME_PROVISIONERare set. The gateway enrolls itself, auto-renews at a fraction of the certificate lifetime, and hot-swaps the renewed certificate onto live listeners. - Static mTLS.
GRPC_TLS_CERT_PATH,GRPC_TLS_KEY_PATH, andGRPC_TLS_CLIENT_CA_PATH, for use when step-ca is unavailable.
Both modes are secure and fail closed. There is no plaintext or self-reported-key path.
Identity verification
Each agent has an endpoint_id, a short fingerprint of its public key (the first 8 bytes of the
SHA-256 of the certificate’s SubjectPublicKeyInfo, written as 16 lowercase hex characters). Because
it is derived from the key, the same agent always produces the same id, and no other agent can
reproduce it without the matching private key.
The gateway computes this fingerprint from the certificate the agent presented during the mTLS
handshake. It compares that against the endpoint_id the agent claims in its Hello, rejects the connection on
a mismatch, then checks the id against a revocation list and rejects revoked devices (identity/fingerprint.rs,
service/endpoint_channel.rs).
Downstream delivery
The gateway sends two kinds of frame down to an agent, and treats both as opaque payloads. It forwards the bytes verbatim and never decodes, inspects, or signs them.
- Commands. Remediation actions from the backend, such as isolate host, kill process, or quarantine file.
- Trust bundles. The “enrollment result” shown on the diagram. A signed set of keys the agent uses to verify the authenticity of a command.
The gateway parses a routing header (the endpoint_id) to pick the destination stream, then
forwards the payload into a downstream frame unchanged (downlink/runner.rs,
service/endpoint_channel.rs). The gateway acts as a dumb relay. Verification happens end-to-end at
the agent using the trust bundle sent earlier (see Command Trust).
Configuration
The gateway is configured entirely from environment variables. This table covers the connection
security variables behind the mTLS scheme and identity verification above. Required means required
within the selected mode. The full set (Kafka tuning, batching, and topic overrides) lives in the
AnyGateway README. These are gateway variables, distinct from the agent’s install-time variables in
the Installation Guide even where a name is shared.
| Variable | Required | Purpose |
|---|---|---|
CA_URL | Yes | Managed. step-ca base URL. With the fingerprint and provisioner set, selects managed mTLS. Takes precedence over the static paths. |
CA_FINGERPRINT | Yes | Managed. SHA-256 hex of the step-ca root, the trust anchor for fetching and pinning the root. |
ACME_PROVISIONER | Yes | Managed. step-ca ACME provisioner name. |
CA_ROOT_PATH | No | Managed. Optional shipped root PEM. When omitted, the root is fetched from the CA URL and pinned against the fingerprint. |
CERT_RENEW_FRACTION | No | Managed. Fraction of the certificate lifetime at which to renew. |
GATEWAY_IDENTITY_DNS | No | Managed. DNS name ordered from ACME and placed in the certificate SAN. |
GRPC_TLS_CERT_PATH | Yes | Static. Gateway TLS certificate. |
GRPC_TLS_KEY_PATH | Yes | Static. Gateway TLS private key. |
GRPC_TLS_CLIENT_CA_PATH | Yes | Static. CA used to validate client certificates. |
REVOKED_ENDPOINTS_PATH | No | Revocation. File of revoked endpoint ids the gateway refuses to serve. Feeds the revocation gate. |
Library inventory
Key third-party libraries the gateway depends on, for traceability when reading the code.
| Library | Used for |
|---|---|
tokio | Asynchronous runtime for all concurrent tasks. |
tonic, tonic-prost, prost | gRPC transport and Protocol Buffers encoding. |
anyproto | Shared AnyOne protobuf contracts. |
rdkafka | Kafka producer and consumer. |
rustls, tokio-rustls, rustls-pemfile | TLS termination and certificate loading. |
instant-acme | ACME client for the managed certificate lifecycle. |
rcgen, x509-parser | Certificate generation and parsing peer certificates. |
reqwest, hyper, hyper-util, http, http-body-util | HTTP client and server plumbing. |
sha2 | Hashing for endpoint identity fingerprints. |
dashmap, arc-swap | Concurrent connection registry and hot-swapping certificates. |
serde, serde_json | Serialization. |
tracing, tracing-subscriber | Logging and diagnostics. |
metrics | Runtime metrics. |
futures, async-trait, tokio-stream | Async utilities. |
Static Analysis API
Repo: AnyOne-StaticAPI (Rust, Axum + Tokio, edition 2024). A file-analysis service the
Endpoint Agent submits hashes and files to for a verdict. The agent answers
“not known-bad” locally first (see Static Analysis Flow); this
service handles the escalations.
The workspace has six crates; two are binaries.
| Crate | Kind | Role |
|---|---|---|
gateway | bin | Axum HTTP API — upload, hash, dedup, enqueue, poll. Owns ACME/mTLS. I/O only, zero analysis. |
worker | bin | Consumes jobs off Redis, runs the YARA-X engine, writes verdicts back. |
queue | lib | Redis client (queue, job state, dedup, verdict cache). |
database | lib | Postgres IOC repo + moka cache. |
shared | lib | Domain types (VerdictCategory, job payloads). |
client | bin | Example CLI (currently a stale stub — POSTs JSON, not multipart). |
Component view
HTTP API
All routes nest under /api/v1 (gateway/src/routes/mod.rs).
| Method | Path | Behavior |
|---|---|---|
GET | /health | {status, uptime_secs}. |
POST | /scan | multipart/form-data, field file, exactly one. Streams to a temp file, SHA-256 on the fly, caps at 100 MiB, renames to samples_dir/{sha256} (content-addressed). Returns {job_id, sha256, status:"queued"}. Codes: 400 / 413 / 500 / 200. |
GET | /scan/{id} | {id} is a UUID; returns {job_id, sha256, status, verdict}; 404 if unknown. |
POST | /lookup | JSON {sha256} → {found, verdict}. Queries the Postgres IOC repo (Malicious on hit, Unknown on miss). |
Data stores
Redis (queue/src/lib.rs) — transient job plumbing:
| Key | Type | Purpose / TTL |
|---|---|---|
sag:jobs | list | The queue. Gateway LPUSH, worker BRPOP (5 s block). |
sag:job:{id} | string (JSON) | Job lifecycle state — what the client polls. TTL 24 h. |
sag:dedup:{sha256} | string | In-flight lock (SET NX EX). TTL 1 h; cleared on completion. |
sag:cache:{sha256} | string | Verdict memo. Malicious = no expiry, Benign = 72 h, Unknown = 1 h. |
Postgres — the IOC hash repository. Table ioc_hashes(id, hash, adversary, first_seen_utc),
queried by hash via IocHashRepository, fronted by an in-memory moka cache
(database/src/cache.rs). Migrations run on gateway startup.
YARA-X engine (worker/src/engine/yara.rs) — pure-Rust yara-x, compiles every *.yar under
the rules dir at init. A match scores 80; EngineRunner aggregates weighted scores: ≥ 80 →
Malicious, else Benign; empty/all-failed → Unknown. VerdictCategory has exactly three variants —
Malicious / Benign / Unknown (no Suspicious).
The README/CLAUDE.md describe a multi-engine cascade (filetype → PE → fuzzy → ML → TI); the current worker runs YARA-X only — the cascade is aspirational.
Scan flow
Submit → dedup-acquire → enqueue → worker BRPOP → mmap sample → YARA (spawn_blocking) → write
verdict + cache → client polls Queued → Running → Done/Failed. Full walk-through with a sequence
diagram is on the Static Analysis Flow page.
Relationship to MARKANYFILTER
The local binary-fuse filter (MARKANYFILTER) lives in the agent, not this repo. It gives a fast
negative answer (no false negatives) so the agent only calls this service on a possible hit or an
Unknown — see the Static Analysis Flow. The artifact is built
offline from this service’s ioc_hashes set (a standalone ops step).
SOC Backend
Repo: AnyOne-SOC-Backend (Rust, axum). The HTTP API and orchestration service behind the SOC
dashboard, and the endpoint’s command/enrollment counterpart. A single axum binary
(anyone-soc-backend), Postgres-backed (schema soc), with best-effort Kafka consumers and
producers. This is where operator intent becomes a signed endpoint command, and where enrollment
and trust-bundle distribution are driven.
Detection/correlation — turning raw
ocsf-eventsintooutput.sigmaalerts — happens upstream and is out of scope; this service consumes the results, not raw telemetry.
Component view
Layering
Strict api/ → services/ → repository/, with domain/ pure types and middleware/ extractors
(src/api/v1/mod.rs). Boot order (src/main.rs): config → PgPool → migrate → seed admin → spawn
Kafka consumers → serve. Axum 0.8, sqlx 0.8, JWT HS256 + argon2, utoipa/Swagger.
Kafka topics
| Topic (default) | Dir | Purpose |
|---|---|---|
output.sigma | consume | SIGMA-detection alerts → Postgres → SSE. |
uba.anomaly.detected | consume | UBA anomalies (Confluent protobuf via Schema Registry). |
endpoint.enrollments | consume | Device enrollments — sole writer of soc.endpoints. |
commands.response | consume | CommandResult acks from the gateway. |
commands.endpoint | produce | Signed operator commands. |
endpoint.enrollment-responses | produce | Trust-bundle blob keyed by endpoint_id. |
rule_updates | produce | Rule lifecycle events. |
Kafka is optional/best-effort. Note it does not consume ocsf-events — alerts arrive
already-detected on output.sigma.
Command dispatch & signing
POST /api/v1/commands (admin) → services/command_dispatch.rs. Builds PROCESS_KILL /
HOST_ISOLATION / FILE_QUARANTINE (domain/command.rs), Ed25519-signs the canonical bytes via
services/signing.rs LocalSigner, publishes to commands.endpoint keyed by endpoint_id
(acks=all, idempotent), and persists soc.command_history. The signing key is loaded in-process
from COMMAND_SIGNING_PRIVATE_KEY_BASE64 (id COMMAND_SIGNING_KEY_ID, default k1) and embedded as
sig_key_id. Gated by COMMAND_DISPATCH_ENABLED (503 otherwise). Signer is a trait, so a KMS
backend can drop in later. See Command Trust.
Trust bundle — hold / validate / distribute (not mint)
The backend ingests a pre-signed bundle from COMMAND_TRUST_BUNDLE_BASE64
(domain/trust_bundle.rs); the anchor private key stays offline and the backend never signs
bundles. At startup it validates that the bundle advertises the active COMMAND_SIGNING_KEY_ID with
a matching public key, or fails fast. On every enrollment it republishes the blob to
endpoint.enrollment-responses keyed by endpoint_id; the gateway forwards it down so endpoints
build their command-verifier key set. Monotonic version gives downgrade protection (enforced
endpoint-side).
Enrollment
enrollment_consumer is the sole writer of soc.endpoints (integrity check that endpoint_id
matches the SHA-256 of the presented public key). One-click issuance
POST /api/v1/bootstrap/enrollment/issue (admin) signs a per-device ECDSA P-256 leaf in process
(services/enrollment_ca.rs; the CA key is age-encrypted at rest) and returns a one-time claimable
link that renders the enrollment .bat. The enrollment CA itself is created offline; mTLS is
terminated at the Gateway against step-ca-signed certs. See
Enrollment.
Other subsystems
SIGMA→OCSF normalizer + custom-rule CRUD; container/system observability (Prometheus, docker-socket-proxy); G-Bridge CACAO playbook proxy; Neo4j ATT&CK knowledge-graph explorer; SSE alert stream. External deps: Postgres, Kafka (+ optional Schema Registry), G-Bridge, Neo4j, Prometheus.
Offline provisioning (not in this repo)
Trust-bundle minting (offline anchor key) and the IOC filter build remain standalone offline
ops steps — this backend only consumes their outputs (a pre-signed bundle; a populated
ioc_hashes set).
Proto Contracts
Repo: AnyProto (protobuf, Buf). The central wire ABI, consumed everywhere as the anyproto
crate (prost messages + tonic stubs). Two concerns live here: the control channel between agent
and gateway, and the OCSF telemetry schema.
Proto tree
Control channel (endpoint/v1/endpoint.proto)
One service, one bidirectional stream — there is no separate enrollment or telemetry service
here (enrollment rides in-band on Hello.shared_enrollment_secret; telemetry lives in ocsf.v1).
service EndpointService {
rpc EndpointChannel(stream EndpointChannelRequest)
returns (stream EndpointChannelResponse);
}
| Direction | Envelope oneof | Messages |
|---|---|---|
| endpoint → server | EndpointChannelRequest | Hello, Heartbeat, CommandResult |
| server → endpoint | EndpointChannelResponse | Command, TrustBundle |
Hello—endpoint_id,mac_addr,public_key(bytes),shared_enrollment_secret,HelloMetadata(hostname, os, agent_version, mac_addresses[], machine_uuid). First frame.Command—endpoint_id+bytes raw(opaque, backend-signed).TrustBundle—endpoint_id+bytes raw(anchor-signed signing-key bundle).CommandResult—command_id,ResultStatus(OK/ERROR/SIG_ERROR/TARGET_REJECTED/EXPIRED),detail.
OCSF telemetry (ocsf/v1/)
The schema is not one message per class. There is a single flat OcsfEvent (event.proto) with
a class_uid discriminator and many optional object fields; each field is annotated with the
class_uid that populates it. The telemetry service is OcsfEventService with unary
OcsfBatch(OcsfBatchRequest{agent_id, events}) → OcsfBatchResponse{accepted, rejected, errors}
(ocsf.proto).
Class UIDs actually used (verified against event.proto — note the corrections from earlier
project docs):
| Class | UID | Field |
|---|---|---|
| FileSystemActivity | 1001 | file / file_result |
| KernelExtensionActivity | 1002 | |
| ModuleActivity | 1005 | module |
| ScheduledJobActivity | 1006 | job |
| ProcessActivity | 1007 | process |
| ScriptActivity | 1009 | |
| RegistryKeyActivity | 201001 | reg_key |
| RegistryValueActivity | 201002 | |
| WindowsServiceActivity | 201004 | |
| ProcessRemediation | 7003 | command_uid |
| NetworkActivity / DnsActivity | 4001 / 4003 | |
| AccountChange / Authentication | 3001 / 3002 | |
| DetectionFinding | 2004 |
Corrections vs older docs: Module is 1005 (1006 is ScheduledJob); remediation is 7003; there is no
999999agent-lifecycle class in the proto.
Streaming/ML messages (EnrichedEvent, WindowResult, AnomalyDetected) and the enrich.v1
IpReputation type also live here but belong to downstream correlation, out of this project’s scope.
Consumption
Consumed as a git dependency tracking main (or pinned to a rust/v… tag):
anyproto = { git = "https://github.com/MarkAny-GaneshaIT/AnyProto", branch = "main" }
Buf codegen (buf.gen.yaml) runs neoeinstein-prost + neoeinstein-tonic + a crate generator; the
generated crate is committed at gen/rust/ and CI regenerates it on push. Cargo features
endpoint-v1, enrich-v1, ocsf-v1 (+ proto_full) let a consumer compile only the packages it
uses.
Telemetry Egress
The primary data plane: raw kernel/OS events become OCSF records and land in Kafka. It spans the Kernel Drivers → Endpoint Agent → Gateway → Kafka.
Sequence
Stages
- Collect.
sources/turn driver and OS signals into a uniformEDREvent(service/src/event.rs). Kernel-callback and minifilter events arrive from the drivers; ETW, the Event Log, WFP tamper watch, and lifecycle events are produced in-agent. - Normalize.
pipeline/normalizercanonicalizes fields (paths, identifiers). - Deduplicate.
pipeline/dedupdrops repeats via a bloom filter (utils/bloom_filter) — cheap suppression of high-volume duplicates before the expensive stages. - Enrich.
pipeline/enricherattaches process-tree ancestry (utils/process_tree) so an event carries its parent chain. - Map to OCSF.
pipeline/mapperemits the flatOcsfEventwith the rightclass_uid(see the UID table in Proto Contracts). - Batch & ship.
pipeline/batchergroups events into anOcsfBatchRequest;channel::telemetrysends it over theOcsfEventServicegRPC (lazy-connected, keep-alive, mTLS). The gateway schema-encodes each event and produces toocsf-events.
Failure handling
If the gRPC send fails, channel::ndjson spools the batch to a local NDJSON file and replays it once
the link recovers — telemetry is not lost across a gateway outage. mTLS certs hot-swap on renewal
without dropping the pipeline (see Enrollment).
Control Channel
A single long-lived bidirectional gRPC stream carries everything that is not bulk telemetry:
identity, liveness, commands, and trust bundles. It runs between the agent’s channel::remotectl
and the gateway’s EndpointService (EndpointChannel RPC), and bridges to Kafka on the gateway
side.
Sequence
Frames
The first frame must be Hello, or the gateway rejects the stream with failed_precondition.
After identity is verified (see Enrollment), the gateway registers the live
downstream sender keyed by endpoint_id in its connection registry — this is what lets a Kafka
command find the right open stream.
| Direction | Frame | Meaning |
|---|---|---|
| up | Hello | Identity + host metadata; opens the stream. |
| up | Heartbeat | Liveness; the gateway “touches” the connection. |
| up | CommandResult | Outcome of a dispatched command → published to commands.response. |
| down | Command | Opaque backend-signed action (see Command Trust). |
| down | TrustBundle | Anchor-signed signing-key list; hot-swapped by the agent. |
Reconnect
The agent reconnects with exponential backoff on stream loss and re-sends Hello. The gateway drops
the stale registry entry when the stream ends, so downstream frames for a disconnected endpoint have
nowhere to route until it re-registers.
Enrollment
How a device gains a transport identity the gateway will trust. A per-device enrollment certificate is issued by the SOC Backend, and the agent proves possession over mTLS. Identity is bound to the key, not the hostname.
Sequence
Identity derivation
endpoint_id = SHA-256(SubjectPublicKeyInfo_DER)[..8] — the first 8 bytes of the SHA-256 of the
full SPKI DER, as 16 lowercase hex chars. Both sides compute it the same way: the agent from its
own keypair (enrollment/fingerprint.rs), the gateway from the presented mTLS leaf cert
(identity/mtls_trust.rs). The gateway enforces verify_fingerprint(claimed_id, spki) and rejects a
mismatch, then applies a revocation-list gate. The SOC backend independently records the enrollment
after an integrity check on the same id.
Credential lifecycle
The SOC backend’s one-click issuance (POST /api/v1/bootstrap/enrollment/issue) signs a per-device
ECDSA P-256 leaf in process against an enrollment CA whose key is age-encrypted at rest; the CA
itself is created offline. The device receives this as its durable X5C enrollment credential; the
private key is stored in the DPAPI-protected keystore (enrollment/keystore.rs). The agent then uses
the X5C credential to obtain and renew its short-lived operational mTLS cert from step-ca, and
enrollment/certmanager.rs hot-swaps renewals without tearing down the
Control Channel or the telemetry link.
In dev, the gateway can instead trust the SPKI self-reported in
Hello.public_keywhenALLOW_INSECURE_HELLO_TRUST=true— never in production.
Command Trust
Commands are authenticated end-to-end, independent of mTLS transport. The SOC Backend signs; the Gateway forwards opaque bytes; the agent verifies. This survives the gateway’s mTLS termination — a compromised or curious gateway cannot forge a command.
Sequence
The trust chain
- Anchor (offline). An Ed25519 anchor key signs the trust bundle. Minting happens with a
standalone offline ops tool; the anchor private key never touches any online service. Its
public key is pinned in the installer and stamped into the enrollment payload
(
AGENT_COMMAND_ANCHOR_PUBLIC_KEY_BASE64). - Trust bundle. An anchor-signed list of currently-valid command-signing public keys with a
monotonic
u64version. The SOC backend ingests a pre-signed bundle (COMMAND_TRUST_BUNDLE_BASE64), validates at startup that it advertises the active signing key, and republishes it on each enrollment toendpoint.enrollment-responses. The gateway forwards it down the Control Channel as aTrustBundleframe. The agent accepts a new bundle only if its version increases (anti-downgrade) and its anchor signature verifies, then hot-swaps it (commands/trust_bundle.rs). - Command. Ed25519-signed by the SOC backend’s in-process signing key (
services/signing.rsLocalSigner, idk1), embedded assig_key_id. The agent’scommands/verifier.rschecks the signature against the keys in the current bundle before dispatch.
Verification & dispatch
commands/dispatcher.rs runs a command only after verifier accepts it. A bad signature returns
CommandResult status SIG_ERROR; other rejections map to TARGET_REJECTED / EXPIRED. Accepted
commands proceed to Remediation. The canonical signing form (canonical_bytes)
is byte-identical across backend and agent — guarded by a frozen cross-repo parity test — so a
signature computed on the backend verifies on the endpoint.
Static Analysis Flow
Tiered file triage. The agent answers “definitely not known-bad” locally in microseconds and only reaches out to the Static Analysis API when it must. The design bias is fail-open: uncertainty never blocks a file.
Sequence
Tiers
- Local filter (
static_api/filter.rs).MARKANYFILTERis axorf::BinaryFuse8over IOC hashes (key = first 8 bytes of the SHA-256, big-endian). A binary fuse filter has no false negatives, socontains() == falsemeans “not a known-bad hash” — answered with zero network. Hot-reloaded every ~30 s via mtime +ArcSwap. Built off-host as an offline ops step (see tier 2). - Confirm a possible hit (
/lookup).contains() == trueis “probably present” (~1/256 false positive), so the agent confirms against the API’s Postgres IOC repo.Malicious→ act;Benign→ done;Unknown→ escalate. Bounded retry, fails open — an unconfirmed positive never blocks. TheMARKANYFILTERartifact itself is built offline from the StaticAPIioc_hashesset (a standalone ops step, not an online service) and hot-reloaded by the agent. - Full scan (
/scan+ poll). A local miss’sUnknown(or a lookupUnknown) escalates to a YARA-X scan: submit the file, pollGET /scan/:iduntilDone/Failed, then read the verdict.
A Malicious verdict drives Remediation; everything else lets the file proceed.
See the Static Analysis API page for the server side of
/lookup, /scan, and the Redis/Postgres stores.
Remediation
Response actions on the endpoint: kill a process, quarantine a file, isolate the host
from the network. An action runs only from a verified signed Command or an
autonomous local verdict, and every action is audited.
Sequence
Actions
| Action | Module | What it does |
|---|---|---|
| Kill | remediation/kill.rs | Terminates the target process (and optionally its tree). |
| Quarantine | remediation/quarantine.rs | Moves the file into a protected QuarantineStore and neutralizes it. |
| Isolate / unisolate | remediation/isolation.rs | Drives the driver’s WFP-based host network isolation. |
Isolation is enforced in the kernel: the executor issues NETWORK_ISOLATE / NETWORK_UNISOLATE
IOCTLs to AnyOneKcallback, with SET_LIFELINE allow-listing the addresses the agent needs to keep talking
to the gateway (so an isolated host can still receive the un-isolate command). See
Kernel Drivers.
Gating & audit
remediation/verdict.rs decides whether an action is warranted; remediation/executor.rs performs
it; remediation/audit.rs records the outcome. Server-dispatched actions arrive verified via
Command Trust and report back a CommandResult up the
Control Channel; autonomous actions (e.g. a local Malicious verdict from
Static Analysis) run under policy without a round trip.