RF

Detection engineering / Security automation

Detection-as-Code Pipeline

A from-scratch compiler and delivery path that turns supported Sigma detections into Wazuh PCRE2 XML, validates every artifact, keeps rule identity stable, and deploys through the Wazuh API behind a dry run.

View repository (opens in a new tab) Compiler trace (opens in a new tab)
Status
Active open source
Role
Sole engineer, architecture through deployment
Stack
Python, pySigma, Jinja2, PCRE2, Wazuh REST API, GitHub Actions
Deployment
Manual dispatch on a self-hosted runner, dry run first

Claim

Translates its supported Sigma subset into traceable Wazuh rules and delivers them through a controlled workflow.

Evidence

58 Sigma rules compile to 216 Wazuh rules across 119 ATT&CK techniques, behind 87 automated tests.

Boundary

Not a complete Sigma backend. Unsupported constructs and oversized DNF expansions fail the build instead of emitting uncertain rules.

The problem

Wazuh needed more than a format converter.

There is no official pySigma backend for Wazuh 4.x, and Wazuh cannot directly express every boolean structure Sigma allows. A useful pipeline has to preserve supported logic, reject the rest loudly, keep rule IDs stable across rebuilds, and treat deployment as a controlled change rather than a file copy.

The failure mode is operational. A detection that compiles wrong looks deployed while silently missing the behavior it was written to catch.

Architecture

Five stages, each able to stop the build.

  1. Author Sigma YAML Single source of truth with UUIDs and ATT&CK tags. 3 hand-authored rules, 55 curated SigmaHQ imports.
  2. Parse pySigma AST Syntax validation and the expression tree the compiler walks.
  3. Compile DNF and PCRE2 Field mapping, De Morgan on negations, lookahead merges, Jinja2 rendering.
  4. Gate CI validation 87 tests, ID and UUID linkage checks, XML well-formedness, security scans.
  5. Deliver Wazuh REST API Dry-run diff on pull requests, authenticated upload, stale-rule reconciliation, manager restart.

Inside the compiler

The walker moves logic, not strings.

pySigma parses each rule into an expression tree. The walker maps Sigma fields to Wazuh decoder fields, converts wildcards into anchored PCRE2, applies De Morgan's law to negations, and distributes nested boolean logic into disjunctive normal form. One DNF alternative becomes one Wazuh rule; the imported mimikatz detection fans out to 23.

evaluate_ast() output for the certutil rule: one OR-alternative, same-field literals merged 
[
  {
    "win.eventdata.image": [
      { "pattern": "(?i).*\\certutil\.exe$", "negate": false }
    ],
    "win.eventdata.commandLine": [
      { "pattern": "(?=.*(?i).*urlcache.*)(?=.*(?i).*split.*)",
        "negate": false }
    ]
  }
]

The homepage shows this rule's full source and generated XML. The repository trace (opens in a new tab) follows every intermediate stage line by line.

Engineering decisions

Chosen around failure modes.

Fail loud on unsupported Sigma

Numeric comparisons, regex and CIDR modifiers, base64 modifiers, field-less keywords, and null values stop the build with a reason. They never emit a rule that looks valid but matches nothing.

Cap DNF growth at 500 clauses

AND distribution over nested ORs is a cartesian product. A hard cap makes a pathologically OR-heavy rule fail the build instead of hanging CI or exhausting memory.

Pin rule identity in a committed registry

id_registry.json maps each Sigma UUID to its Wazuh rule IDs, so detection identity survives reordering and rebuilds. CI checks for collisions and missing linkage, and every generated file carries its source UUID in a comment.

Deploy by diff, never by copy

Pull requests get a dry-run deployment diff. Real deployment authenticates to the Wazuh API, uploads the bundle, reconciles stale rules behind a blast-radius safeguard, and restarts the manager.

Treat the pipeline itself as attack surface

GitHub Actions are SHA-pinned. Gitleaks, CodeQL, Bandit, pip-audit, and Dependabot run in CI, and a written threat model covers the supply chain around the detection pipeline.

Validation

What has been checked.

58Sigma source rules
216Generated Wazuh rules
87Automated tests
119ATT&CK techniques
14ATT&CK tactics, all covered
500DNF clause cap

Deploying against a live Wazuh 4.9 manager exposed a real bug: the API reported rule files under relative_dirname where the parser expected path. The reconciliation logic and its regression tests were fixed against observed behavior, not assumptions.

Known limits

What it does not solve.

  • Partial Sigma coverage. Unsupported modifiers and value types are rejected by design.
  • Field mapping risk. An unmapped field passes through unchanged and may compile into a rule that never fires if no decoder populates it.
  • Combinatorial logic. Rules that exceed the DNF cap must be simplified or handled by a different backend strategy.
  • Platform change. Wazuh 5 adds native Sigma support, which reduces the value of the syntax conversion but not of validation, testing, and controlled deployment.
  • Documented dependency exception. A time-boxed pip-audit exception covers a transitive diskcache CVE while upstream compatibility is constrained.

Source

Inspect it yourself.

The compiler, templates, ID registry, tests, workflows, coverage report, and threat model are all public.

Open the repository (opens in a new tab) Discuss this project