Runtime Security: Catching What the Scanner Missed

Your image scanned clean at build time. That tells you about known vulnerabilities in packages at one moment. It tells you nothing about what the container does at 3am.

A clean image can still spawn a shell, write to a directory it has never written to, or open an outbound connection to an address nobody recognises. Build-time scanning cannot see any of that.

What runtime security watches

Falco observes system calls and raises alerts when behaviour matches a rule. The high-value signals are unglamorous:

• A shell starting inside a container that should never need one
• A process writing to a sensitive path
• An unexpected outbound network connection
• A container running with privileges it did not previously have
• Package management running in production

That first one is worth deploying for on its own. Almost no production container has a legitimate reason to start bash, so the alert has a very low false positive rate and a very high signal.

A rule looks like this

- rule: Shell in web container
  desc: A shell was started inside the web application container
  condition: >
    spawned_process and container
    and container.image.repository contains "myapp-web"
    and proc.name in (bash, sh, zsh)
  output: >
    Shell in web container (user=%user.name command=%proc.cmdline
    container=%container.name)
  priority: WARNING

Readable, reviewable, version-controlled.

Tuning is the work

Out of the box it is noisy, and noise kills adoption. The routine is: run in a staging environment for a week, look at what fires during normal operation, and write exceptions for the legitimate things.

Do the tuning against a real workload. A rule set tuned against an idle system will bury you the day traffic arrives.

Where it fits with everything else

Scanning finds known vulnerabilities before deployment. Runtime rules find unexpected behaviour after. They cover different failure modes, and a zero-day or a compromised dependency shows up only in the second.

For most of my client work, the minimum viable version is: shell-in-container, unexpected outbound connections, and writes to the web root. Three rules, on the servers that matter.

The operational requirement

Alerts need somewhere to go and someone to read them. Runtime alerts arrive at inconvenient hours by definition — that is when the interesting things happen. Route them to something that reaches a human, and be selective enough that being woken is justified.

Resources

• Repo: falcosecurity/falco
• Docs: falco.org/docs
• Video walkthroughs: YouTube: Falco runtime security
• Build-time counterpart: Trivy