Evidence of Intrusions: What the RaptorGRC Audit Log Records and What It Does Not

Illustration for: Evidence of Intrusions: What the RaptorGRC Audit Log Records and What It Does Not

The seventh goal of CISA’s Secure by Design pledge asks a signer to “demonstrate a measurable increase in the ability for customers to gather evidence of cybersecurity intrusions affecting the manufacturer’s products”. Its example approaches name three log categories a baseline product should make available, which are configuration changes and configuration reads, identity events such as sign-in and token creation, and access to or creation of business-relevant data.

This is the goal where the substance is strong and the wording around it has needed the most care. Our pledge page used to say RaptorGRC wrote “every action” to its audit log. It now says it records security-relevant actions and then states the edges, which is the accurate version.

What exists

RaptorGRC writes to an audit log held in its own database. Each entry records an action, the subject it acted on, the acting user’s name and identifier, a category, a timestamp, and a free-form set of properties that the calling code supplies.

Entries are written through a single logging service, which sends them both to the structured application log and to a bounded queue drained into the database by a background service. Administrators read them from an audit log screen in the admin area.

Two things about that are genuinely good. The log is not gated behind an edition or a paid tier, because RaptorGRC has one edition and everything is in it. And it lives in your database, inside your network, so the evidence is yours before anybody knows they need it, and it cannot be withheld by a vendor during an incident. Those two properties are the substance of what this goal is asking for, and we have them.

Why “every action” was the wrong claim

Audit entries are written where a use case calls the logging service. Around six hundred files reference it, so coverage is broad, but there is no general interceptor sitting at the persistence layer that snapshots every change on its way to the database. Coverage is therefore a matter of engineering discipline rather than a structural guarantee, and discipline is uneven by nature.

That design is defensible on its own terms. Deliberate logging usually produces better entries than automatic diffing, because the calling code knows what the action meant, whereas a generic interceptor knows only that a column changed. What it does not support is the inference an investigator most wants to make, which is that an absence of entries proves an absence of activity.

So the claim had to change, and it has. The page now says security-relevant actions rather than every action, and it names this limitation itself.

What the write queue actually does, because I described it wrongly

An earlier draft of this article said the audit queue drops entries under load, with the implication that this was carelessness. That is not what the code does, and the correction is worth more space than the original claim.

The queue is a bounded channel with a capacity of one thousand entries. The full-mode is set to wait, which sounds odd for a queue nothing ever waits on, and the comment in the source explains why. Wait is the only full-mode in which a non-blocking write returns false when the channel is full. Every drop mode returns true and discards an entry silently. Since no producer ever uses the blocking write, nothing waits, and the code gets the one behaviour it wanted, which is a failure it can see.

The alternative would have been to drop the oldest entry, which is the conventional choice and is the wrong one here. Discarding the oldest audit evidence with no signal is exactly the failure an intruder would want. Refusing the newest entry and having the caller record precisely which entry was refused keeps the accepted backlog intact and leaves a trace of the loss.

Shutdown is handled the same way. The queue is completed rather than cancelled, so the writer drains its backlog and exits deterministically, with a ten-second bound and a warning logged if it does not finish in time.

The genuine limitation is narrower than what I originally wrote. Capacity is one thousand. A database write is retried once, and on a second failure that entry is dropped with an error log so that one poison entry cannot stall the queue. Entries can be lost, and every route by which they are lost is signalled. That is the correct trade for a component that must never take the application down, and it is a very different thing from silent discard.

The four gaps

There is no actor address or user agent

An entry names the user. It does not record the IP address the action came from, or the client that made it.

For an intrusion investigation, that is the single most valuable missing field. The question after a compromise is rarely “which account did this”, because the answer is usually a legitimate account. The question is “from where”, and we cannot answer it.

There are no before and after values

The properties bag holds whatever the calling code chose to put in it. Some use cases record the meaningful values and some record only that something happened.

There is no structured record of what a field held before a change and what it held after. For a platform whose job is holding risk assessments and control evidence, an entry saying a record was updated, without saying what changed, does not settle a dispute about whether an assessment was altered after the fact.

Nothing makes the log tamper-evident

There is no hash chain, no signature and no append-only enforcement. An entry has no link to the one before it.

A sufficiently privileged intruder, which is precisely the intruder you are investigating, can delete rows from the audit table and leave the remaining log internally consistent. What we have is evidence of what happened. It is not proof that the record was not altered, and those are different things that get conflated constantly in this market.

There is no shipped route to a SIEM, and no retention control

The application uses a logging framework that reads its sinks from configuration, so an operator can add a syslog or collector sink and forward entries. We do not ship one configured, and we do not document the recipe.

There is also no configurable retention or purge for audit entries. They accumulate. Other parts of the product do implement retention sweeps, so the pattern exists in the codebase and has simply not been applied here.

What a customer should do today

Given the above, three practical things.

1. Treat the audit log as a first source rather than the only one. Pair it with the logs from the layers around the container, which are your reverse proxy, your identity provider and your database, because those carry the addresses and the session detail the application log does not.
2. If you need SIEM ingestion, configure a sink through the logging framework’s configuration rather than waiting for us to ship one, and ask us if you want the exact settings.
3. If your retention obligations require the audit trail to be kept or purged on a schedule, plan for that at the database level, because the application will not do it for you.

The argument that this is enough, taken seriously

There is a defensible position that says an audit log in a self-hosted product does not need hash chaining or its own SIEM connector, because the customer already owns the host, the database and the network, and can apply their own controls at every one of those layers. A customer with a mature logging estate will forward the database and container logs anyway, and duplicating that inside the application is effort spent on a problem the customer has already solved.

There is something in that, particularly for the larger organisations running this product. It is also the reasoning that lets a vendor never build any of it, because there is always a layer below where somebody else could take responsibility.

The pledge is explicit about where that responsibility sits. Software manufacturers should take ownership of their customers’ security outcomes, and providing the artefacts to gather evidence of an intrusion is named as part of that. A customer who has to reconstruct what happened inside the application from database transaction logs is doing our job for us.

The order I would fix them in

CISA accepts a published roadmap as a way of demonstrating progress where a product does not yet collect the logs it should. This is not a roadmap with dates, because I am not in a position to promise dates. It is the order I think is right.

1. Add the actor address and user agent to every entry, because that is the field an investigation reaches for first and cannot get.
2. Guarantee coverage of authentication and authorisation events, meaning successful and failed sign-ins, factor changes, role changes and permission changes, rather than relying on each feature to remember.
3. Add tamper evidence, most simply as a hash chain linking each entry to the previous one.
4. Ship a documented forwarding configuration and a retention setting.

The fair description of goal seven, until at least the first two are done, is that RaptorGRC gives you an audit trail that lives in your network and belongs to you, that its write path fails visibly rather than quietly, and that what it captures is narrower than the phrase “every action” once implied. The pledge page now says that itself, which is the outcome this article was written to reach.

The full tracker is at /secure-by-design. What does and does not leave your network is set out at /transparency.