Audit Investigation

Audit Investigation

Audit records are the primary source of file-level truth in Xferity. When you need to prove what happened to a file — when it was processed, what the outcome was, which run it belonged to — the audit log is where you look.

This page describes how to query, trace, and interpret audit records.

What audit records contain

Each audit event is a structured JSON record written to the append-oriented audit JSONL file.

Fields can include:

Field	Description
timestamp	When the event occurred (ISO 8601)
level	Event severity
flow	Flow name
run_id	Run identifier
correlation_id	Correlation identifier for correlated events
event_type	What happened: file_processed, file_failed, run_started, run_completed, etc.
file_name	Original file name
local_path	Local filesystem path
remote_path	Partner remote path
idempotency_key	Idempotency hash or key used
outcome	success, failure, skipped
error_code	Machine-readable error code when applicable
metadata	Additional structured event metadata

When tamper evidence is enabled, records also include chain_seq, prev_hash, and event_hash.

Tracing a file

The fastest way to look up a file’s history is xferity trace:

xferity trace payroll-2026-03-15.csv

This queries audit records by file basename and returns matching events across all runs. The query sanitizes the input to a basename before lookup.

The trace output shows:

• when the file was first seen
• what happened at each processing stage
• which run each event belongs to
• whether it succeeded, failed, or was skipped as a duplicate

Querying via the audit API

The HTTP API exposes an audit query endpoint:

GET /api/audit?file=<basename>

This returns the same file-lifecycle events as the CLI trace command. The query is authenticated and the file parameter is sanitized.

Example:

GET /api/audit?file=invoice-2026-03-15.xml

Direct file queries

Audit records are stored as newline-delimited JSON (JSONL). Standard tools can query them directly.

Find all events for a specific file:

grep '"invoice-2026-03-15.xml"' /var/xferity/audit/audit.jsonl | jq .

Find failures in a time window:

grep '"outcome":"failure"' /var/xferity/audit/audit.jsonl | \
  jq 'select(.timestamp >= "2026-03-15T00:00:00Z")'

Find all events for a specific run:

grep '"run_id":"<run-id>"' /var/xferity/audit/audit.jsonl | jq .

Sidecar index

For large audit files, Xferity maintains a sidecar index that maps file basenames to byte offsets in the audit JSONL file. This speeds up file-specific lookups significantly without scanning the entire file.

The sidecar index is maintained automatically when audit.query_index_enabled=true. If disabled, lookups scan the full file.

The sidecar index path defaults to the same directory as the audit file with a .idx extension. It can be overridden with audit.sidecar_index_path.

Interpreting tamper evidence

When tamper evidence is enabled:

• each event contains chain_seq, prev_hash, and event_hash
• chain_seq is a monotonically increasing sequence number
• prev_hash is the hash of the previous event’s content
• event_hash is the hash of the current event’s content

This creates a hash-chain where any modification to a prior event breaks the chain continuity for all subsequent events.

Important limits of tamper evidence:

• it provides evidence that records were not modified after writing
• it does not prevent a sufficiently privileged actor on the same host from rewriting the audit file and regenerating a valid chain
• it does not replace an external immutable audit store

For compliance paths requiring stronger guarantees, export audit records to an external immutable store.

Audit retention and retention gaps

Audit retention is controlled by audit.retention_days. When retention is enforced:

• events older than the configured window are removed from the JSONL file
• when tamper evidence is enabled, the chain state resets to the retained dataset
• gaps created by retention are expected and do not indicate tampering

Plan retention policy carefully. Retaining too little creates investigation blind spots. Retaining too much creates large audit files that require the sidecar index for efficient queries.

Audit records and security review

For security review use cases, audit records can answer:

• which files were processed and when
• whether processing was successful or failed
• which partner endpoints were involved
• whether idempotency prevented duplicate processing
• what error occurred when processing failed

Audit records are not a substitute for:

• access control on the Xferity host and storage paths
• external evidence retention for compliance
• SIEM integration for security monitoring

Exporting audit records

Audit JSONL files can be shipped to external systems using standard log forwarding tools (Filebeat, Fluentd, Vector, etc.).

For compliance-sensitive flows, export should be:

• continuous or near-real-time to avoid evidence gaps
• to a system with stronger access controls than the Xferity host itself
• retained for the required duration per the compliance requirement

Related pages

• Audit Logging
• Security: Audit and Non-Repudiation
• Investigating Failures
• Operating Xferity
• Observability