# Owned MCP policy/recovery fixture corpus

This is an **owned, in-process policy/recovery test**, not a production security benchmark, external-server assessment, or full MCP conformance suite. Version 1 runs entirely in memory. It needs Node.js with native ES modules and no packages, accounts, credentials, or network access. The published sample ran with Node v22.17.1.

## Get and run

Save `runner.mjs` and `fixtures.json` in the same directory. Save `results.json` to compare the published run. These files are available at:

- https://inspiredbyfrustration.com/downloads/mcp-security-corpus/runner.mjs
- https://inspiredbyfrustration.com/downloads/mcp-security-corpus/fixtures.json
- https://inspiredbyfrustration.com/downloads/mcp-security-corpus/results.json
- https://inspiredbyfrustration.com/downloads/mcp-security-corpus/README.md

Run from that directory:

```sh
node runner.mjs > local-results.json
```

Exit status is zero only when every expected outcome matches. Invalid fixture packs and execution errors produce a nonzero exit and a JSON error on stderr. Mismatched outcomes produce a report with `failed` above zero and step-level `errors`; they are not silently discarded. A malformed pack fails before any cases execute. `executedAt` is the actual UTC execution timestamp. Repeated runs have a new timestamp; case results are deterministic. No latency, security rating, cost, or real-server availability claim is made.

In the repository, record a new sample with `node scripts/benchmarks/mcp-security-corpus.mjs`; review the changed results before publishing. Verify with `node --test scripts/benchmarks/mcp-security-corpus.test.mjs`. Tests are discovered recursively by the required policy guard runner.

## What executes

The runner serializes each supplied request to JSON, dispatches it through an owned handler, and serializes its reply back to JSON. It models a subset of `tools/list` and `tools/call` envelopes from the [MCP tools specification dated 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/server/tools). That version is an explicit fixture reference, not a statement about the latest protocol version.

The service owns one virtual document, `/workspace/note`. Each case gets fresh state with text `original`, zero writes, an empty deduplication map and its specified injected fault count. The caller identity is supplied separately from tool arguments: `reader`, `writer`, `unknown`, or missing (`null`). These are synthetic labels, not anonymized customer records. Only `writer` may change the document. Only the exact virtual path is allowed; this tests an application allowlist, not an operating-system filesystem sandbox.

`read_note` accepts only `path`; `write_note` accepts only `path`, `text` and an explicit deduplication `key`. Input validation precedes the side effect. Unknown fields, wrong types, missing fields and non-allowlisted paths fail. Duplicate successful writes with the same key and text leave the write counter unchanged; reusing that key with different text fails with `KEY_CONFLICT`. The counter is the observed number of in-memory write operations, not a database audit.

Fault injection returns `UNAVAILABLE` before a side effect for a specified number of eligible calls. The client retries only `read_note`, only on `UNAVAILABLE`, and only up to the case's attempt budget (maximum three). It never automatically retries writes or permission failures. Retry attempts are immediate and deterministic: there is no timing/backoff, outage, network, cancellation or clock measurement. Recovery cases execute subsequent requests against the same case state; other cases cannot share state.

Protocol validation failures use JSON-RPC error envelopes. Permission, transient and key-conflict outcomes use tool results with `isError: true`; successful reads/writes use `isError: false`. This error distinction follows the referenced tools interface; the permission and retry policies are ours.

## Published sample and denominators

Recorded November 4, 2026, at 5:47 p.m. Eastern (see the exact UTC timestamp in `results.json`). The sample contains 17 distinct cases, 24 scripted steps and 27 request attempts, including discovery and rejected malformed requests. Each case executes once; retry attempts are not independent cases. The denominator is this deliberately selected fixture set, not sampled production traffic.

| Category | Passed / cases | What the cases exercise |
| --- | --- | --- |
| Permission boundaries | 4 / 4 | Discovery/read allowed; unknown/missing identity denied |
| Malformed input | 5 / 5 | Invalid JSON/version/arguments/types; unknown tool |
| Denied writes | 3 / 3 | Reader denied; traversal path and identity in arguments rejected |
| Retries | 3 / 3 | Read retry succeeds or exhausts budget; write never auto-retries |
| Recovery | 2 / 2 | Read after exhaustion; explicit write recovery, duplicate and conflict |

All 17 expected case outcomes matched; zero cases failed. Expected denials and injected transient errors count as passes only when the observed response, attempt count, side-effect count and optional returned value match the declared expectation. A 17/17 result therefore says our owned implementation meets these specific policies. It does not mean “zero errors” or prove a target system safe. The JSON retains every raw request/reply, expected result, actual code, attempt count, effects and mismatch list.

The report's `sourceSha256` hashes the executable runner bytes. `fixturesSha256` hashes the parsed fixture object serialized with JavaScript `JSON.stringify`; it is independent of file indentation but not object-key order. To compare, rerun the same runner and fixtures and inspect both hash fields and case outcomes. Hashes identify the sample inputs; they are not a signed provenance or security certification.

## Gaps and adaptation

No real server, vendor, client session, secret, production write, operating-system file, HTTP/SSE/stdio transport, initialization handshake, OAuth flow, token audience, tenant isolation, symlink escape, prompt injection, concurrency, rate limiting or load is tested. There is no externally targetable URL option. Changing the owned implementation and rerunning it does not test your server; adapting to another server requires a separate authorized adapter and a new methodology/results record. Keep those results separate from this controlled sample.

A useful next step is to add your own policy cases, check that a deliberately broken expected outcome exits nonzero, and then design an authorized integration test for the real boundary you operate. The [existing benchmark article](https://inspiredbyfrustration.com/blog/mcp-servers-in-production-benchmark-2026) separates this sample from its historical production account. The downloadable files contain no customer data; no model or human review is represented by the pass count.
