Official conformance bridge

mcptest can run the upstream @modelcontextprotocol/conformance suite as a subprocess and fold its results into the same report a local rubric run produces. The bridge is intentionally optional: when the upstream tool is missing we degrade to local-only checks instead of failing the run.

Run this example. examples/official-conformance.yml runs the local compliance checks the bridge maps onto, the same set it falls back to when the upstream tool is not installed.

mcptest run --config examples/official-conformance.yml

Install

The upstream suite ships through npm. Either install once:

npm install -g @modelcontextprotocol/conformance

or rely on the default npx --yes @modelcontextprotocol/conformance@latest invocation, which fetches the latest published version on demand.

How it works

OfficialBridge in mcptest-core::compliance::official spawns the configured argv, passes --reporter=json --server=<url>, captures stdout, and parses it into OfficialReport. Call to_check_results to translate scenarios into mcptest's uniform CheckResult shape, so reporters and rubric scoring treat upstream and local results the same way.

Rule mapping

The bridge maps upstream scenario_id values onto our rule registry. Unmapped scenarios get a synthesized OFFICIAL-<SANITIZED> id so nothing silently disappears.

Offline fallback

OfficialBridge::check_available probes the binary with --version. If the probe fails (or the call to run returns NotInstalled), the caller is expected to skip the official pass and proceed with local matchers. The CLI wiring for that flag lives in a follow-up ticket.