AI Systems Observatory · In development

Compare models, agents and protocols—with evidence.

We are preparing versioned catalogs and reproducible comparisons. No benchmark rankings have been released here yet.

Benchmark methodology

Rankings will only compare matching evaluation tasks and conditions. Each result must disclose its evaluator, test version, sample size, model snapshot, runtime, success criteria, latency, cost, provenance and limitations. Missing measurements mean “not evaluated,” not zero. Vendor-reported and independently reproduced results will be visibly distinguished.

Capability, security, interoperability and governance evidence are separate dimensions—not one universal score.

Source and freshness status

GC's existing Agent Roster is a curated, versioned source with 32 paid/free model selections across four software-development roles. Its source declares a last manual verification date of August 7, 2026. The monthly scheduled workflow opens a review issue; it does not automatically update rankings or execute performance tests. The Agent Matrix is a separate governance specification, not a benchmark leaderboard.

Public ranking status: No roster recommendations are republished here until the underlying model IDs, provider claims, evaluation evidence, publication rights and current availability have been independently reviewed.

Inspect the public GC registry →