utSources and coverage
Scout uses the documented Hugging Face Hub API to retrieve limited public metadata from selected organizations. The current import requests up to 200 recent repositories per tracked organization. Reaching this limit can leave older models out.
Download formats with recognized suffixes may be grouped with their exact base model; different parameter sizes and fine-tunes stay separate. Cloud API entries are imported separately from the MIT-licensed Models.dev database for selected direct providers. Its metadata and listed USD-per-million-text-token rates are not independent benchmarks. Cloud entries do not imply that downloadable weights are unavailable; local records remain separate.
How we use source data
Cloud specifications and pricing: Models.dev, copyright (c) 2025 models.dev, used under the MIT license. We retain selected structured facts and write our own summaries. We exclude deprecated and future-dated cloud entries; a failed cloud refresh retains prior data with a stale-source notice. Successful updates remove listings no longer present in a provider’s catalog. Model availability and prices should be confirmed with the provider.
We retain model identifiers, maker names, capability and license labels, reported parameter and context sizes, source timestamps, and links. Scout writes its own short summaries. We do not copy full model cards, download model weights, bypass access gates, or copy unrelated benchmark datasets.
Ollama is not connected. OpenRouter imports are paused, and its previously imported records and fields have been removed pending clarification of reuse permissions. A public API or model license alone does not establish permission to reuse every associated piece of content.
Source requests are cached and rate limited. Requests stop on rate-limit responses and resume after the retry window. Hub API documentation ↗ · Rate limits ↗ · Hugging Face terms ↗
Open source radar
The radar discovers AI-related GitHub projects and Hugging Face models and demos using public API metadata. It is separate from model benchmarks and verified release briefs. Adoption is a selection signal, not a review, rating, or guarantee of quality.
GitHub searches now focus on smaller skills, MCP tools, AI agents, local inference, retrieval, and creative utilities. Most searches require 25–5,000 stars and creation within 120 days; one finds smaller established MCP tools. Repositories need a meaningful AI-related description and code pushed within 180 days. New projects with 25 stars qualify; older projects need 100. Forked, archived, disabled and private repositories are excluded.
Hugging Face discovery uses trending and popular model and Space lists. A model needs a recognized task and at least 25 likes or 1,000 reported downloads. Apps need 25 likes and purpose metadata. Fresh finds, the default view, shows recent apps with 25–1,000 likes and GitHub repos with a reported license and 25–5,000 stars; standalone model listings and basic demo wrappers are available through All selected. Unknown-task test uploads stay out. Quantizations, adapters, guides and link directories can be included through More options.
Fresh finds uses repository creation dates within 120 days; an edit to an older project does not make it new. Discovery mix shows two GitHub projects and one Hugging Face app, newest first within each source. Optional adoption sorting uses GitHub stars and Hub likes separately. None is a quality, security, correctness, or model-performance score, and counts can be manipulated.
Checks run about every six hours on our server, with at most nine GitHub searches and four Hub listings per run. Requests have size and time limits, respect rate-limit backoff and access only public endpoints. Source failures preserve earlier selected findings; successful feeds replace their prior entries. The website observes the saved file automatically. Page views and searches do not trigger upstream API calls or disclose visitors’ queries to the sources.
We retain limited listing metadata and original source links, not README text, model cards, code, weights, publisher logos or personal contact details. A public repository does not automatically have an open-source license. License labels are reported when available; check original terms before installing or using a project.
GitHub search API · GitHub API terms · GitHub rate limits · Hugging Face public listing API
Release briefs and automatic discovery
Our release desk checks the approved local model catalog, official OpenAI and Google DeepMind news feeds, and MIT-licensed Models.dev metadata for selected model makers and coding plans. It stores announcement titles, dates, original links, and selected capability signals, not full articles. Shortlisted public model repositories receive additional checks against the official Hugging Face model API. Initial backlog entries are labeled separately from newly detected candidates. Routine repository modification timestamps do not create release posts. Cloud discovery recognizes a new dated event even when its API identifier stays the same; ordinary catalog edit dates do not create posts.
Detection starts a private evidence check. Qualifying briefs publish automatically after specific facts match across two checks at least ten minutes apart. The scanner runs about every fifteen minutes and respects source backoff. The cloud release catalog is checked every six hours, with an earlier confirmation check for newly passing facts. Publication is capped at four automatic briefs per UTC day, with no minimum. Templates report source facts; no paid language model writes these posts. Adapters, repackaged quantizations, unrelated company news, and unsupported announcements are skipped or held. Missing or conflicting evidence prevents publication. Automatic briefs are labeled and have not received an individual editorial review. “Listed” identifies a repository creation or a catalog-reported release date, as labeled beside the date. Neither is an independently verified launch date. “Announced” records the maker’s announcement, without verifying live API access. An owner can pause automation, correct a brief, or remove it. Coverage is selective: some makers have no supported source, and new versions can be missed if their provider never records the change. Catalog-listed capabilities are distinguished from original maker announcements. Pricing, open weights and independent benchmark performance are not inferred from a cloud release listing. The cloud catalog updates separately from release briefs; a cloud announcement does not automatically establish pricing or availability.
OpenAI news feed · Google DeepMind feed · Models.dev metadata · MIT reuse license
Kimi K2.8 Preview and SWE-2 have original, individually checked release notices. Their publisher pages are linked for readers; they are not scraped on a schedule. Cognition’s article text and benchmark tables are excluded from automatic imports. A repository’s software license does not automatically license a separate website.
Facts, estimates, and unknowns
Capabilities and plain-language summaries are derived from source metadata, not an independent performance evaluation. Vision means visual input; image generation means image output. 3D generation alone does not establish CAD capabilities.
Parameter counts use reported metadata when available, otherwise a recognizable nominal size in the model’s name. A 4-bit memory baseline uses total parameters × 4 ÷ 8, in decimal GB. Extra runtime and cache memory are not included, and compatible quantization is not guaranteed.
Hugging Face’s memory estimation reference ↗
“Downloadable” does not mean unrestricted open source. The permissive-license filter recognizes MIT, Apache, BSD, and ISC labels. Read the exact model license before use.
Dates, freshness, and availability
Listing dates are repository creation or catalog addition dates, not confirmed launch dates. Last checked records when Scout retrieved that information. Removed listings remain in the archive; verify availability at the source.
The explorer checks visible-page updates every five minutes, with longer pauses when the source requests backoff. The configured Lenovo service also checks sources in the background, saves updates on disk, and keeps seven daily local backups. A local preview without persistent storage uses its bundled snapshot after a restart.
Compare chart: ratings and speed
Ratings: LMArena’s text leaderboard with style control, from its public leaderboard dataset, used under the CC BY 4.0 license and refreshed daily. Intelligence is LMArena’s overall rating; Coding and Math are its categories. Ratings come from people comparing two anonymous answers and choosing the better one, so they measure preference on real prompts, not scores on a fixed test. Each rating has a margin of error; small gaps are effectively ties. Scout shows the ratings as published and does not adjust or combine them.
Speed is Scout’s own measurement. RTX 3060 speed is output tokens per second from Scout’s model lab, which times Ollama downloads on one NVIDIA RTX 3060 (12 GB); it appears only for downloads reviewed as the exact model. Cloud speed is the median of daily runs that send one short fixed prompt to each top-rated model through OpenRouter and time the answer; results depend on which provider OpenRouter routes to. Prices are listed API rates from Models.dev. Open-weights status comes from the leaderboard’s license field or Scout’s downloadable listing; size comes from Scout’s listing or the model name.
LiveBench task results
Source: LiveBench, by the LiveBench authors. Its data sheet states Apache 2.0 distribution, no copyrights on the data, and no fees or restrictions. We retain the license notice and link each result to its score file and exact configuration. Scout calculates category averages from published task scores; we have not rerun these tests.
The connected editions are November 25, 2025, January 8, 2026, and June 25, 2026. Their mapped scores refresh daily on our server. New benchmark editions and unrecognized model configurations need validation before they are added. A source failure retains the previous results with a visible stale notice. This is selective benchmark coverage, not a score for every model in the catalog.
We never transfer a base-model score to a LoRA, fine-tune, or quantized variant. A local listing can show a result for its exact named model, but the published evaluation may use a hosted provider; local hardware and precision can change results. Image, video, audio, 3D, and embedding tests are not connected yet. Missing evidence stays unscored.
Additional benchmark sources and reuse
We publish selected numerical results and source metadata, not benchmark questions, answers, model responses, or copied articles. Each connected test is listed below with its publisher, reuse license, and permission reference. A model being open source does not give us permission to copy a leaderboard.
EvalPlus HumanEval+ and MBPP+ are legacy published snapshots under Apache 2.0. Run dates are not supplied. They measure Python function writing and remain separate from newer project coding tests.
Epoch AI, “Capabilities & Benchmarking,” epoch.ai/benchmarks/use-this-data, accessed September 12, 2026. Epoch AI’s own evaluation results are provided under CC BY 4.0. Scout selects exact matched models, converts fractional accuracy and standard errors to percentage points, and reformats the results. We retain the model setting label, evaluation date when available, and source record ID. We use mean_score, not the alternative “best across scorers” value.
The connected Epoch evaluations are GPQA Diamond, OTIS Mock AIME 2024–2025, MATH Level 5, SimpleQA Verified, and SWE-bench Verified. Each test stays separate; they are not averaged into a Scout intelligence score. “Epoch evaluation series” refers to the publisher’s collection for that named test, not an identical model budget or pinned harness revision across every run. Settings can vary. Source standard errors are shown when published; one standard error is not a 95% confidence interval.
Aider Polyglot scores come from Aider’s publisher-listed results via Epoch AI’s export. The original Aider data file is in its Apache 2.0 licensed repository; Epoch’s formatting is attributed under CC BY 4.0. We retain the license notice, identify our selection and reformatting, and show available edit-format notes. These results can include different contributors and are labeled publisher-listed, rather than independently verified by Scout.
Epoch’s license for its own work does not cover every third-party leaderboard it hosts. All other external exports stay excluded. Maker-reported results are not imported in this release; they cannot enter the independent or publisher-listed rankings. New sources require a documented reuse permission and identity/methodology review before activation.
The existing server updater checks the approved exports daily. Each feed has its own saved last-check date and retry deadline. A failed refresh preserves that feed’s earlier results and is labeled; other feeds can still update. New, ambiguous, or changed model identities are left unmatched until validated. No paid benchmark API, inference job, or visitor-triggered upstream download is required.
Performance evidence
The Performance page compares published configurations on three separate tests. Code Arena measures human preference for built apps and websites. Agent Arena estimates a model’s contribution to real agent sessions relative to its baseline. Terminal-Bench 4.0 measures successful terminal-task trials with a named coding agent. Ratings, signed indices and completion percentages are never averaged or treated as interchangeable.
Arena’s aggregate leaderboard dataset is published under CC BY 4.0. We select the overall category, preserve original scores and uncertainty intervals, and link each result to the exact source revision. Terminal-Bench uses licensed original repository submissions under Apache 2.0. We keep the result, trial count and tested setup together. No endorsement by either publisher is implied.
Each configuration remains separate, including reasoning effort and agent version where reported. Conflicting configurations are held back until the original records agree. An incomplete or stale comparison cannot display a leading-model card; overlapping uncertainty ranges do not establish a clear winner. Named gaps and source links explain what is missing. Publicly viewable records without clear reuse permission are not copied into our results.
Verified local access requires weight files in the exact model’s official repository. An open-weight model can also be available through a cloud API. Unknown mappings stay unverified. Size follows the publisher’s model parameter count, including inactive experts; individual notes identify extra vision or draft modules. It is not a VRAM estimate. Date filters use reported releases or repository listings and exclude unknown dates.
Our server checks the approved sources every six hours. Results, licenses and source dates are checked before updates appear. Each feed retains its last valid results during a failure and retries after its saved deadline. Visitor requests use our shared cache; they do not initiate source downloads.
How comparisons work
There is no overall intelligence score, paid ordering, or automatic winner. The default ordering is recently listed first; other explicit sorts use names or reported context sizes. Missing measurements do not become zero.
Comparison graphs lead with task performance. The API cost chart uses listed text rates for 1M input and 1M output tokens. Memory and context are supporting specifications; neither establishes better answers.
We import approved results and match exact model identifiers through reviewed mappings. LiveBench mappings use published run configurations; Epoch mappings retain its exact model-version and organization labels. We do not transfer scores from a tested cloud offering to a separate local listing. Users choose a tested configuration when several exist; we do not automatically select the highest score. All bars in a performance chart use the same test edition and a fixed 0–100 axis. For LiveBench, category scores average the tasks in that category. Its General use score averages all complete category scores equally, including agentic coding when that edition includes it. A score is not a percentage of all real-world tasks a model can solve. Missing task results are not treated as zero or averaged away. Small gaps may reflect test variation; we do not claim statistical significance. Only models with known results participate in the named test leader. No shared performance score is assigned across model categories.
Corrections and limitations
A source-backed catalog still reflects which sources we track and the limits of their metadata. We cannot promise complete coverage or zero bias. Missing values remain visible, and previous data may be stale.
To challenge a detail, send a correction with the original source. A public correction history is not yet available.