WHERE THE NUMBERS COME FROM
Every number on GitFind comes from public data. Here is each source, what we take from it, what we do with it, and the credit it is owed. If anything here is about you and you want it changed or removed, see data requests →
§ 1 — sources in use
8 sources
GitHub API ↗
- what we take
- Public repository facts: stars, forks, open issues, last push, archived flag, latest release, commit and contributor counts. For the top 1,000 projects, the usernames and contribution counts of each project’s top 25 contributors.
- what for
- Daily and weekly measurements, the Early Signal Score, the AI Code Index (commit search counts). Contributor usernames are kept internally to measure bus factor and maintainer movement; they are never published in lists or sold.
- terms & credit
- Read through GitHub’s official API with one token, within its rate limits, under GitHub’s Terms of Service.
GH Archive ↗
- what we take
- The public timeline of GitHub events.
- what for
- Discovering newly rising projects. Only counts are kept.
- terms & credit
- Website content CC BY 4.0; the event data itself carries GitHub’s terms.
Hacker News (via the Algolia HN Search API) ↗
- what we take
- How many stories and comments mention a project.
- what for
- The “community buzz” signal and project discovery. We keep counts and links, never Hacker News text.
- terms & credit
- Hacker News content belongs to its authors and Y Combinator; we do not republish it.
npm ↗
- what we take
- Download counts, and whether a package’s maintainer has marked it deprecated.
- what for
- Usage trends and confirmed end-of-life events.
- terms & credit
- npm Open Source Terms; well within the public registry’s usage limits.
PyPI (download counts via pypistats.org) ↗
- what we take
- Download counts, and whether a project is marked archived, deprecated or inactive.
- what for
- Usage trends and confirmed end-of-life events.
- terms & credit
- Download counts derive from PyPI’s public BigQuery dataset, licensed CC BY 4.0. PyPI’s terms forbid selling its users’ personal information; we do not collect any.
crates.io ↗
- what we take
- Download counts, and whether every version of a crate has been yanked.
- what for
- Usage trends and confirmed end-of-life events.
- terms & credit
- crates.io Data Access Policy: at most one request per second, with an identifying user agent.
Vendor pricing pages, and the Internet Archive’s Wayback Machine ↗
- what we take
- Published prices of AI coding tools and model APIs.
- what for
- A dated record of price changes. When a vendor’s page cannot be read directly, an archived copy is used and labelled as such. We keep the price and date, not the page’s text.
- terms & credit
- Page text belongs to each vendor; archived copies to their owners.
News search results ↗
- what we take
- Headlines about projects being funded, acquired, shut down or changing hands.
- what for
- Internal only: a headline nominates a possible event, which is verified before it enters our records. Headlines and snippets are never republished or sold.
- terms & credit
- Used for internal research; not redistributed.
§ 2 — what we never do
- ×we never read private repositories.
- ×we never sell, share or publish lists of people, their usernames, emails or profiles.
- ×we never store commit authors’ email addresses (they are used in memory to count commits per person, then discarded).
- ×we never republish text from Hacker News, news sites or vendor pages.
- ×we never sell access to our rankings or methodology — every project is measured the same way.
§ 3 — the daily record
Once a day we freeze a copy of everything we measure, with a checksum, so that what we said on a given date can be checked later. Numbers that could not be measured are recorded as unknown, never as zero.
How the AI Code Index counts commits, tool by tool: how we count → How the Early Signal Score works: methodology →