What this dataset is, where it comes from, and how to trust it.
trendscope is an open dataset of developer attention: every day it snapshots what developers are reading (Hacker News, Lobste.rs, Dev.to), building (GitHub trending), and downloading (npm, PyPI, crates.io), and publishes the result as JSON/CSV under CC0. The dashboard is a shop window; the dataset — versioned, timestamped, cumulative — is the product.
| Source | Endpoint | What the score means |
|---|---|---|
| hackernews | HN Algolia API (front page + search) | points |
| show_hn | HN Algolia API (tag: show_hn) | points |
| ask_hn | HN Algolia API (tag: ask_hn) | points |
| lobsters | lobste.rs/hottest.json | score |
| devto | dev.to API (top, 1-day window) | reactions |
| producthunt | producthunt.com/feed (Atom, daily launches) | feed rank |
| techmeme | techmeme.com/feed.xml (RSS, editorial news) | feed rank |
| slashdot | rss.slashdot.org/Slashdot/slashdotMain (RSS 1.0) | feed rank |
| arstechnica | feeds.arstechnica.com/arstechnica/index (RSS) | feed rank |
| theverge | theverge.com/rss/index.xml (Atom) | feed rank |
| arxiv | export.arxiv.org/api/query (cs.AI/CL/LG/SE) | arxiv recent |
| github_trending | github.com/trending (HTML) | stars gained today |
| npm | api.npmjs.org/downloads | weekly downloads |
| pypi | pypistats.org API | monthly downloads |
| crates | crates.io API (recent-downloads) | recent downloads |
| lemmy | programming.dev/api/v3 (Lemmy hot) | score |
| krebs | krebsonsecurity.com/feed (RSS, security journalism) | feed rank |
| changelog | changelog.com/news/feed (RSS, dev news) | feed rank |
| freecodecamp | freecodecamp.org/news/rss/ (RSS, tutorials) | feed rank |
| echojs | echojs.com/rss (RSS, JavaScript community) | feed rank |
| infoq | feed.infoq.com (RSS, enterprise dev news) | feed rank |
| theregister | theregister.com/software/headlines.atom (RSS, IT news) | feed rank |
| lwn | lwn.net/headlines/rss (Linux/kernel news) | feed rank |
| hackaday | hackaday.com/feed/ (hardware/maker) | feed rank |
| thenewstack | thenewstack.io/feed/ (cloud-native/devops) | feed rank |
| opensourcecom | opensource.com/feed (Red Hat OSS community) | feed rank |
| soblog | stackoverflow.blog/feed/ (dev culture) | feed rank |
| codinghorror | blog.codinghorror.com/rss/ (classic dev blog) | feed rank |
| martinfowler | martinfowler.com/feed.atom (SWE essays) | feed rank |
| pragmatic | blog.pragmaticengineer.com/rss/ (SWE newsletter) | feed rank |
| bendersky | eli.thegreenplace.net/feeds/all.atom.xml (deep-dive blog) | feed rank |
| cheney | dave.cheney.net/feed (Go/engineering blog) | feed rank |
| lucumr | lucumr.pocoo.org/feed.atom (Armin Ronacher) | feed rank |
| hynek | hynek.me/articles/index.xml (Python ecosystem) | feed rank |
| susam | susam.net/feed.xml (long-form hacker blog) | feed rank |
Loading directory…
| Key | Name | Lane | Metric |
|---|
All endpoints are public and login-free. Scores are source-native: compare within a source, never across them (a point on HN and a download on npm are different units). The cross-source "All" board normalizes by each source's top score — heat, not raw numbers.
scripts/collect.py daily (~06:05 UTC) and commitsdata/daily/YYYY-MM-DD.json/.csv, latest.json/.csv, and appendshistory.csv.url + captured_at — provenance is part of the schema.data/index.json lists all snapshots; the day picker and topic sparklines read it."backfilled": true, "partial": true in those daily files.Items are classified by a keyword map (TOPIC_RULES in collect.py) over title + tags + description — deliberately transparent and deterministic, no ML black box. An item can belong to several topics or none.
Compilation released under CC0; upstream titles/links remain with their owners. Cite as:trendscope — daily open dataset of developer attention, zalize-com, {date}.
Designed, coded and maintained autonomously by two collaborating Devinsessions — one building, one researching and reviewing — as a 7-day experiment in agent-built products. Daily collection, this page, and the weekly digest all run unattended.
docs/methodology.mddescribes how the two sessions split, review and ship — written by them.