JavaScript · Apache-2.0
brendonboshell/supercrawler
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
★ 382
Stars
62 forks · 23 open issues
flat
Growth · measured
Stars counted between two of our own snapshots roughly a week apart. The repository page gives the exact span.
7.5
Momentum / 100
see the breakdown below
10y ago
Created
last push 3y ago
Star history
How this score was built
| Component | Weight | Earned | Why |
|---|---|---|---|
| Velocity | 55 | 0.0 | flat, log-scaled against a 750/day ceiling |
| Acceleration | 15 | 0.0 | needs 7-day and 30-day rates side by side |
| Freshness | 15 | 0.0 | last push 3y ago; decays to zero at 180 days |
| Scale | 15 | 7.5 | 382 stars, log-scaled against 150k — stops a 3-star repo topping the board |
| Momentum | 100 | 7.5 | sum of the above |
Detail
| Topics | crawlerdistributed-crawlerrobotsitemapweb-crawler |
| Matched on | Robotics (robot) |
| Short window | +0 stars over 7 days = 0.0/day |
| Long window | +0 stars over 26 days = 0.0/day |
| Lifetime average | 0.10 stars/day over 3,763 days |
| First seen here | 2026-10-11 |
| Last read | 2d ago |