deniz.in

Markets

Weather

Loading weather

· via GitHub Blog

GitHub's new star history API cuts 2,511 requests to 24 and drops stargazer identities

GitHub replaced paginated stargazer listings with a star history endpoint returning up to 30 weeks of daily counts per request, cutting charting work from thousands of calls to about 24 while removing user identities.

GitHub's new star history API cuts 2,511 requests to 24 and drops stargazer identities

What happened

GitHub has replaced the way tools chart a repository's star history, cutting the number of API calls needed for a popular project from thousands to roughly two dozen while removing user identities from the response entirely. According to the GitHub Blog, the new star history endpoint returns up to 30 weeks of daily star counts in a single request, and building the complete curve for react/react — a repository with about 251,000 stars — now takes about 24 requests instead of the 2,511 the old flow required.

Why stargazer lists became a liability

Maintainers use star history to judge whether a launch, release, or announcement shifted a project's trajectory. To reconstruct that curve through the old API, a tool had to page through the list of everyone who currently stars a repository, keep the timestamps, and throw away the usernames. The old GET /repos/{owner}/{repo}/stargazers endpoint returned at most 100 people per page, with a starred-at timestamp added when a custom media type was requested, so the work scaled with a repository's popularity.

That design caused trouble on two fronts, the GitHub Blog explains. Publicly available identities and timestamps made the lists a target for scrapers running spam and phishing campaigns. Meanwhile, a bug in the pagination implementation meant every request counted all of a repository's stars just to compute how many pages existed, even though the caller received only one page. In the week before GitHub tightened access, nearly 37,000 of these count queries were slow enough to be logged, averaging almost three seconds each, and database safeguards killed more than 1,000 of them, contributing to request timeouts.

There was also a data-model mismatch: star records are deleted when someone removes their star, so the endpoint reflected current state rather than an event log, even though tools were treating it as a time series.

The access change that broke the tools

GitHub's first fix was to restrict stargazer and watcher lists to repository administrators and collaborators; callers without permission now receive an empty response or a 403 Forbidden. That closed the harvesting path and reduced load — stargazer requests dropped about 60 percent and total time spent on the count query fell about 80 percent — but it also cut off tools such as Star History, which had used the old response to compute trends without ever displaying usernames. GitHub says conversations with maintainers and tool builders showed what they actually needed was the curve itself: counts and dates at a useful resolution, not identities.

Why returning fewer fields was not enough

Dropping identities solved privacy but not cost, and the GitHub Blog walks through the rejected alternatives. Keeping one record per star and returning only the timestamp would have let existing tools keep working, but clients would still paginate thousands of records to build an aggregate. Aggregating into daily or weekly buckets before paginating looked better, yet in a straightforward SQL query the LIMIT and OFFSET apply only after the database has found, grouped, and sorted every matching star — so an anonymous request could still scan a huge index to return a handful of buckets. For react/react, that approach would have examined roughly 6 million rows, because all 24 requests would have reprocessed the entire star set.

The design that shipped

The endpoint that launched paginates by calendar week. Each request translates its page and per_page parameters into an indexed created_at range, filters stars down to those weeks, and only then groups the matching rows into daily counts. Because the date filter runs before aggregation, a request processes just the weeks it returns rather than every current star in the repository. Higher page numbers walk further back through a repository's history.

A collaborative fix

According to Tianzhou Chen, who maintains star-history.com and co-founded Bytebase, the disruption to the tool prompted its community to rally around the project. GitHub's community team connected the maintainers with product engineering, and the sides designed the purpose-built endpoint together, turning a breaking change into a joint redesign. His account appears in the GitHub Blog post.

Why it matters

The episode is a tidy case study in fixing privacy and performance at once: the privacy problem was solved not by redacting fields from an existing payload but by redesigning the API contract around what consumers actually needed. It also illustrates a database lesson — shrinking a response does not shrink the work behind it, and filtering has to happen before aggregation or serving cost still scales with repository popularity. Finally, it shows one path for handling the collateral damage of a security-motivated API change: GitHub broke real tools, then worked with their maintainers on a replacement instead of leaving them with a 403.

  • #github
  • #api
  • #privacy
  • #web-api
  • #developer-tools

Related posts