Ask for a month in one request and some of the days come back a fraction full

The symptom: days that look dead but were not

You pull Search Console query data into your own database, a month per request, because that is the shape that keeps the request count sane. Later you chart daily impressions (the number of times one of your pages appeared in a result list someone saw) for a single keyword, and the line falls off a cliff mid-month and comes back.

Check the same range in the Search Console interface and the cliff is not there. Check the total for the month and it is roughly right. Only the day-level detail is wrong.

What you are looking at is not missing data in the sense a monitoring system would catch. Each row you asked for is one query on one day, with its clicks, its impressions and its average position, meaning where your page sat in the result list on average across those appearances, with 1 being the top. On the affected days rows exist; there are just far fewer of them than that day really had. There is no null, no gap, no error code, and no field in the response that says the answer was cut short. An archive built this way looks complete from the inside, which is why this survives code review and shows up months later as a chart nobody can explain.

The mechanism: one ordering across the whole range

A Search Analytics request takes a start date, an end date, a list of dimensions, and a rowLimit. The largest rowLimit the API accepts is 25,000 rows per request, and you page past it with startRow (checked September 2026).

When you ask for the date and query dimensions over a month, the result set Google is working from is every (day, query) pair in that range. It sorts that whole set by clicks, descending, and hands you the top slice. Paging with startRow walks further down the same single ordering; it does not restart per day.

So the truncation is global, not per day. The sort key is clicks, and it ignores which day a row belongs to. Almost every row in a query pull has no clicks at all, so on a range large enough to be cut, the cut lands deep inside an undifferentiated mass of zero-click rows, and there a busy Tuesday's tail and a quiet Sunday's tail are indistinguishable to the sort. The days whose queries carry the most clicks are served deepest, and the rest of the range divides what is left; on a site big enough that clicked rows alone fill the page, even those are cut.

Which days end up starved is not something you can predict from outside the request. It depends on how the clicks are distributed across the whole window you asked for, and it is not simply that the quietest days lose: when we measured a month-chunked pull against a day-by-day pull of the same range in August 2026, the starved days were the most recent ones.

Two consequences worth stating plainly:

• The same day returns a different number of rows depending on what else was in the request. Re-pull last month with the window shifted by a week and the counts change.
• This applies to any multi-day request carrying a date dimension, not just [date, query]. If the response can be truncated, the ordering decides who gets truncated.

Below the row limit, none of this matters. A range whose total rows fit inside one page cannot be cut, so the ordering is irrelevant. The trap only exists at or near the ceiling, which is exactly where a real site with a month of query data sits.

Why this is worse than a small sample

If Search Console handed back a random 25,000 rows, that would be a sample, and a sample is workable. You would know your counts were low, you would know they were low by roughly the same factor everywhere, and shapes over time would still be readable.

This is not that. Comparing the two methods over the same sixteen months in August 2026, the month-chunked pull held under half the rows and about a fifth of the impressions the day-by-day pull found. That shortfall is not spread evenly across the range. It is concentrated in whichever days lost the ordering, and it runs the wrong way:

• The loss is correlated with the measurement.

  A day does not lose a scattering of rows; it loses its tail wholesale, while a neighbouring day keeps its own. Two days in the same archive are not measured to the same depth.

• Drops look steeper than they were.

  A day that genuinely lost clicks has fewer rows above the cut, so it is served less deeply, so the dip in your archive is deeper than the dip in reality. The error amplifies the signal instead of blurring it.

• Long-tail counts become meaningless.

  "How many distinct queries did this site rank for on this day" stops describing the site and starts describing that day's standing inside whatever window you happened to request.

• Nothing marks it.

  A sample you know about is a caveat. A biased cut with no marker is a wrong number presented as a right one, and the reader has no second source to check it against.

The practical version: you set out to build an archive so you could investigate bad days, and the archive is thinnest in places it cannot tell you about.

What it does to your own archive: scripts, connectors and the BigQuery export

1. Hand-rolled scripts.

   Month-chunking is the natural design: fewer requests, easier to resume. It is safe only if you page every chunk to exhaustion with startRow, so that nothing is left sitting behind the cut. Stop at the first page and the same bias reaches [date, page] as readily as [date, query]: a page with steady impressions and no clicks sits below the cut on any request covering more than its own day. We hit exactly that: under a per-month row budget, pages that Search Console showed as live read in our own coverage build as lost. If you already have an archive built this way, re-running the same code over the same range reproduces the same bias exactly; you have to re-pull day by day to fix it.

2. Dashboard connectors.

   A dashboard that reads Search Console live is making API calls of this shape underneath. If a connector asks for your whole date range in one request and stops at the first page, it inherits the bias and renders it as a clean line chart. Worth testing before you trust a daily line: pull one of those days on its own and compare the row count against what the dashboard shows for it.

3. The BigQuery bulk export.

   This one escapes the ordering trap, and it is worth being clear about why: it is a scheduled dump of the underlying data, not a ranked query with a row limit, so there is no ordering to be cut at. If you have it, its daily tables are not distorted this way. They are not complete either. Google withholds the text of rare searches, the ones that could identify a person, from both sources: the API never releases them at all, and the export marks those rows anonymized rather than naming the query (checked September 2026). No query archive from either source is the whole picture. The export's advantage is that what it does contain has not been ranked and cut.

Its limit is a different one. The export only records forward from the day you set it up. It cannot backfill, so it can never give you the sixteen months of history the API will hand over on your first call. It also runs into BigQuery's own storage and query costs, and it needs someone comfortable writing SQL against it. It is the better archive from the day you turn it on, and no archive at all for everything before that.

The fix, and what it costs

Request one day per call. With a single day in the range, the ordering happens inside that day, where the cut costs you that day's tail rather than the whole day. That is the entire fix, and it is not clever. It is just expensive.

The cost is requests. Roughly sixteen months of history is about 490 days, so a first sync goes from around seventeen requests (one per month, for a site small enough that a month fits inside one page) to about 490. Google publishes a per-user limit of 1,200 queries per minute (checked September 2026), so for a single site this is wall-clock time rather than a quota wall. The ceiling that actually binds sits above the account: the quota is shared by every user of the same API client, and we throttle ourselves well below it, at eight requests per second for the whole process, so one site's exhaustive first pull is spending a budget every other site is drawing on that night.

Three things make the trade affordable rather than obvious:

• Only the first pull is exhaustive.

  After that, a sync re-pulls the last seven days (Search Console restates recent days, and its data lags two to three days regardless), plus a full 90-day rebuild once a month and up to 25 missing days healed per run. The archive backfills itself over a few nights instead of making one night enormous. That sync runs nightly on both plans; on a paid project the archive keeps growing, while a free one is trimmed back to 14 days.

• A month may still be requested in one call when the estimate says it cannot fill a page.

  If the independently known per-day counts for that month sum under 18,000 (our margin below the 25,000 page size) and no day in it is already sitting at a ceiling, nothing can be truncated and the ordering cannot hurt you. It is an estimate, not a proof: the known counts are themselves capped measurements and may be undercounts, and the margin is what absorbs that.

• Unknown days are charged at the busiest day we do know about

  , or at the 5,000 ceiling when we know nothing about the site yet, so the estimate errs toward splitting. Guessing wrong in the cheap direction produces a quietly wrong archive; guessing wrong in the expensive direction produces a slow night.

What day-by-day does not fix. In our measurements Google stops a single-day [query] pull at 5,000 rows (August 2026). So on a large site a busy day is still truncated after this change. But it is truncated at its own tail, at the same threshold as every other day, and visibly: a day sitting at exactly 5,000 rows is the ceiling's signature, and we record that day as capped rather than storing 5,000 as though it were a count. Pulling day by day does not make truncation go away. It makes truncation uniform and detectable instead of correlated and silent, which is the difference between a number with a known limit and a number that is simply wrong.