Field notesEngineering
A zero that means we did not look
On 11 August the Today page of our internal admin console reported 0 runs, 0 failed runs and 11 active clients. Every number on it was correct. The page could not tell you which of those zeros it had actually measured.
In production it was measuring 1 of 11 clients.
Two numbers, two different lookups
The two tiles came from two different places. The client count listed every client marked active. The run count asked each client's database, but only the clients whose access key the console could read.
Nine clients had no key configured in production, and one client's key was rejected, so the query skipped them without a word. A client whose name contained a hyphen could never be given a key at all, because a hyphen is not allowed in an environment variable name. A rejected key returned an empty list, identical to a client with nothing to report.
Then the page added up whatever came back and published the total, with no statement of what it covered. Both faults ended at the same character, a zero, so the page read as normal.
The real number
Once the keys were synced, coverage went from 1 to 10 of 11 clients. The most recent run anywhere in the fleet moved from 4 August to 9 August, a run on a workspace the page could not see at all.
What changed
The console now reports what it covered: how many clients were in scope, how many it queried, how many had no credentials and how many failed. A count we cannot vouch for renders as unknown, never as 0. A partial count keeps its number and names the clients it covers.
A script now reads each client's key from the database provider and writes it to our hosting, so the two cannot drift apart by hand again. The Today page itself was removed the same day. Only the page went, and the data behind it stayed.
Coverage reached 10 of 11 that day, not 11.
It came back a month later
On 10 September we audited our errors console the same way. In the window the audit checked, the fleet ran 2,369 agent runs, 93 of them did not finish, and 12 reached the console. Five client workspaces had no run coverage at all while the console showed them clean.
The fix went out the same day, and every unfinished run in its check reached the console, 91 of 91. This time we wrote the rule down. Absent is never counted as a pass, because "could not look" and "looked and found nothing" render identically and mean opposite things. A client with no errors and a client we cannot read are opposite facts.
That audit regenerates its figures rather than quoting them.
What to ask any vendor
When your dashboard shows zero, how do you know it looked? Does it tell you how many of your systems it actually read? What does it show for a system it cannot reach?
Good answers come with a demo. The insights are free. If you want this level of engineering pointed at your operation, start with the free audit. The plan is yours to keep either way.