When I built Pantrify, I added a purchase history almost as an afterthought, so every time Bree or I tick something off the list, a row lands in a purchases table with the item, its category and a purchased_at timestamp. After three months of using it I exported the table, expecting to learn something mildly embarrassing about how much chocolate we eat, and to be fair the chocolate part held up: chocolate and soft drinks were tied as our most-bought items at ten each, and one or the other turned up in more than half of our shopping trips. Toilet paper was also on ten, though, which is where things got a bit more interesting.

Ten packs of toilet paper in twelve weeks would be a genuinely alarming household statistic, especially since the data has us buying it twice on 12 July about an hour and a half apart, but nobody was stockpiling. The staples category in Pantrify sits on the list permanently, so when we're at home and notice we've already got something, we tick it off just to clear it, and you can actually watch that happening in the raw data. On 3 October, eight staples were added to the list in the same second at 7:58am and four of them were ticked within the next eighteen seconds, which I can promise had nothing to do with anyone buying eggs, milk, toilet paper and caramel coffee syrup in eight seconds flat on a Saturday morning.

In hindsight, naming the column purchased_at was fairly generous, because what it really records is the moment someone tapped a checkbox, and that tap means "bought it" some of the time, "we've already got heaps" some of the time and "I'm sick of looking at this" the rest of the time. The opposite problem shows up as well, since by my rough count from the item IDs, about a quarter of everything we've ever added to the list was deleted without being ticked and so never appears in the history at all. Pomegranate molasses has been sitting on the current list for sixteen days without ever being bought, and while I suspect every household has an item like that, as far as the purchase table is concerned it doesn't exist.

None of it matters much for a grocery list, but it's exactly the same mistake I've seen in proper business data, where an order status gets treated as a sale, a "completed" flag gets treated as a delivery, or a field called active_customer turns out to mean "hasn't been manually archived yet". The column name describes what someone hoped the data would mean, while the row itself was created by whatever action happened to trigger it, and if you analyse it at face value you end up with some very confident charts about how often a household buys toilet paper.

The check that would have caught it is boring and quick: before trusting a field, find out what action actually writes it, then look for rows where the timing doesn't make sense, like two purchases of the same thing ninety minutes apart or four items ticked within seconds of being added. For Pantrify, the honest fix is probably a second option when ticking a staple, something like "already have it", so the history can tell buying apart from clearing, although I'm holding off for now because I'd rather see whether either of us would actually use it, and in the meantime I can at least confirm that the chocolate figures are real.