LDBD Dev Log #6 — Nothing Broke While I Was Building

At the end of the last launch retrospective, I spent a single paragraph on the warning email I got on launch night — the one saying I’d burned through the free tier — and the night I spent tracking down why and fixing it. I left it at one line — “having visitors means running into problems like this” — and moved on. In hindsight, that one paragraph is half of this post. The other half came about ten days later, and it was an entirely different kind of trouble.

For the two months I spent building this service, nothing happened. Nothing broke, and it couldn’t have — there were zero visitors. Then I launched, and two incidents that looked completely unrelated happened back to back within two weeks. The first was the server hitting its limit: behind 600 visitors were more than 70,000 function calls a day, and they burned through a month’s free allowance in half a day. The second was getting blocked while trying to tell people about the service. The post I’d spent two months preparing for an overseas community got classified as “AI-written” and died the moment I posted it. Neither was a kind of problem I could have hit while building. This is a record of learning, in two different ways, that building something and telling people about it are completely different jobs.

Part 1 — the server hit its limit first: 600 visitors, 70,000 function calls

The evening the warning email arrived

On the morning of July 13 I posted LDBD to GeekNews (a Korean Hacker News-style community), and for the first time the site had visitors. That evening, with people still streaming in, an email arrived from Vercel. LDBD runs on Vercel, a hosting service — the kind that takes your code, turns it into a website, and serves it to the internet — and on the free plan at the time there was a limit called Active CPU: four hours per month of actual server compute time. Those four hours aren’t how long the site is up; they’re the sum of the time the server spends doing real work. The email said I’d used 100% of them, and that going any further would automatically suspend the project.

Honestly, my first reaction was that it felt unfair. Six hundred people a day isn’t much traffic by web standards. A static blog would serve tens of thousands on a free plan. Burning a month’s allowance within half a day of launch meant something was running abnormally. To keep the site from going down, I upgraded to the paid plan (Pro, $20 a month) that same day, and started digging into the cause that night.

The culprit wasn’t the visitors

I opened Vercel’s Observability — the dashboard that shows which addresses got called how many times and how much CPU each one burned. On Vercel, every page request wakes a small unit of server-side code called a serverless function, which builds the page and hands it back; these function calls came to 72,400 a day. That’s 120 times the day’s 600 visitors. Even if every visitor clicked a hundred pages, you still wouldn’t get to that number.

The rest weren’t people. They were crawlers — the programs search engines and AI companies use to read web pages automatically. In hindsight it was obvious. LDBD has over 1,200 asset pages alone once you count the Korean and English versions together, plus separate profile pages for every participant, and to rank well in search I’d even submitted a sitemap — the list of addresses that tells search engines “here are the pages on our site.” The crawlers took that list and were sweeping all 1,200 pages around the clock, visitors or no visitors. I’d handed out the invitations and then acted surprised that so many showed up.

The CPU broke down as 70% functions, 30% middleware. Middleware is the shared gate every request passes through before it reaches the page code; it handles things like language routing and login checks. The top paths on the function side looked like this.

By call count, the profile page leads; by CPU, the share-card image is number one. That means its cost per call was more than ten times a normal page’s. So I started with the card.

The card I thought I’d cached was redrawing every single time

That little preview image that pops up when you paste a link into KakaoTalk (Korea’s main messenger) or X is an OG card — the name comes from the Open Graph standard. For each prediction, LDBD draws a card holding the asset, direction, and return as an image file, rendered on the server on demand. To keep Korean text from breaking, it needs a font, but the full Korean font is 5MB, too big to ship whole. So it pulls a few-KB sliver of the font — just the characters the card actually uses — from Google Fonts, and fetching two weights can mean up to four external requests per card. Drawing a single card takes about 0.34 seconds on average, which is fairly expensive work from the server’s point of view.

I knew it was expensive, which is exactly why I believed I’d cached it. A cache stores a result once and, when the same request comes again, hands the stored copy back instead of rebuilding it. Next.js (the web framework LDBD uses) has a value called revalidatethat lets you declare “reuse this result for N seconds,” and I’d written exactly that into the card’s code.

But when I checked the responses from the live site, every one of them was being redrawn from scratch. Responses carry a marker telling you whether they came from the cache (HIT) or were freshly built (MISS), and not just the prediction card but all seven kinds of cards built the same way came back MISS. At least in our setup, on these card-image routes — the ones defined purely by file name and location — the revalidatedeclaration wasn’t taking effect in production. The scarier part was that there was no error and no warning. For two months I’d believed the cache was on, while every time a crawler knocked on a card’s address, the whole render ran again, including up to four font downloads.

The fix was to stop trusting the declaration and instead put the instruction directly on the response. I spelled out the cache policy in the response headers so that Vercel’s edge cache — the layer that stores results per address across a worldwide network of relay servers (a CDN) — would hold onto the cards. I split the retention by the nature of each card. A resolved prediction’s card can never change again, so it gets seven days; a card that’s still open or whose numbers are moving gets only five minutes. The service’s own rule — “once resolved, immutable” — became the cache policy verbatim, which is the fix I like best out of this whole episode.

One line reading a cookie switched off the whole cache

After the card came the pages. Next.js has a caching approach called ISR (Incremental Static Regeneration). Instead of rebuilding a page on every request, it builds the finished version ahead of time and serves that, rebuilding only once per set interval (say, 60 seconds). Whether 100 people or 10,000 show up, the server only has to work once every 60 seconds — there’s no better defense against crawler load.

The asset pages already had a 60-second ISR declared. But checking the responses, these too were being rebuilt on every request. The cause was that when a page read its data, it went through cookies()— the function that reads things like the login info stored in the visitor’s browser. The moment you read a cookie, Next.js decides “this page must look different for every viewer” and demotes the whole page to dynamic rendering, rebuilt on each request. My 60-second cache had, again, quietly stopped applying. Twice in one day, I relearned that a cache being off is less dangerous than a cache that’s off while you believe it’s on.

Since asset pages only show public data that has nothing to do with login, ISR came back once I swapped in a separate read-only data client that doesn’t touch cookies at all.

The real trap came next. While converting the prediction detail and profile pages to ISR the same way, I ran into a nasty behavior in Next.js 16. If a page registered for ISR reads cookies()while it’s being built, then — at least in the version I’m on — it doesn’t gently demote to dynamic rendering; it throws a 500 error named DYNAMIC_SERVER_USAGE— a server error where the page doesn’t load at all. Leave even one line of cookie-reading code in a page you’re trying to cache, and the whole page dies.

So I changed the structure. The body of the page serves the same cached copy to everyone, and only the parts that have to differ per viewer — the state of a follow button, a “this is your prediction” marker — get filled in by a separate API call from the browser. Developers call this a client island: most of the page is cached, and only the personalized pieces are filled in on the client. One thing I was careful about: LDBD has a policy that hides the author of an open prediction to prevent copycatting, so I made the cached copy itself hide the author and left the decision of who to reveal it to up to the browser-side API. After deploying, I confirmed that the cached copy of an open prediction page doesn’t leak author info, and that the new API refuses to answer if it’s called without login.

The home page showed “zero verified predictions”

Around this time there was one more side incident. At the top of LDBD’s home page there’s a number showing how many predictions have been verified so far, and at some point I caught it rendering as 0. A figure I’d built up past 128,000 was showing on the home page as “zero verified predictions.”

The cause was fallback code. If the query reading the stats failed, it was written to substitute 0 — and because the home page is cached through ISR and the CDN, the 0 from one failed moment gets frozen into the cache and served to every visitor until the next regeneration. To a first-time visitor, the site looks like it has zero verified predictions.

The lesson from Dev log #2 came right back. Back then the problem was scoring code that would “settle on whatever price if the exact date’s price is missing,” and the conclusion was that quietly succeeding with a wrong value is more dangerous than simply failing. This time the same pattern showed up on the display side. So instead of showing 0 on failure, I made the page regeneration itself fail. When regeneration fails, Next.js keeps serving the last page that succeeded, so visitors see a stale but correct number.

But this safeguard then caused a problem on the deploy side. A deploy includes a step that pre-builds the home page, and at that moment the stats query stumbled once, so the error I’d just added failed the entire deploy. Adding a retry didn’t help — the same query kept failing — so I dug in, and the root cause wasn’t a transient hiccup. To count the verified predictions, it was counting a 128,000-row table exactly, from top to bottom, every single time, and as the table grew that count increasingly timed out. That’s also why the home page showed 0 “at some point.” I changed the counting method to sum the resolved counts across the 36 rows of a table already aggregated per participant. Before switching, I confirmed on the live site that the full count and the summed value matched exactly (both 128,604). In the end the answer was simple: don’t count a big table from scratch every time; add up the values you’ve already counted.

After the fixes

One habit came out of this. Once I’ve set up a cache, I don’t trust the declaration; I check the response headers myself. After deploying, I hit the card, asset, profile, and prediction-detail URLs one by one and watched the x-vercel-cache marker flip from MISS to HIT. Sometimes you get STALE, which means it’s serving a copy whose retention just expired while rebuilding a fresh one behind the scenes, so the visitor doesn’t wait. For two months I’d waved this off with “I declared it, so it must work”; now I only trust what I’ve seen with my own eyes.

The fixes showed up in the numbers, too. On July 13, the day of the incident, the server functions woke 72,000 times and burned 52 minutes of CPU, with the prediction share-card at the top. That one kind of card alone drew 2,400 requests and 5 minutes of CPU a day. After moving the cache into the response headers, I looked again at the week of July 18–25. The same card, across the entire week, came to 1,000 calls and 2 minutes of CPU. Per day that’s about 17 seconds — so from 5 minutes a day to 17 seconds a day, roughly 95% gone. The daily function CPU, which had spiked to 52 minutes on the incident day, sat around 12 minutes a day on average during that quieter week. For context, those 52 minutes are the function-side CPU alone. The Active CPU that counts against the limit adds middleware CPU on top, and the usage accumulated from July 1 to 12 was already on the books. That’s why the month’s allowance overflowed that evening.

That doesn’t mean the top CPU spot went empty. The top CPU spot moved from the card to the prediction detail page — and this isn’t a fault, it’s the structure. In LDBD every prediction gets its own unique address — there are around 130,000 of them — and since the cache is held per address, an address a crawler hits for the first time has to be redrawn once, right then. While crawlers work through those 130,000 addresses for the first time, that first render keeps piling up, coming to 31,000 calls and 33 minutes of CPU a week. It’s not that the cache isn’t working; it’s the cost of having that many addresses for the cache to fill. And the single most expensive job per call is now rendering a participant’s profile page. Three of every four requests finish within a second, but the fourth takes longer. So if you look only at the graph of total server work, it rises and falls right alongside the visitor and crawler counts, making the fixes look pointless — but split it by path and the story is different. The places I touched directly clearly went quiet, and the overall number still moving is because the places I haven’t touched yet, plus newly arriving traffic, keep pulling it up.

I also set up a guard against a repeat. I configured Vercel to alert me when usage hits a certain dollar amount, so that next time something starts running abnormally, I catch it at the alert stage rather than at the warning-email stage. But to be honest about it, the middleware side that made up 30% of the CPU is still untouched, and caching the leaderboard and the feed is still only on my list of candidates. What I fixed this time was the most expensive things, not all of them.

Part 2 — blocked while trying to tell people: an AI detector killed what AI had built

Around the time the server-side incident settled down, the second one began. This one happened not at the server but at the point of trying to reach a community of people, and it took a few days to figure out the cause.

First, some background — the account had been quietly dead

First, some background. I mentioned this in a single line in the launch retrospective: while checking my account before posting to Hacker News (HN for short), the overseas developer community, I found that every comment I’d left was dead. They looked fine to me, but logged out I couldn’t see any of them. Quietly hiding an account so it’s visible only to its owner and no one else is called a shadowban. Not knowing why, I emailed the moderators asking them to restore the account, and when no reply came, I sent it again. No reply ever came, but at some point new comments started showing normally. Without knowing why it died or why it came back, I treated it as fixed and moved on.

Show HN, dead in minutes

On the night of July 22, I posted the piece I’d spent two months preparing. HN has a format called Show HN for introducing something you’ve built, and its rules are fairly strict: no clickbait titles, no hype, and honest limitations spelled out in the body. I followed every one of them. And within minutes the post was marked dead. HN has a mechanism that quietly removes posts judged inappropriate from the list, and mine vanished that way the moment it went up. What made that judgment, what I’d done wrong — I couldn’t tell at the time. Once again, not knowing why, I emailed the moderators.

The moderator’s reply — the culprit was “AI-written text”

The next day, July 23, a reply came from an HN moderator (someone who manages the community), and the whole mystery resolved at once. HN’s software had been auto-classifying my post as genai — “AI-generated text” — and killing it. The shadowban on my account about ten days earlier was for the same reason. The comments I’d left had been polished by an AI, and all of them had been caught as AI text. The moderator’s instruction was clear: don’t use an LLM on anything you post to this community — not even a tiny bit, not even to edit.

The irony

Here I had to stop for a moment. I’m someone with no development experience, and I built this entire service — the site, the bots, the scoring logic — over two months with an AI tool called Claude Code. Writing that fact out honestly was the whole point of the post. Yet once I let an AI polish that post, a different AI detector killed it for being “AI-written.” The first gatekeeper between me and the people I was trying to reach turned out, it seems, not to be a person but a machine.

Rewriting it by hand let it survive

So I rewrote the body from scratch, by hand. I didn’t try to smooth it out; I left the clumsy English as it was. The rewrite was shorter, less polished, the sentences rougher. But it was closer to the order I actually meant to say things in. The AI-polished version wasn’t wrong, but it was too safe and too tidy. I sent that rewrite to the moderator, and he reposted it himself as a new copy. Only, it happened to be evening in the US, so the post — having missed the daytime hours when people are awake — passed by quietly. Three points, three comments, and a little over 100 visitors. Disappointing numbers if you were expecting a big launch, but not zero either. Still, one of those three comments was worth it. Someone asked, seriously, how I’d designed the scoring. How you separate, in numbers, whether a bot just got lucky or actually has skill is the question I’ve held onto longest while building this service. The short exchange over answering it made the two months of preparation count for at least a little.

What this round taught

When things start breaking, it means people finally arrived. For the two months visitors sat at zero, it didn’t matter that the cache was off, that the count query was slow, that the account was dead — nothing happened. The server struggling, the post getting caught by a detector: all of it happened because someone was finally trying to reach this service. You don’t get these incidents while building. That’s the biggest thing the two completely different events have in common.

Open a door to the web and crawlers arrive before visitors — and in far greater numbers. Behind 600 visitors were 70,000 function calls, most of them search-engine and AI crawlers. But these aren’t guests to chase off. They’re traffic I invited by submitting a sitemap so I’d rank in search, and these days getting read by AI is a source of traffic too. You don’t pay for leaving the door open by blocking crawlers; you pay for it with caching — so that being asked for the same page ten thousand times still means building it only once.

You confirm a cache by observation, not by declaration. Writing revalidatedidn’t mean the cache was running. In one place the declaration was ignored outright; in another, one line reading a cookie voided it; and neither raised an error. Checking the response headers with your own eyes takes ten seconds, and the price of not doing it was a month’s CPU.

Don’t paper over a failure with 0. Covering a query failure with 0 makes the screen look fine, but it makes the home page lie. Failing loudly beats being quietly wrong — I confirmed that once more on the display side, after the scoring pipeline.

From now on, community posts get written by hand. At least in that community right now, polished sentences draw suspicion and clumsy ones get through. It’s a little bitter that people get filtered by how they wrote rather than what they built, but there’s something I learned from it too. When I write from scratch by hand, the sentences may be clumsy, but they carry my own thinking more directly. From here on, everything I post to overseas communities gets written that way.

There are problems you can’t know while building, and problems that only appear once you start telling people. These two weeks were my first real lesson in the second kind.