Nimbus

Pull request emails arrived for some pull requests and not others, with no pattern I could see. The notification lived inside the pull request gateway and was reached on exactly one path - the one where the gateway had just created the pull request itself. Two early returns skipped it silently: finding a pull request already open on the branch, and losing a creation race then fetching the winner. Branch names are derived from the session id, so they are stable, which meant any session that ran a second time took the first of those paths and said nothing.

The part worth remembering is that a test named 'notifies once, not once per attempt' passed the entire time. It was asserting the behaviour of the fake gateway rather than the behaviour anybody wanted. I moved notification out to the session runner, which is the only place that knows what the session already had, and keyed it on a pull request number the session was not already carrying. A green suite is evidence that the code does what the tests say, not that the tests say the right thing.

Sign-in codes sent fine from my machine and failed on the deployment, every time, after a 10.7 second wait. My first guess was that the host blocked outbound SMTP. That guess did not survive being reminded that production had sent mail before. The log said ESOCKET on CONN with 'connect ENETUNREACH' against an IPv6 address, so I forced the transport to IPv4 - and nothing changed. The next failure named a different IPv6 address, which was the actual clue: something was choosing, and choosing differently each time.

It was in the mail library. It resolves the hostname itself, concatenates the A and AAAA records, and then takes addresses[Math.floor(Math.random() * addresses.length)] before handing a literal to the socket - by which point an IPv4 preference is meaningless, because there is no name left to resolve. On a host with no IPv6 route, every AAAA record in that list is a coin flip that fails. Resolving to IPv4 myself fixed the flip, but not the shape of the problem, so mail moved onto an HTTPS API instead. Reading the library's source took ten minutes and replaced three confident wrong theories.

Five images filled the per-account limit. Removing all five appeared to empty it, and the next upload was still refused. The browser and the database disagreed about what existed, which is a bad thing for a database to be uncertain about.

The uploads were being started from inside a React state updater. React invokes those twice in StrictMode precisely to expose updaters that are not pure, and each invocation minted fresh local ids and fired its own upload - so one chosen file became two rows, and only one of them carried an id the browser could ever use to delete it. The other was an orphan that counted against the limit until a sweeper found it a day later. Moving the uploads and deletes out of the updater fixed it, and the lesson generalises: a state updater is a pure function of the previous state, and anything with a side effect does not belong in one.

People were being signed out mid-sentence, roughly an hour after signing in, no matter how active they had been. The server side looked correct - every request pushed the stored session another hour into the future, so as far as Redis was concerned an active person stayed active.

The browser was never told. The cookie was written once, at sign in, with a one hour life, and nothing ever rewrote it, so the two clocks disagreed and the shorter one always won. There was also a hard ceiling calculated by multiplying the idle window by twenty four, which meant neither number could be moved without moving the other. The cookie is now rewritten on every request that carries a working session, from the same number the server just used, the idle window and the ceiling are separate settings, and a refreshed cookie is never given more time than the session actually has left.

With the API on one platform and the browser app on another, signing in worked and every request after it was anonymous. The cookie was being set and then never sent again.

Session cookies are SameSite=Lax, and browsers decide 'same site' by registrable domain rather than by host - so a frontend on one vendor's domain calling an API on another's is cross-site, and Lax means the cookie stays home. No CORS header fixes that, and SameSite=None would have given up the exact protection the cookie exists to provide. The answer was to stop fighting it and put both halves under one domain I own, as sibling subdomains. Some problems are configuration; this one was a property of the web, and the only correct move was to arrange the deployment around it.