Where it got interesting
Problems I hit, and how I solved them
The bugs that taught me the most. Each one follows the same loop: reproduce, find the root cause, fix it properly, and verify.
01
Crypto prices in the wrong currency
Data modelling
Problem
Binance quotes every pair in USDT, but users trade with ₹1,00,000 of virtual capital and NSE stocks are priced in rupees. A dollar-denominated BTC price could not be bought with rupees or added to a portfolio value without mixing currencies.
Root cause
The quote currency of the crypto feed did not match the currency of the ledger, and the leaderboard ranks on total portfolio value, so one wrong unit would have skewed every rank.
Fix
Converted USDT prices to INR once, in the worker, at the point ticks are normalised. Everything downstream (the Redis cache, order execution, portfolio value and the leaderboard) only ever sees rupees.
Outcome: stocks and crypto sit side by side in a single ₹ portfolio, and the rest of the system needed no currency logic at all.
02
A leaderboard that couldn't keep up
Performance
Problem
My load test at 500 users showed the leaderboard recompute averaging 1,614 ms (max 2,044 ms), slower than its own 1-second refresh cadence.
Root cause
Recomputation made too many small Redis round trips: prices fetched repeatedly and rank writes sent one by one.
Fix
Rewrote the engine to batch: one MGET and one price fetch per symbol, pipelined ZADD/HSET writes, and batched rank lookups.
Outcome: recompute dropped to 97 ms average (128 ms max), about 16× faster, with p95 fan-out at 129 ms and zero order errors on the re-run.
03
A worker that started but never worked
Debugging
Problem
The worker booted with SUPABASE_URL undefined even though .env was correct, and the Binance feed streamed fine on its own but never ran inside the main worker process. Broker auth also kept failing without a clear error.
Root cause
ES module static imports are hoisted above runtime code, so the Supabase client was created before dotenv had executed. Binance had simply never been wired into the main entry point, and an unquoted # in an API key was being read by dotenv as a comment.
Fix
Moved those imports to dynamic await import() inside main() after configuration loads, wired Binance into the main worker, and quoted the key. For production, the worker and API run through tsx in Docker because the shared package exports raw TypeScript.
Outcome: the worker boots deterministically and both feeds stream into Redis from one process. I now treat import order as part of the design, not an accident.
04
Deploying to EC2 without SSH
Infrastructure
Problem
I couldn't SSH into the server: my campus Wi-Fi blocks outbound port 22. The instance was also a small t3.micro that had to run Redis, the API, the worker and a reverse proxy together, and the custom domain wasn't available to point at it yet.
Root cause
A network policy I couldn't change, a tight memory budget, and TLS certificates that normally depend on a domain name I didn't control at that point.
Fix
Did the entire deployment inside the AWS SSM Session Manager browser terminal, with no SSH. Added a 2 GB swap file, attached an Elastic IP so the address survives restarts, and used a hostname derived from that IP so Caddy could serve real HTTPS without waiting on DNS.
Outcome: the full stack (Redis, API, worker and Caddy) runs on EC2 via Docker Compose over HTTPS, and the frontend on Vercel talks to it.
05
Bugs only end-to-end testing could find
Correctness
Problem
Real orders exposed failures that compiling never showed: cash_balance was reported as ambiguous inside the order function, SHIBUSDT orders were rejected by a price check, and quiet coins like ATOMUSDT were refused as "price feed is stale". Testing was also blocked because anonymous sign-ins wouldn't enable, and the web app clashed with the API on a port.
Root cause
An unqualified column name inside the Postgres RPC; prices for tiny-value coins that didn't fit numeric(18,4); a per-symbol 15-second freshness rule that quiet coins legitimately break; and the web app inheriting PORT=4000 from the root .env.
Fix
Qualified the column, widened fill_price and avg_cost to numeric(28,10), and replaced the per-symbol rule with a feed-wide heartbeat, the same idea already used for NSE. Pinned the web app to port 3000 and tested through a throwaway email user created with an admin script.
Outcome: a 40-user smoke run filled 116 of 116 orders, and the later 500-user load test finished with zero order errors.
06
WebSockets and Redis across two clouds
Real-time
Problem
The worker crashed on start with a Redis connection error, live feeds dropped without warning, and once the frontend and backend lived on different hosts the browser had to reach Socket.IO securely and across origins.
Root cause
Redis wasn't running when the worker started, a raw feed socket can die at any time, and a page served over HTTPS from Vercel can't connect to an insecure or cross-origin socket.
Fix
Ran Redis under Docker Compose, gave the Binance client auto-reconnect with exponential backoff, and made the Breeze client reconnect and re-subscribe on drop. Put Caddy in front of the API for TLS, and locked CORS on both Express and Socket.IO to the Vercel origin.
Outcome: prices flow from the feeds through Redis and Socket.IO to the browser over a secure connection, and they recover on their own after a drop.