Deploy Frontend and Backend Separately: 6 Hard Edges

We deploy frontend and backend separately: the frontend is a static Next.js build on a CDN, while the API, the judge queue and the live voice session run in a long-lived process on a VPS behind a reverse proxy. That split was the right call, and it broke six things a single-origin app never has to think about.

Here is each of those six, in the order we hit them, and what the fix was. None of it is exotic — it is the ordinary tax on two origins and one stateful process.

Why we deploy frontend and backend separately

The decision is not taste. The two halves want opposite infrastructure.

The frontend is a pile of immutable files. It redeploys several times a week, every byte is cacheable, and it scales by existing in more places. That is what a CDN is for, and paying a server to hand out the same JavaScript bundle is paying for nothing.

The backend is the opposite animal:

• A judge run streams per-test-case results for as long as the slowest case takes, over a connection that stays open throughout.
• A voice interview holds a transport open for the length of the interview and keeps per-session state in memory next to it.
• Run state lives in Redis, keyed per run, and something must subscribe to it for a browser that may disconnect and come back.

Every one of those wants a process with a stable identity and no execution-time ceiling. Put it on a platform built around short request handlers and you spend your time fighting the platform instead of the problem. So: static assets at the edge, stateful work on a box we control.

The request path, end to end

The important detail below is what the CDN is not doing: it is not in front of the API. The browser loads the app from one origin, then talks to a second origin for everything dynamic.

The alternative — proxying every API call through the frontend host so the browser sees one origin — makes cookies trivial and costs an extra hop you do not control in the middle of every stream, with its own idle timeout. We chose two origins and paid for them in CORS instead.

1. Cross-origin cookies are the first thing to break

We authenticate with cookies, so every API call is a cross-site request as far as the browser is concerned. Three separate rules have to hold at once, and getting two of three gives you a request that silently arrives unauthenticated.

SameSite, Secure and the Domain attribute

A cookie that must travel to a different origin needs SameSite=None, and MDN is explicit that Secure must also be set when using that value. There is no shortcut here: plain HTTP will not carry the cookie at all.

The Domain attribute is the constraint that outlives your first week. It must be the setting server's own domain or a parent of it — an API host can set a cookie for the apex it belongs to, never for an unrelated registrable domain. So your API hostname is a decision you make once, before launch: the cookie boundary is drawn around it.

The three rules, together

1. Server sets the cookie with Secure and SameSite=None.
2. Client sends credentials: "include" on fetch, and withCredentials on an EventSource.
3. Server answers with Access-Control-Allow-Credentials: true and one exact origin — a wildcard is rejected outright once credentials are involved.

Token rotation gets harder in this shape, because a refresh has to work while a stream is already open. That is written up in refresh token rotation without breaking long sessions.

The preflight nobody budgets for

A cross-origin POST with a JSON content type is not a simple request, so the browser sends an OPTIONS preflight first, and that preflight carries no cookie. If any layer in front of your handler requires authentication, the preflight fails and the real request never happens — and the console error points at the POST, not the OPTIONS.

Two things follow. The preflight must be answerable without auth, and it should carry an Access-Control-Max-Age so the browser stops asking before every call. This bit us hardest at checkout, where an extra round trip per call is the difference between a payment that feels instant and one that feels broken — the rest of that story is in Razorpay webhook verification and payment idempotency.

One more consequence: EventSource cannot send custom headers. It can send cookies, and that is all. For streaming endpoints in a split deploy, cookie auth is not a preference — it is the only option the API gives you.

2. The backend URL is baked in at build time

The client's entire knowledge of where the API lives is one line:

// src/config/api.js
export const API_BASE_URL =
  process.env.NEXT_PUBLIC_BACKEND_URL || DEV_FALLBACK;

That reads like runtime configuration. It is not. The Next.js documentation is blunt: NEXT_PUBLIC_ values are inlined at build time, and after being built, your app will no longer respond to changes to these environment variables. Change it in your dashboard and nothing happens until something rebuilds.

A stable hostname is the whole trick

The fix is not clever plumbing, it is choosing the right thing to put in the bundle. Bake in a hostname you intend to keep forever and move the backend by changing what it resolves to. The moving part stays in DNS, where moving is free, not in a compiled artifact, where it costs a deploy.

What we check before pointing that hostname somewhere new

1. Is the public API hostname genuinely unchanged? If not, this is a frontend release, not an infrastructure change.
2. Does the certificate on the new target cover that exact hostname?
3. Is the cookie Domain still valid from the new host's point of view?
4. Does the CORS allow-list include every frontend origin that exists — including preview deployments?

That last one is a real trap. Preview deploys get a fresh origin per build, so an exact-match allow-list cannot contain them all. Match the preview pattern narrowly, or accept that previews talk to a staging API. What you must not do is relax the allow-list to make previews work — it is the only thing standing between your cookies and any other site.

3. Buffering turns a live stream into a slideshow

The first time we put a reverse proxy in front of the judge stream, results stopped arriving one test case at a time and arrived all at once at the end. Nothing in the application had changed.

Response buffering is on by default — the nginx proxy module reference gives proxy_buffering a default of on, meaning the proxy reads the upstream response into its own buffers as fast as it can before passing it along. For a JSON response that is a feature, since it frees the upstream worker earlier. For a stream whose entire value is arrival timing, it is a bug you installed.

There are two ways to turn it off, and they are not equally good:

• In the proxy, per location. Correct, and invisible to anyone reading the application code.
• From the application, by sending X-Accel-Buffering: no on the response. The same reference documents that an upstream response can switch buffering off per response with that header.

We prefer the header. The handler that opens a stream is the code that knows it is a stream, and a response header travels with that code into every environment, including a proxy someone else configured. Set it in the proxy too, but let the application be the authority.

res.writeHead(200, {
  "Content-Type": "text/event-stream",
  "Cache-Control": "no-cache, no-transform",
  "Connection": "keep-alive",
  "X-Accel-Buffering": "no",
});

4. Read timeouts kill a stream that is merely thinking

A healthy stream can be silent for a long time. A judge run waiting on a slow compile, or a model composing a long answer, puts nothing on the wire for seconds at a stretch. The proxy cannot tell thinking from dead.

proxy_read_timeout defaults to 60 seconds, and — the part that surprises people — it applies between two successive reads, not to the whole response. A stream emitting something every 30 seconds for an hour is fine. A stream that goes quiet for 61 seconds gets closed mid-run.

Raising the timeout on the streaming path is the obvious half of the fix. The half that matters is the heartbeat: a comment line emitted on an interval comfortably shorter than the smallest timeout in the path.

// a comment line: valid SSE, ignored by EventSource,
// and enough traffic to reset every idle timer in the path
const beat = setInterval(() => res.write(": keep-alive

"), 15000);
res.on("close", () => clearInterval(beat));

Why the heartbeat beats the timeout bump

Because you only control some of the timers. Corporate proxies, mobile carriers and whatever middlebox sits in a candidate's office each have their own idea of how long an idle connection lives, and none of them read your config. Traffic is the only argument they all accept. The heartbeat doubles as a liveness signal: a write to a client that vanished fails, so the handler can clean up instead of holding a subscription for a tab that closed ten minutes ago. The full design of that stream is in server-sent events with Redis.

Set the upstream HTTP version explicitly rather than inheriting it. The nginx reference notes proxy_http_version now defaults to 1.1 but defaulted to 1.0 before version 1.29.7, and HTTP/1.0 upstream means no chunked transfer and no keepalive — a strange way to discover your streaming works on one machine and not another.

5. Zero-downtime restarts when a session lasts forty minutes

This is the genuinely hard one, and it is hard because of our own state, not the split. A stateless API restarts in a blip nobody notices. A process holding open streams and live interviews has no polite default — the default is that everyone mid-interview finds out.

Draining instead of killing

On SIGTERM, the goal is to stop taking new work and let existing work finish. Node's HTTP server gets you most of the way: the documentation says server.close() stops the server accepting new connections and closes connections that are not sending a request or waiting for a response. Idle connections get reaped; active ones are left alone.

Which is exactly right, and exactly why close() alone hangs forever on a streaming server. An open SSE response is by definition a connection that is actively sending; it never becomes idle on its own. If shutdown waits for the server to close and one long stream is open, you wait until the supervisor loses patience and sends a harder signal — at which point you did not drain, you delayed the outage.

The sequence we settled on

1. Fail readiness first. The proxy should stop sending new requests before the process stops being able to serve them.
2. Refuse new stream subscriptions, while existing ones keep running.
3. Tell open streams to reconnect. A terminal event carrying the run id, so the client re-subscribes to the replacement process and picks the run back up.
4. Call server.close() and wait for the drain.
5. Hold a hard deadline, then exit anyway. A drain without a deadline is a hang.

Pick that deadline with the keepalive in mind. Node's server.keepAliveTimeout defaults to 65 seconds, so a ten-second drain cuts idle connections the client still believes are reusable, and its next request on one of them fails. Either the budget respects the keepalive, or the keepalive comes down to match it.

Two processes, one public socket

With a resumable client, the restart is ordinary. Start the new process on its own internal address, wait until it reports ready, flip the proxy upstream, then drain the old one. The proxy owns the public socket the whole time, so there is no window where nothing is listening.

The piece worth getting right is "wait until it reports ready", because a fixed sleep is a guess: too short on a bad day, wasted time on a good one. If your supervisor can gate on a readiness notification, use it. systemd's Type=notify expects the service to send READY=1 once startup completes, and Type=notify-reload extends that to reloads — the service sends RELOADING=1 with a monotonic timestamp, then READY=1 when it is serving again. The mechanism matters less than the property: the switch is gated on the process saying so, not on a timer.

6. Supervision has to mean more than "the process exists"

Restart-on-crash with backoff is table stakes, and the backoff matters: a process that crashes on startup and restarts instantly burns a core and buries the log line that explains why.

The failure that costs you a morning is different. The process is running, its PID is right there, and it is wedged — a blocked event loop, a connection pool that never returns connections, a dependency timing out. Every check based on process existence says healthy. Every user says otherwise.

So the checks have to do real work.

• Liveness is a request, not a PID, and it touches the dependency you care about, with a timeout.
• Readiness is separate from liveness. One answers "should traffic come here", the other "should this process be replaced". Conflate them and a slow dependency triggers a restart loop.
• Logs leave the box as a stream, not as a file you discover you rotated badly on the day you need it.
• Resource ceilings are per process, so one runaway request degrades one thing instead of the host.

One public entrance in front of the application is what makes this tractable: a single place where TLS, timeouts and per-path behaviour are decided.

What we would do differently

1. Serve the API location from a tiny runtime endpoint, with the build-time value as a fallback. One fetch on boot buys the ability to move a backend without a frontend release, and makes environments differ in data rather than in artifacts.
2. Put the drain contract in the stream protocol from version one. A "server going away, reconnect with this run id" event is a few lines of code and removes a whole category of deploy anxiety. We added it after the first restart that cost someone an interview — the expensive way to learn it.
3. Version the per-path proxy settings next to the application. Buffering and timeouts for a streaming path are application behaviour that happens to be expressed in proxy configuration. Left on the box, the next machine is subtly different and nobody knows why the stream feels laggy there.

The pattern underneath all six is the same one we keep rediscovering: a split deploy does not create new problems so much as drag implicit assumptions into the open, where you have to state them. Which origin owns the cookie. Which layer decides a timeout. What a client does when a server goes away. Answering those explicitly is most of the work. To see the stateful half of this architecture under load, run a free mock interview and watch results stream in one case at a time.