Running Untrusted Code Safely: A Sandbox Design Guide

Every coding round on Taqari ends the same way: a candidate presses Run, and source code we have never seen executes on our hardware. Running untrusted code safely is the entire problem in that sentence, and it is not solved by putting a container around it. This is the layered design we landed on, why each layer exists, and the mistakes that are easy to make the first time you build a judge.

What follows is design reasoning, not a configuration dump. The specific knobs matter far less than the order you reach for them in.

Why running untrusted code safely is not a container problem

The first instinct is always the same: start a container, run the program, read stdout, tear it down. That gets you a demo. It does not get you a judge, because a container is a process-isolation tool that has some useful security properties — not a boundary designed to hold code written specifically to break out of it.

The difference is who wrote the code. A CI runner executes what your own engineers committed and reviewed. A judge executes whatever anyone who can complete a signup form decided to type. The threat model is closer to a public REPL than to a build server, and not everyone probing it is there for the interview.

What a submission actually is

From the platform's side, a submission is four things: a blob of source text, a language identifier, a session, and a pointer to the test cases it should be graded against. The source text is the dangerous part, and it is dangerous in more ways than "it might be malware":

• It can be slow on purpose — an infinite loop, a fork bomb, a regular expression built to backtrack forever.
• It can be greedy — allocate until the host swaps, write until the disk fills.
• It can be curious — walk the filesystem looking for the expected outputs it is being graded against, or for a credential it can use later.
• It can be talkative — open a socket and turn your infrastructure into an egress point for traffic you do not want your address attached to.
• It can be none of the above and simply wrong, which is the overwhelming majority of runs and the case the system has to stay fast for.

That last bullet is the design constraint people forget. A sandbox that is airtight and takes eight seconds to start has failed, because the candidate in a timed interview experiences it as a broken product.

Ordering the threat model

Vague goals like "make it secure" produce vague designs. Ranking what you are actually defending against produces a list of decisions. Ours, in order:

1. Escape to the host. Rare, catastrophic, and the reason isolation strength is a separate question from resource limits.
2. Lateral movement. Far more likely than a kernel escape: a runner that can open a connection to an internal service is a foothold, even if it never leaves its own container.
3. Resource exhaustion. Near-certain, and usually accidental. Someone writes a recursive function without a base case roughly every day.
4. Grading integrity. Reading the expected output instead of computing it is cheating, not hacking, but it destroys the product either way.

The shape of a run

Before the layers, the path. A submission crosses four boundaries between the editor and the results panel, and each one is doing a different job.

Two properties of that picture do most of the work. The browser never addresses the runner — it talks to our API and reads a stream, nothing else. And the arrow into the runner is the last one pointing right: past that line, nothing the submission does can reach back toward anything that matters.

Layer one: isolation strong enough for hostile input

A default container gives you namespaces and cgroups, which separate what a process can see and how much it can use. Both are genuinely useful. Neither was built to stop deliberate escape attempts, because the kernel underneath is shared, and the kernel is an enormous piece of C reachable through hundreds of system calls.

So the question is not "container or no container" but "how much kernel do I expose". Two ways to shrink it, and they compose:

Cut the syscall surface

Most of what a kernel offers, a program computing the length of the longest palindromic substring will never touch. A syscall filter that denies everything outside a known-good set turns a large attack surface into a small one — the mechanism is seccomp filtering, and container runtimes expose it as a profile you can tighten from their default. Dropping every Linux capability and refusing privilege escalation belongs in the same breath: an interpreter does not need to mount filesystems or load kernel modules.

Or move the kernel out of reach entirely

The stronger option is a sandbox that does not hand syscalls to the host kernel at all — gVisor documents this model well : a user-space kernel that implements the interface itself, or a lightweight virtual machine with its own kernel and a tiny device model. You pay for it in startup time and some syscall-heavy workload performance. For a judge that is usually the right trade, because the workload is short, CPU-bound and syscall-light, which is exactly the shape that suffers least.

Assume the layer fails anyway

Every layer above should be built as if the one below it will eventually be bypassed. A runner that has escaped its sandbox should find itself on a host with no credentials worth stealing, no route to a database, and nothing on disk but the language runtimes. That is a deployment decision, not a sandbox setting, and it is the one that decides whether a bad day is an incident or a headline.

Layer two: no network, ever

This is the cheapest rule in the whole design and the one we would fight hardest to keep.

Give the execution environment no network namespace beyond loopback. No DNS, no outbound TCP, nothing. The effect is disproportionate:

• A submission cannot exfiltrate anything it manages to read, because it has nowhere to send it.
• A submission cannot fetch a solution, a payload, or a second stage at runtime.
• Your infrastructure cannot be used as a relay for outbound scanning or abuse — which is the failure mode most likely to get an entire IP range blocked.
• An escape that lands in the container's network context has nothing to move laterally toward.

The objection is always "but some problems need to call an API". They do not. If you ever add a track that genuinely does, it gets its own execution path with its own allowlist, and it does not relax the rule for the judge.

Layer three: limits that are enforced, not requested

Resource limits are the part most homegrown judges get weakest, because the obvious implementation — start the process, set a timer, kill it when the timer fires — is wrong in several specific ways.

Every limit needs an enforcer outside the sandbox

Anything the submission can influence is not a limit. A timeout implemented inside the runtime the candidate controls is a suggestion. The deadline has to be held by something the code cannot touch, and when it expires the kill has to take the entire process group, not the process you happen to have a handle on. Otherwise the first thing a fork bomb does is orphan itself.

Wall clock and CPU time are different limits

You need both, and for different attacks. A CPU limit catches the busy loop. It does not catch a program that sleeps for an hour, because a sleeping process burns no CPU while still occupying a worker slot — which is a denial-of-service against your queue rather than against your processor. The wall-clock deadline is what bounds occupancy; the CPU limit is what bounds cost.

Processes, descriptors and output all need caps

Memory and time are the limits everyone remembers. The ones that actually get hit in production are less glamorous:

• Process and thread count. Without a cap, one line of code can fill the host's process table. The cgroup v2 controllers are where this and the memory ceiling actually get enforced.
• Open file descriptors. Cheap to exhaust, annoying to diagnose.
• Output size. A loop printing to stdout will happily produce gigabytes. Truncate at the runner, not downstream, or you have simply moved the memory problem into your own pipeline.
• Disk. A writable scratch directory should be small, temporary, and discarded with the container. Everything else mounts read-only.

A timeout is a result, not an error

This one is a product decision as much as an engineering one. When a run exceeds its deadline, the candidate should see "time limit exceeded" on test case four, in the same panel and the same shape as a wrong answer. It should not surface as a failed request or a spinner that never resolves. We stream each case back as it finishes — the mechanics of that are in our write-up on streaming test results with Server-Sent Events and Redis — and a timeout is just another terminal status arriving on that stream.

Layer four: the runner is not on the internet

Everything above is about what the code can do once it is running. This layer is about who gets to start it.

The execution service should accept work from exactly one place: our own backend, over a private network, and never from a browser. This is worth stating plainly because the shortcut is so tempting — the client already knows the source, the language and the test cases, so why not let it submit directly and save a hop?

Because then authentication on the execution service is the only thing standing between the open internet and free compute on your hardware, and compute is the thing people actively hunt for. Keeping the runner unaddressable from outside means an attacker has to get through your application's auth, quota and validation first, and those are layers you can reason about, rate-limit and revoke. The same logic applies to any stateful backend service, which we touched on when writing about scaling real-time infrastructure.

The API in front of it earns its place by doing four boring things before anything is enqueued: confirm the session belongs to the caller, confirm the language is one we support, bound the source size, and check the caller has runs left. Validation that happens after you have already spent a container is not validation.

Layer five: the output is untrusted too

It is easy to relax once the process has exited. Do not — the bytes it produced are still attacker-controlled input, and they are heading for a React tree.

Program output is arbitrary bytes, not a string. It can contain invalid UTF-8, null bytes, terminal escape sequences, and anything that looks like markup. Transporting it in an encoding that survives arbitrary bytes is the simple fix; on our side the runner hands back base64 and the results panel decodes it defensively:

const decodeBase64 = (encodedString) => {
  if (!encodedString) return '';
  try {
    return atob(encodedString);
  } catch (error) {
    // Already decoded, or not valid base64 - show it as-is
    // rather than blanking the candidate's output.
    return encodedString;
  }
};

The fallback matters more than it looks. A decode failure in the middle of an interview must degrade to showing something imperfect, never to an empty panel or a thrown exception that takes the results view down with it.

Then render it as text. Not as markup, not through anything that interprets HTML. The candidate's own compiler error should never be able to inject a tag into the page they are being graded in.

Grading integrity is a separate problem

Sandboxing stops a submission from attacking you. It does not stop it from cheating, and the two get conflated constantly.

The defence is structural rather than adversarial: the expected outputs simply are not present in the execution environment. The runner receives a source file and one test input, produces output, and hands it back. Comparison happens on our side, against data the sandbox never had. There is nothing to find on the filesystem because nothing was ever mounted there.

That property also makes per-case execution genuinely independent, which is what lets cases stream back individually instead of arriving as one batch at the end. Isolation designed for security turned out to buy a better interface too.

What we would tell anyone building this

Five rules, roughly in the order they pay off:

1. Kill the network first. Highest value, zero cost to legitimate users, one configuration line.
2. Put every limit outside the sandbox, and kill process groups rather than processes.
3. Never let a client address the runner. Validate, authenticate and meter in front of it.
4. Design for the layer below failing. An escaped process should find a host with nothing worth having.
5. Treat output as hostile input all the way to the DOM.

None of this is exotic, and that is rather the point. Running untrusted code safely is not one clever trick; it is a short list of unglamorous constraints applied consistently, most of which cost a correct submission nothing at all. The interesting engineering is everywhere else — in the harness that turns a function body into a runnable program, in the streaming, in the interviewer model. The sandbox's job is to be boring, and to stay boring on the day someone decides to test it. You can see the whole thing working end to end in a coding round on Taqari.