I feel that the modern way of hosting apps is too complex and the old cgi-bin way was also suboptimal. I kept the cgi-bin subprocess model and married it with reverse proxying in https://github.com/tarasglek/caddy-reverse-bin/. I believe this way is easier to customize, use and secure than the “modern” corporate stack.

reverse-bin

Imagine if our reverse-proxy corporate future took a different evolutionary path. Imagine that instead of deprecating cgi-bin, we made a successor and taught it to speak multiplexed http2 on stdin instead! Remember, multi-process is great cos that’s also the secret to using operating-system native sandboxes, eg JS, golang and even naughty C++ I wrote before could be sandboxed using something like landlock, pledge, gvisor or in microvm.

My recipe for retro-modern experience:

  1. A configurable modern and somewhat safe Caddy server
  2. Caddy plugin for it to launch reverse proxies on demand.
  3. The reverse proxies can speak http on a tcp socket or preferably a unix socket.
  4. Apps exit on idle just like cgi-bin
  5. For now health checks are used to ensure apps are ready before Caddy forwards traffic to em
first request (cold):
  GET / -> caddy -> detector: "deno app, sock data/app.sock"
                 -> spawn sandboxed app
                 -> wait for data/app.sock
                 -> health check: GET / until 2xx/3xx
                 -> proxy -> app -> response

second request (warm):
  GET / -> caddy -> app already running -> proxy -> response

idle for a while -> caddy stops the app (scale to zero)

Idle apps get killed: 50 hobby apps cost nothing until someone visits.

The caddy plugin is small enough to actually read, which makes it less scary to deploy in prod. I’m not a fantastic go coder, I worked with LLMs to make it as good as I could. Unix subprocess shutdown is surprisingly tricky. This lets me demo the “writing lightweight apps should be fun and safe in a language of your choice” philosophy. The caddy plugin lives in https://github.com/tarasglek/caddy-reverse-bin/. This code doesn’t change much, I consider it basically done. The detector and the hosting package on the other hand still see a lot of changes, those are still maturing, and a lot more vibecoded. I wouldn’t have had time to do any of this without LLMs.

reverse-bin-detector

reverse-bin caddy plugin itself is dumb/minimal: you can hardcode the program, its args, socket and health check right in the Caddyfile. reverse-bin-hosting uses that for caddy’s on-demand TLS: caddy asks a tiny python script “should I get a cert for foo.example.com?” and that script is just a hardcoded reverse-bin app that only says yes if apps/foo exists:

reverse-bin * {
    exec /usr/lib/reverse-bin/allow-domain.py /run/reverse-bin/allow-domain.sock /var/lib/reverse-bin/apps --allowed-suffix .example.com
    reverse_proxy_to unix//run/reverse-bin/allow-domain.sock
    health_check GET /health
}

That’s fine for a few fixed things, but tedious for 50 apps. For actual apps, reverse-bin instead asks a dynamic_proxy_detector program “how do I run this thing?” and it answers with a JSON launch plan. Here’s (a slightly trimmed version of) how reverse-bin-hosting serves every <app>.example.com:

https:// {
    tls {
        on_demand
    }
    # foo.example.com -> /var/lib/reverse-bin/apps/foo
    map {host} {reverse_bin_app_dir} {
        default /var/lib/reverse-bin/apps/{http.request.host.labels.2}
    }
    reverse-bin * {
        dynamic_proxy_detector /usr/lib/reverse-bin/reverse-bin-detector {reverse_bin_app_dir}
    }
}

No exec, no reverse_proxy_to: the detector looks at the app dir and fills those in.

I’m pretty happy with my concept of a detector. Turns out most apps have a common shape, and instead of writing boilerplate for them one can write a detector to detect common patterns and launch them in a sandbox. This concept is still evolving.

Detector recognizes deno/python/static apps with zero config and runs them in watch mode, so editing a file on the server is live immediately. Yes, that’s the naughty val.town edit-in-prod feel, on purpose.

Detector config is just env vars: drop a .env in the app dir with stuff like REVERSE_BIN_COMMAND=./launch.sh, health check path, port, etc. and the detector stops guessing and does what it’s told. Even better, the .env can be SOPS-encrypted into secrets.enc.json and committed to git: the detector decrypts it in memory with the server key and the app only ever sees plain env vars. Add your ssh pubkey as a recipient (I wrote github-to-sops to pull them from GitHub) and you can edit secrets from your laptop without ever touching the server key (docs).

It’s super cool to be able to factor the sandbox out of reverse-bin and let users override it for their needs.

landrun + friends for sandboxing

By default, the detector wraps every app in old-schoolish Linux tools, no docker, no registry, no yaml:

  1. unshare gives each app its own PID namespace with a private /proc (plus IPC and UTS): an app can’t even kill the host or sibling app processes. It’s mapped to the reverse-bin user, not root.
  2. landrun, a tiny wrapper around Landlock, restricts the filesystem: app source is read/execute-only, data/ is the only writable dir, /etc is read-only, and the server keys and other apps’ files just don’t exist as far as the app is concerned.

The sandbox starts in single-digit milliseconds instead of the 100s of milliseconds containers take, so launching apps on-demand per request is actually viable.

Static sites are the strictest: each one spawns its own copy of caddy cos I didn’t wanna special-case serving html in the main caddy. It’s served over a Unix socket and gets no network at all. Executable apps currently get unrestricted networking cos runtimes like Deno/Chromium get grumpy otherwise; restricting bind while still allowing outbound is on the todo list.

Want to give a friend their own app? Hand them an ssh key that only drops into that app’s sandbox. That’s the multi-user ~/public_html feel, minus the part where they can read your homedir.

The policies are inspired by Deno permissions and the smallweb pattern. Full details, incl. the stuff that’s still missing, are in the security posture checklist.

reverse-bin-hosting

reverse-bin-hosting glues caddy, reverse-bin, the detector and the sandbox into a smallweb.run-like setup in a single .deb. I see it as an opinionated way to package and showcase the reverse-bin concept. It bundles deno, uv, sops, age and landrun. A fresh Debian box needs nothing else. Apps are just dirs in /var/lib/reverse-bin/apps/<name> and <name>.example.com routes to them, no deploy step.

Either let caddy do on-demand ACME TLS directly, or bind to localhost and put cloudflared in front. It also ships a GoAccess logs dashboard as just another app, cos of course it does.

Legacy compat is ugly

Btw, http2 does indeed make it possible to write apps without even listening on a socket, eg https://github.com/tarasglek/go-h2c-stdio. This would even allow one to sandbox them in a simple wasm runtime as no sockets are needed. This would’ve been cool, but wasm stdio is too glitchy/immature for this and nobody is gonna write apps this way if major web serving frameworks don’t support it.

For now the http2 stdio hack above is on hold as no software in the world supports that paradigm. The least bad solution is to deploy server processes on-demand by finding available tcp ports smallweb-style (docs, source). It’s racy (something else can take the port before our app binds it) and tricky to secure. I prefer Unix sockets where the app supports them.

I then factored out smallweb’s concept of a “this looks like an app I can run” into a custom detector, that can be forked/extended to whatever app pattern one prefers.

But wait, there is more: the detector can also figure out the shape of an app and sandbox it using modern Linux landlock and namespace tooling.

Then I packaged this up into a smallweb.run-like package and lived happily ever after.

The Unix way is cool

A lot of these sandboxing ideas originated in OpenBSD with pledge/unveil. I wanted to check my approach actually maps back onto OpenBSD and that I didn’t have excessive linuxisms. I tried the same pattern there: OpenBSD still ships a webserver with cgi-bin support, golang runs fine on it, and the process-per-app approach carried over without surprises.

If you can decompose your problem into separate processes, it’s much easier to separate various concerns like security and convenience.

None of this was obvious, I got here via cgi-bin, corporate Linux, val.town and smallweb, see the backstory.