I needed a way to do some programming while offline. These days I feel very unproductive without https://chatcraft.org (the best chat UI for programming) and a good LLM to chat with about coding.

Chatcraft needed a few small fixes to enable llama.cpp support. Here’s how to run models with llama.cpp with chatcraft.org without internet:

Instructions

Install and run llama.cpp. Follow https://github.com/ggerganov/llama.cpp instructions for your platform.

For mac:

1
2
3
4
5
6

# install llama.cpp
brew install llama.cpp
# start llama.cpp server & auto-download a good small (~6GB) model
llama-server    --hf-repo "bartowski/Llama-3-Instruct-8B-SPPO-Iter3-GGUF"       --hf-file Llama-3-Instruct-8B-SPPO-Iter3-Q6_K.gguf
# For more advanced usage I recommend gemma 27b: the smallest smarter-than-gpt-3.5 model (~21GB)
# llama-server    --hf-repo bartowski/gemma-2-27b-it-GGUF       --hf-file gemma-2-27b-it-Q6_K_L.gguf

Setup local chatcraft dev env by following the instructions in the chatcraft repo

1
2
3
4

git clone https://github.com/tarasglek/chatcraft.org/
cd chatcraft.org
pnpm install
pnpm dev

^ will output a development url like http://localhost:5173/, open it. Go to chatcraft settings and add http://localhost:8080/v1 to api providers. Enter a dummy api key.

Enjoy!

TLDR: chatcraft.org is now smooth as butter and maximally performant thanks to throttling LLM chat-completion rendering via requestAnimationFrame().

I read a lot. I enjoy reading on black and white e-readers. Unfortunately many websites make it hard to read them on simple devices. https://markdown.download is my really simple solution to that. Prepend it to any website and curl:

1

curl https://markdown.download/https://dev.to/amnish04/introducing-the-idea-of-web-handlers-in-chatcraft-1b4i

That turns a user-hostile website into a thing that almost anything can render:

It’s amazing how much cleaner the web is when stripped down to essence.

Turns out markdown (or html restricted to what markdown can do) renders fantastic on readers of all types.

Since markdown.download is so utterly simple, I decided to try using it to learn a new-to-me dev tool: val.town. Rest of this post covers what I learned.

I initially was going to publish this project as an npm lib and deploy on cloudflare, but was putting it off because I loathe the amount of bureaucracy that involves. I love the type system in Typescript. I love Node.JS because it’s the least-bad way to write/deploy apps, but much room for improvement remains. My spare time is too valuable to spend it suffering.

val.town: brutally efficient microservice dev-ux

I have an intermittent issue where one of my machines on a specifc network seems to be operational locally on network but not accessible over netbird some of the time. I could not tell if the problem was due to the network the problematic machine was on or due to netbird.

So to debug this I:

1. Setup a grafana node agent to report journald logs to https://grafana.com/

2. Setup to docker to log to journald

3. Wrote ssh-via-cloudflare-tunnel docker-compose stack to tunnel ssh over websocat to internet over cloudflared . Set docker compose to restart-on-failure.

Today netbird had some downtime (tailscale had theirs just a few days before that). I was able to log into grafana, look up the temporary hostname cloudflare assigned to me and ssh in using

ssh -o ProxyCommand="websocat -E --binary - -v %h" -o ServerAliveInterval=10 wss://library-won-nt-gauge.trycloudflare.com

Architecture

Conclusion

This turned out trivial to do. However it took me over 5 years of pondering, discovering ssh ProxyCommand, cloudflared, websocat and deciding to combine them to conveniently expose ssh over web.

I think it’s pretty cool how Cloudflare allows one expose geographically distributed tunnels without requiring any signups. I wish their UX simpler so it was equally easy to expose tunnels with fixed dns. Cloudflare actually has something similar as an official ssh feature , but it requires more hoop jumping.

websocat is amazing. I use websockets a lot more now that I have a robust way to utilize them without writing any code.

Why Would I Want To Do This?

• As a fallback to tailscale/netbird that uses a completely different network topology as in this case
• To temporarily grant ssh to some machine without requiring end-users to install a vpn
• To bypass restrictive gsm/wifi networks that block access to ssh
• As an alternative to port-knocking to obscure ssh (eg hide ssh behind http basic-auth)

How would you improve this?

Checkout ssh-via-cloudflare-tunnel on github.

Let’s face it, managing secrets in software projects can be as thrilling as being stabbed in the eye. Yet, it’s a necessary evil that we all have to deal with.

Problem: we have a set of a developers and set of infrastructure that all needs to share secrets. Would like to minimize infrastructure and keep cognitive load to a minimum so we can focus on writing code. Sure you’ve got AWS Secrets Manager and Hashicorp Vault for the heavy lifting, but that’s like using a tractor to crack a nut. And then there’s the keep-all-your-secrets-in-github-action-ENVs which leads to “push-and-pray” mentality (https://dagger.io/ talks on CI/CD are awesome). Not exactly the pinnacle of security or convenience, right?

Enter SOPS , the cool kid on the block that encrypts your files without the bullshit (You know it’s cool cos it’s the latest in a long line of tech abandoned by Mozilla). But setting it up? Still sucks. This post is about how github-to-sops helps.

libkrun + krunvm

Github randomly recommended me libkrun which is a library backing krunvm . It’s something similar to firecracker, but even simpler.