Taras' Blog on AI, Perf, Hacks

The Case for Secrets as Code: Stop Click-Managing Secrets

Mon, 07 Jul 2025 09:46:32 +0300

We live in the age of software as a service, secret keys are everywhere. Yet somehow it is still normal to click-manage secrets. Smart people share them via signal,smarter people, share them via a password manager like Bitwarden. Super-advanced people punch them into GitHub UI and only let github actions have access to secrets. Infra-devops people spin up Vault or AWS secret manager or similar to inject secrets. In all situations secrets have a lifecycle that is completely disconnected from code, causing the two to get out of sync or worse.

Stupid WASM Hack to Get Native Binaries

Thu, 26 Jun 2025 22:12:27 +0300

I relax by programming. It’s especially satisfying to play with something that changes how I think about a topic. I once discovered eget and realized that most modern software ships as a single binary attached to a GitHub release.

Recently my friend, David, had a further realization that what would make eget better is if it itself was easier to get (ha!). I thought it would be delightful to have a WASI eget cli to fetch native binaries. These days almost everything cross-compiles trivially to WASI command modules. Those are basically little unix cli apps that have a small footprint and can run anywhere (something Java never achieved).

Focus and Context and LLMs

Sun, 08 Jun 2025 10:45:01 +0300

I decided to write down some thoughts on agentic coding and why it’s a very hyped wrong turn.

Let me start with some background on my LLM experience. I adopted LLMs into my work in Aug 2020. I was sold when I saw that GPT-3 could generate usable SQL statements. Something that used to take 4-8 hours of RTFMing, now took 15min. I have since worked on chatcraft.org, various RAG frameworks, etc. I use aider heavily for work, frequently switch models, have been struggling with tool calling during dark ages before MCP.

My E Ink Laptop: ThinkBook Plus Gen 4

Sat, 10 May 2025 11:59:24 +0300

I read a lot, blogs, papers, news, etc. My eyes are sensitive to artificial light so I prefer to read on e-ink. I also like to work outside, so things that try to fight the sun by feebly (relative to sun) shining even more light into my eyes are frustrating. I also find that e-ink screen limitations like slower updates and lack of color to be wonderful for not getting distracted by shiny pictures, videos.

Create Missing RSS Feeds With LLMs

Sun, 27 Apr 2025 19:23:16 +0300

It used to be that blogs all had RSS feeds. Somehow there are more blogs than ever before, but some people do not bother with setting up an RSS feed.

The following blogs have nice content, but are annoying to follow without RSS.

Every time I stumble on one of these, it takes me a while to remember a fun workaround. I decided to write this down as a blog post so I have an easier time looking up the details.

My blog is no longer hostage to some CSS tooling

Sat, 12 Oct 2024 13:04:48 +0300

I’m switching my blog over to a new hugo theme. It looks slightly nicer now, but migrations are hard and there will be some broken posts.

Background

When I was at Mozilla back in pre-2014 years I blogged on their corp wordpress setup. It was awful. I hated the WP editor, I hated the WP themes, I hated the WP plugins.

I had cool blog posts on there, but they were all deleted when I left. I managed to get a backup, but never found the energy to restore it. The posts are on tech which is long obsolete, so the motivation is decreasing with time. I did spent the time to rewrite the coolest blog post I had on there on tracing how page faults work while taking the time to switch from SystemTap to Ebpf.

PSA: eget That Executable From GitHub

Sat, 07 Sep 2024 15:41:39 +0300

tldr: GitHub is where most CLI tools live. eget is a tool that makes it easy to download and install their binaries.

My Black and White Reading Setup

Thu, 05 Sep 2024 12:40:08 +0300

If you’re happy reading on your laptop or Apple device, you can skip this post.

llama.cpp: use local models with chatcraft

Mon, 08 Jul 2024 10:26:27 +0200

I needed a way to do some programming while offline. These days I feel very unproductive without https://chatcraft.org (the best chat UI for programming) and a good LLM to chat with about coding.

Chatcraft needed a few small fixes to enable llama.cpp support. Here’s how to run models with llama.cpp with chatcraft.org without internet:

Instructions

Install and run llama.cpp. Follow https://github.com/ggerganov/llama.cpp instructions for your platform.

For mac:

# install llama.cpp
brew install llama.cpp
# start llama.cpp server & auto-download a good small (~6GB) model
llama-server    --hf-repo "bartowski/Llama-3-Instruct-8B-SPPO-Iter3-GGUF"       --hf-file Llama-3-Instruct-8B-SPPO-Iter3-Q6_K.gguf
# For more advanced usage I recommend gemma 27b: the smallest smarter-than-gpt-3.5 model (~21GB)
# llama-server    --hf-repo bartowski/gemma-2-27b-it-GGUF       --hf-file gemma-2-27b-it-Q6_K_L.gguf

Setup local chatcraft dev env by following the instructions in the chatcraft repo

git clone https://github.com/tarasglek/chatcraft.org/
cd chatcraft.org
pnpm install
pnpm dev

^ will output a development url like http://localhost:5173/, open it. Go to chatcraft settings and add http://localhost:8080/v1 to api providers. Enter a dummy api key.

Enjoy!

Fast groq-hosted LLMs vs browser jank

Sun, 19 May 2024 10:23:07 +0300

TLDR: chatcraft.org is now smooth as butter and maximally performant thanks to throttling LLM chat-completion rendering via requestAnimationFrame().

Developing markdown.download: Exporing val.town, deno, jsr.io

Sun, 24 Mar 2024 20:01:23 +0200

I read a lot. I enjoy reading on black and white e-readers. Unfortunately many websites make it hard to read them on simple devices. https://markdown.download is my really simple solution to that. Prepend it to any website and curl:

curl https://markdown.download/https://dev.to/amnish04/introducing-the-idea-of-web-handlers-in-chatcraft-1b4i

That turns a user-hostile website into a thing that almost anything can render:

It’s amazing how much cleaner the web is when stripped down to essence.

Turns out markdown (or html restricted to what markdown can do) renders fantastic on readers of all types.

Since markdown.download is so utterly simple, I decided to try using it to learn a new-to-me dev tool: val.town. Rest of this post covers what I learned.

I initially was going to publish this project as an npm lib and deploy on cloudflare, but was putting it off because I loathe the amount of bureaucracy that involves. I love the type system in Typescript. I love Node.JS because it’s the least-bad way to write/deploy apps, but much room for improvement remains. My spare time is too valuable to spend it suffering.

val.town: brutally efficient microservice dev-ux

ssh-via-cloudflare-tunnel: Alternative way to expose machines behind NAT

Sun, 10 Mar 2024 14:29:26 +0200

I have an intermittent issue where one of my machines on a specifc network seems to be operational locally on network but not accessible over netbird some of the time. I could not tell if the problem was due to the network the problematic machine was on or due to netbird.

So to debug this I:

Setup a grafana node agent to report journald logs to https://grafana.com/
Setup to docker to log to journald
Wrote ssh-via-cloudflare-tunnel docker-compose stack to tunnel ssh over websocat to internet over cloudflared. Set docker compose to restart-on-failure.

Today netbird had some downtime (tailscale had theirs just a few days before that). I was able to log into grafana, look up the temporary hostname cloudflare assigned to me and ssh in using

ssh -o ProxyCommand="websocat -E --binary - -v %h" -o ServerAliveInterval=10 wss://library-won-nt-gauge.trycloudflare.com

Architecture

graph TD;
    A[ssh client] <-->|Invoke websocat over ProxyCommand| B[websocat client-mode]
    B <-->|WebSocket to Stdio| D(Cloudflare)
    D <-->|HTTPS to WebSocket| E[cloudflared Server]
    E <-->|Reverse proxy over HTTPS| F[websocat server-mode]
    F <-->|Stdio to SSH| G[sshd server]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style B fill:#dfd,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px,
    style E fill:#ddf,stroke:#333,stroke-width:2px,
    style F fill:#dfd,stroke:#333,stroke-width:2px
    style G fill:#f9f,stroke:#333,stroke-width:2px

Conclusion

This turned out trivial to do. However it took me over 5 years of pondering, discovering ssh ProxyCommand, cloudflared, websocat and deciding to combine them to conveniently expose ssh over web.

I think it’s pretty cool how Cloudflare allows one expose geographically distributed tunnels without requiring any signups. I wish their UX simpler so it was equally easy to expose tunnels with fixed dns. Cloudflare actually has something similar as an official ssh feature, but it requires more hoop jumping.

websocat is amazing. I use websockets a lot more now that I have a robust way to utilize them without writing any code.

Why Would I Want To Do This?

As a fallback to tailscale/netbird that uses a completely different network topology as in this case
To temporarily grant ssh to some machine without requiring end-users to install a vpn
To bypass restrictive gsm/wifi networks that block access to ssh
As an alternative to port-knocking to obscure ssh (eg hide ssh behind http basic-auth)

How would you improve this?

Checkout ssh-via-cloudflare-tunnel on github.

github-to-sops: Easy way to manage passwords/keys with Github and SOPS

Wed, 10 Jan 2024 14:58:46 -0800

2025 Disclaimer: I used AI to write some of this blog post. I regret this choice.

Let’s face it, managing secrets in software projects can be as thrilling as being stabbed in the eye. Yet, it’s a necessary evil that we all have to deal with.

Problem: we have a set of a developers and set of infrastructure that all needs to share secrets. Would like to minimize infrastructure and keep cognitive load to a minimum so we can focus on writing code. Sure you’ve got AWS Secrets Manager and Hashicorp Vault for the heavy lifting, but that’s like using a tractor to crack a nut. And then there’s the keep-all-your-secrets-in-github-action-ENVs which leads to “push-and-pray” mentality (https://dagger.io/ talks on CI/CD are awesome). Not exactly the pinnacle of security or convenience, right?

Enter SOPS, the cool kid on the block that encrypts your files without the bullshit (You know it’s cool cos it’s the latest in a long line of tech abandoned by Mozilla). But setting it up? Still sucks. This post is about how github-to-sops helps.

Lightweight Virtualization Metallize Libkrun Vsock

Wed, 30 Aug 2023 12:37:38 +0300

libkrun + krunvm

Github randomly recommended me libkrun which is a library backing krunvm. It’s something similar to firecracker, but even simpler.

Trying pixi: Modern package management for Python

Fri, 25 Aug 2023 17:04:48 +0300

I have been working with Python a lot more recently, and it feels like I spend more time fighting packaging than writing code.

Python’s primary package manager, pip, is roughly equivalent to the best 1990s had to offer(Perl CPAN), it makes it depressingly easy to end up with a broken environment.

Pixi: A modern packaging system for Python

pixi is a modern package manager along the lines of deno/pnpm, but for Python. It’s a single binary that you can download and run. It will install Python + native packages within a single subdirectory. It will use pixi.toml file to track dependencies + pixi.lock to track exact versions of transitive dependencies.

Overlooked on HN: Databases That Process Data Faster Than Memory Bandwidth

Mon, 07 Aug 2023 12:44:29 +0300

13 GiB/s per core!

Sneller posted a blog on HN on how they use AVX-512 to decompress data at 13 gigabytes per second per core.

This a fantastic ad for their “lets turn logs on S3 into cheap database” product. This is a solution I wanted multiple times, will definitely consider them next time the need comes up.

Faster than RAM

Now this post did not get overlooked, but what did get overlooked is that the post engaged the clickhouse CTO. He posted a link to a presentation on how Clickhouse uses compression to process in-memory data faster than RAM bandwidth.

As a result of discussion in these comments, clickhouse might get even faster.

Overlooked on HN: Discovering High-quality Technical Content

Mon, 07 Aug 2023 10:45:27 +0300

I’m gonna start a column on cool blog posts I found, that got 0 or minimal traction. I suspect I will also have no traction doing that 🤦‍♂️.

The Problem

I really enjoy thoughtful writing on deep technical problems. It’s even better when one sees thoughtful comments, that further contribute new directions to throughts presented. HackerNews is where most of that writing tends to land. Unfortunately it tends to not do well vs trendy, click-baity, etc content. Twitter is even worse.

First, a blog post on my tooling for reading HN.

Ukrainian Internet Fun + OpenAI

Sun, 23 Jul 2023 11:43:25 +0300

The Problem

Ukraine had a lot of power outages due to Putin’s bombing of our power infrastructure, I needed to switch to fiber + battery-backup to continue able to able to work.

I’m at a rental apartment and I’m not allowed to drill walls. The idiot that layed the internet into the apartment used a 4-wire cat5 cable to save a few pennies, then cemented it in.

Analyzing Github Stats via Clickhouse via Chat

Sun, 09 Jul 2023 20:43:03 +0300

TLDR: I prompt-engineered the system prompt in chatcraft to turn it into a github analytics tool: github analyst chat.

Longer Story

I suggested to David that we have a newsfeed of new features on chatcraft.org. He replied that we should try to use chatcraft.org to generate those.
Found simonw’s blog post on using a public clickhouse instance.

From Chaos to Control: Overcoming OpenAI Uncertainties with Local Models

Wed, 14 Jun 2023 11:44:09 +0200

chatcraft.org is my open source project for working with GPT. It completely changed how I work. Cool thing about chatcraft is that it’s completely client-side, almost completely stateless(except for sharing) on serverside.

I am also working on a project that uses LLMs to help navigate a knowledge base. Making a wrong tech choice there could kill my project. Few thoughts from that perspective:

The 3 YOLOs of LLM development

“You only live once” (YOLO) is a modern adaptation of the Latin phrase “Carpe diem,” which means “Seize the day.”

Always Bet on Geohot: Tinygrad Will Win

Thu, 25 May 2023 11:23:14 +0300

George Holtz (geohot) just raised $5M for tinygrad and plans to sell a $15,000 Machine Learning AMD Epyc box with AMD 7900 XT video cards. At first glance, this seems like a really risky investment into a low-margin commodity hardware company. AMD can’t compete with NVidia, so how can we expect geohot to compete with NVidia on AMD hardware?

Well, I’ve been following geohot for a long while. He delivers what he promises, but does it years later and on different hardware than initially promised :) Here’s the timeline of my evolution alongside George that informs my conviction:

A Tale of PGVector and Friends: The Joy Of Software Naming

Tue, 23 May 2023 18:20:05 +0300

Ah, the world of software! A place where creativity and collaboration come together to create amazing projects. But sometimes, this world can be a bit… confusing. Let me tell you a story about the frustration of open source naming, featuring our protagonist: PGVector.

Once upon a time, I though that pgvector was a postgresql extension. And that langchain did not support SQL storage out of the box. After reading the source, turned out there were three things named PGVector:

Chatcraft.org: Developer-focused Open Source ChatGPT

Mon, 08 May 2023 08:40:49 +0300

I would like to introduce ChatCraft.org: a developer-focused, open source chatgpt frontend. While it’s my fav GPT UI (as it was created to make my life more convenient), it’s not so much about the features that are complete now as about the fun, collaborative journey ahead. I think we need to explore the design space of GPT-assisted collaboration to write code. The best way to do that is via open source exploration of the concept!

GPT-ARIA: Integrating GPT-3 with Browser Using Accessibility API

Thu, 30 Mar 2023 15:09:04 +0200

https://github.com/thegpvc/gpt-aria

This was the most interesting prototype I’ve explored in my career so far, and I want to thank TheGP for sponsoring this project. Also thanks to Oleksandr Chugai who I partnered with on implementation and Ben Cmejla who helped with prompting.

Inspiration

I’ve been thinking about browser automation since the early days of my career when I worked at Firefox, where I got to work on performance problems and built things like Firefox Telemetry (Hi, HN haters!). With the rise of large language models (LLMs)—and OpenAI’s recent launch of ChatGPT plugins — things are getting exciting around user agents again. My favorite demo so far is natbot, Nat Friedman’s GPT-3-powered browser agent that he launched last October.

Trying Cloudflare Pages: Best Server Tech Since cgi-bin

Sun, 23 Oct 2022 14:28:15 +0300

Cloudflare Pages 😍

I got into Cloudflare pages because it lets you host static websites with all the latest optimizations conveniently and for free. Setting up a static website with a custom domain is easy, and you get the benefits of automatic SSL, CDN, and latest version of HTTP3.

This is how this blog is hosted and I have a few other projects like this. I love the fact that I can get a super-fast static website up and running in a few minutes and I don’t have to worry about infrastructure details.

Curious Case of Maintaining Sufficient Free Space with ZFS

Sun, 10 Jul 2022 12:25:59 +0300

TLDR: ZFS free-space reporting is a lagging indicator.

Background

I use Proxmox VM server backed by a ZFS array of hard drives for various personal infrastructure. I also have security cameras that upload motion-triggered videos to my server (via FTP!).

Problem Description

I would like to use 90% of my available space for most-recent security videos.

Recipe:

Create a dedicated ZFS volume
Setup ZFS quota
Run a cronjob to free space faster than it gets consumed by video uploads.

I set out to find a decent disk-freeing solution. I eventually settled on a python free-disk. (Most of these utils were written in bash, didn’t want to disk having to grasp & modify write-only code).

mmap Page Fault Tracing with bpftrace and ext4

Tue, 15 Mar 2022 13:26:36 +0100

Previous blog post on how to trace Firefox IO using bpftrace via official page_fault_user tracepoint left me a bit unsatisfied with how complicated it turned out. Complexity has potential to be error-prone and the syscall-tracing dependency makes it impossible to trace IO within the main executable.

I decided to try reimplement the trace using my old approach of tracing ext4 functions that handle page-faults. This turned out to be much more robust. This is now documented on my github. It’s ugly in that it’s dependent on internal kernel structures, but it catches 100% of the IO and requires no post-trace syscall-correlation/fudging.

EBPF for Tracing How Firefox Uses Page Faults to Load Libraries

Tue, 22 Feb 2022 12:29:10 +0200

Modern browsers are some of the most complicated programs ever written. For example, the main Firefox library on my system is over 130Mbytes. Doing 130MB of IO poorly can be quite a performance hit, even with SSDs! :).

Few people seem to understand how memory-mapped IO works. There are no pre-canned tools to observe it on Linux, thus even even fewer know how to observe it. Years ago, when I was working on Firefox startup performance, I discovered that libraries were loaded backwards (blog1, blog2, paper, GCC bug) on Linux. Figuring this out was super-painful, involved learning SystemTap and a setting up a just-right kernel with headers and symbols.

Reading NFS at >=25GB/s using FIO + libnfs

Wed, 24 Nov 2021 03:42:10 -0800

My current employer does a lot of really cool systems work that’s covered by NDAs. I recently did some work to integrate a cool open source tool into our workflow. Felt it deserved a blog post.

NFS Testing Requires Parallelism.

I work for Pure Storage. One of the products we make is a scale-out NFS¹ (and S3-compatible) server called FlashBlade.

I was asked to test FlashBlade² performance scaling. I needed to generate NFS read workloads of 15-300 Gigabytes/second. Given that NICs in our lab max out at 100Gbit, (~10GB/s), this required a lab with fast servers and multiple NICs per server. I wanted to make maximum use of hardware to minimize lab space + cost. To make use of multiple NICs I needed multiple NFS connections per NIC. Linux does not make this easy³.

Firefox's Optimized Zip Format: Reading Zip Files Really Quickly

Mon, 22 Nov 2021 06:32:53 -0800

This post is about minimizing amount of disk IO and CPU overhead when reading Zip files.

I recently saw an article about a new format that was faster than zip.

This is quite surprising as to my mind, zip is one of the most flexible and low-overhead formats I’ve encountered.

Some googling showed me that over past 11 years people have noticed that Firefox uses optimized zip files. This inspired me to document thinking behind the optimized zip format I implemented in Firefox in the pre-pandemic 2010. I had a lot of fun writing this code, was surprised that I failed to blog about it.

About

Fri, 02 Aug 2019 11:04:49 +0800

About

Why Google Pixel lags 10x more than Moto Z

Sun, 11 Dec 2016 12:17:23 -0800

In my previous post I made an argument that a modern phone is only as fast as the slowest component: ability of NAND to handle 4k writes. I decided to compare two Android flagships on the opposite ends of random-write-4k benchmark spectrum: Moto Z vs Google Pixel.

I wrote a little fio benchmark driver to fill all available device storage with random 4k writes, print perf stats along the way. Idea is to run the benchmark on /data/ partition, then fill all available space by writing to /storage/emulated/0, then do another round of testing on /data.

Laggy phones and misleading benchmarks

Sat, 12 Nov 2016 12:16:32 -0800

TLDR: You can predict degree of unresponsiveness of a phone via random-write-4k benchmarks. I wish review websites would fill phones to 80-90% prior to running the benchmark, especially on smaller-capacity phones where users are more likely to run out of space.

SQLite vs Phone NAND

I’ve long held a theory that Android lag is almost directly determined by slowness induced by SQLite transactions. This weekend, while researching phones for a family member, I found some supporting evidence.

Using Economics to Help Ukraine Win

Mon, 01 Jan 0001 00:00:00 +0000

I spent most of my life in North America, but I have been living this war in Ukraine. My hope is that I can use my past experience and connections to help Ukraine win this war.

Recently a friend of mine asked what would be the best way to donate $1000. This got me thinking.

Wars are won by convincing the opponent that things are only gonna keep getting worse for them and better for you. Economics is the best way to do that. Germany, Japan could not sustain their aggression in WW2 because they were losing the war economically. USA played a large role in WW2 with amazing logics and manufacturing.