Are we being good ancestors?

— Jonas Salk, referenced from The 10,000-year clock below

The 10,000-year clock will mark time with astronomic and calendric displays and a chime generator designed with the help of Brian Eno that can produce over 3.5 million unique bell chime sequences — one for every day the clock is visited for the next 10,000 years.

Ten thousand years is about the age of modern civilization, so the clock will measure out a future of civilization equal to its past. This assumes our civilization is in the middle of whatever journey we are on — an implicit statement of optimism.

The 10,000-year clock

How do you build a clock that will last 10,000 years? Look at the list given in the article above:

Longevity

10,000-year clocks should display the correct time for the next 10,000 years

• Go slow
• Minimize sliding friction
• Stay clean and dry
• Expect bad weather and earthquakes
• Expect non-malicious human interaction
• Don’t tempt thieves

Maintainability

It should be possible to maintain a 10,000-year clock with little maintenance, using bronze-age technology

• Use familiar materials
• Make it easy to build parts
• Include the manual

Transparency

It should be possible to determine the operational principles of a 10,000-year clock with close inspection

• Allow inspection
• Allow rehearsed motions
• Expect restarts

Evolvability

It should be possible to improve a 10,000-year clock over time

• Separate functions
• Provide simple interfaces

Scalability

It should be possible to build working models of a 10,000-year clock from table top to monumental size using the same design

• Make all parts similar size

Some of these sound similar to software, no?

• Make it easy to build parts
• Include the manual
• Separate functions
• Provide simple interfaces

Indeed, many fundamental principles of nature are true across domains.

This latest release is called κατακλυσμός, and for the savvy transliterator, you can see that means cataclysm!

But its real meaning in Koine Greek is more like flood, which is a better description for this release. The biggest single change, though, is once again another plugin refactor — and we’ll probably do it again. Getting those abstractions right will allow Kai to be designed for the Long Now, and hopefully grow organically into something great that can stand the test of time.

Please note: SOPS/system-level commands modify system state and that and the ISO command is very experimental. Use them with caution on disposable machines as this is still a prototype!

Highlights:

• Refactored the plugin schema (again)
• Lots of ergonomics and UX infrastructure
• Tests for every command implementation!
• Better errors and Nix error bubbling
• system-level commands to control your Nix config!
  • generations/rollback/switch
• ISO create command (for an eventual KaiOS…?)
• SOPS secrets
• backend-scoping and passthrough

Refactoring the plugin schema (again)

Kai is built via plugins similar to how Caddy works. The standard (and currently only useful) plugin, std, is compiled using xkai, which, like xcaddy, just bundles the std package together and provides a generic CLI skeleton around it. As noted in previous posts, getting this plugin structure right is crucial to Kai’s essential goals of being generic, flexible, and modular. The single biggest refactor is changing the schema design to more accurately reflect the separation between commands, blocks, and backend implementations.

Ergonomics

Kai didn’t have much in the way of a help menu, and errors literally just spit out raw Roc constructs (gross!) which made them difficult for humans to read.

Help Menu

Finally, a proper help menu!

kai
A friendly frontend for determinate computing

Usage:
  kai [OPTIONS] <COMMAND> [ARGUMENTS] [--json]

Kai is a tool for providing a simplified interface on top of determinate
systems (mainly Nix) for ease of use. Commands often correspond with a
Kaifile block defining their behavior:

  $ kai shell
  $ cowsay "Hello from Kai!"

  # Kaifile
  shell {
      packages: ["cowsay"]
  }

Commands:
  build     Build an artifact declared in the Kaifile.
  deploy    Build and activate a machine on its declared target.
  image     Build an image from a machine declared in the Kaifile.
  iso       Build a bootable ISO from a machine declared in the Kaifile.
  machine   Build a machine declared in the Kaifile.
  run       Run a task declared in the Kaifile.
  service   Build a service declared in the Kaifile.
  shell     Enter an inline or declared developer environment.
  system    Manage machine-wide state.
  update    Update and lock project dependencies.
  workflow  Run a workflow declared in the Kaifile.
  version   Print version information.

Options:
  -f, --file <PATH>  Use the Kaifile at PATH
      --json         Output JSON Lines
      --no-color     Disable colored output
  -y, --yes          Assume yes for confirmation prompts.
  -h, --help         Print help

Environment:
  KAI_DIR             Project-local workspace directory (default: .kai)

More information: https://github.com/thebrandonlucas/kai

CLI examples on each help menu

Each subcommand has a help menu with its equivalent Kaifile block example if applicable:

kai help build
Build an artifact declared in the Kaifile.

Usage:
  kai [OPTIONS] build [BACKEND] <ARTIFACT> [--json]

Examples:
  kai build <my-artifact>

Kaifile block:
  build my-artifact {
    environment: dev
    run: ["make"]
    output: "result"
  }

Arguments:
  BACKEND  Optional backend name
  ARTIFACT  Artifact name from the Kaifile

Options:
  -f, --file <PATH>  Use the Kaifile at PATH
      --json         Output JSON Lines
  -y, --yes          Assume yes for confirmation prompts.
  -h, --help         Print help

--json Outputs

clig.dev has a great entry on how you should design everything for humans and machines (so tools can be composable), so we have a --json flag that should work for each command:

kai help build --json
{"type":"help","message":"Build an artifact declared in the Kaifile.\n\nUsage:\n  kai [OPTIONS] build [BACKEND] <ARTIFACT> [--json]\n\nExamples:\n  kai build <my-artifact>\n\nKaifile block:\n  build my-artifact {\n    environment: dev\n    run: [\"make\"]\n    output: \"result\"\n  }\n\nArguments:\n  BACKEND  Optional backend name\n  ARTIFACT  Artifact name from the Kaifile\n\nOptions:\n  -f, --file <PATH>  Use the Kaifile at PATH\n      --json         Output JSON Lines\n  -y, --yes          Assume yes for confirmation prompts.\n  -h, --help         Print help"}

I should probably add a --pretty flag or something for humans to be able to use the JSON version if they’d like, but we’ll wait and see if that use case gets justified.

ANSI

We added ANSI coloring infrastructure. Just the very basics, bold for headers, red for errors, etc. The base is there and we can easily add/modify as needed.

Errors

Kai Errors

I think the correct philosophy around error message design is clig.dev’s:

One of the most common reasons to consult documentation is to fix errors. If you can make errors into documentation, then this will save the user loads of time.

Catch errors and rewrite them for humans. If you’re expecting an error to happen, catch it and rewrite the error message to be useful. Think of it like a conversation, where the user has done something wrong and the program is guiding them in the right direction. Example: “Can’t write to file.txt. You might need to make it writable by running ‘chmod +w file.txt’.”

Signal-to-noise ratio is crucial. The more irrelevant output you produce, the longer it’s going to take the user to figure out what they did wrong. If your program produces multiple errors of the same type, consider grouping them under a single explanatory header instead of printing many similar-looking lines.

Consider where the user will look first. Put the most important information at the end of the output. The eye will be drawn to red text, so use it intentionally and sparingly.

If there is an unexpected or unexplainable error, provide debug and traceback information, and instructions on how to submit a bug. That said, don’t forget about the signal-to-noise ratio: you don’t want to overwhelm the user with information they don’t understand. Consider writing the debug log to a file instead of printing it to the terminal.

Make it effortless to submit bug reports. One nice thing you can do is provide a URL and have it pre-populate as much information as possible.

That last one wasn’t considered in this release, but I do love the idea and will add it in the future.

Now, errors that used to look like this:

Program exited with error: PlanningFailed({ backend: "nix", command:
"build", location: None, message: "build requires exactly one config
name", plugin: "std" })

Now, in the new release, look like this:

error: build requires exactly one artifact argument
usage: kai build <ARTIFACT>
example: kai build <my-artifact>

Much better!

Nix Error Bubbling

Since the std plugin in Kai sits on top of Nix, there will always be a subset of errors that Kai won’t be able to capture itself, since there can always be drift between how Kai mirrors the underlying implementation and we’ll never be able to have a perfect one-to-one error map. For example, if the user defines a package foo that is not in nixpkgs, Kai will presently still just write the flake.nix file and try to evaluate it under the hood, resulting in one of those ugly gargantuan Nix errors like:

error:
       … while calling the 'derivationStrict' builtin
         at «nix-internal»/derivation-internal.nix:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'nix-shell'

       … while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'foo' missing
       at /nix/store/…-source/flake.nix:7:15:
            6|       packages = [
            7|               nixpkgs."legacyPackages"."x86_64-linux"."foo"
             |               ^
            8|       ];
       Did you mean one of fio, fjo, folo, foot or fop?

This violates the rules above by showing so much information internal to Nix itself rather than the user’s code, where the violation actually lies. That should be up front and immediately obvious.

The real error is just this:

error: attribute 'foo' missing
at /nix/store/…-source/flake.nix:7:15:
    6|       packages = [
    7|               nixpkgs."legacyPackages"."x86_64-linux"."foo"
     |               ^
    8|       ];
Did you mean one of fio, fjo, folo, foot or fop?

So we attempt to strip this out, but more work/experimentation will need to be done.

Features

import blocks

Kaifiles can now import each other! This is not a whole module system like a programming language, but rather basically just string concatenation under the hood similar to Caddy. We really just needed a way for large Kaifiles to be expanded.

Kaifile imports work like this:

# parts/environment.kai
environment dev {
  packages: ["cowsay", "fortune"]
}

Then, Kaifile:

import "parts/environment.kai"
shell {
  environment: dev
}

system Command and Subcommand Groups

Why should Kai only be used at the project level? What if you wanted to use it to configure your whole system like NixOS does? What if we could start working on combining the UX and flash of Omarchy with the power of NixOS? The first step toward this is to group commands which do this under the system command (i.e. command which manipulate the system). This is the first command which has a few subcommands:

WARNING: These commands are experimental and can modify your system! Advised to use only on throwaway machines.

kai system
Manage machine-wide state.

Usage:
  kai system <COMMAND>

Commands:
  generations  List local system generations.
  rollback     Roll back the current host one system generation.
  switch       Build and activate the declared machine.

One of the most powerful features of NixOS over other operating systems is the ability to rollback your machine’s state. Since most things in NixOS are declarative in a version-controlled file, this provides an enormous advantage over other operating systems which quickly become a tangled layered mess descending into dependency hell. Nix’s rollback allows you to switch between generations of your system’s state if anything goes wrong, or you just liked how one of your previous versions worked better. Paired with a solid UX, I expect this concept to unlock many novel use cases for people that aren’t fully utilizing their NixOS systems. At present, our rollback just goes back to the previous generation, the most useful command to recover from brokenness caused by an upgrade.

The other command added, switch, is what actually does the activation of your Nix derivations to change the state of the software. Under the hood this is nixos-rebuild switch and is probably the most essential and commonly used NixOS command.

Deploy

Another big use-case unlock. If you have a machine in the cloud, it’s extremely convenient to be able to deploy a new config at will. I’ll be adding this to my personal websites — as soon as I can configure them completely using Kai on NixOS machines (more to come!):

kai help deploy
Build and activate a machine on its declared target.

Usage:
  kai [OPTIONS] deploy [BACKEND] <MACHINE> [--json]

Examples:
  kai deploy <machine>
  kai deploy -y <machine>

Kaifile block:
  machine <machine> {
        environment: server
        system: "x86_64-linux"
        target: "root@example.com"
  }

Arguments:
  BACKEND  Optional backend name
  MACHINE  Machine name from the Kaifile

Options:
  -f, --file <PATH>  Use the Kaifile at PATH
      --json         Output JSON Lines
  -y, --yes          Assume yes for confirmation prompts.
  -h, --help         Print help

ISO

WARNING! Experimental, please don’t use on any important machines yet.

Another path to an Omarchy-but-with-Kai is an ISO. With a valid machine config, you should be able to boot into a valid NixOS machine from a Kaifile with this ISO on a flash drive! Eventually, this’ll be spruced up to give a much better UX.

SOPS Secrets

WARNING! Experimental, please don’t use on any important machines yet.

We added sops-nix, a standard tool for safely integrating encrypted secrets in Nix/Git. This is a bit limited as at the moment you have to encrypt the secret — Kai only accepts already-SOPS encrypted secrets and doesn’t handle that itself. However, this unlocks being able to deploy machines with predefined secrets on them.

Backend Escape Hatch

I wanted to allow the NixOS options assignments in flakes resulting from Kaifiles. This is a tricky problem because Nix has a lot of custom options keys for tons of services. Obviously Kai can’t re-hardcode all these. Further, there are some concepts which Kai is tying itself directly to Nix for, like overlays, which Guix doesn’t have (though it does have analogous concepts).

At present, I’m unaware of a good way around the “non-overlapping concepts” problem for backends which should allow the Kai frontend to remain seamlessly usable. So the best approach I could find is to provide an escape hatch for these scenarios:

machine server {
 environment: server
 system: "x86_64-linux"
 users: ["agent"]
 services: ["openssh"]

 backend nix {
   networking.hostName: "kai-server"
   services.fail2ban.enable: true
   networking.firewall.allowedTCPPorts: [22, 443]
 }
}

Everything under backend nix is a restricted set of typed assignments that “lowers” Kaifile code into flake.nix. This is a big flexibility leap forward for Kai which will allow the base cases to remain separately abstracted from the bespoke code in the short term. Long-term, we’ll probably have to find something better.

Global Backend Config

One repo I was attempting to add Kai to did not use the standard nixpkgs branch but rather nixos-unstable-small, a use case I hadn’t accounted for yet. While I’d ultimately like a large set of decisions like this to be configurable by the Kai binary, I think it’s also useful to have it available as a declarative config option. This is of course backend-specific, so we also scope this by its backend:

backend nix {
 packages: "github:NixOS/nixpkgs/nixos-unstable-small"
}

Scoped Architecture Predicates

Similar to the above, this repo also required architecture-specific config. We already had OS-level config e.g.:

on linux {
    ..
}

on macos {
    ..
}

But this doesn’t provide the full host picture, as you also have architectural variation:

on linux x86_64 {
    ..
}

on macos aarch64 {
    ..
}

Future

With all these features, Kai’s surface area is getting bigger. There are a growing number of commands that do very different things. Unix Philosophy says that tools should do one thing well. But the legendary Unix designers had an early-mover advantage in that they got to define much the set of tools that we use today, and therefore they didn’t have to operate as strictly within the set of available tooling. If Kai took the approach of splitting up every concern into sub-binaries, we would both have to ship a binary for every little tool rollback, deploy, switch, generations etc. and you’d have to know that each one of these things was Kai-specific. Nix does some of this with nixos-rebuild etc., but I think in the modern world we don’t want to have to redefine entire systems if we can get the majority of benefits without doing it. Swiss Army knives are closer to the mark for largely capable tools like this.

So I do think packaging things together with the plugin approach is the right one — different plugins can ship different sets of features: we could have a kai-slim, kai-full, one for project-level configs only, one for full OS management, one for Nix, one for Guix, etc.

That said, the plugin system is not being fully utilized: We are neither building out the Guix plugin system nor generalizing std to allow switching between Nix and Guix seamlessly.

I think this will be essential to avoid overfitting ourselves to Nix-only backends. While tempting in the short term, an agnostic design will pay huge dividends over time.

I’m also realizing that Nix can do lots of things and is very flexible. As such, people use it in a thousand different ways, which is something some love and others hate. This directly conflicts with the “one canonical way to do things” approach, which makes it far easier to create standards and form community (which, case in point, Nix really struggles with!).

This is an early canary — and one that we’ll have to pay close attention to mitigating in the future.

Alan Kay’s pithy phrase remains evergreen in my head such that I’m tempted to make it the philosophical motto of the project:

Simple things should be simple, complex things should be possible.

— Alan Kay

Achieving both, though, ain’t easy!

But what great things worth doing ever were?

Onward!

In the beginning all you want are results, in the end all you want is control.

Eskil Steenberg, How I Program in C

Life is long enough, and it has been given in sufficiently generous measure to allow the accomplishment of the very greatest things if the whole of it is well invested.

Seneca, On the Shortness of Life

Most European cathedrals show differences in plan or architectural style between parts built in different generations by different builders. The later builders were tempted to “improve” upon the designs of the earlier ones, to reflect both changes in fashion and differences in individual taste. So the peaceful Norman transept abuts and contradicts the soaring Gothic nave, and the result proclaims the pridefulness of the builders as much as the glory of God. Against these, the architectural unity of Reims stands in glorious contrast. The joy that stirs the beholder comes as much from the integrity of the design as from any particular excellences. As the guidebook tells, this integrity was achieved by the self-abnegation of eight generations of builders, each of whom sacrificed some of his ideas so that the whole might be of pure design. Even though they have not taken centuries to build, most programming systems reflect conceptual disunity far worse than that of cathedrals […] I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.

Fred Brooks, The Mythical Man-Month

This update might be a bit unusual, as I want to write an update but also some a little more personal about my experience of programming.

There are two things that are true of Kai that were never true of a programming project for me before:

1. I’m building something from scratch as its progenitor, that
2. Is completely open source, that
3. I want to last a long time

I’ve taken serious aim at all but (3) before, but the longer I live the more Karl Popper’s maxim of working on long-term problems speaks to me:

I think that there is only one way to science - or to philosophy, for that matter: to meet a problem, to see its beauty and fall in love with it; to get married to it and to live with it happily, till death do ye part - unless you should meet another and even more fascinating problem or unless, indeed, you should obtain a solution. But even if you do obtain a solution, you may then discover, to your delight, the existence of a whole family of enchanting, though perhaps difficult, problem children, for whose welfare you may work, with a purpose, to the end of your days.

Karl Popper, Realism and the Aim of Science

By taking Kai seriously, by striving to make something that provides as close to a strictly better experience in using computers (to the extent achievable, there’s No Silver Bullet), I am forced to think harder than I ever have about what I’m building and why, the alignment between my highest-level end-goals and day to day tasks, how to make incremental progress every day which is aligned with these goals, how to maintain motivation, etc.

The genuine attempt to build something great is a forcing function for practices which lead to mastery.

As such, I’ve been doing a lot of reading (and more writing than I’ve ever done before!).

Sometimes I make mistakes or take some shortcuts of course. I think it can be very helpful to make something badly at first and just get it out there, share it, and get feedback from other really smart people in the community.

Kai is already starting to get fun, real, and useful, so I’m working to ensure it stays that way for a long time.

There’s been two releases since the last update: προιεναι - 0.0.5 (means “to proceed”) and ετυγχανε - 0.0.6 (means “it happened”), and a lot of changes in them.

So many changes, where to start? Let’s just follow Stephen King’s advice for novels on this one, and just skip the boring stuff.

External Features!

Let’s start with the fun stuff relevant to you, dear reader. Here are the new Kai commands added to the StdPlugin since the last update.

Source

Not all things that a project needs are always in nixpkgs. Sometimes you need to declare an external link to a URL tar or something, as was the case for me with aion (also built in Roc), where I had to declare bitcoin-qr (a QR code NPM package I wrote for adding bitcoin payments to your site) URL as a source as this isn’t on nixpkgs. This is really useful and makes Kai far more flexible!

source bitcoin-qr {
  url: "tarball+https://registry.npmjs.org/bitcoin-qr/-/bitcoin-qr-1.4.1.tgz"
}

build aion {
  environment: cli
  source: "src/aion.roc"
  output: "aion"
}

Machine

The machine block allows us to define full NixOS closures which we can then deploy or activate (for already-extant NixOS machines), which when combined with Kai can run services on startup or have particular packages installed.

Invoke with kai machine <name>

# Build a typed NixOS deployment closure with `kai machine agent`.
environment server {
  packages: ["curl"]
}

machine <name> {
  environment: server
  system: "x86_64-linux"
  users: ["<name>"]
  services: ["openssh"]
}

Image

That’s great, but what if we don’t already have a machine and want to create a bootable NixOS image? You can do that too via the image command! Which builds a generic QCOW2 image from a machine definition. For example, kai image agent turns the agent machine in your Kaifile into a portable image that a cloud provider can import.

I was able to test and verify this works on Digital Ocean via aion (below). The definition looks the same as machine, but it is invoked via kai image <name>:

# Build a generic QCOW2 image from the same machine definition.
environment server {
  packages: ["curl"]
}

machine <name> {
  environment: server
  system: "x86_64-linux"
  users: ["<name>"]
  services: ["openssh"]
}

Service

Services are basically programs that run on startup and just continue running in the background. Caddy is a good example: You just want a web server that’s always on and routing requests to your sites. A crucial prerequisite if Kai is going to be useful in spinning up NixOS machines on desktop or in the cloud.

Example:

# A service turns an executable build artifact into a NixOS systemd service.
environment cow {
  packages: ["cowsay"]
}

build cow {
  environment: cow
  run: ["sh", "-c", "cp \"$(command -v cowsay)\" cow"]
  output: "cow"
}

service cow {
  artifact: "cow"
  secrets: []
  restart: on-failure
}

Secret

A secret is basically an environment variable in either your shell, or machine, etc. This allows people to have the same machine but have behavior change based on the environment variables they set or have them set API keys or whatnot that allow them to use services. For example, I used aion to spin up a machine that ran pi coding agent connected to my ppq.ai API key. So in one command I could spin up a NixOS machine based on a Kaifile that had an agent running on it!

Example:

environment cow {
  packages: ["cowsay"]
}

# Build the "cow" program artifact
build cow {
  environment: cow
  run: ["sh", "-c", "printf '#!%s\\ntest -s \"$CREDENTIALS_DIRECTORY/message\"\\nexec %s \"secret loaded\"\\n' \"$(command -v sh)\" \"$(command -v cowsay)\" > cow && chmod +x cow"]
  output: "cow"
}

# Secret values stay outside the Kaifile. For this example, put
# "classified moo" in /run/kai/secrets/message on the machine.
secret message {
  provision: runtime
}

# Systemd passes the file to the service without exposing its contents.
service cow {
  artifact: "cow"
  secrets: ["message"]
  restart: on-failure
}

Internal Updates

Plugin Overhaul (again)

I once more redid the Plugin system, and it still isn’t quite where I want it to be. Getting this right is really important because it’s the gateway to others adding plugins for Kai to create their own plugins and features outside of std. I won’t yet go into the details because it isn’t quite there yet, but suffice to say that I’ve spent a lot of time making this much more self-documenting better abstracted. My goal is to get to the point where new users can “just dive in” making and experimenting with their own plugins.

Tidy.roc

matklad has a wonderful article on automating desired patterns of behavior for your codebase via a tidy script they made at TigerBeetle. This is a really cool concept and one I wished I’d considered sooner. Basically, imagine you just wrote your own linter, not merely to enforce generic code style rules, but also to enforce invariants that you seek to ensure your custom rules are followed with every commit. This is essentially another additional layer of ensuring conceptual integrity for the project, for whatever invariants you care about.

I started simple in Kai, just adding a Tidy.roc which enforces:

1. Every code module has a top-level explainer comment
2. Limit line length to 80 (some really long strings were getting hard to read)

Tigerbeetle’s tidy.zig does so much more, of course, and some things I’d like to do, like ensuring large blobs don’t get committed (once they’re merged in that history, they’re hard to remove!).

But the real value is that I now have a script that I can edit to analyze my codebase on every commit and ensure nothing has violated a growing list specific list of invariants!

Guix Shell

The Mythical Man-Month talks a lot about the importance of maintaining Conceptual Integrity in a program, and much of my research and thinking around Kai has evolved around this concept. Conceptual Integrity is the idea that your program should be aligned conceptually around what it’s supposed to be doing, and that this drifts as programs grow over time and abstractions get leakier.

One suggestion he has for maintaining conceptual integrity is having multiple implementations as a means to do ensure the design does what you want and is not too tied to your specific implementation. After all, one thing we’re trying to explore with Kai is the bounds of a generic determinate system – we need not and should not repeat Nix’s mistakes or tie ourselves too closely to their architectural mistakes.

So, just as a proof of concept for now that Kai will indeed work with two systems, I added just a single Guix shell command, which proved out that the command/backend/implementation architecture works for both Nix and Guix!

You can compile it with nix run .#xkai -- build plugins/guix/GuixPlugin.roc.

Then run ./kai shell guix.

One day, I hope this flexibility will allow us to surgically replace problematic parts of Nix, in an incremental and backward compatible way, with our own implementation of a determinate system!

Fuzzer

I’ve never fuzzed code before, and I’ll admit to ignorance on a lot of the theory and practice, especially with regards to practical benefit. But with Luke Boswell’s new Roc fuzzer library, and given that it was just a small bit of code surface to run for my parser (which fuzzers are apparently pretty good for), I decided to give it a try. Right now it’s only implemented for a portion of the parsing code, but it’s nice to know I have a way of easily and automatically testing thousands of inputs to the parser!

Dogfooding Kai

So far I’ve created two in-progress greenfield projects with Kai with the robot friend. This has proven a lot of the use case for Kai. But I’ll also be adding it to extant projects like lyceum.quest, conllu.lyceum.quest, and this personal website, to prove that Kai can be used both for hosting and deployments, as well as developer environments and builds. Essentially, Kai should be able eventually to cover most common use cases for things that happen around the code, so that you can spend more time doing what you want to do – focus on the code itself.

Aion

I built a very vibe coded (do not use yet!) tool to dogfood Kai with that I wanted for myself, a way for me to spin up NixOS servers on DigitalOcean at-will which spin up machines according to their Kai configuration. With this I was able, for example, to spin up an agent that “just works” and allows me to build things on that box with environment secret variables, services, packages, and just declarative computer setups in general, via just a single Kaifile!

This feels like a huge milestone, because one of the goals of Kai is to be able to augment/replace the traditional interface of using NixOS. If you can easily spin up simple to read declarative, version-controllable operating systems, it’s a massive game-changer for the headache of maintaining and remembering how you set up software environments. Your operating system setup becomes self-documenting, portable, shareable, trivially modifiable, trivially rollback-able, and now, because it’s not written in pure Nix, understandable!

The greater eventual ambition with this will be to allow people to use something like a KaiOS as a desktop as well, just like you can with NixOS. Then, maybe we can steal some of the Omarchy mindshare and prove to DHH that Nix-for-Desktop can be made easy ;)

There are a lot of knobs to think about before doing this in earnest, but we’ve already come a long way.

It’s completely open source and you can use it with your own Digital Ocean account (they’re one of the few that allow you to spin up custom NixOS servers).

Gump

With all the recent problems with Github lately, I figured I’d try my hand at a (again, very vibecoded) version-control system with a few twists.

I’m calling it Gump (short for git dump!). The goals are:

1. Self-hostable alternative to Github.
2. Allows for portable projects, not just portable code. i.e., issues, pull requests, wiki updates, etc. should all be concepts built into and portable and them selves version-controlled, like Fossil but for decentralized work like git.
3. Treats jj concepts as first-class as opposed to git. Like Nix, most people who’ve spent the time to figure out jj seem to far prefer it as a major innovation over git. Plus, unlike Nix but very much like Kai, jj is completely backward compatible with git and your teammates don’t even need to know you’re using it.
4. Easy to use CLI, usable by agents with configurable permissions.

The main goal, of course, is simply to work on Kai. It’s going to be extremely opinionated and bespoke to my wants, if I keep working on it at all. It’s going to do weird stuff like probably only use public key encryption for auth (the fact that most websites still do “check your inbox for an email code” is insane and backward). Or take matklad’s advice of queueing PR merges by default instead of waiting for the merge button to be green, then reviewing, then clicking merge.

Who knows? Maybe this will even lead to building a jj-based generic magit style editing experience too. There’s so much to improve upon.

There are a lot of problems we forgot because we were all just using git and Github for so long that we forgot to question many basic assumptions.

There are of course a whole host of things Github does really well that would be extremely difficult to replicate, but it’s also been really bad lately and the coding world is waking up to the idea that maybe it was not-so-great idea to make all of open source code in the world dependent on a single company.

I don’t want to distract too much from Kai though, and if I actually start using/depending on this I’ll need to throw the slop code away and rebuild something proper from scratch, but Kai development comes first and foremost and I need to stay focused on the bigger goal, but still, this is a fun idea to pursue.

Bootstrapping and Playing with Kai

You can try Kai without installing it by running nix run github:thebrandonlucas/kai -- version. That is how Aion and Gump keep their custom Kai plugins reproducible without requiring a global installation, while this package is not yet on nixpkgs.

Notes and Learnings

Making Code Smaller

Jonathan Blow has a talk where he has an interesting section about how “deleting code is always a good thing”. I’m not sure I agree completely, but a common theme I’m noticing in all these investigations is that conceptual integrity, and an understanding of the entire system is paramount to maintaining good code.

One aspect of understanding things well is having less things to understand. Deleting code while keeping it readable is one way to do this. This also means deleting anything that feels superfluous, inessential, etc. It means careful abstraction when helpful, and no abstraction when not.

Lines of code I think can generally be a helpful metric for whether code is getting smaller but it’s not the full story. If your code is very DRY but opaque, the true workings of it buried under layers and layers of indirection, what was the point?

Testing Must Be Done Up Front

SQLite creator Richard Hipp says in his excellent Software Should Work talk that the secrets to making SQLite probably the most-used piece of software in the world are:

1. Extreme portability. As Hipp says, “it’s just a file you can email to your friends!”
2. It just works. He primarily attributes this to an extreme approach to testing. In fact, there is far more test code in SQLite than code being tested!

I will admit with some shame that I do not have any test yet other than a single integration test that doesn’t block CI. At first, I just blindly told the bot to add tests “in the style of ‘How to Test’” which was silly and lazy and didn’t solve my essential problem: Helping to know what my code did (free documentation) and more importantly give me confidence that it worked.

The main reason I don’t have much testing is that I’m heavily debating and considering my strategy. How to Test says that tests should be cheap, operate at the boundaries of your interface, operate only on data (i.e. only do CPU work, no I/O), and therefore be fast, that they should be very easy to maintain (or else you won’t want to do them), and more.

That’s a lot of things to think about! Roc thankfully makes much of that easy by having a ! marker to declare effectful and effectfree (pure) code, and so if you just trivially separate the code which performs effects from the code which just operates on pure data (same input always results in same output). This means if you just test the pure functions your tests will run fast. But I was finding that my tests were adding a lot of surface area, that I didn’t understand how they worked or what they were doing, and that they therefore didn’t give me much confidence and in fact hampered my understanding of the code.

So, in the meantime, I wrote a zig build kaifiles command which just runs all the Kaifiles in the examples folder end-to-end. I figured that since the examples demonstrate and run through basically all syntax using the simple cowsay program, having those work and actually seeing their output gives me a lot of confidence that at least the happy path is consistently working. I didn’t block zig build ci with these since they’re neither pure nor fast, which may be a mistake, and that has bitten me (while writing this, I realized the command failed because I hadn’t run it in awhile!), so I may change that despite it being effectful.

But a major next step for me is figuring out and working hard on the right fast test boundary so I can start building more confidence moving forward that Kai doesn’t have many bugs.

I should probably also add an invariant to tidy.roc to verify that every block of Kai’s StdPlugin Kaifiles are used. As we add new blocks, the examples won’t include them all automatically without some rule.

Dependency Cultures

Richard Feldman’s Dependency Cultures talk at Software Should Work really took me out of my Javascript bubble and got me asking deeper questions about what tools I should be using.

There are real downsides to the Tigerbeetle approach of manually maintaining all of your own tools and needing to understand everything about your program. I don’t have a great grasp of assembly or the theory of electricity and yet I can still write code to make computers do things. Tools and dependencies do exist for a reason.

But.

There are two edges to this sword. If you take a short enough view, never having sought or gained a deeper understanding of how the things you use work, you’ll always be spending your time learning things which don’t last and are always changing.

The more time I’ve spent in the room with the creators of great things, they tend to have a deep understanding of the foundational structures of the tools they use and are building on, and as a result are much more versatile and capable. As new technologies slowly die in favor of newer technologies, they are less and less affected, because the deeper down into the foundations you go the less the knowledge you gained becomes irrelevant.

There’s a line between the practical utility of leveraging new technologies and the foundational stability of learning the old ones, and I think in tech by our very nature we’re drawn to the former more than we generally should be.

Often, if you think about the software you love the most that’s lasted the longest, it’s usually by people who spend a lot of time learning deeply.

The Tools You’re Using Are Enough

Relatedly, one of matklad’s articles linked to a John Carmack discussion about “just using the tools you already have”. Since this is being built in a language for which the compiler is being rebuilt, I run into a lot of bugs. So, because:

a) I’m excited to help in some small way contribute to this new language. b) I want to use more reliable, fast tools (who doesn’t?) and Roc’s model fits my concerns well. c) There’s less mental overhead and more conceptual integrity in using one tool for me. d) It’s fun and I get to do a bunch of mini-projects in the language. e) Not all code is equally important. The code that checks your code is less important than the code it’s checking, and I don’t need to anguish over every tiny detail.

Trunk-based Development

This article struck an idea in my mind that I can’t get out: every commit should add value to the codebase. This is hard to think about how to do properly, since features rarely lend themselves to single-commit completion! But there are tricks to do so for bigger features as noted in the article, and even in reviewing agent constructed code the idea of constantly merging things to master has been helping tremendously in avoiding not only cumbersome merge conflicts but just keeping the full program in my head, which is becoming more important as features keep getting added. The rapid development speed increases in adding more features provided by agents means that a lot of discipline is required to monitor them, reign them in, maintain control over what the code should actually be doing, and simplifying it. Agents have a huge bias in favor of bloat, and I’ve actually found it pretty hard to have them keep things simple, small, and easy to understand. Trunk-based development helps by reducing the number of concepts I’m having to keep up with at once.

Thoughts on Commits

Different projects have different philosophies on what a commit is, it seems, which has consequences for how your repo history is written and what history you allow to be merged. Mitchell Hashimoto, for example, thinks 1 PR equals 1 commit. Therefore a commit is the mergeable, PRable unit of change. So, the process would be something like: work on your feature locally, adding WIP commits or whatever as you go, then, amending, rebasing, fixup’ing, etc. when you’re done into a single commit that you push and merge.

According to Hashimoto, that final mergeable product should be +50/-50 ideally (easy to review, small incremental change) but no greater than +500/-500 (beyond that it starts getting difficult for a human to review).

Matklad has a looser definition, in that commit history can show the progress of how the final idea came to be. So a commit doesn’t need to necessarily be an isolated independent unit of work, the merge commit can be that, but it should tell a cohesive, easy to understand story about how you got there (and therefore you should still rebase/amend/fixup your WIP or exploration commits into something comprehensible by later review). He finds this useful as you can more easily trace the why and how for features and bugs when doing git blame, and if you have the merge commits, you already have the “final commit” at which that feature got merged.

I’ve been tending to favor Matklad’s approach on this, and trying to balance/reconcile that with this article’s trunk-based development approach where each commit can still provide independent value.

Also, I just started using jj, the new darling VCS of most who take the time to use it, it seems, and that also requires a huge rethinking of how to do all these things.

Regardless, I’m trying to pay much more attention now to the utility of the history.

AI Use

Matklad’s “Loop Closing” Strategy

Vibe coding has been both great and terrible for me. It’s difficult to find the right middle ground between having the robot do everything for you and doing everything yourself old-school. Obviously typing every for loop yourself should no longer be necessary. But I do think I should at least read every loop, and if I can’t understand the code (which I frequently can’t) that’s a red flag for me. This is becoming increasingly important and increasingly my bar: if I can’t understand the code as I read through it, it’s not good enough. This means that I either need to add comments (I plan on adding many more comments as part of cleaning up this codebase) or name variables something better, or either abstract more or even sometimes abstract less (abstracting can make internals unnecessarily opaque).

But one strategy I’ve been trying that’s been working as a nice middle ground (though I still need a bit more discipline) is:

1. Create a .gitignore’d directory called plans.
2. Inside plans, create a folder for my-feature-plan or my-bug-fix.
3. Inside my-feature-plan, create a spec.md.
4. Describe, in as much detail with as many real and specific examples you can of how you’d like the thing to work, the feature, fix, or refactor.
5. Tell the AI to create a plan.md which follows repository rules and splits the work into small, reviewable, independently value-adding chunks (Using Mitchell Hashimoto’s rule of +50/-50, no more than +500/-500 – a human (me!) has to read all the code after all!). Tell it to give examples of what it intends to do.
6. Review and refine that plan (I always find I underspecify and the AI didn’t get what I meant exactly).
7. Tell it “Do first chunk”, review that chunk, merge into master. Sometimes if I’ve got multiple things going I’ll be lazy and just have it do all the chunks at once but even with all the invariants, CI checks on every commit, and commit rules, I find this still bites me and I have to end up having it redo things. So I’m becoming more convinced that smaller chunks, each reviewed by a human before moving on, is still the way.

Focus, Motivation, Mistakes

Some final thoughts.

Focus Focus Focus!

Focus is extremely important yet really hard. I have to play all sorts of tricks on myself because the internet as it stands is adversarial by default: that is, more helpful than harmful to my overall happiness and effectiveness unless I actively take steps to make it a positive. I shouldn’t have to work to make a computer a net positive for me (which is a big part of what I want Kai to help with). This universal constant in my life should be a net positive by default, or at least trivial to make it one.

Maintaining Motivation

Read this fantastic blog on programming and just a lot of good general life advice.

Sometimes, I’ll find that reading and re-reading my code with the purpose of refactoring just doesn’t sound fun. Or thinking about testing strategy or whatever. And then I don’t want to do it. And then I find myself trying to reach for Yet Another Side Project instead of just making the software I want most (Kai) great.

I’m trying to employ several tricks to get around this, such as:

• Work in smaller and smaller chunks
• Walk away from the computer when tired and, if I still want to feel like I’m making progress, read a book on programming that would help (e.g. The Mythical Man-Month or The Cathedral and the Bazaar) read a blog article, or watch a talk.
• Sharing! Writing blog posts takes a lot of effort, even relatively simple or sloppy, plain-English ones like this one. But even just slight positive feedback from real humans who would like to use this is uber-motivating. And, it helps me think through things at a high-level again and remind myself what I’m really trying to do and forces me to understand the system I built better.
• Frequent releases (Mechanical Habits).
• Working on automating processes. E.g. creating a release script which makes releasing as simple as zig build release ....
• Releasing frequently. Doing “the whole vertical” from top to bottom for every small change or small set of changes, that is, writing code, running it, testing it, releasing it, and writing about it, all frequently, makes the whole process run smoother and be less stressful. If you deploy every small change to prod every day, prod deploys accumulate less cruft, become muscle memory, and each one has a lower surface area for problems. Many eyes make all bugs shallow much faster in that case. Plus, it’s more exciting for both you and people who like your project!
• Dogfood your project into another project. I can’t tell you how exciting it was to just say to an agent: “Build thing X. For anything that isn’t code itself, you are only allowed to use kai. No bash scripts, no python helpers, no nix. Just kai” and have it work! It’s a bit clunky, but in Aion and Gump, you really can do everything you need to do, from develop to deploy, using Kai!

Some other great sources I’m borrowing from are:

• matklad’s How to Test
• Mitchell Hashimoto

Mistakes

I’ve already found myself making a lot of mistakes with Kai:

• Overreliance on AI. It takes a lot of discipline to not just keep building and instead fixing what you already have and trying to make it great. That said, if I spend a whole year working on Kai in isolation, just to rip it apart and throw it away in favor of some new implementation when someone points out a completely better architecture, I won’t feel bad about not trying to achieve perfection on the first go before even sharing it.
• Not reading code enough (Don’t write bugs!). And, as a result, having some of the internals of things like the parser or assembler remain a bit fuzzy in my brain.
• Getting distracted. I think it was an overall positive to vibe code Aion and Gump (and vibing was absolutely the correct decision in their case – anything more would have meant spending an unjustifiable amount of time on two projects I may might 1. throw away or 2. distract from Kai too much).

A lot of upfront work went into ensuring modular design, especially so that new features could be added relatively cleanly under a contract that I liked. The abstractions aren’t perfect yet by any means, and it may still be a little confusing for a newcomer writing their own plugin. We’ll work more on that in some future release.

But I did feel it was good enough to finally build more of the features that would put kai to good use. That said, here are the features currently available in kai. These examples are all available in the latest code release:

As before, you could enter shells with certain programs:

# The simplest possible Kaifile.
#
# Run `kai shell` to enter this environment on Linux or macOS.
shell {
  packages: ["cowsay", "fortune"]
}

But now we have a more generic environment builtin which can be used to describe the same package set across the other new keys. For example: you can now create tasks and run them via kai run <task name>

# Run `kai run moo` to make the cow say a pithy quote!
environment cow {
  packages: ["cowsay", "fortune"]
}

task moo {
  environment: "cow"
  run: ["sh", "-c", "fortune | cowsay"]
}

Try running this with kai run moo!

You can also run tasks together as part of a workflow (think CI). Run the below via kai run cowvolution:

# Workflows compose tasks and builds into one repeatable command.
# Run this one with `kai workflow cowvolution`.
environment cow {
  packages: ["cowsay"]
}

task normalcow {
  environment: "cow"
  run: ["cowsay", "moo"]
}

task antiquitycow {
  environment: "cow"
  run: ["cowsay", "Non nobis solum nati sumus."]
}

task victoriancow {
  environment: "cow"
  run: ["cowsay", "doth mother know you weareth her drapes?"]
}

workflow cowvolution {
  steps: [
    "run normalcow",
    "run antiquitycow",
    "run victoriancow",
  ]
}

What if you want to package things together into a portably build-able derivation? You can! Here’s a little shell script which will combine cowsay and fortune to have the fortune piped into whatever cowsay has to say. Run via kai build <output-name> (in this case wisecow)

# You can build artifacts using the build command!
#
# This one combines `cowsay` and `fortune` to give you a
# `wisecow` binary.

environment cow {
  packages: ["cowsay", "fortune"]
}

build wisecow {
  environment: cow
  run: [
    "sh",
    "-c",
    "printf '#!%s\\n%s | %s\\n' \"$(command -v sh)\" \"$(command -v fortune)\" \"$(command -v cowsay)\" > wisecow && chmod +x wisecow"
  ]
  output: "wisecow"
}

Finally, we have overlays, to help people run things that may not be in nixpkgs (kai itself needs this ability)

# Overlays add packages that are not provided directly by nixpkgs.
shell {
  packages: ["rocpkgs.nightly"]
  overlays: ["github:thebrandonlucas/roc-overlay"]
}

That’s it for now! Stay tuned!

• You had to write custom parsing logic (terrible!)
• They weren’t strict definitionally and the concept of a Backend was unclear and vague (does that just mean nix? What if there is more than one requirement?)
• They were unvalidated. You could “do whatever you wanted” and received no help from xkai when you did something illogical
• Plugin modules themselves were not modular/reusable/easy to make

This release now updates that so the plugins should be much simpler to write, parsing is handled for you, a few validations are done by xkai to help ensure you built it right, and Backend was given a much stronger definition that represented the system much better.

Additionally, I made a custom DSL! Even though the previous embedded roc DSL was pretty thin, I realized I could make things much simpler if I made a custom one. The primary motivation to do it in an embedded language was to inherit things like conditional logic, but I really only needed it for switching between systems (e.g. if system == linux then install ...) and that alone did not justify the extra complexity. Plus, it would require every user to have both a determinate system installed like nix and the roc compiler. Instead, we remove that extra 30MB or so and the final kai binary comes out to about 600KB! This does come at the cost of a lot of gutsy config parsing and validation, which is annoying, but I think the tradeoff is worth it.

But the best part is now, the simplest config.kai (which I’ll soon rename Kaifile) looks like this:

shell {
    pkgs: ["cowsay"]
}

With just this, you’ll be able to define reproducible dev environments! For changing behavior on different systems:

on linux {
    shell {
        pkgs: ["cowsay"]
    }
}

on macos {
    shell {
        pkgs: ["pokemonsay"]
    }
}

Much simpler and straightforward than a flake.nix, that’s for sure!

Onward!

• Its usefulness
• Its modularity
• Its evangelistic efforts
• Its ease of use

I believe this has become orders of magnitude more important in the Age of AI, as code (which was never the most expensive piece of the software development lifecycle anyway) has become incredibly cheap to produce. I also believe, perhaps controversially, that the principles of software development as established by its progenitors are even more relevant, with perhaps some tweaks and modifications.

A recurring theme, whether it be in such classics as the Unix Philosophy or The Mythical Man-Month or The Cathedral and the Bazaar or modern works like Working in Public, is that good software should be modular. Think of the success of LEGO. A wonderful toy that embodies the spirit of open source software. LEGO’s have outlived many a toy because their design allows them endless reconfigurability and modifiability. If grandpa builds a castle from LEGO’s, then dad can add a dragon, and son, having slain it, can finally give the freed princess’ tower a long-overdue remodeling. This means that the efforts of the tower have not been lost, from grandpa to father to son, that there is a deeply human history and personalization to that tower.

Code works the same way. Linux is the undisputed king of infrastructure OS, and will probably overcome Windows and Apple: as the latter continue to get worse, piling on more and more new bloatware on already inferior foundations, their software will become more and more intractable to fix while the former continues to get more featureful, more useable, more battle-tested.

Thus, a modular approach will allow the code to be more flexible, interoperable, changeable, robust, and last much longer.

Making Kai Modular

The three concerns for kai modularity outlined in previous articles are:

• Allowing replacement of any standard command with custom logic
• Allowing commands to be added since we can’t anticipate every use case
• Allowing piecemeal replacement of underlying nix or guix pain points (such as the famously slow nix evaluator) with custom interoperable solutions

I modeled the design after caddy, the best web server technology I’ve ever used, which has great defaults, is fantastically easy to use and configure, and is extremely modular.

I created a separate binary, called xkai (analagous to caddy’s xcaddy) which creates custom kai binaries which execute the behavior specified in a custom Plugin.roc file. For example, the “core library” of kai is StdPlugin which looks like this:

import kai.Plugin as PluginApi

# The standard plugin is pure. kai.roc imports its typed configuration
# constructor; only the generic executor performs the resulting actions.
# TODO: it's explicitly limited to nix for now. This might be long-term
# desirable behavior, but we may want to consider providing guix support
# out of the box as a fallback. Or keep it as a separate Guix plugin/binary.
StdPlugin := [].{
    ShellConfig : { pkgs : List(Str) }

    nix : PluginApi.Backend
    nix = PluginApi.Backend.{ name: "nix" }

  # Only support the nix backend for now
    backends = [nix]

  # We have one command, shell, which just creates a nix dev shell
  # with a flake and enters it.
    shell_command : PluginApi.Command
    shell_command = PluginApi.Command.{
        argv: [],
        backends,
        name: "shell",
    }

  # Whereas `shell` is a portable `Command` variable, `nix_shell`
  # is the actual implementation that tells the `kai` binary resulting
  # from `xkai build` on this plugin which side-effects to perform.
  #
  # Separating these two concepts is important - if we supported, say,
  # `guix shell`, then we could have one command in kai which would run
  # `guix shell` or `nix shell` depending on which system the user wanted
  # to use. So it's very flexible!
    nix_shell : PluginApi.Implementation
    nix_shell = PluginApi.Implementation.{
        actions: [
            WriteConfigUtf8({ path: ".kai/flake.nix" }),
            Exec({
                args: ["develop", "path:.kai#default"],
                command: nix.name,
            }),
        ],
        backend: nix,
        command: shell_command,
        requirement: Program(nix.name),
    }

    shell : ShellConfig -> {
        implementation : PluginApi.Implementation,
        rendered_config : Str,
    }
    shell = |shell_config| {
        implementation: nix_shell,
        rendered_config: StdPlugin.render_nix(
            shell_config.pkgs,
        ),
    }

  # Render the actual flake, which will be written to a managed
  # directory called `.kai`
    render_nix : List(Str) -> Str
    render_nix = |pkgs| {
        package_lines = pkgs.map(
            |pkg| "              nixpkgs.\"legacyPackages\".\"x86_64-linux\".\"${pkg}\"",
        )
        lines = [
            "{",
            "  inputs.nixpkgs.url = \"github:NixOS/nixpkgs/nixos-unstable\";",
            "  outputs = { nixpkgs, ... }: {",
            "    devShells.x86_64-linux.default = nixpkgs.legacyPackages.x86_64-linux.mkShell {",
            "      packages = [",
        ].concat(package_lines).concat([
            "      ];",
            "    };",
            "  };",
            "}",
        ])
        Str.join_with(lines, "\n")
    }
}

# Plugins are pure, so we can do cheap, no I/O tests!
expect {
    module_config = StdPlugin.shell({ pkgs: ["hello"] })
    module_config.implementation == StdPlugin.nix_shell and
        module_config.rendered_config.contains(".\"hello\"")
}

This means that plugin creators need only write this plugin file, run xkai build on it, and they have their own custom kai binary! Which is, of course, how I will also be developing kai.

Thinking through this design has been by far the most difficult aspect of this project so far, and required a lot of research up front. But I am profoundly satisfied with this architecture.

Although kai is still very much a prototype which needs much better testing, interface, and of course, feature work, this is beginning to feel like a very solid and extensible foundation which will allow not just me, but anyone, to easily contribute meaningfully to this in the future.

Onward!

A lot of work went into doing that very basic thing that is a near-zero improvement over nix itself, because so much went into the underlying design that I’m hoping will make Kai a Zero to One tool: an upgrade to the interface of deterministic computers so delectable and powerful that it will make the human experience, tooling interop, and agentic efficiency orders of magnitude better.

Finding a Boundary

One of the main ideas behind Kai is for the user to manage a simple config file, kai.roc, which can serve as a simplified interface for interacting with your determinate system (for the vast majority, Nix under the hood). This is hard, because:

• A) Nix has so many layers of the stack, e.g.:
  • Nix the language, used by:
  • Nix the package manager, used by:
  • NixOS, along with:
  • The nixpkgs package repository

That’s a lot of layers interoperating in a “closed” way because:

• B) Nix built these monolithically, such that it’s very hard to take one piece of that stack and use it for anything useful outside of the broader ecosystem. This means two things:
  • Boundaries are difficult to isolate at the conceptual level (essential complexity baked into the Nix ecosystem).
  • Boundaries are difficult to isolate at the code level (accidental complexity due to years upon years of cruft, ad-hoc development, and no real competition leading to a lack of need for interop).

All of this together forms a natural walled garden: unlike Apple, which intentionally fights its users by hiding code for the sake of (Apple believes) reduced complexity and financial incentives, Nix seems to have created one by accident. While explicitly encoding preferences for open software (via, for example, needing to manually modify configs to set a flag allowUnfree = true) and a philosophy which advocates maximal user control over their systems, in practice it is so complex and abstruse for the average daily user, with so many layers, that anyone who wishes to improve upon or isolate a single piece faces a very effective barrier to general adoption because, in sum:

• A) Users can’t figure out how to use it, and:
• B) Developers don’t have a middle ground between improving on the current ecosystem (thus adopting its current problems and working backward to solve a limited subset of things it can), or creating new systems (which, despite the profundity of the idea, is so difficult to do—see all the many layers above!—that only one has even tried: Guix, and its community is very small).

Enter Kai

Kai proposes a middle way. Kai acknowledges two fundamental facts:

1. Nix dominance
2. Nix “Accidental Complexity”

Nix has the largest package ecosystem in the world. A stunning achievement for a package manager which isn’t the most well used, and an easy testament to the power of how reproducible software allows you to ship. Aside from that, it has seen slow increased adoption over the years despite continually fracturing communities and continuously deprecating APIs: even frontier LLMs get frequently confused about the best way to use Nix, because so much of the documentation teaches deprecated practices! Finally, there are no real competitors: imagine trying to build a completely unique package manager which uses a custom language and build a custom operating system from it while maintaining the largest package ecosystem on Earth, without being a for-profit company. Despite valid criticisms of Nix, this is a staggering achievement! One that is extremely difficult to replicate.

On the other hand, Nix suffers from so many problems which don’t need to exist. I sometimes wonder if its creator, Eelco Dolstra, thought that some enterprisers would take his idea and carry it out to permanently improve the world by eliminating the whole class of software deployment problems, in the way that, say, Rust did for programmers. Maybe he, like Benjamin Franklin when he named the poles of electricity positive and negative, thought that someone would take his discovery and formalize it more appropriately and give them better names, whereas instead, we are just stuck with these terms, and we are still stuck with Franklin’s conceptual mistake of inverting the logical poles. And thus we are still stuck with many of the tradeoffs of Nix that are probably not worth having, despite the scope of the downstream improvements of using it over our current imperative systems being enormous and fundamental. Here we are over twenty years later, still building on top of inferior foundations. A huge portion of this I think is to be blamed on Nix’s design at basically every level.

Taking these into account, Kai proposes to do the following:

1. Be backward compatible with Nix. Indeed, to simply use Nix directly under the hood at first.
2. Remove as much accidental complexity from using Nix as possible. Primarily through simplified UX and useful missing features or abstractions.

Alan Kay’s motto is my North Star here: “Simple things should be simple; complex things should be possible.” That is, make Nix (via Kai) as easy as possible to use while allowing users to do anything they’d normally be able to do in Nix (even modify/create raw Nix files where they find they can’t do something in pure Kai).

This has resulted in me spending a lot of time seeking the appropriate boundaries to expose. After settling on Roc as the config language to use, I posted my idea on the Roc forum, and to my amazement, Luke Boswell created a working prototype of my idea, roc-blueprint, overnight!

This was great, as roc-blueprint provided an example of a general, pure, typed, and portable backend which could then be lowered into a compatible backend of choice (i.e. any reproducible system). In reality there are only two: Nix and Guix. Then this lowered Roc representation (itself another platform) could be used to instantiate and run actual respective commands. For example, here’s a program that provides a portable representation of a Nix shell which contains the pokemonsay program:

app [main!] {
    pf: platform "https://github.com/lukewilliamboswell/roc-platform-template-zig/releases/download/1.0.0/AnZoxzoGPtSGQ15EQh6pBeeaHJ7aizP9MQhK81dES3Uq.tar.zst",
    blueprint: "../../blueprint/package.roc",
}

import pf.Stdout
import blueprint.Blueprint
import blueprint.Requirement
import blueprint.Target
import blueprint.Environment
import blueprint.Nix

pokemonsay : Requirement
pokemonsay = Requirement.new({
    id: "pokemonsay",
    display_name: "Pokemonsay Program",
})

workspace : Blueprint.Draft
workspace = Blueprint.workspace({
    name: "pokemonsay shell",
    target_systems: [Target.X86_64Linux],
    envs: [
        Environment.new({
            name: "default",
            requirements: [pokemonsay],
        }),
    ],
})

nix_config : Nix.Config
nix_config = Nix.config({
    nixpkgs: Nix.github_input(
        "nixpkgs",
        "NixOS",
        "nixpkgs",
        "nixos-unstable",
    ),
    bindings: [
        Nix.bind(
            pokemonsay,
            "nixpkgs",
            ["pokemonsay"],
        ),
    ],
})

main! : List(Str) => Try({}, _)
main! = |_args| {
    valid = Blueprint.validate(workspace) ? |errors| BlueprintInvalid(errors)
    source = Nix.render(valid, nix_config) ? |errors| NixInvalid(errors)
    Stdout.line!(source)?
    Ok({})
}

Cool! But notice any problems?

1. Boilerplate-galore! The actual flake.nix this renders is fewer lines than the config! (21 vs. 54)

# Generated by roc-blueprint and roc-blueprint-nix. Do not edit.
{
  "description" = "Development environments for pokemonsay shell";
  "inputs" = {
    "nixpkgs" = {
      "url" = "github:NixOS/nixpkgs/nixos-unstable";
    };
  };
  "outputs" = { nixpkgs, ... }:
    {
      "devShells" = {
        "x86_64-linux" = {
          "default" = nixpkgs."legacyPackages"."x86_64-linux"."mkShell" {
            "packages" = [
              nixpkgs."legacyPackages"."x86_64-linux"."pokemonsay"
            ];
          };
        };
      };
    };
}

So the resulting config file is more complex than what it is simplifying, with this caveat: we define the portable (blueprint platform) and the non-portable (blueprint_nix platform) in the same file, the benefit being we can reuse blueprint programmatically and theoretically have it port to any number of determinate system protocols under the hood (eventually, one config could boil down to a Nix or Guix system). Another benefit is we get strong typing: unlike flake.nix, the individual pieces are strongly typed and thus linters and checkers could be built upon this to much greater effect. So these are some great benefits, the strong typing and portability alone we may even call Zero to One benefits, but we have only achieved goal (1) above:

Be backward compatible with Nix. Indeed, to simply use Nix directly under the hood at first.

But we’ve made (2) worse, adding complexity for arguably marginal gains for the average user:

Remove as much accidental complexity from using Nix as possible. Primarily through simplified UX and useful missing features or abstractions.

There are also other problems: for example the user must have a compatible Roc compiler installed, and they have to do roc main.roc to run it.

And even if we solve the above, this is an incomplete solution: the platform design is great, but must we really expect every user to just import this into their own programs, and write the logic to lower it?

I thought hard about what to do here, because the pros of this design are real, and I wanted to keep it for the time being as programmers might find this level of control useful, but for the vast majority of people a simpler config is in order. So I wrote one abstraction layer on top of it. Where the previous design looked like this:

[portable blueprint] -> [nix blueprint] -> [nix files]

The current one adds:

[kai.roc config] -> [portable blueprint] -> [nix blueprint] -> [nix files]

The kai.roc sacrifices a bit more detail and control for simplicity and looks like this:

app [config] {
    kai: platform "../../platform/config.roc",
}

import kai.Kai

config : Kai.Config
config = {
    name: "pokemonsay shell",
    shell: {
        pkgs: ["pokemonsay"],
    },
}

Much better! 13 total lines, and only 5 are config that the user needs to consider in the base case! But the idea is that the user will be able to move down and use the tool at any layer of abstraction as suits their needs. I’m not sure whether I’ll keep roc-blueprint at all yet, as the benefits of an intermediate representation between high-level abstraction and low-level control must be weighed against the need for maintenance simplicity, and I’m not sure yet of the value proposition when the simpler model exists.

To achieve this simplicity over roc-blueprint, we make some assumptions on:

• The distinction between a validated blueprint and a Draft workspace
• The Environment concept
• The machine architecture (we remove Target.X86_64Linux from above, for example)

But I think many of these decisions can appropriately be outsourced to under-the-hood management by a CLI tool, which is the second part of the solution.

For example, we don’t have to encode the architecture in the config mandatorily if the CLI tool kai can just save those details in the config (or just default to building for all default/common systems).

Right now, the CLI can only do kai shell. It is dependent on your machine having Nix (and Nix is still hardcoded in) and most egregiously on being x86_64 Linux (sorry!), but this will be modularized away later.

Mitchell Hashimoto built an excellent Zig overlay flake which allows you to get the nightly version of the Zig compiler with a single Nix command. At the time of this writing, Zig 0.16.0 is the latest stable release. Some projects intentionally track Zig’s rapidly changing development branch instead. Ziglings, for example, currently requires a sufficiently recent 0.17.0 development build and does not compile with Zig 0.16.0! Although Zig’s official binary can run on NixOS, integrating it reproducibly and ensuring host linking works correctly requires additional Nix setup. If zig-overlay didn’t exist, I’d have had to roll a custom solution just for myself. An overlay/flake is thus useful for any project not considered stable to allow anyone to have a reproducible/pinnable or just bleeding-edge environment for any version indexed by the overlay. It makes compatibility and environment setup orders of magnitude easier to achieve. No more hunting down binaries! (or hunting down and patching if you’re on NixOS).

The newer and more in flux the project, the more useful this is. And with the proven utility of the zig-overlay template, I decided to build one for an even newer programming language, Roc. At present, Roc releases nightly binaries of the compiler. They are currently replacing the Rust compiler with a new scratch Zig rewrite, which has breaking changes to Roc’s design such that code and platforms targeting the old compiler are usually incompatible with the new compiler, and have not made a stable release of the Zig compiler release yet. Thus, there’s a lot of incompatibility between apps which were built using the old and new Roc compilers, since the new compiler introduces a lot of backward-incompatible language features. So, being a nix enthusiast, I thought it would be helpful to create a roc-overlay modeled after zig-overlay, allowing tracked versions of the nightly compiler to be selected by a developer, or even better, declared as an input in flake.nix and pinned to an exact repository revision in flake.lock for given projects so the developer doesn’t even have to think about which nightly to use.

roc-overlay Contents

roc-overlay provides a Nix flake for the official Roc new-compiler binaries. It installs prebuilt archives and does not build from source. It mirrors zig-overlay in overall structure. Use it like this:

# To get the latest nightly as seen by the repo,
# run one of these:

# Create a shell with the compiler.
nix shell github:thebrandonlucas/roc-overlay
roc version

# Run one-off commands directly from the flake.
nix run github:thebrandonlucas/roc-overlay -- version

Including the overlay in your flake.nix:

   {
     inputs = {
       nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
       roc-overlay.url = "github:thebrandonlucas/roc-overlay";
     };

     outputs = { nixpkgs, roc-overlay, ... }:
       let
         system = "x86_64-linux";
         pkgs = import nixpkgs {
           inherit system;
           overlays = [ roc-overlay.overlays.default ];
         };
       in
       {
         devShells.${system}.default = pkgs.mkShell {
           packages = [ pkgs.rocpkgs.nightly ];
         };
       };
   }

Hopefully you find this overlay useful. Please open an issue if you find anything that needs improvement.

[The software deployment problem] is about getting computer programs from one machine to another—and having them still work when they get there.

The Purely Functional Software Deployment Model, Eelco Dolstra

Nix allows you to setup software on your computer in such a way that your setup is reproducible, meaning your setup on machine A can be exactly the same as your setup on machine B – as long as you have Nix.

To most people, learning Nix is a pain, due to the new concepts, sparse and outdated documentation, and community infighting.

But I think Nix can make using computers fun and powerful and less painful, once you learn how to handle its’ edges.

One fun way we can use Nix is to stitch together programs like Legos and have them interact with each other in a reproducible way. Let’s create an example!

First install Nix if you haven’t.

Say we have a fun little bash script like this:

#!/usr/bin/env bash

# filename: pokefortune.sh

message="${1:-}"
pokemon="${2:-slowking}"

# Silence Perl locale warnings.
export LC_ALL=C

# Generate a fortune if user did not pass a message.
if [[ -z "$message" ]]; then
  message=$(fortune)
fi

echo "$message" | pokemonsay -p "$pokemon" -n

It optionally takes in a message and a Pokédex number, and prints out that Pokémon and message using the pokemonsay program. If either the message or the Pokemon aren’t specified, it uses a default Pokemon and the program fortune to generate the message.

Save this in a file pokefortune.sh, make it executable, then run it:

# Make the bash script executable
chmod +x pokefortune.sh

./pokefortune.sh

If you’re like most people, you probably don’t have pokemonsay or fortune installed on your system, so you’ll likely see something like this:

./pokefortune.sh: line 6: fortune: command not found
./pokefortune.sh: line 7: pokemonsay: command not found

Therefore this script, which works on my system because I have these programs installed, isn’t reproducible on your system. Let’s create a Nix derivation to make it so. Create a file called pokefortune.nix and copy the following:

# filename: pokefortune.nix

{
  pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/nixos-24.05.tar.gz") { },
}:
pkgs.writeShellScriptBin "pokefortune" ''

message="''${1:-}"
pokemon="''${2:-slowking}"

# Silence Perl locale warnings.
export LC_ALL=C

# Generate a fortune if user did not pass a message.
if [[ -z "$message" ]]; then
  message=''$(${pkgs.fortune}/bin/fortune)
fi

echo $message | ${pkgs.pokemonsay}/bin/pokemonsay -p "$pokemon" -n
''

Now run:

nix-build pokefortune.nix

This may take awhile, especially if this is your first time running Nix. Let’s look at the output:

nix-build src/scripts/nix/pokefortune.nix
unpacking 'https://github.com/NixOS/nixpkgs/archive/nixos-24.05.tar.gz' into the Git cache...
this derivation will be built:
  /nix/store/vg4zmghqcnhjbs8kqhx04xixvm36d3ik-pokefortune.drv
these 9 paths will be fetched (2.65 MiB download, 26.91 MiB unpacked):
  /nix/store/0vdf5mpd762bw53rgl5nkmhvzq8n4m0d-file-5.45
  /nix/store/77cdqhqprqbciyhzsnzmsk7azbk2xv6r-fortune-mod-3.20.0
  /nix/store/d0i8idmbb4jji9ml01xsqgykrbvm7dss-gnu-config-2024-01-01
  /nix/store/0554jm1l1qw1pcfqsliw91hnifn11w8m-gnumake-4.4.1
  /nix/store/2wp235bg03gykpixd9v2nyxp08w8xq8a-patchelf-0.15.0
  /nix/store/c5x32idp600dklz9n25q38lk78j5vwxb-pokemonsay-1.0.0
  /nix/store/pba53n11na87fs4c20mp8yg4j7qx1by2-recode-3.7.14
  /nix/store/zix67r268ihi4c362zw7c0989z12jmy7-stdenv-linux
  /nix/store/2329271b42wh6b6yhl7jmjyi0cs4428b-update-autotools-gnu-config-scripts-hook
copying path '/nix/store/d0i8idmbb4jji9ml01xsqgykrbvm7dss-gnu-config-2024-01-01' from 'https://cache.nixos.org'...
copying path '/nix/store/c5x32idp600dklz9n25q38lk78j5vwxb-pokemonsay-1.0.0' from 'https://cache.nixos.org'...
copying path '/nix/store/0vdf5mpd762bw53rgl5nkmhvzq8n4m0d-file-5.45' from 'https://cache.nixos.org'...
copying path '/nix/store/0554jm1l1qw1pcfqsliw91hnifn11w8m-gnumake-4.4.1' from 'https://cache.nixos.org'...
copying path '/nix/store/2wp235bg03gykpixd9v2nyxp08w8xq8a-patchelf-0.15.0' from 'https://cache.nixos.org'...
copying path '/nix/store/pba53n11na87fs4c20mp8yg4j7qx1by2-recode-3.7.14' from 'https://cache.nixos.org'...
copying path '/nix/store/2329271b42wh6b6yhl7jmjyi0cs4428b-update-autotools-gnu-config-scripts-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/zix67r268ihi4c362zw7c0989z12jmy7-stdenv-linux' from 'https://cache.nixos.org'...
copying path '/nix/store/77cdqhqprqbciyhzsnzmsk7azbk2xv6r-fortune-mod-3.20.0' from 'https://cache.nixos.org'...
building '/nix/store/vg4zmghqcnhjbs8kqhx04xixvm36d3ik-pokefortune.drv'...
/nix/store/vsv2spw517cwq791fl3f8iymm6hshhyq-pokefortune

Nix fetched the packages we specified in our .nix file: pokemonsay, fortune, and some we didn’t specify: such as file, patchelf, and recode, which one or both of the other two packages depends on. Then it copied them locally, built a binary, and placed it at /nix/store/vsv2spw517cwq791fl3f8iymm6hshhyq-pokefortune/bin. We can confirm this by running it:

/nix/store/vsv2spw517cwq791fl3f8iymm6hshhyq-pokefortune/bin

You should see something like this:

To be reproducible, Nix ensures that it knows about every single dependency needed to create a package at all times. Because of that, you can do fun things like this:

# Quickly enter a shell with the dependencies to create and display images.
nix-shell -p graphviz chafa

# Query the derivation's dependency graph, create a .png from it and display it:
    nix-store --query --graph $(nix-build pokefortune.nix) | \
    dot -Tpng -o pokefortune-dependency-graph.png && chafa pokefortune-dependency-graph.png

Resulting in a view of every single dependency (the “closure”) that our pokefortune program requires:

How cool is that! We can see the whole dependency tree for anything packaged with Nix!

You have just done something very powerful with Nix: You’ve created a reproducible derivation that anyone who has the package manager installed on their system can use, which they couldn’t before.

This is just a taste of what Nix can do, but I hope that the potential is clear. As Farid Zakaria says in his blog post “Learn Nix the Fun Way”:

Hopefully, seeing the fun things you can do with Nix might inspire you to push through the hard parts.

There is a golden pot 💰 at the end of this rainbow 🌈 awaiting you.

• https://fzakaria.com/2024/07/05/learn-nix-the-fun-way

This article is directly inspired by Farid Zakaria’s blog post Learn Nix the Fun Way. Check out his excellent blog here.

Special thanks to Russell Weas and veracius for their input to this article and code

References:

• https://edolstra.github.io/pubs/phd-thesis.pdf
• https://web.archive.org/web/20171017151526/http://aptitude.alioth.debian.org/doc/en/pr01s02.html
• https://fzakaria.com/2024/07/05/learn-nix-the-fun-way
• https://zero-to-nix.com/start/nix-run/

When developing a Bitcoin payment flow, there are multiple ways a user can expect to be able to pay. They might want to pay an on-chain address or Lightning invoice, they may be scanning a QR Code from their phone, copy/pasting from a wallet, or using a WebLN browser extension. Creating an intuitive interface that also captures all the possible ways a user can pay is one of the fundamental UX challenges of developing an application that can receive payments in Bitcoin.

This flexibility opens up many exciting use cases, but often comes at the cost of being able to easily develop a smooth experience for the user. The greater the developer’s cognitive load, the more difficult it is to create intuitive UX.

This project aims to provide everything needed to allow standard Bitcoin & Lightning Network payments out of the box. It handles creating the proper URIs from just an address or invoice, and favors creating unified URIs whenever possible. Styles are highly customizable and images can be embedded. It also includes polling functionality – a callback can be passed as a property of the element to periodically check for payment.

Check it out on Github and feel free to play around with the options and create your own here.

In addition, because it’s a web component, you should just be able to drop it into any framework and have it work, whether it’s pure HTML, Sveltekit, React, making it very versatile. Examples are included in the Github repo.

Here’s a quick demo of how to use it in Sveltekit:

Install:

yarn add bitcoin-qr

Then:

<script lang="ts">
  import { defineCustomElements } from 'bitcoin-qr/loader';

  defineCustomElements();
</script>

<bitcoin-qr
 id="qr"
 width="300"
 height="300"
 bitcoin="bc1p4mqqpxdyg3l0tvpv9xaesw5rsqv7fqpcn9ydawx3e8yr4enmc3qq7m59vd"
 lightning="lno1zrxq8pjw7qjlm68mtp7e3yvxee4y5xrgjhhyf2fxhlphpckrvevh50u0q2a8y2qt9jeu3pj7p3u58rxs98yu93dxjlmy3eqp24qze09anfr6cqszm40686ujqq7t9jsjjf42tl2pryv3ds9f4hm6hysk7lvswdm70ntsqvlgc5nm4px6k3q2wmj4xun784sss2jn00q0w4dhydmln92wvash6jckffu0wmetwptj38f5g09k8pznuk4wq2df046yv0c5m3dakwgfz5ths2ymm3z6vnehvxj6urcarufa7uecwqqst9x4063hdpxq4frmupnsam9ahg"
 parameters="amount=0.00001&label=bitcoin-qr.blu.cx&message=bitcoin-qr.blu.cx"
 image="./assets/bitcoin.svg"
 is-polling="false"
 poll-interval="1000"
 type="svg"
 corners-square-color="#000000"
 corners-dot-color="#000000"
 corners-square-type="extra-rounded"
 dots-type="classy-rounded"
 dots-color="#000000"
 debug="true"
 poll-callback={callbackExample}
/>

I built this about a year ago but didn’t post about it because there was more I wanted to do and test before putting it out there. But there’s always something else to do, isn’t there?

If you find any issues, feature requests, or suggestions for improvement, feel free to create an issue in Github.

Hope this saves you some time!

#!/bin/bash

# Script to copy files from nb drafts to site code, then commit and push

# --- Check environment variables ---
if [ -z "$SITE_GIT_REPO_DIR" ]; then
  echo "ERROR: SITE_GIT_REPO_DIR is not set."
  exit 1
fi
if [ -z "$NB_FINAL_DRAFT_DIR" ]; then
  echo "ERROR: NB_FINAL_DRAFT_DIR is not set."
  exit 1
fi

if [ -z "$NB_FINAL_DRAFT_IMAGES_DIR" ]; then
  echo "ERROR: NB_FINAL_DRAFT_IMAGES_DIR is not set."
  exit 1
fi

if [ -z "$SITE_CODE_MD_DIR" ]; then
  echo "ERROR: SITE_CODE_MD_DIR is not set."
  exit 1
fi

if [ -z "$SITE_CODE_MD_IMAGES_DIR" ]; then
  echo "ERROR: SITE_CODE_MD_IMAGES_DIR is not set."
  exit 1
fi

# --- Copy files ---

# Copy markdown files
echo "Copying files from $NB_FINAL_DRAFT_DIR to $SITE_CODE_MD_DIR..."
cp -r "$NB_FINAL_DRAFT_DIR"/* "$SITE_CODE_MD_DIR"/

# Check if the copy command was successful.
if [ $? -ne 0 ]; then
  echo "ERROR: Failed to copy files from $NB_FINAL_DRAFT_DIR to $SITE_CODE_MD_DIR."
  exit 1
fi

# Copy image files
echo "Copying files from $NB_FINAL_DRAFT_IMAGES_DIR to $SITE_CODE_MD_IMAGES_DIR..."
cp -r "$NB_FINAL_DRAFT_IMAGES_DIR"/* "$SITE_CODE_MD_IMAGES_DIR"/

# Check if the copy command was successful.
if [ $? -ne 0 ]; then
  echo "ERROR: Failed to copy files from $NB_FINAL_DRAFT_IMAGES_DIR to $SITE_CODE_MD_IMAGES_DIR."
  exit 1
fi

# --- Git Commit and Push ---

# Navigate to the Git repository directory
cd "$SITE_GIT_REPO_DIR"

# Check if it's a git repository
if [ ! -d ".git" ]; then
  echo "ERROR: $SITE_GIT_REPO_DIR is not a git repository."
  exit 1
fi

# Add the changed files (only the md and images directories)
echo "Adding files to git..."
git add "$SITE_CODE_MD_DIR" "$SITE_CODE_MD_IMAGES_DIR"

# Check if there are any changes to commit
if ! git diff --cached --quiet --exit-code; then
  echo "Committing changes to git..."
  git commit -m "Update content"

  # Push the changes to the remote repository
  echo "Pushing changes to remote origin..."
  git push origin master

  # Check if the push command was successful.
  if [ $? -ne 0 ]; then
    echo "ERROR: Failed to push changes to remote origin."
    exit 1
  fi

  echo "Successfully updated remote repository."
else
  echo "No changes to commit."
fi

echo "Script completed."
exit 0

Automating Deployments to VPS

I also wanted to automate deployments to the VPS I was using on every push. So essentially, my website would update every time the script ran. So I put this deploy.yml in .github/workflows:

name: Deploy to VPS

on:
  push:
    branches:
      - master

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Install Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install dependencies
        run: npm install

      - name: Build SvelteKit app
        run: npm run build

      - name: Deploy to VPS
        uses: appleboy/scp-action@v0.1.7
        with:
          host: ${{ secrets.VPS_HOST }}
          username: ${{ secrets.VPS_USER }}
          password: ${{ secrets.VPS_PASSWORD }}
          source: "build/*"
          target: "/var/www/blu"
          rm: true
          strip_components: 1

Note that the VPS_HOST, VPS_USER, and VPS_PASSWORD must be defined repository/environment secrets in Github. Also note the action we’re using at the end to do the actual deploy, appleboy/scp-action. Two parameters were important for what I was trying to do. First, rm: true, which deletes the target folder (/var/www/blu in this case) before uploading the data. In the scenario where I delete or move a file in the new build, the old file would’ve remained present without that parameter set, which I didn’t want.

Second, the strip_components: 1 is also important here. Without it, the build directory was getting placed inside /var/www/blu, instead of the contents of the build directory moving into blu. So strip_components specifies that while I want to use build/* as the source of my files, I only want what’s in it, i.e. build/*, so don’t put anything after the first slash (e.g. build) into /var/www/blu.

That’s it! Now every time I update the content of the site (such as with this post), I just copy it over to final, then run the sync script, and voila!

I’ve been using it fairly happily for awhile, but it booted into a black screen recently. I had a flash drive plugged in and I had also installed new programs using dnf (The Fedora package manager, which is the distro Asahi uses), and after some reddit searching it appears that the problem was caused by something having to do with dnf update. After restarting the computer, it showed the Apple logo, Asahi logo, and Fedora logo in sequence as usual, and then never booted past a black screen. Unfortunately I wasn’t really able to “fix” it, but I was at least able to get into a terminal screen, mount a flash drive, backup my data, and then do a hard reset.

On boot, press and hold FN + CTRL + OPTION + F2. This should take you into a terminal-based login screen. Login with your normal user credentials. Then plugin a flash drive. Identify the device name by typing lsblk. It should appear as /dev/sda or something of the like. You should be able to identify it by its’ file size (e.g. 32GB, etc.). Then, mount the device. First, you should create a mount point using mkdir, such as in /mnt:

Create a mount point:

sudo mkdir `/mnt/mydrive`

Mount the device:

sudo mount /dev/sda /mnt/mydrive

Copy files from your home directory (or wherever you have data stored you want to keep) to the flashdrive:

cp -r ~/* /mnt/mydrive

Unmount and eject the drive when done:

sudo umount /mnt/mydrive
sudo eject /dev/sda

Repartition the Hard Drive

Now, you’ll need to do a complete uninstall/reinstall of Asahi Linux by repartitioning the hard drive on your Mac.

I followed this tutorial video to help me repartition the Mac properly, and it worked like a charm. Then, I was able to go through the same process of reinstalling Asahi again and using my Apple Silicon machine with Linux.

1. I use it on a lot of different computers and often want to add to notes that only exist on computer A while I’m on computer B.
2. Since everything I write on my website is in Markdown, I want to have the ability to have what I place in that notebook automatically update the website (or at least update it after running some command)

Solving Problem 1 will be the focus in this article. In a subsequent post, I plan to detail solving problem 2.

Requirements

I want a system that automatically syncs files across devices that have it installed on and connected to at an interval I define when the machine comes online.

Syncthing

After prompting Perplexity.ai about the issue, two methods which appealed to me are Syncthing and Git. Using Git would’ve been more manual and time-consuming, and since Syncthing is a very popular tool and fully open source, I went with that option.

Installing

Syncthing has a great “Getting Started” page that gave me everything I needed to know to use it. Just download the app on two devices. Select the architecture you’re using from their releases page here. Follow their Getting Started page and you should have all the basics to get up and running.

Helpful Notes

Use Cases

Syncthing is not really meant to be a backup software like rsync, because changes made on one machine will synchronize to the other machines, and therefore if one machine deletes a file it will also be deleted on the other machine. It’s meant to be a sharing software keep documents on different computers synchronized, as the name suggests. Which means that it should really be used for things like media libraries or as an alternative to cloud storage solutions like Box or Google Drive, or when needing to edit documents on multiple machines, as in my case with nb.

Mobile Devices

I wanted one of my devices to be my phone, so just note that for mobile (i.e. by mobile I mean Android or a security-hardened derivative like GrapheneOS — sorry iPhone fans, but their anti open source ethos precludes you from using this one) you have to use a fork of their now deprecated Android app. You can download the apk file here.

Folder Types

There are three types of folders:

• Send Only
• Receive Only
• Send and Receive

This is useful for sharing files while still setting some basic permissions for them. Send Only is great for sharing with people/devices that you don’t want to be able to modify your local copy via their modifications; Receive Only is the opposite — it can only be shared with, and yet changes from the sending folder will not modify changes in the receiving folder.

The FAQ is Very Helpful

Initially when using Syncthing I was confused about a few things (Should I use this for backups? Why does syncing take so long? How does it work? Will it ever connect to anything other than my own machines and can they see my data?). The FAQ answered all my questions very nicely and in detail, so be sure to give that a read.

Note For Obsidian Users

Obsidian is an (unfortunately closed-source) note-taking app that pairs nicely with nb. I use nb on all my desk/laptop machines and use Obsidian on mobile since trying to use a terminal on mobile would be a terrible experience. After installing Syncthing-Fork on your mobile device and syncing the relevant folder, you can open that folder as your Vault in Obsidian. Before making any changes in Obsidian, however, you should disable a setting in Syncthing. Click on the synced folder, scroll down to “Watch for changes”, and disable it. This will make it so changes to that synced directory will only attempt to sync changes to the others on an hourly basis. Obsidian saves files to disk extremely frequently, basically on every keystroke, and this caused Syncthing to behave strangely or simply not sync any changes made from Obsidian to the other devices. This is a bit unfortunate because there’s a possibility that you could update the same file on your mobile device and another machine within that hour, causing a conflict. But fortunately Syncthing will save both versions of the file in a “conflict” file and you will have both versions so that there’s no content loss.

We shall not cease from exploration
And the end of all our exploring
Will be to arrive where we started
And know the place for the first time.

• T.S. Eliot, from “Little Gidding,” Four Quartets. Originally published 1943.

This past year, I’ve experimented with several different methods of creating and managing a personal website. I didn’t merely want the most convenient solution, or the solution most likely to buy popularity more easily via its platform or network effects. I wanted a solution that was in line with:

• 1. A growing set of principles I want to practice see more of in software I use and create.
• 2. A set of very unique and specific features I desired.

The pursuit of principles and features, of course, give way to the brutal constraints of reality to some degree. I don’t have infinite time. And so the desire to obtain the features I want while maintaining those principles must be balanced with the constraints of time and effort. After experimenting with several iterations of website-building, I have in many ways “arrived where I started”, but now feel more confident, more sure-footed, more “aligned”, and more excited about what even a simple little personal website could be.

First Site — A DIY Compromise

I started out as most Gen-Z developers start, I think. You want to be a developer, you need a portfolio. You want to be a web developer? Your portfolio is your website. And so my first website was just that. I built it during many skipped classes in college, while I was still painstakingly and inefficiently learning HTML, CSS, and Javascript. It was designed to be artsy and beautiful and to get me a job as a web developer. And so I didn’t post or write about anything, just listed what I felt would be most impressive to recruiters. But, after proving to yourself that you can indeed get a job and you can indeed make enough to pay your rent and not be a broke college kid forever, you start to realize there’s more important things in life than just trying to impress recruiters. You realize that a personal website can be so much more. It can be a reflection of yourself.

And so, with that in mind, I built my first “real” attempt at a personal site. I built it in Sveltekit (which was in unstable version 3 at the time), used the MDSveX library so I could write my articles in Markdown. This was a proper blogging site that also listed projects and “About Me” and “Contact” and whatnot, however there were several problems:

• Hosting: It was hosted on Netlify, which meant that there were often cold starts when visiting it. This also meant that I wasn’t hosting the site myself, which wasn’t a big deal to me at the time, but developed into one over the course of the year.
• Features: The site was low on features and bells and whistles, but this was mainly because I just didn’t take the time to implement them. The fact that everything was written in Sveltekit meant that I could have, but I wasn’t taking the time to really think about what I wanted implemented and then come up with a good plan to do it.
• Preprocessing the Markdown: I didn’t quite understand how to set up the preprocessor in MDSveX properly, which does the job of converting the Markdown to HTML when building the site. What I think was happening was this resulted in pages having to convert the Markdown to HTML on the fly during runtime, and so the bigger the article, the slower the page load, but I never truly investigating this since I decided to redo the website from the ground up anyway. To the end user, this just felt like the page was frozen for awhile, which was really annoying UX.
• Content and Organization: I wasn’t sure of how I wanted things organized, and I only had a few articles, blogs, and quotes on there. The way I would write my content was by writing the article in Markdown directly on the site and then just committing and pushing the code to Github, and from there it would build and deploy to Netlify automagically. That was nice, but I would prefer to be able to write the article in another program (so that I could have “draft” versions of it and whatnot) and then just commit + push when ready in a single command/click sort of way.

All these problems contributed to a year of experimentation that left my actual ability to write things a bit in limbo.

Substack

I started out by just wanting something that worked, and worked well, for writing, without having to worry about anything else. Substack is the obvious choice for this. It’s famously anti-censorship, has an intuitive UI, a large audience, most of my own subscriptions worth reading are from Substack, easy subscription tiers and connection to Stripe for payments, easy email subscriptions. What’s not to like?

There was one thing that just ended up being a deal breaker for me. Probably a bizarre requirement to most people, but I just couldn’t in good conscience do it. At the time, I was considering requiring subscriptions and payments for some of the articles I wrote, and on Substack, there’s no option for bitcoin payments. The saddest thing of all is, it appears that there was a way to pay with bitcoin while Substack was using OpenNode’s API, but don’t any longer. I suppose that’s a price of vendor lock-in. It is now Substack’s official policy to not support any bitcoin/crypto payments. I work in the Lightning Network because the payments use case of bitcoin is one of if not it’s most exciting aspect. It’s probably one of the strongest ways bitcoin can positively influence how we use the internet. To only allow credit card subscriptions, and not bitcoin lightning micropayments, for paywalled articles if I did any would be sad and hypocritical of me. There are other reasons of course, like the fact that Substack isn’t open source and that I wouldn’t actually own my own data. The idea that my blog dies when the company dies was another major negative.

So what other options were there that fit my increasingly stringent criteria?

Ghost

Enter Ghost, a website which markets itself as essentially the more principled and feature-rich version of Substack (which I believe they are). They are fully open source and you can run the blog software locally yourself, deploy it on a VPS of your choice, setup your own email provider, etc. If you choose to pay for them to host it for you, it’s only $9 a month and that includes the ability to bring your own domain, email subscriptions, etc. A huge improvement to me, and a project and team that I was (and still am) very excited about.

They even have a marketplace of templates and integrations, and given their open-source spirit I was hopeful they’d have some sort of bitcoin integration on there — and there is one…that you have to pay for. It’s called Scribsat; here’s the Ghost Forum post about it. They charge a monthly fee steeper than Ghost’s hosting fees itself to allow you to accept bitcoin lightning payments on Ghost. That didn’t really fit the bill for me, either, and I don’t want to be forced to depend on another service to accept the payments for me. For what it’s worth, this does seem to be a hot topic on the Ghost Forum, and I hope that someone will take up the call of developing something people can use for this. Ghost is a great tool and company, used by Bitcoiners, and would be powerful to many if properly combined with bitcoin payments.

I considered working on this myself, but there is significant overhead that I just didn’t and don’t have the time (for now) to take up.

For awhile, the lack of bitcoin-based payment options was worth it, and I paid for a full year’s subscription even though I wasn’t sure if I’d use it for long (I didn’t, but I consider it a tithe to open-source software). I loved the writing and publishing experience, which was extremely simple, and there were a good number of default themes to choose from. However, none that I loved, and I found myself longing for more customizability. For most bloggers who can pay a few bucks a month or have the wherewithal to self-host, I would definitely recommend Ghost.

Inspiration Draws From Disparate Sources

Shall I tell you the secret of the true scholar? It is this: Every man I meet is my master in some point, and in that I learn of him.

• Ralph Waldo Emerson, Greatness

My website was hosted on Ghost for a couple months, and it actually allowed me to write an article or two and post a quote here and there, and I was starting to give up on monetizing it (was I ever really doing it for that anyway?).

But around this time I was coming into greater contact with online presences that helped me to think differently, and though they all differ radically in occupation, style, and personal beliefs, they share an energy for creating good things meant to last; a passion for thinking long-term and for following their hearts over easy money and fame. A desire to achieve greatness in some form or other, and to have the revenues of their considerable efforts paid in something deeper than cash.

One of those I stumbled across was Luke Smith, a thoroughly original person, whose unique admixture of Twitter-like meme humor, dead-serious power usage of FOSS-only tech, deep historical, philosophical, and linguistic knowledge, and opinionated take on Christianity fascinated me, even if I didn’t agree with many of his opinions and means of presenting that information. It’s rare to find a person occupying all those particular spheres simultaneously, particularly in technology. From him I learned to ask for more from my software, to seek and value the right to and practice of ownership, whether that’s in owning your own land, owning your own money (bitcoin, being debt-free), or owning your own technology to the highest degree possible. He even removed all credit card-based ways to donate to him on his website, as they weren’t in keeping with his principles. I used his tutorial website on becoming an Internet Landlord by self-hosting a website on your own DNS, for example, to build and host the version of this site at the time of writing. Another thing that is important to him is to have technology be a tool you use, rather than you be its’ tool. To that end, he helped encourage me to try to build and use software that will last, and focus less on the ephemeral. Not an easy pursuit in the software world, open source or otherwise!

Another figure who influenced this blog is Henrik Karlsson, whose article A blog post is a very long and complex search query to find fascinating people and make them route interesting stuff to your inbox had a deep impact on me and the direction of my thinking about how a blog should be made. It is excellent, both the idea and the read.

Zig creator Andrew Kelley, also comes to mind, whose article Why We Can’t Have Nice Things helped me realize the incredible waste that often goes into making products worse when money alone is the goal.

Visakan Veerasamy’s article We Were Voyagers is also an amazing and deeply encouraging read, and while it didn’t directly influence how I built this website, it further influenced the “do something useful to others with your life in a spirit of gratitude” approach I’m trying to take to life in general, and will have an effect on the prose and content of the site if nothing else.

But the final major influence was gwern.net, whose beautiful and carefully crafted website design is matched by its rich content and elegant prose. It’s filled with a hundred little features, optimizations, and design decisions that make it a joy to use. For example, it has custom page previews which allow you to see a preview of where a link will take you when hovering over it. It has tables of contents and footnotes for each article, and the footnotes become sidenotes on wide screens for convenience. He takes the time to design artsy dropcaps for each article. And something I found very interesting that I’ve never seen before, is his extensive use of local archives to fight against linkrot, which is an important problem if we wish to preserve something of ourselves for the future. Read his about page for the site to learn more.

All told, Gwern’s website appears to me an exercise in how to create an optimal personal website for the long-run. He summarizes the fundamental question of building a site from the link above:

What does it take to present, for the long-term, complex, highly-referenced, link-intensive, long-form text online as effectively as possible, while conserving the reader’s time & attention?

This version of my site, though still nascent, is based on a fuzzy mesh of these influences, and I will make the attempt to improve it and make it more resilient and higher quality over time.

Knowing the Place for the First Time

Who was I kidding? I’m a web developer. I should’ve known from the beginning it would feel wrong to me to outsource that task, despite the work it may have taken. So after a year of exploration, I’m back at doing the same thing I did before — building the website in Sveltekit and MDsveX. But hopefully this time better, and having learned a lot about how to practically achieve that in addition to learning my own reasons for trying to do so in the first place.

Before, building my own website was a cool thing to put on my resume. But this time I have my reasons.

What Now?

My primary reason for building this site is to document everything I do that I think might be of use to someone else (or my future self). I want something that I built myself and which I have full control over, whose format I have the freedom to modify or restructure at any time (which I couldn’t do in my previous methods). I want something that allows me to publish articles, blogs, daily journals, quotes I find interesting, projects I’m working on, what have you, in as frictionless a manner as possible while still retaining maximum expressive possibilities. Which, in essence, means I have to build things myself, and that there will be a high up-front cost to doing so.

So be it!

Here’s Gibbon on the importance of reading and writing to civilization, and what distinguished the Romans from the “barbarians”, i.e. the Germans:

The Germans, in the age of Tacitus, were unacquainted with the use of letters; and the use of letters is the principal circumstance that distinguishes a civilized people from a herd of savages incapable of knowledge or reflection. Without that artificial help, the human memory soon dissipates or corrupts the ideas intrusted to her charge; and the nobler faculties of the mind, no longer supplied with models or with materials, gradually forget their powers; the judgement becomes feeble and lethargic, the imagination languid or irregular. Fully to apprehend this important truth, let us attempt, in an improved society, to calculate the immense distance between the man of learning and the illiterate peasant. The former, by reading and reflection, multiplies his own experience, and lives in distant ages and remote countries; whilst the latter, rooted to a single spot, and confined to a few years of existence, surpasses, but very little, his fellow-labourer the ox in the exercise of his mental faculties. The same, and even a greater, difference will be found between nations than between individuals; and we may safely pronounce that, without some species of writing, no people has ever preserved the faithful annals of their history, ever made any considerable progress in the abstract sciences, or ever possessed, in any tolerable degree of perfection, the useful and agreeable arts of life.

– Decline and Fall of the Roman Empire, Ch. IX, p. 242

He’s an extremely ancient figure, so much so that no one really knows when he lived, typically placing him somewhere between 1500 to 1000 BC. Zoroastrianism was the religion of the ancient Persians, and dominated much of the Middle East for a millennium, until being overtaken by Islam.

Aside from many of its very interesting ideas about free will, heaven and hell, creation and anti-creation, etc., one idea I found especially refreshing to find in minds that long predate Jesus, Socrates, Confucius, and even the first philosopher – was the rejection of asceticism; the abstinence of sensual pleasures which often also leads to the withdrawal of societal participation and sense of social responsibility. Zoroastrianism apparently differs directly with the founders of many of our modern religions in this regard. To quote Gibbon:

But there are some remarkable instances, in which Zoroaster lays aside the prophet, assumes the legislator, and discovers a liberal concern for private and public happiness, seldom to be found among the grovelling or visionary schemes of superstition. Fasting and celibacy, the common means of purchasing the Divine favour, he condemns with abhorrence, as a criminal rejection of the best gifts of Providence. The saint, in the Magian religion, is obliged to beget children, to plant useful trees, to destroy noxious animals, to convey water to the dry lands of Persia, and to work out his salvation by pursuing all the labours of agriculture.

One may wonder how much pointless human suffering was caused by the elevation of self-inflicted pain to an act smiled upon by God. This isn’t to diminish the merits of well-directed sacrifice – a sacrifice to some social purpose – just sacrifice for sacrifice’s sake.

Continuing with Gibbon:

We may quote from the Zendavesta a wise and benevolent maxim, which compensates for many an absurdity. ‘He who sows the ground with care and diligence, acquires a greater stock of religious merit, than he could gain by the repetition of ten thousand prayers.’

While the rejection of asceticism doesn’t exist without the opposing danger of hedonism, and while I’m sure a closer look at Zoroastrianism would reveal many strange features, it is interesting to see a religion that explicitly celebrates the application of our efforts toward positive social utility. If you’re going to be judged by God, it’s nice to be judged by whether you do good in the world, instead of being judged simply by the degree to which you abstain from participating in its evils.