Made by Mikal

High level themes from CloudCon 2025

Post author:mikal
Post published:September 11, 2025
Post category:Conference

I've just been in Sydney for a couple of days for CloudCon 2025. I think depending on how you count this is my third one of these events -- the event has changed names at least twice, so its actually a little hard to work out the lineage of the event. This year's conference was noticeably smaller than last years which is confusing to me for an event which is so competitively priced and branded so heavily with the hot topics dejour. That said the event was well run, in a good venue, and well worth the time. That is, this event really deserves more support than its getting. There were some clear themes from the event for me: ClickHouse is cool. Or at least I think so. ClickHouse observability certainly has potential, but I think the underlying SQL database is actually the most interesting bit. ClickHouse is also investing heavily in the Australian market right now, so I suspect they're seeing strong traction here. No one talks about "devops" any more, because its become a meaningless term where everything is devops if you squint at it right. Instead people are using the term "platform engineering", which doesn't appear to have…

Converting from setup.py and PBR to pyproject.toml

Post author:mikal
Post published:September 9, 2025
Post category:Python

I make no claim to be an expert at this, but I did just need to convert a project from a slightly complicated setup.py / PBR configuration to pyproject.toml and thought I should write up where I landed. I say “slightly complicated” because there are a few very OpenStacky things I like to do in these things. Specifically:

version numbers are driven by git tags not hard coded in the configuration file.
console scripts are a thing.
I often include data files in the built package.

So here’s an example of all of those things that is working ok for me:

(more…)

I was unaware of Debian’s extrepo command…

Post author:mikal
Post published:August 18, 2025
Post category:Debian

…and its actually a pretty neat idea!

Now admittedly, I encountered extrepo when one of the repos being added to a Docker container build I care about was being unreliable, but that caused me to dig into what it is and how it works and I quite like the idea.

The basic idea is that there are people out there who publish packages for Debian, either as hilarious curl scripts which you’re meant to pipe into a root shell, or as just .deb files which you download and manually install. Instead of being terrible though, what if there was a tool which knew how to provide apt repository configurations for those things? Then, apt would do the right things when the packages changed and so forth. Those configurations could come from a central source which you could also trust to provide the GPG keys that signed those packages. Neat yeah?

(more…)

Edward Zitron is right about why companies struggle long term

Post author:mikal
Post published:July 2, 2025
Post category:Work

I spent a lot of last year trying to understand why companies treated the employees they rely on for success so poorly — The Man Who Broke Capitalism; Lights Out; The Idea Factory; AI Snake Oil; Leaders Eat Last; and so on are all part of that journey. At the time I was a bit fixated on Jack Welsh and his long term impact on General Electric — which I would summarize as being overwhelmingly negative. It was a classic example of managing to short term profit metrics, instead of for long term sustainable growth by delighting your customers.

Ultimately this is why I chose to take a break from working for corporate America, as being treated like a replaceable cog in a profit machine wasn’t really working for me.

Then Edward Zitron wrote this blog post which really resonated with me…

(more…)

Getting Digitech / Bresser 6in1 weather stations working with rtl_433, MQTT, and Home Assistant

Post author:mikal
Post published:June 24, 2025
Post category:HomeAssistant

Please note: this post is a re-write of a previous post about my Vevor weather station that failed after a few months because of ineffective weather seals on the outdoor unit. Given the poor warranty service from Vevor, I instead went and bought a weather station from a local retailer. In my case a Jaycar Digitech XC-0434, which also appears to be sold online as a Bressser 6in1 weather station.

Now that we’ve defeated QNAP’s slightly broken udev, we can run a Docker container with rtl_433 in it to wire up our Digitech / Bresser 6in1 weather station to Home Assistant via MQTT. First off, we need a Docker container running rtl_433, which assumes you’ve already setup the udev rule mentioned in the previous post, even if you’re not using a QNAP!

(more…)

The Complete Robot (again)

Post author:mikal
Post published:June 23, 2025
Post category:Book

I’ve read this book a few times, but honestly the Foundation TV series has left me with a new enthusiasm to re-read some Asimov stuff. I have previously read the entire extended multi-author series, and honestly a fair few of them sucked — especially the ones by other authors — so this time I have the luxury of being a bit more picky. Worse, Asimov remixed the robot stories several times into various volumes, and it can be quite confusing. The Complete Robot contains all the robot stories, and replaces I, Robot (reading one, reading two), The Rest of the Robots; Robot Dreams; and Robot Visions. It also contains a couple of previously unpublished stories.

(more…)

Title: The Complete Robot
Author: Isaac Asimov
Genre: Robots
Release Date: 1982
Pages: 688

Training isn’t a work perk, its essential operational risk mitigation

Post author:mikal
Post published:June 13, 2025
Post category:Work

I’ve been thinking a bit about training at work recently, largely in the context of having spent the last twenty years working for US technology companies. I think effectively all of these companies made a pretty big mistake — they viewed training of employees as a perk much like vacation, book budgets, or t-shirts. They advertise their training programs as part of their recruitment process, and just like other perks they’re cut when times get a bit grim. However, that’s not actually why employers should train their people. We train people so that they have the skills they need to do their jobs — especially when things get real and aren’t working out to plan.

There are definitely industries who have good examples of this sort of risk reduction training done well — airlines and the military both engage in regular training activities that ensure that when things get exciting the people know what to do. This includes leaders being trained on how to make decisions that are likely to result in the desired outcomes.

(more…)

I really wanted to like etcd, but Andy Pavlo was right

Post author:mikal
Post published:June 9, 2025
Post category:Databases etcd Shaken Fist

Andy Pavlo of the CMU Database Group is well known for saying that while NoSQL databases acquire cyclical popularity, all databases eventually iterate back to a SQL interface — it happened with MongoDB and Google’s BigTable for example.

I think I have hit that point with etcd. Initially I ported from MySQL to etcd because I really wanted the inexpensive distributed locking and being able to watch values. However, I never actually watch values in my code any more, and I now spend a huge amount of my time maintaining what my code calls “caches”, but which I can now see are just poorly implemented secondary indexes. The straw that broke the camel’s back was https://github.com/etcd-io/etcd/issues/9043, which changed etcd’s defaults to only being able to return 1.5mb in a RPC request.

I therefore think it might be time for me to port back to a real SQL database, perhaps keeping etcd to manage distributed locks. Perhaps.

(more…)

Databases built on object stores are officially interesting

Post author:mikal
Post published:June 4, 2025
Post category:Databases

…even if none of my friends seem to think so.

I’ve been off on a bit of a tangent recently. Its a slow burn tangent, that I am pretty sure was kicked off by this Geek Narrator podcast episode about the design of Turbo Puffer with Simon Eskildsen:

The basic idea is that you can build very large scale database systems using only the primitives provided by an object store such as Amazon S3. Now, the performance might also suck, but you can alleviate some of that with a good caching layer and in return you get massive scale. This first video caused me to discover the work of Andy Pavlo, who was interviewed by the same podcast:

(more…)

The simplest boot target for the Kerbside SPICE VDI proxy CI

Post author:mikal
Post published:June 1, 2025
Post category:Kerbside OpenStack Shaken Fist

For the last couple of years I have been working on a SPICE protocol native proxy called Kerbside. The basic idea is to be able to provide SPICE Virtual Desktop Interface (VDI) consoles to users from cloud platforms such as Shaken Fist, OpenStack, or oVirt. Think Citrix, but for Open Source cloud platforms. SPICE is attractive here because it has some features that other more common VDI protocols like VNC don’t have — good cut and paste support, USB device pass-through, multiple monitor support, and so on. RDP has these, but RDP was not a supported VDI protocol when using qemu on Linux with KVM until incredibly recently — literally the last couple of months.

(In terms of clouds that Kerbside supports, I think it would be relatively trivial to also support Proxmox, KubeVirt, or a list of static manually created virtual machines, but there’s only so many things one Mikal can do at once…)

Some of these cloud platforms have supported SPICE consoles for a while, but generally with warts. OpenStack for example only exposes them as HTML5 transcoded sessions with reduced functionality. oVirt exposes them via a “proxy” which is just squid (or equivalent), but its fairly dumb — it exposes the underlying hypervisor details to the client for example. I thought I could do better than that.

(more…)