Growing Desperate

Just some white guy. I used to write about software stuff. Nowadays I’m looking for ways to put out the myriad dumpster fires people like me set, stoke, and pour gasoline on since forever ago.

Read this first

The fundamental issue

This is the most coherent take I’ve seen on the current state of the left/right, Republican/Democratic divide in the U.S. today.

I had a hard time not pull-quoting the whole damn thing, but here’s one of many head-nodders:

The American creed has no more devoted adherents than those who have been historically denied its promises, and no more fair-weather friends than those who have taken them for granted.

Spot. Fucking. On.

Read the whole thing.

View →


RabbitMQ cluster on kubernetes with StatefulSets

UPDATE 2018-1-8: Don’t do it this way. Use the k8s support built in to either the official autocluster plugin (for RabbitMQ 3.6.x) or the new built-in peer discovery feature in RabbitMQ 3.7.x+.

UPDATE 2017-3-18: Improved the postStart command based on further testing and refinement.

Since I couldn’t find a blog post showing how to do this, I told myself I should write one up once I figured it out. So here it is.

Kubernetes has a relatively new feature called StatefulSets that is designed to make it easier to run containerized services that are inherently stateful. And RabbitMQ is nothing if not inherently stateful. RabbitMQ clusters double down on the whole statefulness thing. After a few years of experience trying to force RabbitMQ clusters into stateless container infrastructure, I was curious to try embracing the statefulness instead. Paint with the grain and all that.

The first...

Continue reading →


You don’t (always) get to decide when you pay down technical debt

“[I]f you cut corners in the short term, you’re borrowing against the future maintainability of the software and that at some point, not unlike a credit card, you’ll pay down that debt by going back and fixing the hacky solutions that were put in place.” - Ben Lakey in this Programmers StackExchange answer.

The quote above is one of the best explanations for the term “technical debt” that I’ve run across. I like that it’s succinct, and I like it’s credit card analogy. Like credit card debt’s increasing minimum payments, loading up on technical debt will represent an ever-increasing drag on your productivity. This plays out in the “How does this work again? I wish we had docs,” and “Uh oh, we broke feature X when we added feature Y. I wish we had tests,” type of problems that you routinely run into if you are cutting corners.

But even worse than credit card debt (whose collectors will...

Continue reading →


Deploying Clojure apps with Docker and Immutant 2

While Docker is a very cool technology that holds a lot of promise for the future of devops, it’s not always the best fit for JVM-based apps. The standard approach of baking every runtime dependency into the image and running each container as a self-contained system gets out of hand rather quickly from a resource utilization perspective. How many JVMs can your server fit into memory at once?

At Democracy Works we wanted to find a way to share JVMs, for this and other reasons1. We use Immutant as our production app server for this. Our first attempt to combine Docker and Immutant 1 worked but it was a bit clunky. It involved building .ima (Immutant archive) files in the CMD’s of the app containers and then doing a –volumes-from for each (now-stopped) app container when we ran the Immutant container. Especially in dev, it was hard to get all the pieces to line up just so. We no longer...

Continue reading →


CoreOS, Docker, and batteries

When the CoreOS folks announced Rocket, it was with much concern over the “new direction” of the Docker project, as they saw it. They felt compelled to launch a competitor to Docker because Docker was straying from its roots as being merely a container standard and runtime.

Something rang a bit hollow as I read that original announcement, and it was not the theory that CoreOS was really responding to Docker encroaching on the market they were trying to capture. Others have speculated on this at length, and there probably is something to it. But no, I think the thing that rang hollow to me was CoreOS’ face-value, negative reaction to Docker’s “batteries included, but removable” philosophy. That finally came into focus today as Ubuntu announced Snappy Ubuntu Core (or is it “Ubuntu Snappy” or “Ubuntu Core Snappy”?).

Snappy Ubuntu Core is positioning itself in the same space as CoreOS (and...

Continue reading →


Recruiting Software Developers: Language Matters

When I first proposed that we start doing all of our new development at Democracy Works in Clojure instead of Ruby, one of the first concerns brought up by my colleagues was developer recruitment. The assumption–which I agreed with at the time–was that pulling from a much smaller pool of talent would hamper our ability to find people with experience in our language-of-choice (let’s leave aside for now the debate over whether that should be a priority in the first place). It was probably the biggest concern we had with moving to the new language.

Then a funny thing happened1. As soon as we became “a Clojure shop,” smart, experienced developers wanted to work with us. Rather than being yet-another-Rails-shop and having to convince highly-sought-after Ruby developers why they should work here instead of there, we were one of the few places offering developers who had discovered this new...

Continue reading →