The Secret Lives of Data, a visualisation of the Raft algorithm

One of the big problems in computer science is distributed consensus. This is the problem of getting a set of nodes in a network (or distributed system) to agree on something: perhaps a value, an action, or a record of history. Some of the nodes in this system will be faulty, and drop messages, so you need to be able to work around that as well. This turns out to be very difficult to solve.

There are two commonly-used algorithms for solving this problem: Paxos and Raft. Both have been mathematically proven to work to solve the consensus problem.

As part of my day job (Project Calico), we use etcd as a distributed database layer, which implements the Raft algorithm for consensus. Since Raft was explicitly designed to be understandable (as opposed to Paxos’s reputation for inscrutability), I thought it was worth trying to understand how it works.

I came across this visualisation, which explains how the Raft algorithm works. It takes you from the basics of the consensus problem, through the design of the Raft algorithm and explains how it copes when the network starts to fail. I found it really interesting, and I think it’s well worth a read.

Previewing notes from nvALT in Marked

I find nvALT to be an indispensable note-taking application. I have thousands of plain-text notes, but it’s still incredibly fast and easy to look up a specific note. I also lean heavily on Marked for previewing notes – particularly complex notes with lots of links and images.

Until recently, I’ve been using a Keyboard Maestro macro from Patrick Welker to take a note from nvALT and preview it in Marked. The AppleScript in the macro takes the title of a note, converts it to a filename, and passes the filename to Marked.

That works in about 95% of cases, but I’ve encountered two problems:

  1. OS X does strange things with file separators (the colon and the slash). Having a colon or slash in the title of a note means that it isn’t picked up by this script.
  2. The script assumes that all my notes have the same extension: .md. This is almost always the case, but sometimes notes appear with the .txt or .mdown extension. I could play whack-a-mole with file extensions, but it’s easier to have the script do it for me.

I’ve written a Python script to replace the AppleScript, which seems to solve both of these problems. I’ve been using it for the last few weeks, and now I’d like to share it.

Continue reading →

One-step paste in the iOS Simulator

On the latest episode of The Talk Show, John Gruber lamented the two-step paste process into the iOS Simulator (about fifteen minutes in). It goes like this:

  1. In OS X, you copy some text, which gets saved to the OS X clipboard.
  2. You switch to the iOS Simulator, and paste, which saves it to the iOS clipboard.
  3. You long click in iOS within the Simulator, and paste again (this simulates the iOS paste action), which causes the text to be pasted in iOS.

Having to paste twice is mildly annoying, but it’s easy to fix. I have a Keyboard Maestro macro that lets me paste directly into the copy of iOS running in the Simulator.

The macro itself is very simple:

It’s just a tweaked version of Gabe Weatherhead’s Paste as Typed Text macro, which is useful for pasting passwords into fields where pasting is disabled. I have it inside a macro group, which is configured to only be enabled in the iOS Simulator.

For it to work, you need to have the hardware keyboard connected in the iOS Simulator (Hardware > Keyboard > Connect Hardware Keyboard). This is what allows you to type in the iOS Simulator with your regular keyboard, and lets Keyboard Maestro pass in keystrokes.

There are a few other notes:

  • I have this mapped to ⌘V. This is the same as the default Paste shortcut in the Simulator. In my usage, the conflict hasn’t caused any problems, but if it does, you can override the default Paste shortcut in the “Shortcuts” tab of the Keyboard Preference Pane.

  • Occasionally I find apps in OS X that break the Paste as Typed Text. This macro has always worked for me in the iOS Simulator, but that’s not to say it will always work everywhere.

  • This will only work for text, not images or files.

  • I’ve only tested this with the iOS Simulator that comes with Xcode 6. Apple has a habit of changing the Simulator between versions of Xcode, so it might not work if you use a different version. (Since the macro bypasses the Simulator, I’d expect it to keep working, but no guarantees.)

Electoral reform

A common criticism of the UK voting system (First Past the Post or FPTP) is that the share of the popular vote and the share of Parliamentary seats is often wildly different. This was true again in yesterday’s election.

I made this chart, which shows the difference between a party’s share of the seats, and their share of the popular vote. It’s not great:

difference between % seats and % votes

Seat and vote counts from the BBC.

Both the major parties did well from FPTP, as is usually the case. The other major beneficiary was the SNP, which isn’t surprising. The smaller parties (Greens, UKIP and Lib Dems) often suffer from having a voting base that’s spread across the country, but not concentrated in many areas. That makes it harder to win in individual seats. Since the SNP only run candidates in Scotland, I’m sure that helped them here.

The disparity between the large and the small parties is not new, nor is the disparity between votes and seats. But it is disappointing.

In 2011, I supported a change to the Alternative Vote (although I was too young to actually vote). I still think it would have been a good idea, but I worry that the failure of that campaign has shut out electoral reform for a generation. And as long as one of the two main parties is in government, they have no incentive to change the system, because it treats them so well.

I strongly believe that we need to scrap FPTP, and bring in a new, more proportional voting system. It will make our politics fairer, give smaller parties a proper voice, and go some way to correcting these disparities. But I’m at a total loss to see how we might do that.

Some exam advice

As the exam season approaches, I see a gentle trickle of hits to my exam advice for first-year maths students. It’s been a year since I wrote it, and I still agree with almost all of it.

Looking it over, I think a lot of it is applicable to other technical subjects, so I’ve tidied it up and tried to make the advice a little less maths-specific. Rather than letting it get lost in my blog archives, I’ve also given it a new, permanent page.

I hope people continue to read it and find it useful, and I wish the best of luck to everybody sitting exams in May.

The Level: a deck designed to set you in motion

After I wrote about my standing desk last year, I got a nice email pointing me to this Indiegogo project.

It’s a board that you place under your feet while you’re at a desk, and it has a curved underside to keep it moving and tilting. As the board tilts, you move your feet and maintain the blood flow in your legs.

Here’s what it looks like:

It’s an interesting idea, and I can see the potential health benefits. I’d want to try one in person before I committed to backing it – I suspect I might just find it more annoying than anything else – but I’m definitely intrigued. They’re well past their goal, and just a few days left to go on the project. I’ll keep an eye on it, and follow the reviews when the board is released to backers.

In the meantime, my low-tech solution for maintaining circulation in my legs is to take regular breaks. Whether sitting or standing, it’s good to get up and walk around the room at least once an hour. It’s probably not as good as continual gentle motions, but it’s cheap and it’s easy.


I really like Pygments. It’s a Python module for applying syntax highlighting to code in printed documents. I use it on this blog, and when I had to include code listings in my university coursework, I used it with the Minted package in LaTeX.

But recently I’ve needed to generate standalone code listings that aren’t part of a larger document. I was using the command-line each time, but to make it a little easier, I’ve written a small web app instead. Here’s a screenshot:

I enter the code in the first box, select the language, and click the button. The HTML and rendered preview appear at the bottom of the page. Really simple.

The code is all on GitHub, and below I’ll explain why I wrote it.

Continue reading →

Adding Python to the OS X firewall

If you ever get the warning “Do you want the application “python” to accept incoming connections?” on OS X, then somebody on Stack Overflow has helpfully written some instructions to keep it quiet.

My current side project is a bookmarking app written in Flask, and I get this message every time the app restarts. It’s mildly annoying, and I’m glad to see the back of it.

Tidying up my 1Password

I’ve been creating online accounts for years, and saving them in 1Password, but everything was a bit scattered. I’d never stopped to organise or tidy up my database. I also had a lot of weak or duplicate passwords that I’d never changed.

Last week, I set about cleaning up my 1Password database. This was more than just changing some old passwords: it was a thorough and deep-level reorganisation.

Continue reading →

A Flask app for taking notes

This is a pretty good plaintext notes web app. I’d been looking for something to use to take notes at work, and I’ve been trying this for about a month. I think I’ll keep using it.

It has all the features I need: Markdown support, full text search, and it runs on Windows. As a bonus, because it’s open source, I can easily tweak it where I want. (Some features were added in later posts.)

Flask is a small web server framework for Python that I’ve been using for a couple of recent projects. I was considering rolling my own notes app, but this one is pretty good.

← Older Posts