alexwlchan - page 2


Displaying tweets in Keynote

Every so often, I want to use a tweet in some slides I’m making (I have three in my PyCon UK slides for Friday). If I’m doing this, I want to make it clear that the text I’m using is a tweet, not just a generic quote. Tweets have quite a distinct visual style, and give a very clear way to find the original author.

Twitter gives you an “Embed Tweet” button for using on web pages, but I’m not sure if you can use this in Keynote or PowerPoint — and given it has to make a network call to display the tweet properly, do you want to rely on it in a presentation?

Screenshots are better, but still not ideal — you lose the text, so your presentation becomes less accessible. You can also get fuzzy text if you have to resize the tweet or took a small screenshot.

Far better to draw it using your app’s drawing tools as a static image, which is exactly what I do in Keynote. Then the text is directly embedded (more accessible), and text always looks nice and crisp. This is what the effect looks like, with a single tweet per slide (more than one gets distracting):

It’s on the small side for text on a slide, but I’ve found it to work well if deployed sparingly.

If you’d like to use these templates, I’ve uploaded the Keynote file that has both these slides, and templates for creating more. It will probably work in PowerPoint, although I don’t have a copy of PowerPoint to test with.

Enjoy!


Using hooks for custom behaviour in requests

Recently I’ve been writing a lot of scripts with python-requests to interact with a new API. It starts off with a simple GET request:

resp = requests.get('http://example.com/api/v1/assets', params={...})

I want to make sure that the request succeeded before I carry on, so I throw an exception if I got an error responses:

resp = requests.get('http://example.com/api/v1/assets', params={...})
resp.raise_for_status()

If I get an error, the server response may contain useful debugging information, so let’s log that as well (and actually, logging it might be generally useful):

resp = requests.get('http://example.com/api/v1/assets', params={...})

try:
    resp.raise_for_status()
except requests.HTTPError:
    logger.error('Received error %s', resp.text)
    raise
else:
    logger.debug('Received response %s', resp.text)

And depending on the API, I may want even more checks or logging. For example, APIs that always return an HTTP 200 OK, but embedded the real response code in a JSON response. Or maybe I want to log the URL I requested.

If I’m making lots of calls to the same API, repeating this code gets quite tedious. Previously I would have wrapped requests.get in a helper function, but that relies on me remembering to use the wrapper.

It turns out there’s a better way — today I learnt that requests has a hook mechanism that allows you to provide functions that are called after every response. In this post, I’ll show you some simple examples of hooks that I’m already using to clean up my code.

Read more →


Control Centre: one step forward, two steps back

I’m not much of an iOS power user, and these days, most new features go straight over my head. As such, there wasn’t much in iOS 11 to interest me, and it took me a while to get round to upgrading.

One thing I was looking forward to was the new Control Centre. The ability to customise controls could come in handy, and doing away with the separate pages seemed like an easy win. Plus, I think the new version just looks nicer.

Now I’ve been using it for several weeks, I’m more ambivalent. Customisation has been really useful — I’ve done away with the unused calculator shortcut, and brought in Low Power Mode, which I use all the time. Most of the buttons look good and are easy to hit, and I’m having much more success with the chunky brightness and volume sliders. But as it advances in one area, so it slips in another. I have two big problems with the new Control Centre.

Read more →


Four ways to underline text in LaTeX

Because I’m old-fashioned, I still write printed documents in LaTeX, and I still think hyperlinks should be underlined. In general, I’m glad that underlines as a form of emphasis have gone away (boldface or italics are much nicer) — but I have yet to be convinced to drop underlines on hyperlinks.

Sometimes I have to write printed documents that contain hyperlinks, which begs the question: how do you write underlines in LaTeX? Finding an underline I like has proven surprisingly hard — in this post, I’ll show you the different ways I’ve tried to underline text.

Using the \underline command

Without installing any packages, you can just use the \underline command. Here’s an example:

I visited \underline{Berlin} in \underline{Germany}.

and the rendered output:

The underline on “Berlin” is nice and tight — but notice how the underline on “Germany” is lower than “Berlin”. That’s to accommodate the descender on the “y”. (A descender is any part of a letter that extends below the baseline of the text. For example, “p”, “y” and “j” all have descenders, but “a”, “i” and “x” don’t.)

The inconsistency is what I don’t like about this approach. It’s fine for one-off underlines, but in a larger document, the inconsistency gets very obvious, and I don’t like how it looks.

Read more →


Using pip-tools to manage my Python dependencies

At last year’s PyCon UK, one of my favourite talks was Aaron Bassett’s session on Python dependency management. He showed us a package called pip-tools, and I’ve been using it ever since.

pip-tools is used to manage your pip dependencies. It allows you to write a top-level summary of the packages you need, for example:

$ cat requirements.in
pytest >= 1.4
requests

Here I want a version of pytest that’s at least 1.4, and any version of requests.

Then I run pip-compile, which turns that into a full requirements.txt:

$ pip-compile
$ cat requirements.txt
certifi==2017.7.27.1      # via requests
chardet==3.0.4            # via requests
idna==2.6                 # via requests
py==1.4.34                # via pytest
pytest==3.2.2
requests==2.18.4
urllib3==1.22             # via requests

I can install these dependencies with pip install -r requirements.txt.

The generated file is pinned: every package has a fixed version. This means that I get the same versions whenever I run pip install, no matter what the new version is. If you don’t pin your dependencies, your package manager may silently install a new version when it’s released – and that’s an easy way for bugs to sneak in.

Instead, check in both files into version control, so you can see exactly when a dependency version was changed. This makes it easier to see if a version bump introduced a bump.

There are also comments to explain why you need a particular package: for example, I’m installing certifi because it’s required by requests.

I’ve been using pip-tools since Aaron’s recommendation, and it’s been really nice. It’s not had an earth-shattering impact on my workflow, but it shaves off a bunch of rough edges. If you do any work with Python, I recommend giving it a look.

For more about pip-tools itself, I recommend Better Package Management by Vincent Driessen, one of the pip-tools authors. This human-readable/pinned-package distinction is coming to vanilla pip in the form of Pipfile, but that was in its infancy last September. pip-tools has been stable for over two years.


Recently, I’ve been trying to push more of my tools inside Docker. Every tool I run in Docker is one less tool I have to install locally, so I can get up-and-running that much faster. Handily, there’s already a Docker image for running pip-tools.

You run it as follows:

$ docker run --volume /path/to/repo:/src --rm micktwomey/pip-tools

It looks for a requirements.in in /src, so we mount the repo in that directory — this gives the container the ability to read the file, and write a requirements.txt back into a file on the host system. I also add the --rm flag, which cleans up the countainer after it’s finished running.

If you already have Docker, this is a nice way to use pip-tools without installing it locally.


Alongside Docker, I’ve been defining more of my build processes in Makefiles. Having Docker commands is useful, but I don’t want to have to remember all the flags every time I use them. Writing a Makefile gives me shortcuts for common tasks.

This is the Make task I have for updating a requirements.txt:

requirements.txt: requirements.in
    docker run --volume $(CURDIR):/src --rm micktwomey/pip-tools
    touch requirements.txt

To use it, run make requirements.txt.

The first line specifies the Make target (requirements.txt), and tells Make that it depends on requirements.in. So when the Make task is invoked, it checks that the .in file exists, and then whether the .in file was updated more recently than .txt. If yes — the .txt file needs rebuilding. If no — we’re up-to-date, there’s nothing to do.

The second line runs the Docker command explained above, using the Make variable $(CURDIR) to get the current directory.

Finally, touch ensures that the last modified time of requirements.txt is always updated. pip-tools will only change the modification time if there are changes to the dependency pins — I change it manually so that make knows the task has run, and the “should I run this task” logic explained above doesn’t spin endlessly.

Once I have this Make task, I can invoke it from other tasks — for example, build tasks that install from requirements.txt — and so it gets run when required, but without an explicit action from me. It’s just another step that happens transparently when I run make build.

If you’d like to see an example of this in use, check out the Makefile changes in the same patch as this post.


NPR: Graduation Readers At MIT Go The Extra Mile To Pronounce Names Correctly

I really enjoyed this NPR story about how MIT try to get accurate pronunciations of names at commencement. They’re really trying to make sure they get every name right:

When we feel that we just aren’t comfortable with the pronunciation of the name, we will send an email to 60, 40, 75 students, sort of depends on the year, saying I would like to pronounce your name correctly at commencement. Could you please call my voicemail and leave your name slowly twice? And, you know, thank you very much.

The piece is only short — a little under five minutes — and worth listening to in full. The transcript struggles to convey the nuances of pronunciation, it’s hard in written text!

My name is on the simpler side, so it’s rare for my name to be mangled — but I still notice and appreciate when somebody is trying to get the names right.

On the flipside, something I find particularly tiresome is people who don’t just mispronounce names, but try to joke about doing so. “Oh dear, isn’t Welsh so hard to pronounce, woe is me.” An inability to even try to pronounce certain names isn’t funny or clever, and laughing about it just draws attention to your laziness. Growing up near Wales, I heard a lot of jokes from English people about the unpronouncability of Welsh names – this from a country that has names like Gloucester and Worcestershire.

If you’re introducing somebody at an event, it’s polite to ask how their name is pronounced. You don’t have to be perfect, but you should at least try.


What happens when you overengineer a static site?

When I started this site, I was using Jekyll (well, Octopress). Frequent issues with maintaining a working Ruby installation caused me to look elsewhere for a Python solution. For a while, I used Pelican, but licensing issues and a sense that the project had been abandoned by maintainers led me to write my own static site generator (SSG). Recently, I’ve come full circle and returned to Jekyll.

Writing my own SSG was a fun exercise, but a bit of a time sink. So I started thinking about switching back to something I didn’t have to maintain myself – and the two popular choices seem to be Jekyll or Hugo.

Today, I’m very happy using Docker, which can handle the problems of keeping a working Ruby install. Jekyll has an edge on longevity, and the plug-in architecture offers lots of room for customisation. As far as I know, Hugo doesn’t have plug-in support – I’ve built up some pretty esoteric features on this site, so customisation is a must-have.

With the newest rewrite, I wanted to treat this like a proper software project. Builds in Docker, continuous testing in CI, and automated deployments. It took more work, but it means I don’t just have a pile of hacked-together scripts.

If you’re interested, I’ve put my entire Jekyll setup on GitHub. It has all of my Jekyll config, the Docker containers I use to build the site, and a bunch of interesting plug-ins – check out the README for more details. Over time, I might write up some of the interesting bits as standalone blog posts.

Hopefully this is the last rewrite I’ll be doing in a while – so I wanted to do this one properly.


Some useful Git commands for CI

I spend a lot of time writing build scripts for interacting with Git repos. But Git’s documentation is notoriously opaque, and a lot of my Git knowledge comes from word-of-mouth rather than reading the docs. In that spirit of sharing, these are few of the Git commands I find useful when writing build scripts.

Read more →


Ode to docopt

Every week, we have an hour of developer learning at work – maybe a talk, a workshop, or some other session about of topic of interest to developers. Last week, we did a round of lightning talks. We’re quite a mixed group of developers – in background, technical stacks, and what we actually work on – so coming up with a topic that’s useful to all can be tricky.

For my slot, I decided to wax lyrical about the docopt library. Once upon a time, I was sceptical, but it’s become my go-to library for any sort of command-line interface. Rather than fiddling with argparse and the like, I just write a docopt help string, and the hard work is done for me. I’ve used it in multiple languages, and thought it might be handy for other devs at work. Ergo, this talk.

You can download my slides as a PDF, or read the notes below.

Read more →


A Python module for lazy reading of file objects

At work, we often pass data around via large files kept in Amazon S3 – XML exports from legacy applications, large log files, JSON dumps of Elasticsearch indexes – that sort of thing. The services that deal with these files run in Docker containers on AWS, and they have limited memory and local storage.

Downloading large files into memory is slow, expensive, and often unnecessary. Many of these files contain a list of records, which we want to process one-at-a-time. We only need to hold a single record in memory at a time, not the whole file.

Python can do efficient line-by-line processing of local files. The following code only reads a line at a time:

with open('very-long-file.txt') as f:
    for line in f:
        do_stuff_with(line)

This is more efficient, and usually results in faster code – but you can only do this for local files, and the only delimiter is a newline. You need a different wrapper if you want to do this for files in S3, or use a different delimiter – and that’s what this module does. It goes like this:

import boto3
from lazyreader import lazyread

s3 = boto3.client('s3')
s3_object = client.get_object(Bucket='example-bucket', Key='records.txt')
body = s3_object['Body']

for doc in lazyread(body, delimiter=b';'):
    print(doc)

The code isn’t especially complicated, just a little fiddly, but I think it’s a useful standalone component.

I was mildly surprised that something like this doesn’t already exist, or if it does, I couldn’t find the right search terms! If you know an existing module that does this, please let me know.

You can install lazyreader from PyPI (pip install lazyreader), or see the README for more details.