Today I Learned (TIL)

TIL stands for today I learned. This is a collection of small, practical things I’ve learnt while writing software, which I thought were worth remembering and sharing with other people.

If you want to follow along, these posts have their own RSS feed.

• Drawing a diagonal banner over the corner of an image

• Find the shortest prefix to identify a string in Ruby

  The built-in Abbrev module can calculate a set of unambiguous abbreviations for a set of strings, and then you can look for the shortest result for each string.

• How to get the target of an HTTP redirect with curl

• What is the author_name in the list of tags on a Flickr photo?

  When you call the flickr.photos.getInfo API, each tag is attributed to an author. The author_name is their username, not their realname.

• How to get a user’s email address with the Flickr API

  The flickr.profile.getProfile API returns somebody’s email address, but only if you’re allowed to see it.

• Making an “under construction” element in pure CSS

  Using a repeating-linear-gradient as the border gets you something that looks a bit like hazard tape.

• What errors can you get from hyperlink.parse?

  Mostly you get hyperlink.URLParseError, but you can occasionally get a ValueError as well.

• HTML strings may not be equivalent if you minify them

  There’s a lot of whitespace in HTML which looks irrelevant at first glance, but may be significant and cause the document to render differently.

• How do the ispublic, isfriend and isfamily flags work in the Flickr API?

• Use std::io::IsTerminal to detect if you’re running in the terminal in Rust

  This allows me to suppress ANSI escape codes if the output is going somewhere other than the terminal.

• Use $_ to get the last argument to the previous bash command

  This allows you to write commands like mkdir myfolder && cd $_ or git init myrepo && cd $_.

• With Flask-Login, you want current_user == None, not current_user is None

  current_user is a proxy object that happens to be wrapping None, but isn’t actually None.

• How to get the expiry date of an HTTPS certificate

• Write to the middle of a file with Python

  Open the file with mode r+ to be able to seek around the file and write to it.

• Create solid-colour image placeholders to show before an image loads

• Get an image from a video with ffmpeg

• Get the embedded artwork from an MP3 file

  Use the command eyeD3 [MP3_FILE] --write-images [FOLDER].

• Convert an animated GIF to an MP4 with ffmpeg

• Using the Wikimedia Commons API to tell if a file has been deleted

• Build a URL with query string parameters with curl

  A combination of --get and --data/--data-urlencode allows you to write curl commands which are readable and expressive.

• How to get the selected item in Finder using AppleScript

• Removing letterboxing from a video screenshot with ImageMagick

  Using -trim will remove the black portions and leave you the unletterboxed image.

• How to check the quality of a network connection

  Using an NWPathMonitor and inspecting the value of NWPath.status, NWPath.isExpensive and NWPath.isConstrained can tell you what sort of connection you’re running on.

• How to highlight Python console sessions in Jekyll

  Adding a couple of options to the console lexer (console?lang=python&prompt=>>>) gets you syntax highlighting for a Python console session.

• Not all coal is the same

• How to simulate an [Errno 54] Connection reset by peer when using pytest

  You can run a TCP server in the background using a fixture, and using the SO_LINGER socket option can reset the connection.

• What does “insomnolent” mean?

• Open a Safari webarchive from Twitter/X without being redirected

  Disabling JavaScript when you open the webarchive file will prevent you from redirecting you to twitter.com.

• Writing a file in Swift, but only if it doesn’t already exist

  Adding .withoutOverwriting to your write() call will prevent you from overwriting a file that already exists.

• What’s inside a Safari webarchive?

  The inside of a .webarchive file is a binary property list with the complete responses and some request metadata.

• Get a Palette colour as a command-line argument with Clap

  Wrapping a Palette:Srgb in a struct and implementing FromStr for the struct allows you to take hexadecimal colours as command-line inputs.

• How to get the filename/size without downloading a file in curl

  You can do some fun stuff with the --write-out flag and variables.

• There’s a musical that tells you the number of minutes in a year

  The song Seasons of Love from Rent starts with the line “Five hundred twenty-five thousand, six hundred minutes”.

• How to get a tally of tally counts in SQLite

  Using a nested query allows me to perform a two-level aggregation of the values in a column – how many values appear once, how many twice, and so on.

• How to get a list of captures from the Wayback Machine

  Use the CDX Server API to get a list of captures for a particular URL.

• How to take a screenshot of a page in the Wayback Machine

  Using Playwright to take screenshots and adding some custom styles gets a screenshot of a page without the Wayback Machine overlay.

• How to count how many Discord messages were sent on a given day

  Using the During filter gives me a count of how many messages were being sent.

• How to see the HTTP requests being made by pywikibot

  To see exactly what HTTP requests were being made, I modified the library so that betamax would record requests.

• How to embed an inline SVG in a CSS rule

  Modern browsers allow you to embed the SVG almost as-is, with just a couple of characters that need escaping – no base64 required!

• How to shuffle an array in a Jekyll template

  If you want an array in random order, you can use the sample filter to get a random sample of the same size as the original array.

• Checking if a URL has changed when you fetch it over HTTP

  When you make an HTTP request, you can use the If-Modified-Since header to get a 304 Not Modified if nothing has changed since your last request.

• The open command can ask questions

  If you pass an argument that can’t be easily identified as a file or a URL, open will ask you what to do next. This may be a surprise if you were trying to use it in a script.

• Why is Pillow rotating my image when I save it?

  Images can have orientation specified in their EXIF metadata, which isn’t preserved when you open and save an image with Pillow.

• How to change the name of an internal link in an Obsidian table

  Escaping the pipe like [[filename\|display text]] allows you to customise the of a link in a table.

• Getting a boto3 Session for an IAM role using Python

  Why I use Sessions in boto3, and the Python function I use to create them.

• My config for running youtube-dl

  The flags and arguments I find useful when I’m using youtube-dl.

• How much will Mastodon instances try to re-send messages?

• Get my Netlify bandwidth usage from the API

• How to restrict a page to specific IP addresses

• Use the -n/-i flags to avoid overwriting files with cp and mv

• How to parse URLs in JXA

• Using errexit and arithmetic expressions in bash

• How to check when an HTTPS certificate expires

• The COUNT(X) function only counts non-null values

• What happens when you replace a photo on Flickr?

• Exclude files from Time Machine with tmutil addexclusion

• You need to call resp.close() to close the file opened by send_file()

• Add the -v flag to see what rm is deleting

• Custom error pages in Flask

  You can use app.error_handler to add custom responses for HTTP status codes, so the errors match the look and feel of the rest of the site.

• Use the {% raw %} tag to describe Liquid in Liquid

  If you’re trying to write about using Liquid tags in a Liquid-based site, wrapping your tags in the {% raw %} tag will prevent them being rendered.

• What characters are allowed in titles on Wikimedia Commons?

• Run a script on macOS on a schedule using a LaunchAgent

• Beware of using test -n with command expansion

• How to do offline geo-lookups of IP addresses

  MaxMind offer databases you can do to look up IP addresses without sending the address off to a remote service.

• How to tally combinations of values across multiple columns

• How to move files when you need sudo on the remote server

• How to gracefully restart a gunicorn app

• How to find the longest common suffix in a list of strings in Python

• How to simulate shell pipes with the subprocess module

• Use the {% capture %} tag to assign complex strings to variables

  If you want to get a string that’s semi-complicated to construct, you can put a “mini-template” in the {% capture %} tag to build it over multiple lines.

• How to create a footer that’s always at the bottom of the page

• WordPress URLs that get hammered by spammers

• Manage MP3 metadata from iTunes with eyed3

• Sort a list of DOM elements by sorting and calling appendChild()

• How to create flag emojis for countries in Python

• Use the IMAGE function to insert an image into a spreadsheet

• How to style a <details> element differently depending on whether it’s open or closed

• Pausing the animation of <svg> elements can affect child <svg> elements differently in different browsers

• Create a directory before you cp or mv a file to it

• Running the Netlify CLI in GitHub Actions

• Use shlex.split() to parse log files quickly

• Use the regex library to get Unicode property escapes in Python

• Get and manipulate the contents of a page in Safari with "do JavaScript"

• SVGs are only rendered on GitHub if you use an <img> that points to another file

• Animate an attribute of an element with <animate>

• Run a randomly selected subset of tests with pytest

  By reading the code for the pytest-random-order plugin, I was able to write a new plugin that runs a random subset of tests.

• Getting a tally of SQLite values as a CSV

• Using sqlite-utils to convert a CSV into a SQLite database

  You can use sqlite-utils on the command line to create a SQLite database from a CSV file.

• Python’s sqlite3 context manager doesn’t close connections

  The sqlite3.connect(…) context manager will hold connections open, so you need to remember to close it manually or write your own context manager.

• How to delete albums

• Live Text is aware of how hyphenation works (kinda)

• Go between M-IDs and filenames on Wikimedia Commons

• Telling mechanize how to find local issuer certificates

  Calling browser.set_ca_data(cafile=certifi.where()) will tell where mechanize can find some local SSL certificates.

• Why I prefer XML to JSON in the Wikimedia Commons APIs

  The XML-to-JSON conversion leads to some inconsistent behaviour, especially in corner cases of the API.

• How to do resumable downloads with curl

• Find files that use a particular SDC field

• Why the term “snak” keeps appearing in the Wikidata API

• The acronym “woe” in the Flickr API stands for “Where On Earth”

• Use Unicode property escapes to detect emoji in JavaScript

• Finding the original page for a post on Mastodon

  Following the logged-out 302 Redirect takes you to the original post.

• Use concurrency gates to prevent concurrent deployments

• How to profile Swift code

• How to use hex colours with the palette crate

  You can use Srgb::from_str() to parse a hexadecimal string as a colour in the palette crate.

• Calley-ope (calliope) Syndrome is pronouncing a word wrong because you’ve only ever read it on the page

• Use git check-ignore to debug your .gitignore

  Running git check-ignore --verbose <PATH> will tell you which rule applies to a given path, and where that rule is defined.

• Installing mimetype on Alpine Linux

• How do Dreamwidth posts IDs work?

  They were deliberately non-sequential as an anti-spam technique. It’s no longer required, but it’s in the codebase now and hasn’t been changed since it was written.

• DynamoDB: Conditional updates on nested fields

• How to iterate over the lines of an InputStream in Scala

• Manipulating URL query parameters in JavaScript

• The Content-Disposition header can be used to name a downloaded file

• How to suppress installing rdoc/ri docs when running gem install

• Replace black/white parts of an image with ImageMagick

• How to set the clock on a Horstmann Electronic 7 water heater

• Pushing with --force-with-lease is safer than with --force

  It checks the remote state of the origin hasn’t changed since you last fetched, so you don’t risk overwriting anybody else’s commits.

• Create compact JSON with Python

  Calling json.dumps(…, separators=(',', ':')) will reduce the amount of whitespace in the final string.

• List all Git object IDs and their type

• Why does Hypothesis try the same example three times before failing?

• How to set the clock on a Tricity Bendix SIE454 electric cooker

• How to set the clock on a Worcester 28CDi boiler

• Get an RSS feed of external audio posts on Tumblr

  If you add /podcast to a Tumblr site, you get a podcast-like RSS feed for all the external audio posts on that site.