Today I Learned (TIL)

TIL stands for today I learned. This is a collection of small, interesting things I’ve learnt, which I thought were worth remembering and sharing with other people.

If you want to follow along, these posts have their own RSS feed.

• How to find the Pygments token type for a short name

  Look at the source code of pygments.token.

• Remove the microsecond precision from a datetime in Python

  Call datetime.replace(microsecond=0).

• Look at the __annotations__ to learn about the definition of a TypedDict

• Don’t show Dock icons from apps on another device

  The name of this feature is “Handoff”, and that’s where you’ll find the setting for it.

• Get the avatar URL for an Instagram page

  Use gallery-dl --get-urls "https://www.instagram.com/{page_name}/avatar".

• Using the Tumblr v1 API doesn’t require auth

  You can get information from https://{username}.tumblr.com/api/read.

• The “MCP” in Archivematica stands for “Master Control Program”

  It’s nothing to do with generative AI.

• Use the -v flag to get verbose output from Go tests

  This prints all t.Log and t.Logf calls, even if the test succeeds or times out.

• Get a list of values in a JSON object with jq

  The equivalent to Python’s dict.values() is jq '[.[]]'.

• Seeing the public node key of a Tailscale node

  Use tailscale status --self --json or tailscale debug netmap.

• Using Go to write to a file, but only if it doesn’t exist yet

  Opening a file with os.O_CREATE|os.O_EXCL will ensure you only create the file if it doesn’t already exist.

• The person who runs the Cambridge NTP servers has an excellent email address

• The @ symbol was added to Morse code in 2004

  It was added in May 2004, it’s the first new symbol since the Second World War, and the French have a cute name for it. The rest of Morse code has some surprising omissions.

• Discard a variable in a JavaScript object spread by assigning it to _

• Repeatedly run flaky Go tests with stress

  It runs your test hundreds of times, which can be useful for finding flaky failures.

• How to play with SQLite functions without real data

  You can run a SELECT function(…); query without any tables.

• How to list the tests that will be run by a go test command

  Use go test -list={pattern}.

• My preferred options for SmartyPants in Python

  smartypants.smartypants(…, Attr.default | Attr.u)

• Get the avatar URL for a Bluesky user

  Make a request to the app.bsky.actor.getProfile endpoint, passing their handle as the actor parameter.

• Looking up posts in the Bluesky API

  Install the atproto package, construct a client with your username/password, then call the get_post_thread method with your at:// URI.

• Using zipstream to stream new zip files to object storage with boto3

  You can construct a zipstream.ZipFile, add files, then wrap it in a file-like object to upload it with S3.upload_fileobj.

• Using Linode object storage and boto3

  If you’re calling put_object, you need the config option request_checksum_calculation = "when_required".

• Adding a string to a tarfile in Python

  Wrap the string in an io.BytesIO first, then create a TarInfo object and pass them both to addfile() to add the string to your tarfile.

• Disable HTTP Basic Auth for certain pages in Caddy

  Define a matcher that negates the routes you want to be public, then use that with your basic_auth directive.

• Collecting pytest markers from failing tests

  You can annotate tests with markers, and using the pytest_collection_modifyitems and pytest_terminal_summary hooks you can get a list of markers for tests that failed.

• You can change the size of tabs on web pages with the tab-size property

• Group nodes in a Mermaid flowchart by putting them in a subgraph

• Show a list of checkboxes in a WTForms form

  Subclass SelectMultipleField and override the widget and option_widget fields.

• Combine arrows in Mermaid by using an invisible node

• Some countries have “poison centers” as part of their healthcare service

  Poison centres are specialist services offering expert medical advice when you’re poisoned. They’re common in countries like the US, but barely visible to the public in the UK.

• Redacting sensitive information from gunicorn access logs

  Create a subclass of gunicorn.glogging.Logger, and redact information in the atoms() method.

• Listen for the popstate event to see when the user clicks the “back” button

  More generally, it fires whenever the user is navigating the session history.

• Python’s f-strings support = for self-documenting expressions

  The f-string f"{x=}" expands to f"x={x}".

• Comparing two files in a bash script

  Inspect the exit value of cmp --silent.

• Use typing.getargs() to get a list of typing.Literal[…] values

• Ruby’s range can iterate over more than just numbers

  You can iterate over a range between two String values, because Ruby’s String does intelligent increments of alphanumeric strings.

• The error “No aggregated item, sequence was empty” comes from Jinja2

  You get this error message if you try to use Jinja2’s filters to get the min/max of an empty sequence.

• How to run a task on a schedule on macOS

  Create a LaunchAgent with a StartCalendarInterval or StartInterval that defines how often you want the task to run.

• Get a string representation of a Python traceback with traceback.format_exc()

• Go’s compiler is smart enough to spot division by zero errors

• Parsing JSON in Go with a required field

• Use text=true with subprocess functions to get stdout/stderr as str, not bytes

• Get the duration of a video file with mediainfo

  Use mediainfo --Inform='Video;%Duration%' [VIDEO_FILE].

• Print a comma-separated number in Python with {num:,}

  You can use {num:,} to insert a comma every three digits, {num:_} to insert an underscore every three digits, and {num:n} to insert a locale-aware digit separator.

• The Panamanian Golden Frog communicates by semaphore

• Use rm -v to see which files are being removed

• You can set/update the total of a progress bar tqdm after it starts

  Update the .total attribute, then call .refresh().

• How to stream lines from stdout with subprocess

  Use subprocess.Popen() with bufsize=1 and text=True, then you can interate over the lines with proc.stdout.

• What does the word “gubernatorial” mean?

  It’s anything related to a governor, and mostly used for the governors of US states.

• How do I find photos of a person on Flickr?

  https://www.flickr.com/people/{path_identifier}/photosof

• How does Flickr’s getLicenseHistory handle photos with no license changes?

• Python 3.14 includes a pi-related Easter egg

  You can start the interpreter with 𝜋thon!

• Where does AirDrop save files on macOS?

  Look in /private/tmp.

• How to check if Tailscale is running

  Use tailscale status and look for the BackendState key.

• To filter the results of a SQLite tally for values with a certain frequency, use a HAVING instead of a WHERE clause

  SELECT col_name, COUNT(*)
FROM tbl_name
GROUP BY col_name
HAVING COUNT(*) > 100;

• When fixing mojibake, use ftfy.fix_and_explain() to understand how it’s fixing a piece of text

• Why isn’t delete_where() deleting rows in sqlite-utils?

  The delete_where() function doesn’t auto-commit, so you need to wrap it with db.conn or something else to trigger the commit.

• console.log() holds a reference to an object, not a copy of it

  When you view an array/object with console.log(), you see the contents at the time you expand it, not as it existed when you called console.log().

• How do you write ratios in the aspect-ratio property?

  When you define an aspect ratio as x/y, you can only use numbers for x and y.

• You can set an output mode for SQLite

• Python 3.13 throws a ResourceWarning for SQLite databases that aren’t explicitly closed

• How to install exiftool in GitHub Actions

• How to get a macOS file/folder icon in Swift

  Use NSWorkspace.shared.icon to get the icon as an NSImage, then you can save it to disk or do something else with it.

• The British Forces Post Office (BFPO) and the Loamshire Regiment

  The BFPO sends post to the armed forces and MoD personnel, but they’ll never send it to the Loamshire Regiment, which is just a placeholder name for documentation.

• Disable JavaScript in an <iframe> by setting the sandbox attribute

• Create an animated placeholder box with CSS

• Get the dimensions of a video file with mediainfo

• Get video dimensions on macOS with built-in tools

  If the video file is indexed by Spotlight, you can use mdls to get the width and height of a video file.

• Downloading avatars from Tumblr

  There’s an API endpoint that lets you download avatars in a variety of sizes.

• What does Flickr return in the flickr.photo.getSizes API for videos?

  The video’s owner will get the URL to the original video file as “Video Original”, otherwise you should look for the largest video size.

• Bitly will delete your account if you don’t use it for three years

• A basic socket server in Ruby

  My first bit of socket programming is a Ruby server that reads lines from the socket, and prints them. Not useful on its own, but a stepping stone to more exciting things!

• Getting and setting the cover image from an EPUB file

• How to find the content.opf in an EPUB file

• Google will delete your account if you don’t use it for two years

• How to find platform-specific directories

  Two Python libraries for this task are appdirs and platformdirs, which tell you the location of the platform-specific cache directory and other similar directories.

• Restricting routes to pre-approved IP addresses in Caddy

• Use shutil.copyfileobj and xb to avoid overwriting files when copying in Python

• How to tally the attributes of the top N rows in a SQLite table

  Use a WITH clause to do a nested query for the top N rows in the table, then do a tally over that result.

• How to get the IP address of a device in my Tailnet

  Use tailscale status --json and filter the output using jq.

• How to find the biggest files backed up by Backblaze

  Look at the file /Library/Backblaze.bzpkg/bzdata/bzfilelists/bigfilelist.dat.

• There are limits on the styles you can apply with :visited

  Because the :visited selector will tell you whether somebody has been to a URL, browsers limit what styles you can apply to such links – to prevent somebody nefarious stealing your browsing history.

• Editing a filename in Finder will convert it to NFD

  Even if the filename looks the same, it may be invisibly converted to a different sequence of bytes.

• How can I work out what program is keeping a disk open?

  Use sudo lsof and grep for the name of the disk you’re trying to eject.

• You can reset the start of a regex in Ruby

  The \K escape is the “Match-Reset Anchor”, which resets the start of the reported match, and skip any previously consumed characters.

• How to get the expiry date of an HTTPS certificate with Ruby

  Connect to the domain using net/http, then you can inspect the peer_cert/not_after property on the response.

• Debugging some confusing behaviour with find and xargs

  Use the --verbose flag to see what xargs is doing; don’t rely on find to return files in a consistent order.

• How to install an asset from a GitHub release

  Use gh release download, which includes a pattern matcher if you want to pick a specific asset.

• Creating a reverse proxy to a multi-site server with Caddy

  You need to add Host headers and HTTPS configuration to your reverse_proxy block.

• How to find all of Apple’s system icons

  You need to look for files named *.icns inside any subdirectory of CoreTypes.bundle.

• Drawing a diagonal banner over the corner of an image

• Find the shortest prefix to identify a string in Ruby

  The built-in Abbrev module can calculate a set of unambiguous abbreviations for a set of strings, and then you can look for the shortest result for each string.

• How to get the target of an HTTP redirect with curl

• What is the author_name in the list of tags on a Flickr photo?

  When you call the flickr.photos.getInfo API, each tag is attributed to an author. The author_name is their username, not their realname.

• How to get a user’s email address with the Flickr API

  The flickr.profile.getProfile API returns somebody’s email address, but only if you’re allowed to see it.

• Making an “under construction” element in pure CSS

  Using a repeating-linear-gradient as the border gets you something that looks a bit like hazard tape.

• What errors can you get from hyperlink.parse?

  Mostly you get hyperlink.URLParseError, but you can occasionally get a ValueError as well.

• HTML strings may not be equivalent if you minify them

  There’s a lot of whitespace in HTML which looks irrelevant at first glance, but may be significant and cause the document to render differently.

• How do the ispublic, isfriend and isfamily flags work in the Flickr API?

• Use std::io::IsTerminal to detect if you’re running in the terminal in Rust

  This allows me to suppress ANSI escape codes if the output is going somewhere other than the terminal.

• Use $_ to get the last argument to the previous bash command

  This allows you to write commands like mkdir myfolder && cd $_ or git init myrepo && cd $_.

• With Flask-Login, you want current_user == None, not current_user is None

  current_user is a proxy object that happens to be wrapping None, but isn’t actually None.

• How to get the expiry date of an HTTPS certificate with Python

  Connect to the domain using the socket module, then use the getpeercert() method on the connection to get information about the HTTPS certificate.

• Write to the middle of a file with Python

  Open the file with mode r+ to be able to seek around the file and write to it.

• Create solid-colour image placeholders to show before an image loads

• Get an image from a video with ffmpeg

• Get the embedded artwork from an MP3 file

  Use the command eyeD3 [MP3_FILE] --write-images [FOLDER].

• Convert an animated GIF to an MP4 with ffmpeg

• Using the Wikimedia Commons API to tell if a file has been deleted

• Build a URL with query string parameters with curl

  A combination of --get and --data/--data-urlencode allows you to write curl commands which are readable and expressive.

• How to get the selected item in Finder using AppleScript

• Removing letterboxing from a video screenshot with ImageMagick

  Using -trim will remove the black portions and leave you the unletterboxed image.

• How to check the quality of a network connection

  Using an NWPathMonitor and inspecting the value of NWPath.status, NWPath.isExpensive and NWPath.isConstrained can tell you what sort of connection you’re running on.

• How to highlight Python console sessions in Jekyll

  Adding a couple of options to the console lexer (console?lang=python&prompt=>>>) gets you syntax highlighting for a Python console session.

• Not all coal is the same

• How to simulate an [Errno 54] Connection reset by peer when using pytest

  You can run a TCP server in the background using a fixture, and using the SO_LINGER socket option can reset the connection.

• What does “insomnolent” mean?

• Open a Safari webarchive from Twitter/X without being redirected

  Disabling JavaScript when you open the webarchive file will prevent you from redirecting you to twitter.com.

• Writing a file in Swift, but only if it doesn’t already exist

  Adding .withoutOverwriting to your write() call will prevent you from overwriting a file that already exists.

• What’s inside a Safari webarchive?

  The inside of a .webarchive file is a binary property list with the complete responses and some request metadata.

• Get a Palette colour as a command-line argument with Clap

  Wrapping a Palette:Srgb in a struct and implementing FromStr for the struct allows you to take hexadecimal colours as command-line inputs.

• How to get the filename/size without downloading a file in curl

  You can do some fun stuff with the --write-out flag and variables.

• There’s a musical that tells you the number of minutes in a year

  The song Seasons of Love from Rent starts with the line “Five hundred twenty-five thousand, six hundred minutes”.

• How to get a tally of tally counts in SQLite

  Using a nested query allows me to perform a two-level aggregation of the values in a column – how many values appear once, how many twice, and so on.

• How to get a list of captures from the Wayback Machine

  Use the CDX Server API to get a list of captures for a particular URL.

• How to take a screenshot of a page in the Wayback Machine

  Using Playwright to take screenshots and adding some custom styles gets a screenshot of a page without the Wayback Machine overlay.

• How to count how many Discord messages were sent on a given day

  Using the During filter gives me a count of how many messages were being sent.

• How to see the HTTP requests being made by pywikibot

  To see exactly what HTTP requests were being made, I modified the library so that betamax would record requests.

• How to embed an inline SVG in a CSS rule

  Modern browsers allow you to embed the SVG almost as-is, with just a couple of characters that need escaping – no base64 required!

• How to shuffle an array in a Jekyll template

  If you want an array in random order, you can use the sample filter to get a random sample of the same size as the original array.

• Checking if a URL has changed when you fetch it over HTTP

  When you make an HTTP request, you can use the If-Modified-Since header to get a 304 Not Modified if nothing has changed since your last request.

• The open command can ask questions

  If you pass an argument that can’t be easily identified as a file or a URL, open will ask you what to do next. This may be a surprise if you were trying to use it in a script.

• Why is Pillow rotating my image when I save it?

  Images can have orientation specified in their EXIF metadata, which isn’t preserved when you open and save an image with Pillow.

• How to change the name of an internal link in an Obsidian table

  Escaping the pipe like [[filename\|display text]] allows you to customise the of a link in a table.

• Getting a boto3 Session for an IAM role using Python

  Why I use Sessions in boto3, and the Python function I use to create them.

• My config for running youtube-dl

  The flags and arguments I find useful when I’m using youtube-dl.

• How much will Mastodon instances try to re-send messages?

• Get my Netlify bandwidth usage from the API

• How to restrict a page to specific IP addresses

• Use the -n/-i flags to avoid overwriting files with cp and mv

• How to parse URLs in JXA

• Using errexit and arithmetic expressions in bash

• How to check when an HTTPS certificate expires

• The COUNT(X) function only counts non-null values

• What happens when you replace a photo on Flickr?

• Exclude files from Time Machine with tmutil addexclusion

• You need to call resp.close() to close the file opened by send_file()

• Add the -v flag to see what rm is deleting

• Custom error pages in Flask

  You can use app.error_handler to add custom responses for HTTP status codes, so the errors match the look and feel of the rest of the site.

• Use the {% raw %} tag to describe Liquid in Liquid

  If you’re trying to write about using Liquid tags in a Liquid-based site, wrapping your tags in the {% raw %} tag will prevent them being rendered.

• What characters are allowed in titles on Wikimedia Commons?

• Run a script on macOS on a schedule using a LaunchAgent

• Beware of using test -n with command expansion

• How to do offline geo-lookups of IP addresses

  MaxMind offer databases you can do to look up IP addresses without sending the address off to a remote service.

• How to tally combinations of values across multiple columns

• How to move files when you need sudo on the remote server

• How to gracefully restart a gunicorn app

• How to find the longest common suffix in a list of strings in Python

• How to simulate shell pipes with the subprocess module

• Use the {% capture %} tag to assign complex strings to variables

  If you want to get a string that’s semi-complicated to construct, you can put a “mini-template” in the {% capture %} tag to build it over multiple lines.

• How to create a footer that’s always at the bottom of the page

• WordPress URLs that get hammered by spammers

• Manage MP3 metadata from iTunes with eyed3

• Sort a list of DOM elements by sorting and calling appendChild()

• How to create flag emojis for countries in Python

• Use the IMAGE function to insert an image into a spreadsheet

• How to style a <details> element differently depending on whether it’s open or closed

• Pausing the animation of <svg> elements can affect child <svg> elements differently in different browsers

• Create a directory before you cp or mv a file to it

• Running the Netlify CLI in GitHub Actions

• Use shlex.split() to parse log files quickly

• Use the regex library to get Unicode property escapes in Python

• Get and manipulate the contents of a page in Safari with "do JavaScript"

• SVGs are only rendered on GitHub if you use an <img> that points to another file

• Animate an attribute of an element with <animate>

• Run a randomly selected subset of tests with pytest

  By reading the code for the pytest-random-order plugin, I was able to write a new plugin that runs a random subset of tests.

• Getting a tally of SQLite values as a CSV

• Using sqlite-utils to convert a CSV into a SQLite database

  You can use sqlite-utils on the command line to create a SQLite database from a CSV file.

• Python’s sqlite3 context manager doesn’t close connections

  The sqlite3.connect(…) context manager will hold connections open, so you need to remember to close it manually or write your own context manager.

• How to delete albums

• Live Text is aware of how hyphenation works (kinda)

• Go between M-IDs and filenames on Wikimedia Commons

• Telling mechanize how to find local issuer certificates

  Calling browser.set_ca_data(cafile=certifi.where()) will tell where mechanize can find some local SSL certificates.

• Why I prefer XML to JSON in the Wikimedia Commons APIs

  The XML-to-JSON conversion leads to some inconsistent behaviour, especially in corner cases of the API.

• How to do resumable downloads with curl

• Find files that use a particular SDC field

• Why the term “snak” keeps appearing in the Wikidata API

• The acronym “woe” in the Flickr API stands for “Where On Earth”

• Use Unicode property escapes to detect emoji in JavaScript

• Finding the original page for a post on Mastodon

  Following the logged-out 302 Redirect takes you to the original post.

• Use concurrency gates to prevent concurrent deployments

• How to profile Swift code

• Getting the base directory of an sbt project

  Some notes on printing sbt settings, so you can use them as the input to another script.

• How to use hex colours with the palette crate

  You can use Srgb::from_str() to parse a hexadecimal string as a colour in the palette crate.

• Calley-ope (calliope) Syndrome is pronouncing a word wrong because you’ve only ever read it on the page

• Use git check-ignore to debug your .gitignore

  Running git check-ignore --verbose <PATH> will tell you which rule applies to a given path, and where that rule is defined.

• Installing mimetype on Alpine Linux

• How do Dreamwidth posts IDs work?

  They were deliberately non-sequential as an anti-spam technique. It’s no longer required, but it’s in the codebase now and hasn’t been changed since it was written.

• DynamoDB: Conditional updates on nested fields

• Add a Git co-author credit with “Co-authored-by” in your commit message

• How to iterate over the lines of an InputStream in Scala

• Using TransportAPI and geopy to get the distance between stations

• Manipulating URL query parameters in JavaScript

• The Content-Disposition header can be used to name a downloaded file

• How to suppress installing rdoc/ri docs when running gem install

• Replace black/white parts of an image with ImageMagick

• How to set the clock on a Horstmann Electronic 7 water heater

• Pushing with --force-with-lease is safer than with --force

  It checks the remote state of the origin hasn’t changed since you last fetched, so you don’t risk overwriting anybody else’s commits.

• How to use xargs for parallel processing

• Create compact JSON with Python

  Calling json.dumps(…, separators=(',', ':')) will reduce the amount of whitespace in the final string.

• List all Git object IDs and their type

• Python throws a TypeError if you return a non-string from a custom __repr__ or __str__ method

  It fails with the error “__repr__/__str__ returned non-string”.

• Why does Hypothesis try the same example three times before failing?

• Rust macros are smarter than just text substitution

  This is a safety feature that prevents macros expanding in an unexpected way.

• How to set the clock on a Tricity Bendix SIE454 electric cooker

• How to set the clock on a Worcester 28CDi boiler

• You can use shutil.which to check if an executable is in your PATH

  This is useful for checking if something’s installed.

• Germany has longer telephone numbers than the rest of the world

• Get an RSS feed of external audio posts on Tumblr

  If you add /podcast to a Tumblr site, you get a podcast-like RSS feed for all the external audio posts on that site.