Notes
Notes are snippets of information that I find useful and wanted to write down, but aren’t long or original enough to be articles. I expect most people will find these notes by browsing my list of topics or from a search engine.
Road signs in the Soviet union don’t have circular heads
Their heads are more a sort of rounded triangle shape, a bit like an oval or an egg.
Setting up golink in my personal tailnet
I created a macOS LaunchAgent to start golink automatically whenever my desktop Mac restarts.
Create a file atomically in Go
Use
os.CreateTempto create a temporary file in the target directory, then do an atomic rename once you’ve finished writing.Get a map of IP addresses for devices in my tailnet
The SQLite command line shell will count your unclosed parentheses
If the prompt starts with
(x1or(x2, it means you’ve opened some parentheses and not closed them yet.Use SQL triggers to prevent overwriting a value
A trigger lets you run an action when you
INSERT,UPDATEorDELETEa value.Testing date formatting with date-fns-tz and different timezones
Override the
TZenvironment variable in your tests.The “strangler” pattern is named after a tree, not an act of violence
It’s named after the strangler fig tree, which wraps itself a host tree and gradually kills them.
Place with the same name, but different etymology
Use
systemctl is-activeto determine if a service is runningRemoving a self-hosted runner from GitHub Actions
My Git config
How I set up Git on a new computer.
Who is Bufo the frog?
Unpicking the history of a Slack icon.
Python f-strings cheat sheet
Some Python f-string examples that I find helpful.
Ignore AI upscaled YouTube videos with yt-dlp
Filter for formats that don’t include
-sr(“super resolution”) in their format ID.Collapsing whitespace in a Liquid template
Output an empty string with stripped whitespace, that is
{{- "" -}}.The LastModified date of an S3 multipart upload is when the upload started, not when it finished
Python has a builtin tool to do rot13 (among other things)
Look in the codecs module.
How to find the Pygments token type for a short name
Look at the source code of
pygments.token.Remove the microsecond precision from a
datetimein PythonCall
datetime.replace(microsecond=0).Look at the
__annotations__to learn about the definition of a TypedDictDon’t show Dock icons from apps on another device
The name of this feature is “Handoff”, and that’s where you’ll find the setting for it.
Get the avatar URL for an Instagram page
Use
gallery-dl --get-urls "https://www.instagram.com/{page_name}/avatar".Using the Tumblr v1 API doesn’t require auth
You can get information from
https://{username}.tumblr.com/api/read.How to expire a Tailscale node key faster than the min expiry
Use
tailscale debug set-expire --in=<duration>.Use the
-vflag to get verbose output from Go testsThis prints all
t.Logandt.Logfcalls, even if the test succeeds or times out.Get a list of values in a JSON object with jq
The equivalent to Python’s
dict.values()isjq '[.[]]'.Seeing the public node key of a Tailscale node
Use
tailscale status --self --jsonortailscale debug netmap.Don’t nest a
<Script>in a<Head>with Next.jsIf you do, the rendered page won’t include the script anywhere.
Using Go to write to a file, but only if it doesn’t exist yet
Opening a file with
os.O_CREATE|os.O_EXCLwill ensure you only create the file if it doesn’t already exist.The @ symbol was added to Morse code in 2004
It was added in May 2004, it’s the first new symbol since the Second World War, and the French have a cute name for it. The rest of Morse code has some surprising omissions.
Discard a variable in a JavaScript object spread by assigning it to
_Repeatedly run flaky Go tests with
stressIt runs your test hundreds of times, which can be useful for finding flaky failures.
How to play with SQLite functions without real data
You can run a
SELECT function(…);query without any tables.How to list the tests that will be run by a
go testcommandUse
go test -list={pattern}.My preferred options for SmartyPants in Python
smartypants.smartypants(…, Attr.q | Attr.D | Attr.e | Attr.u)Get the avatar URL for a Bluesky user
Make a request to the
app.bsky.actor.getProfileendpoint, passing their handle as theactorparameter.Using zipstream to stream new zip files to object storage with boto3
You can construct a
zipstream.ZipFile, add files, then wrap it in a file-like object to upload it withS3.upload_fileobj.Adding a string to a tarfile in Python
Wrap the string in an
io.BytesIOfirst, then create aTarInfoobject and pass them both toaddfile()to add the string to your tarfile.Collecting pytest markers from failing tests
You can annotate tests with markers, and using the
pytest_collection_modifyitemsandpytest_terminal_summaryhooks you can get a list of markers for tests that failed.You can change the size of tabs on web pages with the
tab-sizepropertyGroup nodes in a Mermaid flowchart by putting them in a subgraph
Combine arrows in Mermaid by using an invisible node
Some countries have “poison centers” as part of their healthcare service
Poison centres are specialist services offering expert medical advice when you’re poisoned. They’re common in countries like the US, but barely visible to the public in the UK.
Redacting sensitive information from gunicorn access logs
Create a subclass of
gunicorn.glogging.Logger, and redact information in theatoms()method.Python’s f-strings support
=for self-documenting expressionsThe f-string
f"{x=}"expands tof"x={x}".Comparing two files in a bash script
Inspect the exit value of
cmp --silent.Use
typing.getargs()to get a list oftyping.Literal[…]valuesRuby’s range can iterate over more than just numbers
You can iterate over a range between two
Stringvalues, because Ruby’sStringdoes intelligent increments of alphanumeric strings.The error “No aggregated item, sequence was empty” comes from Jinja2
You get this error message if you try to use Jinja2’s filters to get the min/max of an empty sequence.
How to run a task on a schedule on macOS
Create a
LaunchAgentwith aStartCalendarIntervalorStartIntervalthat defines how often you want the task to run.Get a string representation of a Python traceback with
traceback.format_exc()Go’s compiler is smart enough to spot division by zero errors
Parsing JSON in Go with a required field
Use
text=truewithsubprocessfunctions to get stdout/stderr as str, not bytesGet the duration of a video file with
mediainfoUse
mediainfo --Inform='Video;%Duration%' [VIDEO_FILE].The Panamanian Golden Frog communicates by semaphore
You can set/update the
totalof a progress bartqdmafter it startsUpdate the
.totalattribute, then call.refresh().How to stream lines from stdout with
subprocessUse
subprocess.Popen()withbufsize=1andtext=True, then you can interate over the lines withproc.stdout.What does the word “gubernatorial” mean?
It’s anything related to a governor, and mostly used for the governors of US states.
Python 3.14 includes a pi-related Easter egg
You can start the interpreter with
𝜋thon!Where does AirDrop save files on macOS?
Look in
/private/tmp.How to check if Tailscale is running
Use
tailscale statusand look for theBackendStatekey.To filter the results of a SQLite tally for values with a certain frequency, use a
HAVINGinstead of aWHEREclauseSELECT col_name, COUNT(*)
FROM tbl_name
GROUP BY col_name
HAVING COUNT(*) > 100;Why isn’t
delete_where()deleting rows in sqlite-utils?The
delete_where()function doesn’t auto-commit, so you need to wrap itwith db.connor something else to trigger the commit.console.log()holds a reference to an object, not a copy of itWhen you view an array/object with
console.log(), you see the contents at the time you expand it, not as it existed when you calledconsole.log().You can set an output mode for SQLite
How do you write ratios in the
aspect-ratioproperty?When you define an aspect ratio as
x/y, you can only use numbers forxandy.Python 3.13 throws a
ResourceWarningfor SQLite databases that aren’t explicitly closedHow to install exiftool in GitHub Actions
How to get a macOS file/folder icon in Swift
Use
NSWorkspace.shared.iconto get the icon as anNSImage, then you can save it to disk or do something else with it.The British Forces Post Office (BFPO) and the Loamshire Regiment
The BFPO sends post to the armed forces and MoD personnel, but they’ll never send it to the Loamshire Regiment, which is just a placeholder name for documentation.
Disable JavaScript in an
<iframe>by setting thesandboxattributeGet the dimensions of a video file with
mediainfoGet video dimensions on macOS with built-in tools
If the video file is indexed by Spotlight, you can use
mdlsto get the width andheightof a video file.Downloading avatars from Tumblr
There’s an API endpoint that lets you download avatars in a variety of sizes.
Bitly will delete your account if you don’t use it for three years
A basic socket server in Ruby
My first bit of socket programming is a Ruby server that reads lines from the socket, and prints them. Not useful on its own, but a stepping stone to more exciting things!
Google will delete your account if you don’t use it for two years
How to find platform-specific directories
Two Python libraries for this task are appdirs and platformdirs, which tell you the location of the platform-specific cache directory and other similar directories.
Use
shutil.copyfileobjandxbto avoid overwriting files when copying in PythonHow to tally the attributes of the top N rows in a SQLite table
Use a
WITHclause to do a nested query for the top N rows in the table, then do a tally over that result.How to get the IP address of a device in my Tailnet
Use
tailscale status --jsonand filter the output usingjq.There are limits on the styles you can apply with
:visitedBecause the
:visitedselector will tell you whether somebody has been to a URL, browsers limit what styles you can apply to such links – to prevent somebody nefarious stealing your browsing history.How can I work out what program is keeping a disk open?
Use
sudo lsofand grep for the name of the disk you’re trying to eject.You can reset the start of a regex in Ruby
The
\Kescape is the “Match-Reset Anchor”, which resets the start of the reported match, and skip any previously consumed characters.How to get the expiry date of an HTTPS certificate with Ruby
Connect to the domain using
net/http, then you can inspect thepeer_cert/not_afterproperty on the response.Debugging some confusing behaviour with
findandxargsUse the
--verboseflag to see whatxargsis doing; don’t rely onfindto return files in a consistent order.How to install an asset from a GitHub release
Use
gh release download, which includes a pattern matcher if you want to pick a specific asset.How to find all of Apple’s system icons
You need to look for files named
*.icnsinside any subdirectory ofCoreTypes.bundle.Drawing a diagonal banner over the corner of an image
Find the shortest prefix to identify a string in Ruby
The built-in
Abbrevmodule can calculate a set of unambiguous abbreviations for a set of strings, and then you can look for the shortest result for each string.How to get the target of an HTTP redirect with curl
Making an “under construction” element in pure CSS
Using a
repeating-linear-gradientas thebordergets you something that looks a bit like hazard tape.What errors can you get from
hyperlink.parse?Mostly you get
hyperlink.URLParseError, but you can occasionally get aValueErroras well.Use
std::io::IsTerminalto detect if you’re running in the terminal in RustThis allows me to suppress ANSI escape codes if the output is going somewhere other than the terminal.
Use
$_to get the last argument to the previous bash commandThis allows you to write commands like
mkdir myfolder && cd $_orgit init myrepo && cd $_.With Flask-Login, you want
current_user == None, notcurrent_user is Nonecurrent_useris a proxy object that happens to be wrappingNone, but isn’t actuallyNone.How to get the expiry date of an HTTPS certificate with Python
Connect to the domain using the
socketmodule, then use thegetpeercert()method on the connection to get information about the HTTPS certificate.Write to the middle of a file with Python
Open the file with mode
r+to be able to seek around the file and write to it.Get an image from a video with ffmpeg
Convert an animated GIF to an MP4 with ffmpeg
Build a URL with query string parameters with curl
A combination of
--getand--data/--data-urlencodeallows you to write curl commands which are readable and expressive.How to get the selected item in Finder using AppleScript
Removing letterboxing from a video screenshot with ImageMagick
Using
-trimwill remove the black portions and leave you the unletterboxed image.How to check the quality of a network connection
Using an
NWPathMonitorand inspecting the value ofNWPath.status,NWPath.isExpensiveandNWPath.isConstrainedcan tell you what sort of connection you’re running on.How to highlight Python console sessions in Jekyll
Adding a couple of options to the
consolelexer (console?lang=python&prompt=>>>) gets you syntax highlighting for a Python console session.Not all coal is the same
How to simulate an
[Errno 54] Connection reset by peerwhen using pytestYou can run a TCP server in the background using a fixture, and using the
SO_LINGERsocket option can reset the connection.What does “insomnolent” mean?
Open a Safari webarchive from Twitter/X without being redirected
Disabling JavaScript when you open the webarchive file will prevent you from redirecting you to twitter.com.
Writing a file in Swift, but only if it doesn’t already exist
Adding
.withoutOverwritingto yourwrite()call will prevent you from overwriting a file that already exists.What’s inside a Safari webarchive?
The inside of a
.webarchivefile is a binary property list with the complete responses and some request metadata.There’s a musical that tells you the number of minutes in a year
The song Seasons of Love from Rent starts with the line “Five hundred twenty-five thousand, six hundred minutes”.
How to get a tally of tally counts in SQLite
Using a nested query allows me to perform a two-level aggregation of the values in a column – how many values appear once, how many twice, and so on.
How to get a list of captures from the Wayback Machine
Use the CDX Server API to get a list of captures for a particular URL.
How to take a screenshot of a page in the Wayback Machine
Using Playwright to take screenshots and adding some custom styles gets a screenshot of a page without the Wayback Machine overlay.
How to embed an inline SVG in a CSS rule
Modern browsers allow you to embed the SVG almost as-is, with just a couple of characters that need escaping – no base64 required!
The
opencommand can ask questionsIf you pass an argument that can’t be easily identified as a file or a URL,
openwill ask you what to do next. This may be a surprise if you were trying to use it in a script.Getting a boto3 Session for an IAM role using Python
Why I use Sessions in boto3, and the Python function I use to create them.
My config for running youtube-dl
The flags and arguments I find useful when I’m using youtube-dl.
Use the
-n/-iflags to avoid overwriting files withcpandmvHow to parse URLs in JXA
Using
errexitand arithmetic expressions in bashThe COUNT(X) function only counts non-null values
Exclude files from Time Machine with
tmutil addexclusionYou need to call
resp.close()to close the file opened bysend_file()Add the
-vflag to see whatrmis deletingCustom error pages in Flask
You can use
app.error_handlerto add custom responses for HTTP status codes, so the errors match the look and feel of the rest of the site.Use the
{% raw %}tag to describe Liquid in LiquidIf you’re trying to write about using Liquid tags in a Liquid-based site, wrapping your tags in the
{% raw %}tag will prevent them being rendered.Run a script on macOS on a schedule using a LaunchAgent
Beware of using
test -nwith command expansionHow to tally combinations of values across multiple columns
How to move files when you need sudo on the remote server
How to gracefully restart a gunicorn app
How to find the longest common suffix in a list of strings in Python
How to simulate shell pipes with the subprocess module
Use the
{% capture %}tag to assign complex strings to variablesIf you want to get a string that’s semi-complicated to construct, you can put a “mini-template” in the
{% capture %}tag to build it over multiple lines.How to create a footer that’s always at the bottom of the page
WordPress URLs that get hammered by spammers
Sort a list of DOM elements by sorting and calling
appendChild()How to style a
<details>element differently depending on whether it’s open or closedPausing the animation of <svg> elements can affect child <svg> elements differently in different browsers
Create a directory before you
cpormva file to itRunning the Netlify CLI in GitHub Actions
Use shlex.split() to parse log files quickly
Use the regex library to get Unicode property escapes in Python
Get and manipulate the contents of a page in Safari with
"do JavaScript"SVGs are only rendered on GitHub if you use an
<img>that points to another fileAnimate an attribute of an element with <animate>
Run a randomly selected subset of tests with pytest
By reading the code for the
pytest-random-orderplugin, I was able to write a new plugin that runs a random subset of tests.Getting a tally of SQLite values as a CSV
Using sqlite-utils to convert a CSV into a SQLite database
You can use sqlite-utils on the command line to create a SQLite database from a CSV file.
Python’s sqlite3 context manager doesn’t close connections
The
sqlite3.connect(…)context manager will hold connections open, so you need to remember to close it manually or write your own context manager.Live Text is aware of how hyphenation works (kinda)
Telling mechanize how to find local issuer certificates
Calling
browser.set_ca_data(cafile=certifi.where())will tell where mechanize can find some local SSL certificates.How to do resumable downloads with curl
You want the
--continue-at -flag, which will resume the transfer from the size of the already-downloaded file.Use concurrency gates to prevent concurrent deployments
How to profile Swift code
Getting the base directory of an sbt project
Some notes on printing sbt settings, so you can use them as the input to another script.
Calley-ope (calliope) Syndrome is pronouncing a word wrong because you’ve only ever read it on the page
A Python function to ignore a path with .git/info/exclude
If your Python script creates a file that you don’t want to track in Git, here’s how you can ignore it.
Use
git check-ignoreto debug your.gitignoreRunning
git check-ignore --verbose <PATH>will tell you which rule applies to a given path, and where that rule is defined.Installing mimetype on Alpine Linux
Conditional updates on nested fields in DynamoDB
Add a Git co-author credit with “Co-authored-by” in your commit message
How to iterate over the lines of an InputStream in Scala
Manipulating URL query parameters in JavaScript
How to suppress installing rdoc/ri docs when running
gem installReplace black/white parts of an image with ImageMagick
Pushing with
--force-with-leaseis safer than with--forceIt checks the remote state of the origin hasn’t changed since you last fetched, so you don’t risk overwriting anybody else’s commits.
How to use xargs for parallel processing
Create compact JSON with Python
Calling
json.dumps(…, separators=(',', ':'))will reduce the amount of whitespace in the final string.Custom 404 responses in Finatra
A snippet for returning a custom 404 response in a Finatra app when somebody requests a missing page.
List all Git object IDs and their type
Python throws a TypeError if you return a non-string from a custom
__repr__or__str__methodIt fails with the error “
__repr__/__str__returned non-string”.Why does Hypothesis try the same example three times before failing?
Rust macros are smarter than just text substitution
This is a safety feature that prevents macros expanding in an unexpected way.
You can use
shutil.whichto check if an executable is in your PATHThis is useful for checking if something’s installed.
Germany has longer telephone numbers than the rest of the world
Custom CSS in ResophNotes