Skip to main content

How to simulate shell pipes with the subprocess module

I had a shell command featuring a pipe that I wanted to replicate with subprocess:

youtube-dl --get-id "$PLAYLIST_URL" \
  | xargs -I '{}' -P 5 youtube-dl '{}'

I could try to create this command as a string, pass it to…, shell=True), and I hope I’ve used shlex.quote() correctly – but that’s dangerous and error-prone.

I found a better approach in a Stack Overflow answer by Taymon: use subprocess.PIPE to pass stdout/stdin between processes.

get_ids_proc = subprocess.Popen(
    ["youtube-dl", "--get-id", youtube_url],

    ["xargs", "-I", "{}", "-P", "5", "youtube-dl", "{}"],