Zero Touch

The Summer of CCNA course produces daily livestream recordings on YouTube. Every day, someone needs to check if there is a new one, download it, think of a title, upload it to Vimeo with the right privacy settings, update the config files that tell the learning platform what videos exist, and do all of that for both the premium and free tiers. Eight steps, every single day, for the entire summer.

I was that someone.

It is the kind of work that feels productive while you are doing it and utterly mechanical once you step back and look at it. Download. Rename. Upload. Edit JSON. Zip. Re-upload. Repeat. The kind of task that makes you start Googling "can I automate this" at 11 PM on a Tuesday, which is how most of my projects begin.

The script I ended up building is called watch_livestreams.py, and it does everything the manual process did, but hourly and without me.

It starts by querying the YouTube Data API with OAuth. This is the only way to see unlisted videos, and the daily Q&A check-ins are unlisted. Normal scraping would miss them entirely, so OAuth was non-negotiable. The script classifies each video as either a Q&A or a 90 Minute session based on title patterns, extracts the date, and skips anything it has already processed.

Then it downloads the video and its auto-generated English subtitles using yt-dlp. The subtitles get parsed and sent to OpenAI's GPT-4o-mini, which reads through the transcript and generates a concise, descriptive title. This was one of those features I almost did not include because it felt like overkill, but the titles it produces are consistently better than what I would come up with at the end of a long day. The AI does not just summarize the topic; it pulls out the specific thing that made the session interesting.

After renaming the file, the script uploads it to Vimeo via the tus protocol in 128 MB chunks. Sets the privacy to embed-only with domain whitelisting so the videos can only be played on the academy sites. Places it in the correct monthly folder. All through the Vimeo API.

Then it updates the JSON config file, commits, and pushes to a private GitHub repo. GitHub Actions picks up the push and deploys the config to Cloudflare KV within about fifteen seconds. The SCORM shell in LearnWorlds fetches this config on every page load.

That last part is the piece I am most proud of. The SCORM packages were uploaded to LearnWorlds once, and they never need to be touched again. They contain all the UI code but no video data. The data comes from Cloudflare at runtime. New video goes live without anyone logging into LearnWorlds, without re-zipping a SCORM package, without re-uploading anything. Zero touch.

State is saved after every step. If the script crashes mid-run or the network drops during a Vimeo upload, the next hourly run picks up exactly where it left off. Nothing is ever lost.

The pipeline started on my Mac, running as a cron job. This worked fine until I realized I did not want my laptop to be a critical piece of infrastructure. If I closed it, or traveled, or just let it sleep, the pipeline stopped.

So I moved it to an LXC container on a Proxmox host. Ubuntu 24.04, 2 GB of RAM, 2 cores, 20 GB of disk. Tiny, dedicated, and always on.

This is where things got interesting.

The first problem was line endings. I developed on macOS, and somewhere in the pipeline the shell scripts picked up Windows-style CRLF line endings. When I ran setup.sh on the Linux container, bash could not even find itself. The error said it was looking for bash\r—which is a sentence that probably tells you everything about how that afternoon went. The fix was dos2unix and a .gitattributes file that enforces LF endings. Simple in retrospect. Maddening in the moment.

The second problem was SSH deploy keys. The pipeline needs access to two GitHub repos, and GitHub does not allow the same deploy key on multiple repositories. I learned this when the API returned a 422 and a message that the key was "already in use." The fix was generating a second key and creating SSH host aliases so each repo connects through the right identity. One of those problems that is obvious once you know, and invisible until you do not.

The third problem replaced the old cron job with a systemd timer. This gave me a five-minute watchdog that kills the process if any API call hangs, persistence across reboots, and proper logging through journalctl. The timer fires at five past every hour, and if the container was down when a run was missed, it triggers immediately on boot. This is the kind of thing that makes systemd worth learning, even if the learning curve is its own separate project.

And then YouTube broke everything.

Not YouTube exactly. yt-dlp updated and suddenly required a JavaScript runtime called deno to solve YouTube's signature challenges. The container did not have deno installed. Every hourly run from 10 PM on July 28 through the following morning failed with the same error, fourteen times in a row. The July 28 livestream was detected correctly every single time—the YouTube Data API part worked fine—but the download step kept failing silently. No alert. No notification. Just a line in the logs that nobody was reading.

Installing deno fixed the JavaScript runtime issue. But then YouTube hit back with something worse. "Sign in to confirm you're not a bot." The container lives in a datacenter, and YouTube's bot detection specifically targets datacenter IP ranges. My Mac at home on a residential IP could download the same video without any issues. The container, three states away in a datacenter, could not.

Cookies were the obvious fix, except that cookies exported from my Mac were tied to my Mac's IP. Using them from the container's different IP just made YouTube reject them faster.

The solution I landed on is a bit of a Rube Goldberg machine, but it works. I create a SOCKS proxy from my Mac through the container—ssh -D 1080—and then launch Firefox on the Mac routed through that proxy. When I log into YouTube in this Firefox window, the traffic exits through the container's datacenter IP. The cookies are created from the right IP. I export them, transfer them to the container, and yt-dlp uses them going forward.

The fourteen silent failures overnight were the thing that convinced me the pipeline needed a notification system. It is one thing to build automation that runs without you. It is another thing entirely to build automation that fails without you knowing.

I added a Slack integration—an incoming webhook that fires on four conditions. Cookie expiration, which includes remediation steps right in the alert. Vimeo upload failures. Runs where every download failed. And successful processing, because it is nice to see a green checkmark come through at 3 AM confirming that the system did its job.

The notification function fails silently if the webhook is not configured. The pipeline does not care whether Slack is listening. But I do.

The thing I keep coming back to is this: the hard part of automation is never the happy path. Writing the script that detects, downloads, titles, uploads, and pushes config—that was a weekend. Maybe two. Everything after that was edge cases. Line endings. Deploy key uniqueness. JavaScript runtimes appearing as surprise dependencies. IP reputation systems that treat your automation like a threat.

Each of those problems taught me something I would not have learned from a tutorial. The line endings problem taught me about .gitattributes. The deploy key problem taught me about SSH host aliases. The datacenter IP problem taught me about SOCKS proxies and how YouTube's bot detection actually works under the hood. None of these were in the original plan.

The pipeline processes about five to ten videos per month, and each one takes roughly two minutes from detection to student availability. The entire infrastructure runs on free tiers. The monthly cost is about one cent, and most of that is OpenAI's token charges for generating a title.

Eight manual steps. Zero now. But the number that actually matters is the one I did not measure—the number of hours I spent building the thing that saves me ten minutes a day.

I think I came out ahead. Probably.