# YouTube RSS

A local app that watches YouTube channels Dan is subscribed to, downloads
new uploads as mp3s via yt-dlp, and serves them as a private podcast RSS
feed that Dan subscribes to in Overcast on his iPhone.

## Location

```
/Users/dan/Documents/PARA/2 Areas of Responsibility/Sound Computer/Personal Apps/YouTube RSS
```

This sits alongside a separate local app (different project, not in this
folder) that generates smart playlists from Dan's mp3 library. Both are
part of a growing collection of small personal services under
`Personal Apps/` — worth keeping that in mind if asked to help organise,
document, or extend "the other local apps" generally.

## How it works

1. `channels.txt` lists YouTube channel IDs, one per line
2. `watcher.py`, on each run:
   - Fetches each channel's built-in RSS feed
     (`https://www.youtube.com/feeds/videos.xml?channel_id=X`) — no API key
     needed
   - Compares entries against `episodes.json` to find videos not yet handled
   - For new videos, shells out to `yt-dlp -x --audio-format mp3` to download
     just the audio into `downloads/`
   - Records each attempt in `episodes.json` (even failures, with
     `filename: null`, so they aren't retried forever automatically)
   - Regenerates `feed.xml`, a standard podcast RSS 2.0 file, from
     `episodes.json` + whatever's actually present in `downloads/`
3. A `launchd` job (`com.dan.ytrss.server.plist`) runs
   `python3 -m http.server 8000` persistently in this folder, serving
   `feed.xml` and everything in `downloads/`
4. A second `launchd` job (`com.dan.ytrss.watcher.plist`) runs
   `watcher.py` at 4am and 4pm daily
5. Dan's Mac and iPhone are connected via Tailscale (already configured).
   Overcast itself fetches feeds and episode audio from Overcast's own
   servers, not from Dan's phone directly — so a plain Tailscale-only
   address doesn't work, since Overcast's servers aren't part of Dan's
   tailnet and can't reach a private `100.x.x.x` address. The feed is
   exposed publicly via **Tailscale Funnel** instead, at
   `https://macbook-air-2023.tail67203.ts.net/feed.xml` — that hostname is
   hardcoded as `BASE_URL` in `watcher.py`. Funnel proxies that public HTTPS
   URL to the local `python3 -m http.server 8000` (see `tailscale funnel
   status` to check it's still active). This does mean the feed and
   downloaded audio are technically reachable by anyone with the URL, not
   just devices on Dan's tailnet — mitigated only by the hostname being
   obscure/unguessable, not by any real access control.

## Setup status — complete

All eight original setup steps are done: yt-dlp installed, `channels.txt`
has at least one channel (CRpTAC), Tailscale connected, manual run
verified, both launchd jobs installed and confirmed running, Tailscale
Funnel enabled and persistent, and Overcast subscribed at
`https://macbook-air-2023.tail67203.ts.net/feed.xml`.

One non-obvious fix needed along the way, worth knowing if similar issues
crop up: both launchd jobs invoke `/usr/bin/python3`, which needed **Full
Disk Access** granted (System Settings → Privacy & Security → Full Disk
Access) before it could read/write anything in this project's folder,
since it lives under `~/Documents`. Without that grant, both jobs fail
immediately with `PermissionError: [Errno 1] Operation not permitted` —
this is a macOS privacy restriction on background processes, not a bug in
`watcher.py`. Manual `python3 watcher.py` runs from Terminal don't hit this
since Terminal already has its own access, which spawned processes
inherit.

The app is now in normal day-to-day operation — see the "Day-to-day
maintenance" section of `README.md`.

## Key files

- `watcher.py` — the whole app; single file, stdlib only except for the
  `yt-dlp` subprocess call. Config constants (BASE_URL, podcast title, etc.)
  are at the top.
- `channels.txt` — channel IDs to watch, `#` for comments
- `episodes.json` — append-only-ish record of every video seen: title,
  published date, source URL, downloaded filename (or null if it failed)
- `feed.xml` — generated output, don't hand-edit, `watcher.py` overwrites it
  every run
- `watcher.log` — plain text log, timestamped, appended each run
- `com.dan.ytrss.*.plist` (project root, not a `launchd/` subfolder) —
  copies of what's installed in `~/Library/LaunchAgents/`; if these are
  edited, the installed copies also need updating and the jobs reloaded
  (see README.md). Both reference `/usr/bin/python3` (the older
  system Python, 3.9.6) rather than the Homebrew `python3` used in manual
  testing - confirmed to work fine since `watcher.py` only uses stable
  stdlib features. `YTDLP_BIN` in `watcher.py` is a full path
  (`/usr/local/bin/yt-dlp`) rather than relying on `PATH`, since launchd
  jobs don't get the Homebrew `/usr/local/bin` PATH entry that interactive
  shells do.

## Known limitations / things not yet built

- No automatic cleanup — `downloads/` grows forever, nothing expires old
  episodes
- No episode duration in the RSS feed (skipped to avoid extra dependencies
  like mutagen/ffprobe)
- Failed downloads (e.g. transient yt-dlp/YouTube errors) are retried
  automatically on the next run. Videos skipped for being older than
  `ONLY_DOWNLOAD_AFTER` are recorded the same way (`filename: null`) but are
  never retried, since that skip is intentional
- `BASE_URL` is hardcoded to the Mac's Tailscale Funnel hostname
  (`https://macbook-air-2023.tail67203.ts.net`), not its Tailscale IP, so it
  survives IP changes. It would need updating (and `watcher.py` re-run to
  regenerate `feed.xml`) only if the machine or tailnet is ever renamed.
  Funnel itself needs to stay enabled and running (`tailscale funnel --bg
  8000`) for the feed to be reachable — check with `tailscale funnel
  status` if the feed stops loading in Overcast
- The feed and downloaded audio are technically reachable by anyone on the
  public internet with the URL, since Funnel exposes them outside Dan's
  tailnet (Overcast needs this — its servers fetch feeds/audio themselves,
  not from Dan's phone). Privacy currently relies on the hostname being
  obscure, not on any real access control. Dan was offered the option of
  adding a secret path segment for tighter privacy and chose to skip it for
  now
- No dedup by title/content, only by YouTube video ID — a re-upload of the
  same video under a new ID would download again

## Conventions when working on this project

- Keep it dependency-light — stdlib Python plus the `yt-dlp` CLI is the
  whole stack on purpose, so it's easy for Dan to understand and fix
  himself without a virtualenv
- Preserve the manual-test-first workflow: changes should be runnable via
  `python3 watcher.py` directly in Terminal before touching the launchd
  jobs
- If adding config options, put them as constants at the top of
  `watcher.py` rather than a separate config file, to match the existing
  style
- Full setup and maintenance instructions live in `README.md` in this same
  folder — update it alongside any change that affects setup or day-to-day
  use