Plankton

Self-hosted media manager, running on a Raspberry Pi next to Jellyfin.

Search for a movie across several torrent aggregators, pick the right release, download it, and file it automatically into the Jellyfin library, all from a web interface.

Rewritten in Rust. The Python version is kept under the python tag; its README documents the search data contract, which remains the spec.

Status

Version 0.2: search a title (TMDB for movies and shows, AniList for anime), see its releases from your own indexers, pick one and download it, from a web interface behind a login, on a Raspberry Pi behind ProtonVPN. Finished downloads are filed into the Jellyfin library and seeded from there, without a second copy, up to a target ratio. The interface shows each disk's free space. Detailed plan: docs/ROADMAP.md.

• Downloads and progress tracking
• Login
• Web interface (desktop)
• Automatic filing into the Jellyfin library
• Search: catalog, then releases from indexers you describe (JSON or RSS)
• Free space per disk
• Library management: list and delete what was filed

Development

Once per clone, enable the project's git hooks (formatting, commit message format, clippy). Branches, pull requests and commit conventions are described in CONTRIBUTING.md.

git config core.hooksPath .githooks

cp .env.example .env
mkdir -p media        # every disk in PLANKTON_DISKS must exist
cargo run -- create-user <name>   # asks for a password
cargo run
curl localhost:2727/health

Locally, add PLANKTON_COOKIE_SECURE=false to .env: the session cookie is otherwise HTTPS only.

CI

The repository lives on a self-hosted Forgejo forge, https://git.thibot.fr/thibault/plankton. Its workflows are in .forgejo/workflows/ (.github/workflows/ holds the GitHub versions):

• ci.yml: formatting, clippy, tests and the web build, on every pull request and after each merge into a version branch.
• image.yml: on a version branch's pull request into main, builds both binaries and the image; once merged, publishes it and updates the Pi.

They run on a Forgejo runner labelled pc, on the developer's PC, in host mode: jobs use the PC's toolchains (rustup, cross, node, podman) and keep their caches. To set one up:

# A registration token: repository Settings → Actions → Runners.
forgejo-runner register --instance https://git.thibot.fr --token <token> \
  --name my-pc --labels pc:host
forgejo-runner daemon

Web interface

The web interface lives in web/ (Svelte 5, Tailwind CSS 4, daisyUI 5). npm run build writes it to web/dist/, which Plankton serves at the root: read from disk by debug builds, embedded in the binary by release builds. The API is under /api.

cd web
npm ci
npm run build   # then open http://localhost:2727 with `cargo run` running
npm run dev     # or: live reload on http://localhost:5173, API proxied to :2727
npm run check   # type checking

Debug builds and tests compile without web/dist/; a release build refuses to.

Three screens. Search finds a title in the catalog and lists its releases on every indexer, once its group is opened (a season first for shows), with filters on sources, size, seeders and resolution. Select opens the release in Magnet, with the library target of the search already filled in; a magnet can also be pasted there directly. Downloads follows them live.

Ready-made requests for every endpoint live in the Bruno collection under bruno/ (see CONTRIBUTING.md).

Variable	Default
PLANKTON_DISKS	required	Library disks, separated by :.
PLANKTON_PORT	2727	HTTP API.
PLANKTON_DATA_DIR	./data	SQLite, BitTorrent session and DHT state.
PLANKTON_MIN_FREE_GB	10	Reserve kept free on each disk.
PLANKTON_BT_PORT	random	BitTorrent (TCP) and DHT (UDP) port.
PLANKTON_BT_PORT_FILE	unset	File holding the port (gluetun), instead.
PLANKTON_DEFAULT_RATIO	1.0	Target seed ratio.
PLANKTON_METADATA_TIMEOUT_SECS	60	How long POST /api/metadata waits for peers.
PLANKTON_JELLYFIN_URL	—	Jellyfin as Plankton reaches it. With the key, the library is scanned after each change.
PLANKTON_JELLYFIN_API_KEY	—	Jellyfin: Dashboard → API Keys. Both or neither.
PLANKTON_COOKIE_SECURE	true	Session cookie over HTTPS only.
PLANKTON_TMDB_API_KEY	—	TMDB's API Read Access Token (or v3 key), to search movies and shows. Without it, only anime.
PLANKTON_CATALOG_LANGUAGE	fr-FR	Language of TMDB's titles and overviews.
PLANKTON_INDEXERS_DIR	{data_dir}/indexers	Indexer definitions (*.json), see docs/INDEXERS.md.

Each torrent goes entirely onto a single disk, staging included (<disk>/.plankton/staging), then is filed with a rename on that same disk: never a copy. GET /health reports the free space of each disk, and returns 503 if the database stops responding or a disk is gone.

API

✅ implemented, the rest is planned. Everything but /health and /api/auth/login answers 401 without a session; any other path serves the web interface.

Method	Route
POST	/api/auth/login	Opens a session (cookie). ✅
POST	/api/auth/logout	Closes it. ✅
GET	/api/auth/me	The logged-in user. ✅
POST	/api/metadata	Files of a magnet, without downloading. ✅
POST	/api/downloads	Downloads the selected files of a magnet. ✅
GET	/api/downloads	State and progress of all torrents. ✅
GET	/api/downloads/{id}	One torrent. {id} = infohash. ✅
PATCH	/api/downloads/{id}	Pause, resume, change the files or the ratio. ✅
DELETE	/api/downloads/{id}	Removes it. ?delete_files=true deletes the files. ✅
GET	/api/events	Live updates (server-sent events). ✅
GET	/api/storage	Space of each disk, and what a new download can use. ✅
GET	/api/catalog/search?q=	Movies, shows (TMDB) and anime (AniList) by title. ✅
GET	/api/catalog/{kind}/{id}	One title: IMDb id, seasons, other names. ✅
GET	/api/indexers	Loaded indexer definitions, and the broken files. ✅
POST	/api/indexers/reload	Reads the definitions folder again. ✅
POST	/api/indexers/check	Searches a known title on each indexer: is it working? ✅
GET	/api/releases?kind=&id=	Torrents of a catalog title on every indexer, filtered and merged. ✅

curl -X POST localhost:2727/api/metadata -H 'content-type: application/json' \
  -d '{"magnet": "magnet:?xt=urn:btih:..."}'
# {"info_hash": "...", "name": "...", "total_bytes": 276445467,
#  "files": [{"index": 1, "path": ["Big Buck Bunny.mp4"], "bytes": 276134947}, ...],
#  "proposal": {"kind": "movie", "title": "Big Buck Bunny", "year": null, "season": null}}

index is what will select the files to download. path lists the folders then the file name. proposal is where the download would be filed in the library, guessed from the name: kind is movie, show or anime. 400 for an invalid magnet, 504 if no peer answers within PLANKTON_METADATA_TIMEOUT_SECS.

curl -X POST localhost:2727/api/downloads -H 'content-type: application/json' \
  -d '{"magnet": "magnet:?xt=urn:btih:...", "files": [0, 2], "ratio": 1.5,
       "target": {"kind": "movie", "title": "Big Buck Bunny", "year": 2008, "season": null}}'
# 202 {"info_hash": "...", "name": "...", "disk": "/mnt/hdd/jellyfin",
#      "output_folder": "/mnt/hdd/jellyfin/.plankton/staging/<infohash>",
#      "target": {...}, "library_folder": "/mnt/hdd/jellyfin/films/Big Buck Bunny (2008)",
#      "selected_bytes": 310520, "needed_bytes": 276445467, "ratio": 1.5}

files are indices from /api/metadata. ratio defaults to PLANKTON_DEFAULT_RATIO. target defaults to the proposal of /api/metadata; it gives films/Title (Year), shows/Title (Year)/Season NN or anime/… on the chosen disk, with : turned into - and the other characters exFAT refuses removed. The torrent goes entirely onto the disk with the most room left, counting what downloads in progress will still write. needed_bytes can exceed selected_bytes: pieces are downloaded whole, so an unselected file sharing a piece with a selected one is written too, and exFAT allocates a file up to the last byte written.

400 for an invalid magnet, file index, ratio or target (empty title), 409 if the torrent is already added, 422 if a file name cannot exist on exFAT (: * ? " < > | \), 507 if no disk has room, 504 if no peer sends the metadata in time.

curl localhost:2727/api/downloads/<infohash>
# {"info_hash": "...", "name": "...", "disk": "...", "output_folder": "...",
#  "files": [1], "target": {...}, "library_folder": "...", "ratio": 1.0, "state": "downloading", "added_at": "...",
#  "engine": "live", "error": null, "selected_bytes": 276134947,
#  "downloaded_bytes": 251658240, "progress": 0.91, "finished": false,
#  "uploaded_bytes": 0, "current_ratio": 0.0, "download_bytes_per_sec": 30408704,
#  "upload_bytes_per_sec": 0, "peers": 33, "eta_secs": 0}
curl -X DELETE 'localhost:2727/api/downloads/<infohash>?delete_files=true'   # 204

GET /api/downloads returns the same objects, oldest first. state is Plankton's (downloading, seeding, done), engine is librqbit's (initializing, live, paused, error, or missing if the torrent is no longer in the session). uploaded_bytes is the total since the torrent was added, across pauses and restarts (saved every minute and at shutdown). DELETE keeps the files unless delete_files=true, which deletes the whole staging folder, or once filed, the torrent's files in the library (not the others of the same folder) and its parts folder. 400 for an invalid infohash, 404 for an unknown one.

A finished download is filed into the library within 5 seconds: its selected files are renamed into library_folder, the unselected ones (which librqbit still needs for the pieces they share) into the hidden <disk>/.plankton/parts/<infohash>/. It then seeds from the library, and state becomes seeding. Everything stays on the same disk, so nothing is copied. If a file is already there (another release of the same movie), nothing moves: the download keeps seeding from staging and filing_error says why.

With PLANKTON_JELLYFIN_URL and PLANKTON_JELLYFIN_API_KEY, Plankton then asks Jellyfin to scan its libraries (POST /Library/Refresh), as it does after deleting a filed download with its files. A Jellyfin that does not answer only gives a warning in the logs.

Once current_ratio reaches ratio (uploaded bytes over selected bytes, paused or not), the download leaves the session: its files stay in the library, its parts folder is deleted, and state becomes done. A done download can no longer be changed (409), only removed.

curl -X PATCH localhost:2727/api/downloads/<infohash> -H 'content-type: application/json' \
  -d '{"paused": false, "files": [0, 2], "ratio": 2}'
# 200, the new status (same object as GET)

Each field is optional, but at least one is required. Asking for the current state (pausing a paused torrent) is not an error. A new selection must fit on the torrent's disk (507 otherwise), and cannot be changed while librqbit is still checking the files (409, try again) or once filed (409). Deselected files keep what was already downloaded.

curl -N localhost:2727/api/events
# event: downloads
# data: [{"info_hash": "...", "progress": 0.21, ...}]

A downloads event carries the same list as GET /api/downloads: once on connection, then each time something changes (checked every second). The stream ends when the server shuts down. Behind a reverse proxy, disable response buffering (flush_interval -1 in Caddy).

Storage

curl localhost:2727/api/storage
# {"disks": [{"path": "/mnt/hdd/jellyfin", "available": true,
#             "total_bytes": 1000203087872, "free_bytes": 412316860416,
#             "reserve_bytes": 10000000000, "usable_bytes": 402316860416,
#             "pending_bytes": 2147483648, "room_bytes": 400169376768},
#            {"path": "/mnt/sd/jellyfin", "available": false, "total_bytes": null,
#             "free_bytes": null, "reserve_bytes": 10000000000, "usable_bytes": null,
#             "pending_bytes": 0, "room_bytes": null}],
#  "totals": {"total_bytes": 1000203087872, "free_bytes": 412316860416, ...}}

usable_bytes is the free space minus the reserve (PLANKTON_MIN_FREE_GB). pending_bytes is what the unfinished downloads on that disk will still write: librqbit does not preallocate, so the free space does not show it yet. room_bytes (usable minus pending) is what a new download can use: the figure POST /api/downloads picks the disk with, and refuses with 507 when no disk has enough. An unplugged disk is available: false, with no sizes, and is left out of totals.

Catalog

Searching a title comes before searching its torrents: it gives the exact title, year, IMDb id and seasons that releases are matched against.

curl 'localhost:2727/api/catalog/search?q=frieren'
# {"results": [{"kind": "anime", "id": 154587, "title": "Frieren: Beyond Journey's End",
#               "original_title": "Sousou no Frieren", "year": 2023, "poster_url": "..."},
#              {"kind": "movie", "id": 1, "title": "...", "year": 2021, ...}],
#  "errors": []}
curl localhost:2727/api/catalog/show/2
# {"kind": "show", "id": 2, "title": "...", "imdb_id": "tt...",
#  "seasons": [{"number": 1, "name": "Season 1", "episodes": 10, "year": 2022}, ...]}

Movies and shows come from TMDB (id is TMDB's), anime from AniList (id is AniList's; each season is its own title there, like in release names). TMDB's Japanese animation is left out when AniList answers. Exact title matches come first. If a source fails, its error is in errors and the other's results are still there; 502 if every source failed. Without PLANKTON_TMDB_API_KEY, only anime are searched, and movie or show details answer 503. Answers are kept 10 minutes in memory.

Indexers

Plankton ships a search engine for torrent indexers, but no indexer: each one is described by a JSON definition file that you write and keep in PLANKTON_INDEXERS_DIR. Sites that answer in JSON or with an RSS feed can be described. The format is in docs/INDEXERS.md. Definitions are read at startup; after changing them, reload:

curl -X POST localhost:2727/api/indexers/reload
# {"indexers": [{"id": "sample-movies", "name": "Sample movies", "enabled": true,
#                "kinds": ["movie"], "search": ["imdb", "title"], "type": "json"}],
#  "errors": [{"file": "broken.json", "error": "missing field `name` at line 1 column 2"}]}

GET /api/indexers answers the same without reading the folder. A broken file is listed in errors (and logged) and left out; the others still load.

To see whether each enabled indexer works, POST /api/indexers/check searches a well-known title on it (never from the cache):

curl -X POST localhost:2727/api/indexers/check
# {"indexers": [{"id": "sample-movies", "status": "ok", "count": 12, "ms": 420, "error": null},
#               {"id": "sample-anime", "status": "failed", "count": 0, "ms": 15000,
#                "error": "no complete answer within 15 s"}]}

status is ok (releases found), empty (it answered, but nothing usable) or failed.

Releases

The torrents of a catalog title (kind and id as in /api/catalog), from every enabled indexer of that kind, asked at once:

curl 'localhost:2727/api/releases?kind=show&id=2&season=1'
# {"releases": [{"name": "Sample.Show.S01.1080p.WEB-DL.x265", "indexers": ["sample-shows"],
#                "magnet": "magnet:?xt=urn:btih:...", "info_hash": "0123...",
#                "size_bytes": 1950000000, "seeders": 120, "leechers": 8,
#                "published_at": "2023-11-14T22:13:20Z", "link": null,
#                "quality": {"resolution": "1080p", "source": "WEB-DL", "codec": "x265",
#                            "hdr": null, "dv": false, "languages": []},
#                "season": 1, "last_season": null, "pack": true}],
#  "indexers": [{"id": "sample-shows", "status": "ok", "count": 14, "filtered": 9, "ms": 380,
#                "error": null}],
#  "target": {"kind": "show", "title": "Sample Show", "year": 2019, "season": 1}}

season (from 1) fills the season variables of the definitions' URLs. It is required for shows: only season packs holding that season are kept (single episodes are left out). For anime, single episodes and other seasons are left out: the entry's season is season, else the one its names say (2nd Season), else 1. Releases an indexer found by title (not by IMDb id) must name the title, its original title or another of its names, and for a movie carry a year within one of the title's. filtered counts what each indexer lost to these rules. Each release name is read for its quality, languages, season and whether it is a pack.

The same torrent found on several indexers is listed once, with all of them in indexers. Releases nobody seeds (seeders is 0) are left out; unknown seeders are null and come last. An indexer that fails does not fail the search: it is failed in indexers, with the reason. Each indexer's results are kept 2 minutes in memory. target is where a release of this title is filed, as POST /api/downloads takes it. Details in docs/INDEXERS.md.

Accounts

curl -c cookies.txt -X POST localhost:2727/api/auth/login -H 'content-type: application/json' \
  -d '{"username": "thibault", "password": "..."}'
# {"id": 1, "username": "thibault", "role": "admin"}, plus the session cookie
curl -b cookies.txt localhost:2727/api/downloads

The session lasts 30 days. With "remember": false (the unchecked "Remember me" box of the web interface), the cookie has no Max-Age and goes away when the browser closes, and the session expires after a day at most.

Accounts are created from the command line, never through the API: plankton create-user <name> (the first account is admin), and plankton set-password <name>, which also closes all the user's sessions. Both ask for the password on the terminal (at least 8 characters). In Docker: docker compose exec -it plankton plankton create-user <name>.

Passwords are hashed with argon2id. A session lasts 30 days; its cookie is HttpOnly and SameSite=Strict, and the database only keeps a SHA-256 of it.

A failed login always takes one second. After 5 failures in 15 minutes from the same address, POST /api/auth/login answers 429 (with Retry-After) without checking anything. Behind Caddy, the address is the last one of X-Forwarded-For, trusted only from loopback or a private network. Requests that change something (POST, PATCH, DELETE) are refused with 403 when a browser sends them from another site (Origin different from Host, or Sec-Fetch-Site: cross-site).

Deployment

Each release (a version branch merged into main) publishes the image git.thibot.fr/thibault/plankton, on the forge's registry, for arm64 (the Pi) and amd64 (PCs, Intel and AMD). Next to compose.yaml and .env:

docker compose pull     # the image for this machine's architecture
docker compose up -d

latest follows the releases; each one is also tagged with its version (0.1.0…), to go back to it.

Automatic update

After publishing, CI calls https://plankton.thibot.fr/hooks/deploy. On the Pi, webhook checks that the request is signed with a secret shared with the forge, then runs deploy/update.sh (the pull above). Without the secret, the request is refused and nothing runs.

Once, on the Pi (the deploy/ folder of the repository in ~/plankton/):

sudo apt install webhook
# The shared secret: in .env, and on the forge as DEPLOY_HOOK_SECRET
# (repository Settings → Actions → Secrets).
echo "DEPLOY_HOOK_SECRET=$(openssl rand -hex 32)" >> ~/plankton/.env
sudo cp ~/plankton/deploy/plankton-hook.service /etc/systemd/system/
sudo systemctl enable --now plankton-hook

Caddy forwards /hooks/ to it (see below). journalctl -u plankton-hook shows the calls and the output of update.sh; deploy/update.sh also updates by hand.

Test version

To try a version before its release, cross-compile on the PC and build the image on the Pi. One command does it all, and asks for the Pi's SSH password once:

scripts/deploy.sh               # build, check the glibc floor, rsync, restart
scripts/deploy.sh --build-only  # same, without touching the Pi

The test image takes the name of the release one: the next docker compose pull puts the release back.

PI (default pi) and PI_DIR (default plankton) set the SSH host and the folder on it. The steps it runs:

npm --prefix web ci && npm --prefix web run build   # embedded in the binary
CROSS_CONTAINER_ENGINE=podman cross build --release --target aarch64-unknown-linux-gnu
rsync -R target/aarch64-unknown-linux-gnu/release/plankton \
  Dockerfile .dockerignore compose.yaml pi:~/plankton/
ssh pi 'cd ~/plankton && mkdir -p data && docker compose up -d --build'

--build matters: the Dockerfile copies the binary into the image, so a plain restart would run the previous version.

On the Pi, Plankton writes directly to the mergerfs branches (/mnt/internal and /mnt/hdd/jellyfin), not into /media: it picks the disk for each torrent (see ROADMAP, mergerfs spike). /mnt/internal is owned by root: create once sudo install -d -o 1000 -g 1000 /mnt/internal/.plankton. PUID/PGID default to 1000. data/ must exist before the first start, otherwise Docker creates it as root and Plankton cannot write to it. The API only listens on 127.0.0.1:2727: Caddy serves it over HTTPS (see below).

Cross-compilation

cross (with podman) rather than local cross-compilation: Arch's aarch64-linux-gnu toolchain ships glibc 2.44, while Raspberry Pi OS bookworm has 2.36. A binary built locally would refuse to start on the Pi.

The official cross image itself moved to Ubuntu 24.04 (glibc 2.39): Cross.toml replaces it with a Debian bookworm toolchain (docker/cross-aarch64.Dockerfile). cross then has to build an image: with Docker it requires buildx, podman does not, hence CROSS_CONTAINER_ENGINE=podman. If podman fails on networking (pasta), add CROSS_BUILD_OPTS=--network=host CROSS_CONTAINER_OPTS=--network=host. After this kind of change, clear target/release: build scripts compiled by the previous image would no longer run.

The same goes for PCs and servers (x86_64, Intel and AMD), with docker/cross-x86_64.Dockerfile:

TARGET=x86_64-unknown-linux-gnu scripts/deploy.sh --build-only

Check the glibc floor of the resulting binary:

aarch64-linux-gnu-objdump -T target/aarch64-unknown-linux-gnu/release/plankton \
  | grep -o 'GLIBC_[0-9.]*' | sort -Vu | tail -1

VPN

BitTorrent traffic goes through a gluetun container, with network_mode: "service:gluetun" on the Plankton container: the process only sees the VPN interface, so no leak is possible.

compose.yaml runs gluetun with ProtonVPN over WireGuard, on P2P servers only (PORT_FORWARD_ONLY). It needs a paid Proton plan and a WireGuard key:

1. On account.protonvpn.com, Downloads → WireGuard configuration: platform Router or GNU/Linux, NAT-PMP (Port Forwarding) on, any P2P server. Copy the PrivateKey line of the generated file.
2. On the Pi, next to compose.yaml, a .env file readable by you only:

   install -m 600 /dev/null .env
   echo 'WIREGUARD_PRIVATE_KEY=<the key>' >> .env
   

   Never commit it: anyone with the key uses your Proton account.
3. docker compose up -d --build, then docker logs gluetun: look for Public IP address is … (a Proton address) and port forwarded is ….

Plankton's API is published by gluetun (127.0.0.1:2727), since Plankton shares its network.

The forwarded port is what lets peers connect to you, so it decides uploads and much of the speed. Proton gives a new one with each connection. gluetun writes it to gluetun-port/forwarded_port, which Plankton reads (PLANKTON_BT_PORT_FILE): it listens on it and announces it to trackers and the DHT. When the port changes or disappears (gluetun reconnecting or restarted), Plankton stops within 15 seconds and exits with an error; Docker starts it again, it waits up to 2 minutes for the new port, and fastresume spares a full recheck. docker logs plankton shows the reason.

Without gluetun, the in-app equivalent is SessionOptions.bind_device_name (SO_BINDTODEVICE, covers DHT, trackers and peers), but it requires CAP_NET_RAW and does not protect as well as a network namespace.

Port forwarding decides your speeds. Without a port forwarded by the VPN provider, no incoming connection arrives: downloading works, but in passive mode, which is much slower.

Caddy

Caddy runs on the Pi itself (not in Docker) and serves the API as https://plankton.thibot.fr. In /etc/caddy/Caddyfile:

plankton.thibot.fr {
	# Disk paths and free space: no need to show them to the world. Docker and
	# local checks call 127.0.0.1:2727 directly.
	respond /health 404

	# The deploy hook (deploy/hooks.yaml): only requests signed by CI do
	# anything.
	reverse_proxy /hooks/* 127.0.0.1:9000

	reverse_proxy 127.0.0.1:2727 {
		# /events streams: send each event at once, no buffering.
		flush_interval -1
	}

	header {
		Strict-Transport-Security "max-age=31536000"
		X-Content-Type-Options "nosniff"
		X-Frame-Options "DENY"
		Referrer-Policy "no-referrer"
		-Server
	}
}

Then caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile and sudo systemctl reload caddy. Caddy gets the certificate by itself.

Plankton relies on what Caddy sends along: the original Host (compared to the browser's Origin for cross-site requests) and X-Forwarded-For (the client address for the login rate limit; Caddy reaches Plankton through Docker's network, a private address, which is what makes Plankton trust it). The session cookie is Secure: log in through https://, not through an SSH tunnel.