feat: data lifecycle rewrite, download clients, wanted list, and central catalog index
Build & publish Arch package / arch-package (push) Successful in 2m12s
Search index maintenance / maintain-index (push) Successful in 2h22m28s

Ships the fresh-start schema cleanup: rebuilt explore catalog index
pipeline (dump import, artifact fetch/build, incremental listen-count
refresh), a new download subsystem (Lidarr/Prowlarr/qBittorrent/SABnzbd/
slskd/yt-dlp providers, staging, reconciliation, wanted list), and the
supporting schema/query/store changes across backend and frontend.

Also includes two smaller follow-ups: bump the central index's
rebuild-after cadence from 90 to 180 days, and remove the Explore
"library only" online/offline toggle entirely (frontend-only, no
backend counterpart) rather than carry unused UI/state.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2Agd9af5hE7qzti2ackiS
This commit is contained in:
2026-08-06 17:12:01 -04:00
co-authored by Claude Sonnet 5
parent d0d86f85d5
commit e190fd75b9
165 changed files with 31088 additions and 5192 deletions
+28 -9
View File
@@ -4,7 +4,7 @@ name: Search index maintenance
# trigger below runs the same command:
#
# no completed import -> build (first run, or resume a partial one)
# import older than 3mo -> rebuild (re-import from the newest dump)
# import older than 6mo -> rebuild (re-import from the newest dump)
# otherwise -> refresh (fold in new incremental listens)
#
# A refresh is cheap and no-ops when nothing new has been published, so
@@ -13,7 +13,7 @@ on:
push:
branches: [main]
schedule:
# Weekly update pass. The 3-month rebuild is triggered by the same
# Weekly update pass. The 6-month rebuild is triggered by the same
# command when it notices the import has aged out.
- cron: '0 4 * * 1'
workflow_dispatch:
@@ -91,7 +91,12 @@ jobs:
- name: Build tools
working-directory: /src
run: go build -o /usr/local/bin/ ./cmd/indexbuild ./cmd/indexexport
# The dump importer is behind the `indexbuild` tag so it is not
# linked into the app binary; cmd/indexbuild carries the same tag
# and will not build without it.
run: |
go build -tags indexbuild -o /usr/local/bin/ ./cmd/indexbuild
go build -o /usr/local/bin/ ./cmd/indexexport
- name: Maintain index
id: maintain
@@ -122,12 +127,26 @@ jobs:
if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true'
run: |
set -eu
version="$(date -u +%Y%m%d)"
base="${SERVER_URL}/api/packages/${OWNER}/generic/yellowjacket-core-index/${version}"
for f in core-index.db.zst core-index.db.zst.sha256; do
echo "Uploading $f -> $version"
curl --fail-with-body --user "${OWNER}:${PACKAGE_TOKEN}" \
--upload-file "/tmp/$f" "${base}/${f}"
pkg="${SERVER_URL}/api/packages/${OWNER}/generic/yellowjacket-core-index"
# Published twice: under a dated version for history, and under
# the fixed "latest" version the client fetches. Clients cannot
# discover the newest dated version on their own — the package
# listing API requires a token, while a plain file GET does not
# — so "latest" is what makes an anonymous first run possible.
#
# A generic package rejects re-uploading a filename that already
# exists, so "latest" is deleted before being rewritten. It is
# absent on the very first publish, hence the tolerated 404.
curl --silent --show-error --user "${OWNER}:${PACKAGE_TOKEN}" \
--request DELETE "${pkg}/latest" || true
for version in "$(date -u +%Y%m%d)" latest; do
for f in core-index.db.zst core-index.db.zst.sha256; do
echo "Uploading $f -> $version"
curl --fail-with-body --user "${OWNER}:${PACKAGE_TOKEN}" \
--upload-file "/tmp/$f" "${pkg}/${version}/${f}"
done
done
- name: Summary