Compare commits
87
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
97f2eb85ee | ||
|
|
0821deb877 | ||
|
|
31ada14111 | ||
|
|
20139394f3 | ||
|
|
eb139cf872 | ||
|
|
ae82fd2233 | ||
|
|
3c3197df4b | ||
|
|
e16bd245bd | ||
|
|
887a9324b4 | ||
|
|
fcb484ead5 | ||
|
|
48de41cd69 | ||
|
|
66a6ee63ab | ||
|
|
10660c8168 | ||
|
|
441b67daaa | ||
|
|
026f26bdf6 | ||
|
|
73dc80bdc9 | ||
|
|
760021ea5a | ||
|
|
63ec068add | ||
|
|
a2ff0aed4c | ||
|
|
12e75ee24c | ||
|
|
792e87298b | ||
|
|
266e7032dd | ||
|
|
d6b48fb3ac | ||
|
|
3bf27e3fd5 | ||
|
|
185eb1b125 | ||
|
|
b3556d825c | ||
|
|
bf4f352117 | ||
|
|
1062b7c0bc | ||
|
|
48abecb830 | ||
|
|
e1c07438e9 | ||
|
|
6e563f3846 | ||
|
|
186f6a5839 | ||
|
|
590a0d86dd | ||
|
|
36af7090d9 | ||
|
|
3e142f8c35 | ||
|
|
3d375adab1 | ||
|
|
e3d492e130 | ||
|
|
e6f30b6e43 | ||
|
|
351798fd66 | ||
|
|
40984f6086 | ||
|
|
786d9c6110 | ||
|
|
0019310ca4 | ||
|
|
1940cb548f | ||
|
|
37e3373db9 | ||
|
|
8d5d8af297 | ||
|
|
9ce79ee416 | ||
|
|
b3a0814f24 | ||
|
|
2c576fa1e8 | ||
|
|
544dbdb4db | ||
|
|
087eb77875 | ||
|
|
6fb7b5ea11 | ||
|
|
369810e06b | ||
|
|
e51cb13662 | ||
|
|
3d65da0529 | ||
|
|
52cbef27c4 | ||
|
|
c03c0b8ec4 | ||
|
|
8c48105ca3 | ||
|
|
1c4d6ca9a1 | ||
|
|
6bf832a4ba | ||
|
|
4f8257ef72 | ||
|
|
b505959934 | ||
|
|
d0250a2133 | ||
|
|
de2b324e20 | ||
|
|
2c78b58207 | ||
|
|
a9852c18a0 | ||
|
|
409bfd5e89 | ||
|
|
0eeef6048e | ||
|
|
4fc0cdeab7 | ||
|
|
eb059a3d71 | ||
|
|
dcabec8b1d | ||
|
|
b3737d30af | ||
|
|
0bfa2136be | ||
|
|
b1cdef8769 | ||
|
|
d661836347 | ||
|
|
28eecf0a97 | ||
|
|
e8690476bd | ||
|
|
7e0be8fa30 | ||
|
|
1b05dde382 | ||
|
|
29299d17da | ||
|
|
57fbbdf0d2 | ||
|
|
df2e9ea777 | ||
|
|
c99c8efa11 | ||
|
|
904786b941 | ||
|
|
b6651310ea | ||
|
|
da38b865fc | ||
|
|
ced537ecf2 | ||
|
|
e14a34fccf |
@@ -1,7 +1,7 @@
|
|||||||
name: Build & publish the Android APK
|
name: Build & publish the Android APK
|
||||||
|
|
||||||
# The fifth workflow, and the second that publishes. It builds a signed
|
# The fifth workflow, and the second that publishes. It builds a signed
|
||||||
# fat APK (arm64-v8a + x86_64) on every version tag and puts it in
|
# arm64-v8a APK on every version tag and puts it in
|
||||||
# Gitea's *generic* package registry, which — unlike the repository — is
|
# Gitea's *generic* package registry, which — unlike the repository — is
|
||||||
# readable without credentials. That is what lets an Obtainium client
|
# readable without credentials. That is what lets an Obtainium client
|
||||||
# poll a plain URL with no token and no public mirror of the source.
|
# poll a plain URL with no token and no public mirror of the source.
|
||||||
@@ -129,6 +129,18 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
v="${v#v}"
|
v="${v#v}"
|
||||||
|
|
||||||
|
# v0.0.0 is semantic-release's version floor, not a shipment —
|
||||||
|
# see the bootstrap step in release.yml. It is skipped cleanly
|
||||||
|
# rather than failing the guard below, because a 45-minute red
|
||||||
|
# run against a tag that was never meant to ship is noise, and
|
||||||
|
# this is the most expensive of the four workflows a tag fires.
|
||||||
|
if [ "$v" = "0.0.0" ]; then
|
||||||
|
echo "v0.0.0 is the version floor, not a release; nothing to build"
|
||||||
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
# Android orders releases by an integer and refuses anything
|
# Android orders releases by an integer and refuses anything
|
||||||
# not greater than what is installed. 1.3.1 -> 10301, which
|
# not greater than what is installed. 1.3.1 -> 10301, which
|
||||||
# increases as long as minor and patch stay below 100.
|
# increases as long as minor and patch stay below 100.
|
||||||
@@ -143,9 +155,24 @@ jobs:
|
|||||||
|
|
||||||
echo "version=$v" >> "$GITHUB_OUTPUT"
|
echo "version=$v" >> "$GITHUB_OUTPUT"
|
||||||
echo "code=$code" >> "$GITHUB_OUTPUT"
|
echo "code=$code" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "tag=v$v" >> "$GITHUB_OUTPUT"
|
||||||
echo "building $v (versionCode $code)"
|
echo "building $v (versionCode $code)"
|
||||||
|
|
||||||
|
# Releases restarted at 0.0.1 when they became automatic (plan
|
||||||
|
# 017), so versionCode restarted at 1 — *below* the 10300 an
|
||||||
|
# installed 1.3.0 build carries. Android refuses a downgrade
|
||||||
|
# outright, and the only remedy is an uninstall, which takes the
|
||||||
|
# user's library with it. Said here because this is the file
|
||||||
|
# that computes the number.
|
||||||
|
if [ "$code" -lt 10600 ]; then
|
||||||
|
echo
|
||||||
|
echo "note: versionCode $code is below the 10600 that v1.6.0 shipped."
|
||||||
|
echo " An existing install must be removed before this one will"
|
||||||
|
echo " install, and that removal takes its library with it."
|
||||||
|
fi
|
||||||
|
|
||||||
- name: Go toolchain
|
- name: Go toolchain
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
if [ ! -x /cache/tool/go/bin/go ] || ! /cache/tool/go/bin/go version | grep -q "$GO_VERSION"; then
|
if [ ! -x /cache/tool/go/bin/go ] || ! /cache/tool/go/bin/go version | grep -q "$GO_VERSION"; then
|
||||||
@@ -156,6 +183,7 @@ jobs:
|
|||||||
/cache/tool/go/bin/go version
|
/cache/tool/go/bin/go version
|
||||||
|
|
||||||
- name: Node toolchain
|
- name: Node toolchain
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
||||||
@@ -167,6 +195,7 @@ jobs:
|
|||||||
# but still spends minutes verifying, so the guards are what make
|
# but still spends minutes verifying, so the guards are what make
|
||||||
# this cheap on every run after the first.
|
# this cheap on every run after the first.
|
||||||
- name: Android SDK and NDK (cached)
|
- name: Android SDK and NDK (cached)
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
mkdir -p "$ANDROID_HOME/cmdline-tools"
|
mkdir -p "$ANDROID_HOME/cmdline-tools"
|
||||||
@@ -225,7 +254,8 @@ jobs:
|
|||||||
# `$GITHUB_ENV` — where the `env:` dump is only masked for values
|
# `$GITHUB_ENV` — where the `env:` dump is only masked for values
|
||||||
# that are *verbatim* a secret, so a trimmed one could print in
|
# that are *verbatim* a secret, so a trimmed one could print in
|
||||||
# clear — or repeating the trimming logic in both.
|
# clear — or repeating the trimming logic in both.
|
||||||
- name: Build the signed fat APK
|
- name: Build the signed APK
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
working-directory: /src
|
working-directory: /src
|
||||||
env:
|
env:
|
||||||
KEYSTORE_B64: ${{ secrets.ANDROID_KEYSTORE_B64 }}
|
KEYSTORE_B64: ${{ secrets.ANDROID_KEYSTORE_B64 }}
|
||||||
@@ -277,6 +307,18 @@ jobs:
|
|||||||
size=$(stat -c %s "$keystore")
|
size=$(stat -c %s "$keystore")
|
||||||
magic=$(od -An -N4 -tx1 "$keystore" | tr -s ' ' | sed 's/^ //')
|
magic=$(od -An -N4 -tx1 "$keystore" | tr -s ' ' | sed 's/^ //')
|
||||||
echo "keystore: $size bytes, first four bytes: $magic"
|
echo "keystore: $size bytes, first four bytes: $magic"
|
||||||
|
|
||||||
|
# The fingerprint of the decoded file, so "is the secret the
|
||||||
|
# keystore I have locally?" is answerable without guessing.
|
||||||
|
# A hash of a *public* certificate store gives nothing away,
|
||||||
|
# and the alternative is comparing byte counts by eye.
|
||||||
|
#
|
||||||
|
# sha256sum ~/path/to/yellowjacket-release.jks
|
||||||
|
#
|
||||||
|
# A password that is right for one keystore and wrong for
|
||||||
|
# another is indistinguishable from a wrong password, and this
|
||||||
|
# is the line that distinguishes them.
|
||||||
|
echo " sha256: $(sha256sum "$keystore" | cut -d' ' -f1)"
|
||||||
case "$magic" in
|
case "$magic" in
|
||||||
"30 82"*) echo " header: PKCS12 (keytool's default since JDK 9)" ;;
|
"30 82"*) echo " header: PKCS12 (keytool's default since JDK 9)" ;;
|
||||||
"fe ed fe ed") echo " header: legacy JKS" ;;
|
"fe ed fe ed") echo " header: legacy JKS" ;;
|
||||||
@@ -291,6 +333,20 @@ jobs:
|
|||||||
echo " password length after trimming: ${#pass}" >&2
|
echo " password length after trimming: ${#pass}" >&2
|
||||||
sed 's/^/ keytool: /' /tmp/ks.err | head -5 >&2
|
sed 's/^/ keytool: /' /tmp/ks.err | head -5 >&2
|
||||||
echo >&2
|
echo >&2
|
||||||
|
|
||||||
|
# A password pasted *with its shell quotes* is the one
|
||||||
|
# remaining cause that looks identical to a wrong password:
|
||||||
|
# the secret is two characters longer than the password and
|
||||||
|
# nothing in the error says so. Naming it is safe --
|
||||||
|
# stripping the quotes and carrying on would not be, since a
|
||||||
|
# password may legitimately contain them.
|
||||||
|
unquoted=$(printf '%s' "$pass" | sed "s/^['\"]//;s/['\"]$//")
|
||||||
|
if [ "$unquoted" != "$pass" ] &&
|
||||||
|
keytool -list -keystore "$keystore" -storepass "$unquoted" >/dev/null 2>&1; then
|
||||||
|
echo " ** it opens with the surrounding quotes removed. **" >&2
|
||||||
|
echo " Re-paste ANDROID_KEYSTORE_PASSWORD without them." >&2
|
||||||
|
echo >&2
|
||||||
|
fi
|
||||||
echo "Check it locally with the same two values:" >&2
|
echo "Check it locally with the same two values:" >&2
|
||||||
echo " printf %s \"\$SECRET_B64\" | base64 -d > /tmp/k.jks" >&2
|
echo " printf %s \"\$SECRET_B64\" | base64 -d > /tmp/k.jks" >&2
|
||||||
echo " keytool -list -keystore /tmp/k.jks -storepass '<password>'" >&2
|
echo " keytool -list -keystore /tmp/k.jks -storepass '<password>'" >&2
|
||||||
@@ -323,6 +379,7 @@ jobs:
|
|||||||
|
|
||||||
- name: Verify the APK
|
- name: Verify the APK
|
||||||
id: apk
|
id: apk
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
working-directory: /src
|
working-directory: /src
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
@@ -333,9 +390,14 @@ jobs:
|
|||||||
ls -la "$apk"
|
ls -la "$apk"
|
||||||
"$bt/aapt2" dump badging "$apk" | sed -n '1p;/application-label:/p;/native-code/p'
|
"$bt/aapt2" dump badging "$apk" | sed -n '1p;/application-label:/p;/native-code/p'
|
||||||
|
|
||||||
# Both ABIs, or the artifact is not the fat APK it claims to be.
|
# arm64 and *only* arm64. x86_64 Android cannot run this app
|
||||||
"$bt/aapt2" dump badging "$apk" | grep -q "native-code: 'arm64-v8a' 'x86_64'" || {
|
# (modernc's raw lstat against Android's seccomp filter, which
|
||||||
echo "the APK does not carry both ABIs" >&2; exit 1; }
|
# is every x86_64 device and not merely the emulator), so an
|
||||||
|
# x86_64 slice would be ~31 MB that runs nowhere -- and its
|
||||||
|
# reappearance would mean someone had put the ABI back in
|
||||||
|
# app/build.gradle without knowing that.
|
||||||
|
"$bt/aapt2" dump badging "$apk" | grep -q "native-code: 'arm64-v8a'$" || {
|
||||||
|
echo "the APK's ABI set is not exactly arm64-v8a" >&2; exit 1; }
|
||||||
|
|
||||||
# The identity the pipeline exists to keep stable.
|
# The identity the pipeline exists to keep stable.
|
||||||
"$bt/aapt2" dump badging "$apk" | grep -q "versionCode='${{ steps.version.outputs.code }}'" || {
|
"$bt/aapt2" dump badging "$apk" | grep -q "versionCode='${{ steps.version.outputs.code }}'" || {
|
||||||
@@ -359,6 +421,7 @@ jobs:
|
|||||||
# file, so `latest` is deleted first. Credentials are the same
|
# file, so `latest` is deleted first. Credentials are the same
|
||||||
# OWNER/PACKAGE_TOKEN pair arch-package.yml publishes with.
|
# OWNER/PACKAGE_TOKEN pair arch-package.yml publishes with.
|
||||||
- name: Publish to the Gitea package registry
|
- name: Publish to the Gitea package registry
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
working-directory: /src
|
working-directory: /src
|
||||||
env:
|
env:
|
||||||
VERSION: ${{ steps.version.outputs.version }}
|
VERSION: ${{ steps.version.outputs.version }}
|
||||||
@@ -394,3 +457,18 @@ jobs:
|
|||||||
echo
|
echo
|
||||||
echo "Obtainium URL:"
|
echo "Obtainium URL:"
|
||||||
echo " $base/latest/yellowjacket.apk"
|
echo " $base/latest/yellowjacket.apk"
|
||||||
|
|
||||||
|
# The generic registry is what Obtainium polls; the release page is
|
||||||
|
# what a person looks at. Same file, already built and already
|
||||||
|
# verified by the step above — so this cannot publish something the
|
||||||
|
# signature check would have refused.
|
||||||
|
- name: Attach the APK to the release
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
env:
|
||||||
|
TAG: ${{ steps.version.outputs.tag }}
|
||||||
|
VERSION: ${{ steps.version.outputs.version }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
./scripts/release-asset.sh "$TAG" bin/yellowjacket.apk \
|
||||||
|
"yellowjacket-${VERSION}-android-arm64.apk"
|
||||||
|
|||||||
@@ -1,8 +1,23 @@
|
|||||||
name: Build & publish Arch package
|
name: Build & publish Arch package
|
||||||
|
|
||||||
|
# Keyed on the tag, not on main. It used to publish on every push,
|
||||||
|
# deriving a version from `git describe` — so the registry accumulated a
|
||||||
|
# package per merge and none of them corresponded to anything a user
|
||||||
|
# could be told to install. release.yml decides what a release is now,
|
||||||
|
# and this builds the tag it cuts.
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [main]
|
tags: ["v*"]
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
version:
|
||||||
|
description: "Version to build (default: the latest v* tag)"
|
||||||
|
required: false
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: arch-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
arch-package:
|
arch-package:
|
||||||
@@ -17,6 +32,7 @@ jobs:
|
|||||||
REPO: ${{ github.repository }}
|
REPO: ${{ github.repository }}
|
||||||
OWNER: ${{ github.repository_owner }}
|
OWNER: ${{ github.repository_owner }}
|
||||||
SHA: ${{ github.sha }}
|
SHA: ${{ github.sha }}
|
||||||
|
REF_NAME: ${{ github.ref_name }}
|
||||||
# Arch registry name (the "$repo" in clients' pacman.conf). Arbitrary label.
|
# Arch registry name (the "$repo" in clients' pacman.conf). Arbitrary label.
|
||||||
ARCH_REPO: stable
|
ARCH_REPO: stable
|
||||||
steps:
|
steps:
|
||||||
@@ -26,8 +42,9 @@ jobs:
|
|||||||
# gtk3 was v2's stack and is now only the `-tags gtk3` escape hatch.
|
# gtk3 was v2's stack and is now only the `-tags gtk3` escape hatch.
|
||||||
# These must match the PKGBUILD's depends=() — makepkg installs
|
# These must match the PKGBUILD's depends=() — makepkg installs
|
||||||
# nothing itself, so a mismatch fails at link time, not at check time.
|
# nothing itself, so a mismatch fails at link time, not at check time.
|
||||||
|
# jq is scripts/release-asset.sh's, not the build's.
|
||||||
pacman -Syu --noconfirm --needed \
|
pacman -Syu --noconfirm --needed \
|
||||||
base-devel git go nodejs pnpm curl sudo \
|
base-devel git go nodejs pnpm curl sudo jq \
|
||||||
webkitgtk-6.0 gtk4 alsa-lib
|
webkitgtk-6.0 gtk4 alsa-lib
|
||||||
|
|
||||||
- name: Create unprivileged build user
|
- name: Create unprivileged build user
|
||||||
@@ -36,15 +53,43 @@ jobs:
|
|||||||
install -d -o builder -g builder /build
|
install -d -o builder -g builder /build
|
||||||
echo 'builder ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/builder
|
echo 'builder ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/builder
|
||||||
|
|
||||||
|
# v0.0.0 is semantic-release's version floor, not a shipment — see
|
||||||
|
# the bootstrap step in release.yml. A clean skip rather than a
|
||||||
|
# failure: a red run against a tag that was never meant to ship is
|
||||||
|
# noise, and this is one of the four workflows that would otherwise
|
||||||
|
# fire on it.
|
||||||
|
- name: Resolve the version
|
||||||
|
id: version
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
v="${{ inputs.version }}"
|
||||||
|
[ -n "$v" ] || v="$REF_NAME"
|
||||||
|
case "$v" in v*) ;; *) v="v$v" ;; esac
|
||||||
|
|
||||||
|
if [ "$v" = "v0.0.0" ]; then
|
||||||
|
echo "v0.0.0 is the version floor, not a release; nothing to build"
|
||||||
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "tag=$v" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "building $v"
|
||||||
|
|
||||||
- name: Clone repo at the pushed commit
|
- name: Clone repo at the pushed commit
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
# Token auth works for private repos and needs no SSH key in CI.
|
# Token auth works for private repos and needs no SSH key in CI.
|
||||||
sudo -u builder git clone \
|
sudo -u builder git clone \
|
||||||
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" \
|
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" \
|
||||||
/build/yellowjacket
|
/build/yellowjacket
|
||||||
|
# A tag push carries the tag's own commit in $SHA, so this checks
|
||||||
|
# out exactly what was tagged. pkgver() then reads the tag from
|
||||||
|
# the clone's own git history.
|
||||||
sudo -u builder git -C /build/yellowjacket checkout --detach "$SHA"
|
sudo -u builder git -C /build/yellowjacket checkout --detach "$SHA"
|
||||||
|
|
||||||
- name: Build package with makepkg
|
- name: Build package with makepkg
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
cd /build/yellowjacket/packaging/arch
|
cd /build/yellowjacket/packaging/arch
|
||||||
# Point the PKGBUILD at this local clone / exact commit; pkgver() then
|
# Point the PKGBUILD at this local clone / exact commit; pkgver() then
|
||||||
@@ -54,6 +99,7 @@ jobs:
|
|||||||
makepkg -f --noconfirm --cleanbuild
|
makepkg -f --noconfirm --cleanbuild
|
||||||
|
|
||||||
- name: Publish to the Gitea Arch registry
|
- name: Publish to the Gitea Arch registry
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
cd /build/yellowjacket/packaging/arch
|
cd /build/yellowjacket/packaging/arch
|
||||||
# makepkg also produces a -debug package (detached symbols); end users
|
# makepkg also produces a -debug package (detached symbols); end users
|
||||||
@@ -67,3 +113,20 @@ jobs:
|
|||||||
--upload-file "$pkg" \
|
--upload-file "$pkg" \
|
||||||
"${SERVER_URL}/api/packages/${OWNER}/arch/${ARCH_REPO}"
|
"${SERVER_URL}/api/packages/${OWNER}/arch/${ARCH_REPO}"
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# The pacman registry is for people who have added it to pacman.conf;
|
||||||
|
# the release page is for everyone else. Same file, and it is
|
||||||
|
# already built.
|
||||||
|
- name: Attach the package to the release
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
env:
|
||||||
|
TAG: ${{ steps.version.outputs.tag }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
cd /build/yellowjacket/packaging/arch
|
||||||
|
for pkg in yellowjacket-*.pkg.tar.zst; do
|
||||||
|
case "$pkg" in
|
||||||
|
yellowjacket-debug-*) continue ;;
|
||||||
|
esac
|
||||||
|
/build/yellowjacket/scripts/release-asset.sh "$TAG" "$(pwd)/$pkg"
|
||||||
|
done
|
||||||
|
|||||||
+17
-3
@@ -1,6 +1,6 @@
|
|||||||
name: CI
|
name: CI
|
||||||
|
|
||||||
# The other three workflows package and publish; none of them test
|
# The other five workflows package, publish or release; none of them test
|
||||||
# anything, so a green tick on this repo used to mean "the Arch package
|
# anything, so a green tick on this repo used to mean "the Arch package
|
||||||
# built", which is not the question anyone was asking. This is the
|
# built", which is not the question anyone was asking. This is the
|
||||||
# workflow that gates.
|
# workflow that gates.
|
||||||
@@ -9,9 +9,23 @@ name: CI
|
|||||||
# before being written here, so every step below is a transcription of
|
# before being written here, so every step below is a transcription of
|
||||||
# something observed working rather than something expected to.
|
# something observed working rather than something expected to.
|
||||||
|
|
||||||
|
# **A branch push and its PR are the same commit, and testing it twice
|
||||||
|
# costs the only runner there is.** `branches: ['**']` here meant every
|
||||||
|
# PR booked four runs — `check` and `e2e` for the branch push, then both
|
||||||
|
# again for `refs/pull/N/head` — on a host with capacity 1, where the
|
||||||
|
# queue is shared with an index build that can hold it for three hours.
|
||||||
|
#
|
||||||
|
# `pull_request` covers feature branches, and `main` is kept because a
|
||||||
|
# post-merge run is the record of the trunk's health. Since main now
|
||||||
|
# refuses direct pushes, that run happens exactly once per merge.
|
||||||
|
#
|
||||||
|
# The trade is explicit: a branch pushed with **no** PR open gets no CI.
|
||||||
|
# That is consistent with the workflow this repo committed to — every
|
||||||
|
# change goes through a PR — and the signal returns the moment one is
|
||||||
|
# opened, on the same commit.
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: ['**']
|
branches: [main]
|
||||||
pull_request:
|
pull_request:
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
|
|
||||||
@@ -78,7 +92,7 @@ jobs:
|
|||||||
# Cloned by hand rather than with actions/checkout: that is a JS
|
# Cloned by hand rather than with actions/checkout: that is a JS
|
||||||
# action and needs node inside the job container before any step
|
# action and needs node inside the job container before any step
|
||||||
# has had a chance to install it. Same approach as the other
|
# has had a chance to install it. Same approach as the other
|
||||||
# three workflows in this directory.
|
# other workflows in this directory.
|
||||||
- name: Clone repo at this commit
|
- name: Clone repo at this commit
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
|
|||||||
@@ -0,0 +1,176 @@
|
|||||||
|
name: Attach the desktop build to the release
|
||||||
|
|
||||||
|
# The Arch package goes to the pacman registry and the APK to the generic
|
||||||
|
# one, but a release page with nothing on it to download is a release page
|
||||||
|
# nobody can use. This builds the plain Linux x86_64 binary and attaches
|
||||||
|
# it, so "get the latest version" has an answer that needs no package
|
||||||
|
# manager at all.
|
||||||
|
#
|
||||||
|
# **Linux only, and macOS is not an oversight.** `GOOS=darwin
|
||||||
|
# CGO_ENABLED=0` fails at `wails/v3/pkg/mac: build constraints exclude all
|
||||||
|
# Go files` — the darwin backend is Objective-C behind cgo, so a .app
|
||||||
|
# needs a macOS host, and the runner is a Linux container. That is
|
||||||
|
# exactly why the Homebrew formula builds from source on the user's own
|
||||||
|
# Mac, and it stays the macOS channel.
|
||||||
|
#
|
||||||
|
# Windows *does* cross-compile (GOOS=windows CGO_ENABLED=0 succeeds in a
|
||||||
|
# couple of seconds — nothing in the audio, database or webview path needs
|
||||||
|
# cgo there), and is deliberately not published: no Windows build of this
|
||||||
|
# app has ever been run, and no tier here can exercise one. Shipping it
|
||||||
|
# would be a promise nothing in this repo can keep. Revisit when someone
|
||||||
|
# has actually booted it.
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags: ["v*"]
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
version:
|
||||||
|
description: "Version to build and attach (default: the latest v* tag)"
|
||||||
|
required: false
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: desktop-assets-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
linux:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
container:
|
||||||
|
image: ubuntu:24.04
|
||||||
|
volumes:
|
||||||
|
- /home/logan/docker/gitea/data/runner/cache/tool:/cache/tool
|
||||||
|
- /home/logan/docker/gitea/data/runner/cache/pnpm-store:/cache/pnpm-store
|
||||||
|
env:
|
||||||
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
||||||
|
SERVER_URL: ${{ github.server_url }}
|
||||||
|
REPO: ${{ github.repository }}
|
||||||
|
SHA: ${{ github.sha }}
|
||||||
|
REF_NAME: ${{ github.ref_name }}
|
||||||
|
DEBIAN_FRONTEND: noninteractive
|
||||||
|
GO_VERSION: '1.25.0'
|
||||||
|
npm_config_store_dir: /cache/pnpm-store
|
||||||
|
steps:
|
||||||
|
# The same set ci.yml's check job installs: the app is cgo, and
|
||||||
|
# without alsa.pc oto/v3 fails at `pkg-config --cflags -- alsa`
|
||||||
|
# before anything is compiled.
|
||||||
|
- name: System packages
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
apt-get update -qq
|
||||||
|
apt-get install -y -qq --no-install-recommends \
|
||||||
|
ca-certificates curl git jq build-essential pkg-config \
|
||||||
|
libwebkitgtk-6.0-dev libgtk-4-dev libasound2-dev
|
||||||
|
|
||||||
|
- name: Clone repo at this commit
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
git clone --quiet \
|
||||||
|
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" /src
|
||||||
|
git -C /src checkout --quiet --detach "$SHA"
|
||||||
|
git config --global --add safe.directory /src
|
||||||
|
git -C /src log --oneline -1
|
||||||
|
|
||||||
|
- name: Resolve the version
|
||||||
|
id: version
|
||||||
|
working-directory: /src
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
v="${{ inputs.version }}"
|
||||||
|
if [ -z "$v" ]; then
|
||||||
|
case "$REF_NAME" in
|
||||||
|
v*) v="$REF_NAME" ;;
|
||||||
|
*) v=$(git describe --tags --abbrev=0 --match 'v[0-9]*') ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
case "$v" in v*) ;; *) v="v$v" ;; esac
|
||||||
|
|
||||||
|
# v0.0.0 is semantic-release's version floor, not a shipment —
|
||||||
|
# see the bootstrap step in release.yml. Nothing is built for
|
||||||
|
# it, and this is a clean skip rather than a failure because a
|
||||||
|
# red run against a tag that was never meant to ship is noise.
|
||||||
|
if [ "$v" = "v0.0.0" ]; then
|
||||||
|
echo "v0.0.0 is the version floor, not a release; nothing to build"
|
||||||
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "tag=$v" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "version=${v#v}" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "building $v"
|
||||||
|
|
||||||
|
- name: Go toolchain
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
if [ ! -x /cache/tool/go/bin/go ] || ! /cache/tool/go/bin/go version | grep -q "$GO_VERSION"; then
|
||||||
|
mkdir -p /cache/tool && rm -rf /cache/tool/go
|
||||||
|
curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" | tar -C /cache/tool -xz
|
||||||
|
fi
|
||||||
|
echo "/cache/tool/go/bin" >> "$GITHUB_PATH"
|
||||||
|
/cache/tool/go/bin/go version
|
||||||
|
|
||||||
|
- name: Node toolchain
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
||||||
|
apt-get install -y -qq --no-install-recommends nodejs
|
||||||
|
corepack enable
|
||||||
|
node --version
|
||||||
|
|
||||||
|
# `make build-prod` is the production task: -trimpath and -w -s are
|
||||||
|
# already in it, so only the version stamp is passed, through the
|
||||||
|
# LDFLAGS_EXTRA variable this repo added to build/linux/Taskfile.yml.
|
||||||
|
# (`wails3 build` has no -ldflags of its own; that was v2.)
|
||||||
|
- name: Build
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
env:
|
||||||
|
TAG: ${{ steps.version.outputs.tag }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
export PATH="/src/scripts/toolbin:$PATH"
|
||||||
|
commit=$(git rev-parse --short HEAD)
|
||||||
|
|
||||||
|
go generate ./...
|
||||||
|
go tool wails3 task build \
|
||||||
|
LDFLAGS_EXTRA="-X 'main.version=${TAG}' -X 'main.commit=${commit}'"
|
||||||
|
|
||||||
|
# Described, never run: main.go has no flag parsing, so any
|
||||||
|
# invocation here would try to open a window in a container with
|
||||||
|
# no display and hang the job rather than printing a version.
|
||||||
|
test -x bin/yellowjacket
|
||||||
|
ls -la bin/yellowjacket
|
||||||
|
file bin/yellowjacket || true
|
||||||
|
|
||||||
|
# The .desktop file and the icon go in the tarball because without
|
||||||
|
# them the binary is a window with no menu entry — the Arch package
|
||||||
|
# installs both, and this is the same app for people not using it.
|
||||||
|
- name: Package the tarball
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
env:
|
||||||
|
VERSION: ${{ steps.version.outputs.version }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
dir="yellowjacket-${VERSION}-linux-amd64"
|
||||||
|
mkdir -p "/tmp/$dir"
|
||||||
|
cp bin/yellowjacket "/tmp/$dir/"
|
||||||
|
cp packaging/arch/yellowjacket.desktop "/tmp/$dir/"
|
||||||
|
cp frontend/src/assets/images/icons/music/compact-disc.svg \
|
||||||
|
"/tmp/$dir/yellowjacket.svg"
|
||||||
|
tar -C /tmp -czf "/tmp/${dir}.tar.gz" "$dir"
|
||||||
|
ls -la "/tmp/${dir}.tar.gz"
|
||||||
|
|
||||||
|
- name: Attach it to the release
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
env:
|
||||||
|
TAG: ${{ steps.version.outputs.tag }}
|
||||||
|
VERSION: ${{ steps.version.outputs.version }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
./scripts/release-asset.sh "$TAG" \
|
||||||
|
"/tmp/yellowjacket-${VERSION}-linux-amd64.tar.gz"
|
||||||
@@ -14,6 +14,15 @@ on:
|
|||||||
push:
|
push:
|
||||||
tags:
|
tags:
|
||||||
- "v*"
|
- "v*"
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
version:
|
||||||
|
description: "Version to sync (default: the pushed tag)"
|
||||||
|
required: false
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: homebrew-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
sync-formula:
|
sync-formula:
|
||||||
@@ -30,10 +39,25 @@ jobs:
|
|||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: Compute version and tarball checksum
|
- name: Compute version and tarball checksum
|
||||||
|
id: version
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
TAG="${GITHUB_REF_NAME}" # e.g. v1.3.0
|
TAG="${{ inputs.version }}"
|
||||||
VERSION="${TAG#v}" # e.g. 1.3.0
|
[ -n "$TAG" ] || TAG="${GITHUB_REF_NAME}" # e.g. v0.0.1
|
||||||
|
case "$TAG" in v*) ;; *) TAG="v$TAG" ;; esac
|
||||||
|
VERSION="${TAG#v}" # e.g. 0.0.1
|
||||||
|
|
||||||
|
# v0.0.0 is semantic-release's version floor, not a shipment —
|
||||||
|
# see the bootstrap step in release.yml. Skipped cleanly rather
|
||||||
|
# than failing: this one would otherwise push a formula for a
|
||||||
|
# version that does not exist into a *public* tap.
|
||||||
|
if [ "$VERSION" = "0.0.0" ]; then
|
||||||
|
echo "v0.0.0 is the version floor, not a release; nothing to sync"
|
||||||
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
TARBALL="${SOURCE_TARBALL_BASE}/${TAG}.tar.gz"
|
TARBALL="${SOURCE_TARBALL_BASE}/${TAG}.tar.gz"
|
||||||
|
|
||||||
echo "Fetching ${TARBALL}"
|
echo "Fetching ${TARBALL}"
|
||||||
@@ -53,6 +77,7 @@ jobs:
|
|||||||
echo "SHA256=${SHA256}" >> "$GITHUB_ENV"
|
echo "SHA256=${SHA256}" >> "$GITHUB_ENV"
|
||||||
|
|
||||||
- name: Render the formula with the new version and checksum
|
- name: Render the formula with the new version and checksum
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
src="packaging/homebrew/Formula/yellowjacket.rb"
|
src="packaging/homebrew/Formula/yellowjacket.rb"
|
||||||
@@ -66,6 +91,7 @@ jobs:
|
|||||||
cat yellowjacket.rb
|
cat yellowjacket.rb
|
||||||
|
|
||||||
- name: Push to the Homebrew tap repo
|
- name: Push to the Homebrew tap repo
|
||||||
|
if: steps.version.outputs.skip == 'false'
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
git clone "https://x-access-token:${TAP_TOKEN}@github.com/${TAP_REPO}.git" tap
|
git clone "https://x-access-token:${TAP_TOKEN}@github.com/${TAP_REPO}.git" tap
|
||||||
|
|||||||
@@ -7,11 +7,29 @@ name: Search index maintenance
|
|||||||
# import older than 6mo -> rebuild (re-import from the newest dump)
|
# import older than 6mo -> rebuild (re-import from the newest dump)
|
||||||
# otherwise -> refresh (fold in new incremental listens)
|
# otherwise -> refresh (fold in new incremental listens)
|
||||||
#
|
#
|
||||||
# A refresh is cheap and no-ops when nothing new has been published, so
|
# **There is deliberately no `push` trigger, and restoring one is a
|
||||||
# running it on every push to main is safe.
|
# decision rather than a cleanup.** A refresh is individually cheap, so
|
||||||
|
# running it on every push to main looked free; what it actually does is
|
||||||
|
# put an unattended job that mutates the only copy of a ~205 GB catalog
|
||||||
|
# on the same trigger as an ordinary code change, on a runner with
|
||||||
|
# capacity 1.
|
||||||
|
#
|
||||||
|
# That is not hypothetical. On 2026-08-17 `fix(database): retire a table
|
||||||
|
# whose shape the schema moved past` landed on main, green — the CI
|
||||||
|
# database is deliberately in the older encoding, so the stale-shape
|
||||||
|
# repair judged its `explore_index` stale and dropped it, and this job
|
||||||
|
# fell back to a full import from the dumps. `fix(database): never
|
||||||
|
# retire the catalog the index build derives` stops that specific repair
|
||||||
|
# and cannot undo it. Every push to main then booked another `budget`
|
||||||
|
# (3h) of the one runner while ordinary CI queued behind it.
|
||||||
|
#
|
||||||
|
# So the rule this file is an instance of: **a job that mutates state
|
||||||
|
# which cannot be rebuilt in ten minutes is triggered deliberately, not
|
||||||
|
# by a push.** The weekly cron keeps the catalog current, and
|
||||||
|
# workflow_dispatch resumes or forces a build — indexbuild picks up from
|
||||||
|
# its checkpoint either way, so nothing is lost by not running on every
|
||||||
|
# merge. See docs/index-cache.md for the snapshot and the restore.
|
||||||
on:
|
on:
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
schedule:
|
schedule:
|
||||||
# Weekly update pass. The 6-month rebuild is triggered by the same
|
# Weekly update pass. The 6-month rebuild is triggered by the same
|
||||||
# command when it notices the import has aged out.
|
# command when it notices the import has aged out.
|
||||||
@@ -33,6 +51,10 @@ on:
|
|||||||
|
|
||||||
# Runs share one persistent working directory, so they must not overlap.
|
# Runs share one persistent working directory, so they must not overlap.
|
||||||
# A push landing mid-build waits rather than corrupting the checkpoint.
|
# A push landing mid-build waits rather than corrupting the checkpoint.
|
||||||
|
#
|
||||||
|
# That directory holds the only copy of a catalog nothing can cheaply
|
||||||
|
# re-derive: see docs/index-cache.md for the snapshot it takes and the
|
||||||
|
# restore, which is minutes against the hours a rebuild costs.
|
||||||
concurrency:
|
concurrency:
|
||||||
group: search-index
|
group: search-index
|
||||||
cancel-in-progress: false
|
cancel-in-progress: false
|
||||||
|
|||||||
@@ -0,0 +1,182 @@
|
|||||||
|
name: Release
|
||||||
|
|
||||||
|
# The sixth workflow, and the one that decides whether the other three
|
||||||
|
# run at all. On every push to main it reads the Conventional Commits
|
||||||
|
# since the last tag, and if any of them is releasable it writes the
|
||||||
|
# changelog, pushes the tag, and creates the Gitea release whose body is
|
||||||
|
# that changelog section. The publishing workflows are keyed on `v*`, so
|
||||||
|
# the tag push is what starts them.
|
||||||
|
#
|
||||||
|
# **Why the tag is pushed with PACKAGE_TOKEN and not the Actions token.**
|
||||||
|
# Gitea, like GitHub, does not start a workflow from a ref pushed by a
|
||||||
|
# workflow's own token (go-gitea#33123). The token is what decides this,
|
||||||
|
# not the workflow — so semantic-release is handed a repositoryUrl
|
||||||
|
# carrying a *user* PAT, and the resulting push is attributed to a person
|
||||||
|
# and triggers the `v*` workflows normally.
|
||||||
|
#
|
||||||
|
# That limitation is used deliberately in the bootstrap step below, where
|
||||||
|
# a tag that must *not* trigger anything is pushed with the Actions token
|
||||||
|
# instead.
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
# Cutting a tag is not a thing to cancel halfway: a superseded run must
|
||||||
|
# finish, not be killed between `git push --tags` and the release POST.
|
||||||
|
concurrency:
|
||||||
|
group: release-main
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
release:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
container:
|
||||||
|
image: ubuntu:24.04
|
||||||
|
env:
|
||||||
|
SERVER_URL: ${{ github.server_url }}
|
||||||
|
OWNER: ${{ github.repository_owner }}
|
||||||
|
REPO: ${{ github.repository }}
|
||||||
|
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
|
||||||
|
DEBIAN_FRONTEND: noninteractive
|
||||||
|
steps:
|
||||||
|
- name: System packages
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
apt-get update -qq
|
||||||
|
apt-get install -y -qq --no-install-recommends ca-certificates curl git jq
|
||||||
|
|
||||||
|
- name: Node toolchain
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
||||||
|
apt-get install -y -qq --no-install-recommends nodejs
|
||||||
|
node --version
|
||||||
|
|
||||||
|
# By hand rather than actions/checkout, like the other five: that is
|
||||||
|
# a JS action and needs node inside the container before any step has
|
||||||
|
# installed it. The full history is required — semantic-release
|
||||||
|
# reads tags and walks commits, and a shallow clone silently makes
|
||||||
|
# every release look like the first one.
|
||||||
|
- name: Clone repo at this commit
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
git clone --quiet \
|
||||||
|
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" /src
|
||||||
|
# -B main rather than --detach, which the other five workflows
|
||||||
|
# use: semantic-release resolves the release branch and then
|
||||||
|
# pushes a commit and a tag to it, and a detached HEAD is a
|
||||||
|
# worse starting point for both than a local branch named after
|
||||||
|
# the one being released. Pinned to this commit, not to
|
||||||
|
# whatever main points at by the time the container started.
|
||||||
|
git -C /src checkout --quiet -B main "${{ github.sha }}"
|
||||||
|
git config --global --add safe.directory /src
|
||||||
|
git -C /src log --oneline -1
|
||||||
|
|
||||||
|
# Nothing currently pushes a `chore(release):` commit — main is a
|
||||||
|
# protected branch, so .releaserc.yml carries no @semantic-release/git
|
||||||
|
# and the release page is the changelog. This guard is kept for the
|
||||||
|
# day someone adds that plugin back: without it the commit-back is a
|
||||||
|
# push to the branch this workflow runs on, and the loop is a release
|
||||||
|
# per release. Six lines against that is cheap.
|
||||||
|
- name: Skip a changelog commit, if one ever exists
|
||||||
|
id: guard
|
||||||
|
working-directory: /src
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
subject=$(git log -1 --format='%s')
|
||||||
|
case "$subject" in
|
||||||
|
"chore(release):"*)
|
||||||
|
echo "this is the release commit itself; nothing to do"
|
||||||
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# semantic-release calls the first release of a repo with no tags
|
||||||
|
# 1.0.0, and offers no option to say otherwise. A floor tag is the
|
||||||
|
# only way to start at 0.0.1, so this creates one — once, ever.
|
||||||
|
#
|
||||||
|
# **It is pushed with the Actions token on purpose.** v0.0.0 is a
|
||||||
|
# floor, not a shipment: pushing it with a user PAT would start the
|
||||||
|
# Arch, Homebrew and Android workflows for a version that does not
|
||||||
|
# exist. The very limitation the header describes is what makes
|
||||||
|
# this inert.
|
||||||
|
- name: Seed the version floor
|
||||||
|
if: steps.guard.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
env:
|
||||||
|
ACTIONS_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
git fetch --quiet --tags origin
|
||||||
|
|
||||||
|
if [ -n "$(git tag --list 'v[0-9]*')" ]; then
|
||||||
|
echo "floor already set; newest tag is $(git describe --tags --abbrev=0 --match 'v[0-9]*')"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Prefer the Actions token because a ref it pushes starts no
|
||||||
|
# workflow, which is the whole point for a tag that is a floor
|
||||||
|
# rather than a shipment. Falling back to the PAT is safe
|
||||||
|
# rather than merely convenient: all four publishing workflows
|
||||||
|
# skip v0.0.0 explicitly, so the worst case is four jobs that
|
||||||
|
# start and immediately say there is nothing to build.
|
||||||
|
token="${ACTIONS_TOKEN:-$PACKAGE_TOKEN}"
|
||||||
|
[ -n "$ACTIONS_TOKEN" ] || echo "note: GITEA_TOKEN is unset; using the PAT"
|
||||||
|
|
||||||
|
# **On the parent, not on HEAD.** The floor marks what has
|
||||||
|
# already been released, so tagging the commit being pushed
|
||||||
|
# leaves nothing between the floor and HEAD — semantic-release
|
||||||
|
# then correctly reports there is nothing to release, which is
|
||||||
|
# exactly what the first run of this workflow did. HEAD^ is the
|
||||||
|
# first parent, so on the merge commit this fires for it is main
|
||||||
|
# as it was before the merge, and everything the merge brought
|
||||||
|
# in is releasable.
|
||||||
|
floor=$(git rev-parse "${{ github.sha }}^" 2>/dev/null || true)
|
||||||
|
if [ -z "$floor" ]; then
|
||||||
|
echo "HEAD has no parent, so no commit can precede the floor" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "no v* tag exists — seeding v0.0.0 so the first release is 0.0.1"
|
||||||
|
git tag v0.0.0 "$floor"
|
||||||
|
git push --quiet \
|
||||||
|
"https://x-access-token:${token}@${SERVER_URL#https://}/${REPO}.git" \
|
||||||
|
refs/tags/v0.0.0
|
||||||
|
echo "seeded v0.0.0 at $floor (parent of ${{ github.sha }})"
|
||||||
|
|
||||||
|
# Pinned rather than installed into the repo: this is a Go project
|
||||||
|
# and a package.json at its root invites the npm plugin and every
|
||||||
|
# tool that looks for one. conventional-changelog-conventionalcommits
|
||||||
|
# is in the list because both the analyzer and the notes generator
|
||||||
|
# name that preset and neither depends on it.
|
||||||
|
#
|
||||||
|
# **That preset is held at 9 and the reason is worth keeping.** At
|
||||||
|
# 10 it is silently incompatible with the writer that
|
||||||
|
# release-notes-generator@14 pulls in (^8): every release note comes
|
||||||
|
# out as a bare `## 0.0.1 (date)` heading with **no sections and no
|
||||||
|
# commits under it**, and nothing errors. The version would have
|
||||||
|
# been right, the tag would have been right, every job would have
|
||||||
|
# been green, and the release body would have been empty. Check the
|
||||||
|
# notes, not the exit code, before moving any of these.
|
||||||
|
- name: Run semantic-release
|
||||||
|
if: steps.guard.outputs.skip == 'false'
|
||||||
|
working-directory: /src
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
git config user.name "yellowjacket-ci"
|
||||||
|
git config user.email "yj@yellowjacket.app"
|
||||||
|
|
||||||
|
npx --yes \
|
||||||
|
-p semantic-release@25 \
|
||||||
|
-p @semantic-release/commit-analyzer@13 \
|
||||||
|
-p @semantic-release/release-notes-generator@14 \
|
||||||
|
-p @semantic-release/changelog@7 \
|
||||||
|
-p @semantic-release/exec@7 \
|
||||||
|
-p conventional-changelog-conventionalcommits@9 \
|
||||||
|
semantic-release \
|
||||||
|
--repository-url "https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git"
|
||||||
@@ -84,3 +84,7 @@ build/android/build/
|
|||||||
build/android/.gradle/
|
build/android/.gradle/
|
||||||
build/android/gen/
|
build/android/gen/
|
||||||
build/android/overlay.json
|
build/android/overlay.json
|
||||||
|
|
||||||
|
# Written by @semantic-release/changelog purely to carry the release notes
|
||||||
|
# into scripts/gitea-release.sh; the release page is the changelog.
|
||||||
|
.release-notes.md
|
||||||
|
|||||||
-118
@@ -1,118 +0,0 @@
|
|||||||
# Work log
|
|
||||||
|
|
||||||
Temporal memory: what happened and what's next. Structure lives in
|
|
||||||
`CLAUDE.md`, operational instructions in `.pi/skills/yellowjacket-dev/`,
|
|
||||||
measured discoveries in `.planning/NOTES.md`. Don't duplicate those here.
|
|
||||||
|
|
||||||
## Current state
|
|
||||||
|
|
||||||
Plan 005 (agent development harness) is **complete — all seven
|
|
||||||
phases**. Everything from phase 1 onward is still **uncommitted**: one
|
|
||||||
large but coherent working-tree diff, nothing pushed.
|
|
||||||
|
|
||||||
All four tiers verified green from a cold, cleaned state:
|
|
||||||
`make ui-test` 313 passed, `make lint` 0 issues × 3 configurations,
|
|
||||||
`make test` green × 3 passes, `make e2e` 19 passed. Both CI jobs
|
|
||||||
verified green in a bare `ubuntu:24.04` container, including 19/19 on
|
|
||||||
WebKit.
|
|
||||||
|
|
||||||
**Committed and pushed** as `5ca6cad` (the harness) + `ccacd67` (a CI
|
|
||||||
fix), and **green on the real runner**: job `check` ~4 min, job `e2e`
|
|
||||||
~3 min with 19/19 chromium *and* 19/19 webkit. One commit rather than
|
|
||||||
seven because the working tree was the end state, not per-phase
|
|
||||||
snapshots — `Makefile`, `CLAUDE.md` and `lefthook.yml` are touched by
|
|
||||||
nearly every phase, so a split would have been fabricated history.
|
|
||||||
|
|
||||||
Still unverified, because no run has failed yet: the
|
|
||||||
`actions/upload-artifact` step (`continue-on-error`, so it cannot mask
|
|
||||||
a real failure) and whether pnpm honours `npm_config_store_dir` for
|
|
||||||
store caching. Worth checking the next time a spec legitimately fails.
|
|
||||||
|
|
||||||
- [ ] `gitea_ci`'s `job_logs` returns 404 on Gitea 1.27.1 — the endpoint
|
|
||||||
is not exposed. Logs come from the VPS instead: `zstdcat` the file
|
|
||||||
under `gitea/actions_log/<owner>/<repo>/<xx>/<task_id>.log.zst`,
|
|
||||||
and note `zstdcat` is not in the gitea container, so
|
|
||||||
`docker cp` it out first. Job status is `action_run_job.status`
|
|
||||||
(1 success, 2 failure, 4 skipped, 5 waiting, 6 running).
|
|
||||||
Probably belongs in the `gitea` skill, not here.
|
|
||||||
|
|
||||||
Open items deliberately not fixed: WAV tags are write-only
|
|
||||||
(`TestWAVTagsAreNotReadableYet`), `themeStore.loadFromBackend`'s failure
|
|
||||||
handler cannot recover, `backend/playlist` has no CRUD suite.
|
|
||||||
|
|
||||||
## Log
|
|
||||||
|
|
||||||
### 2026-08-11 — cold skill run, then phase 7 (CI)
|
|
||||||
|
|
||||||
- **Followed the skill cold first**, as the last session asked. It
|
|
||||||
works: app up from a wiped `.dev/`, an undocumented flow driven
|
|
||||||
(queue panel + shuffle, asserted on `QueueModeChanged`), stopped —
|
|
||||||
~1 minute, no dead ends. One real config bug: `outputDir` in
|
|
||||||
`.playwright/cli.config.json` resolves against **cwd**, not the
|
|
||||||
config file's directory (only `initScript` does that), so snapshots
|
|
||||||
were landing above the repo and a *stale* one from the previous
|
|
||||||
session answered `ls -t` instead. That cost a DOM walk to disprove a
|
|
||||||
regression that did not exist. Four smaller doc gaps fixed
|
|
||||||
(`sandbox-seed` already runs `testdata`; `ui-setup`/`e2e-setup` were
|
|
||||||
undocumented prerequisites; `snapshot` prints a path; `dev-stop`
|
|
||||||
leaves the browser open), plus `dev-headless.sh`'s own banner, which
|
|
||||||
was suggesting the bare `window.go` call its next paragraph warns
|
|
||||||
against.
|
|
||||||
- **Built both CI jobs as container scripts before writing any YAML**,
|
|
||||||
then transcribed the YAML back out and re-ran it to prove the
|
|
||||||
transcription. Push-and-see is a bad loop on a self-hosted runner.
|
|
||||||
- **It found a real bug immediately**: `make lint` omitted
|
|
||||||
`webkit2_41` on all three passes, so it was linting configurations
|
|
||||||
nothing builds. Invisible on Arch (which still ships
|
|
||||||
`webkit2gtk-4.0.pc`), fatal on Ubuntu 24.04. Tag sets now match
|
|
||||||
`make test`.
|
|
||||||
- **Both open decisions settled by measurement**: ALSA `null` PCM for
|
|
||||||
audio (no daemon; the elapsed clock really advances), dead-address
|
|
||||||
stub for the explore artifact (and setting it for the *app* run, not
|
|
||||||
just seeding, is worth 8x on suite wall clock). **WebKit is a
|
|
||||||
required step** — it had never been run anywhere, so one throwaway
|
|
||||||
container run replaced a coin flip with 19/19 at +11 s.
|
|
||||||
|
|
||||||
### 2026-08-10 — phase 6, pi affordances
|
|
||||||
|
|
||||||
- Added `.pi/skills/yellowjacket-dev/` as a directory rather than a flat
|
|
||||||
file: only the description is always in context, so `SKILL.md` stays
|
|
||||||
short enough that reading it whole is never a decision, and the deeper
|
|
||||||
material sits in `references/{harness,fixtures,ui-tier,schema-change}.md`.
|
|
||||||
- Settled the CLAUDE.md-vs-skill split **grammatically, not topically**,
|
|
||||||
because a topical split is what rots — every new fact gets two
|
|
||||||
plausible homes. Three docs, three tenses: NOTES.md is past
|
|
||||||
(measured, dated, append-only), CLAUDE.md is present (what the system
|
|
||||||
is), the skill is imperative (what to run). A new paragraph's tense
|
|
||||||
decides where it goes.
|
|
||||||
- The five gotchas (binding timeouts, first-run wizard, `pkill -f`,
|
|
||||||
seeds-by-running, WebKit-is-CI-only) went **inline in SKILL.md**, not
|
|
||||||
into a reference: you need them before the failure, not after.
|
|
||||||
- Trimmed CLAUDE.md's "Fixtures and the headless harness" section by
|
|
||||||
about half — the command sequences and gotchas it was carrying are now
|
|
||||||
the skill's, and leaving both would have created exactly the duplicate
|
|
||||||
description this repo has a standing rule against.
|
|
||||||
- Added `make skill-check` / `scripts/skill-check.sh` + a pre-commit
|
|
||||||
hook: every command in `.pi/**/*.md` must be a real `make` target, so
|
|
||||||
the Makefile stays the source of truth for invocation and a renamed
|
|
||||||
target fails a commit instead of misleading an agent later. Verified
|
|
||||||
it fails (it caught its own not-yet-created target) and passes.
|
|
||||||
- Added the `/e2e` prompt template: promoting a hand-driven
|
|
||||||
`playwright-cli` session into a spec is a transcription with four
|
|
||||||
fixed substitutions (refs → testids, sleeps → `waitForEvent`, raw
|
|
||||||
`window.go` → `callBinding`, short fixture → `LONG_TRACK`), plus three
|
|
||||||
runs — pass, pass again, pass after a DB restore — because the usual
|
|
||||||
failure is a spec depending on state the hand-driving left behind.
|
|
||||||
- One shell trap: under `set -euo pipefail`, `x="$(make -pqRr | …)"`
|
|
||||||
fails the whole assignment, because `make -q` exits non-zero when a
|
|
||||||
target is out of date and `pipefail` propagates it.
|
|
||||||
|
|
||||||
### Earlier
|
|
||||||
|
|
||||||
Phases 1–5 of plan 005: fixture generator and manifest, headless launch
|
|
||||||
and seeds, the event bridge + `data-testid` pass + `backend/testctl` +
|
|
||||||
`e2e/`, the Vitest component tier + `make bindings-check`, and the
|
|
||||||
`events.Emit` wrapper with its in-process service-event tests. Recaps
|
|
||||||
and the five "verified end to end" blocks are in
|
|
||||||
`.planning/plans/active/005-agent-development-harness.md`; the lessons
|
|
||||||
are in `.planning/NOTES.md`.
|
|
||||||
@@ -96,6 +96,15 @@ reference, because you need them *before* the failure, not after.
|
|||||||
Run against the `bulk` seed a measurement session left behind and a
|
Run against the `bulk` seed a measurement session left behind and a
|
||||||
third of them fail (13 of 36, when it was measured), in a list that
|
third of them fail (13 of 36, when it was measured), in a list that
|
||||||
reads exactly like a regression in whatever you are holding. `make dev-headless SEED=default` first.
|
reads exactly like a regression in whatever you are holding. `make dev-headless SEED=default` first.
|
||||||
|
- **The catalog is stubbed out locally now, like CI.**
|
||||||
|
`dev-headless.sh` defaults `YJ_CORE_INDEX_URL` to a dead address
|
||||||
|
because it was the only launcher that did not — `seed-sandbox.sh` and
|
||||||
|
`ci.yml` always have. Without it the app downloads the real ~1M-row
|
||||||
|
Explore catalog into the run's `YJ_HOME`, and specs that stage their
|
||||||
|
own catalog rows then search a million real ones and fail *locally
|
||||||
|
only*, which reads as a regression and is an environment. Pass
|
||||||
|
`YJ_CORE_INDEX_URL=<real url>` when you want the real catalog to
|
||||||
|
explore by hand.
|
||||||
- **…and the suite spends state it cannot always give back.**
|
- **…and the suite spends state it cannot always give back.**
|
||||||
`view-lifecycle.spec.ts` **skips an autotag album** on every run, out
|
`view-lifecycle.spec.ts` **skips an autotag album** on every run, out
|
||||||
of the eleven the seed has, and does not put it back — so around the
|
of the eleven the seed has, and does not put it back — so around the
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ make android-setup # SDK pieces + the yj-test AVD, idempotent
|
|||||||
Then:
|
Then:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make android # fat APK (arm64 + x86_64) -> bin/yellowjacket.apk
|
make android # arm64-v8a APK -> bin/yellowjacket.apk (~16 MB)
|
||||||
make android-emulator # boot headless in the background, wait for boot
|
make android-emulator # boot headless in the background, wait for boot
|
||||||
make android-install # adb install -r
|
make android-install # adb install -r
|
||||||
make android-smoke # launch, then assert the same pid survives 10s
|
make android-smoke # launch, then assert the same pid survives 10s
|
||||||
@@ -63,6 +63,17 @@ command line and kills it, silently dropping the rest of your compound
|
|||||||
command. The emulator is addressed by its saved pid in
|
command. The emulator is addressed by its saved pid in
|
||||||
`.dev/emulator.pid`, same discipline as `make dev-stop`.
|
`.dev/emulator.pid`, same discipline as `make dev-stop`.
|
||||||
|
|
||||||
|
**adb is addressed by AVD name, not by whatever is plugged in.** The
|
||||||
|
script resolves `ANDROID_SERIAL` from `ro.boot.qemu.avd_name` before
|
||||||
|
any device command, because a second emulator (another project's, or
|
||||||
|
this one's own corpse left `offline` by a previous run) makes a bare
|
||||||
|
`adb` fail with "more than one device" — which `cmd_install` reported
|
||||||
|
as *"no device — run 'make android-emulator' first"* immediately after
|
||||||
|
that had succeeded. Serials are assigned in boot order and change
|
||||||
|
between runs, so the AVD name is the identity. Set `ANDROID_SERIAL`
|
||||||
|
yourself and it is honoured; one device that is not ours (a phone) is
|
||||||
|
taken as the target.
|
||||||
|
|
||||||
## Things that cost a cycle
|
## Things that cost a cycle
|
||||||
|
|
||||||
- **`ANDROID_HOME` must carry a platform, and Arch's does not.**
|
- **`ANDROID_HOME` must carry a platform, and Arch's does not.**
|
||||||
@@ -135,11 +146,57 @@ FATAL | Avd's CPU Architecture 'arm64' is not supported by the QEMU2
|
|||||||
Google dropped cross-architecture emulation; there is no flag. The
|
Google dropped cross-architecture emulation; there is no flag. The
|
||||||
options are an arm64 host, a physical device, or `adb connect` to one.
|
options are an arm64 host, a physical device, or `adb connect` to one.
|
||||||
|
|
||||||
Two consequences worth holding onto. The x86_64 half of the fat APK is
|
**The x86_64 ABI is therefore gone from the build** (`abiFilters` in
|
||||||
*only* useful for emulators, and cannot work on any Android until
|
`build/android/app/build.gradle`, `android:package` rather than
|
||||||
modernc fixes this — including x86 Chromebooks. And the tombstone is at
|
`package:fat` in the Makefile, and a `native-code: 'arm64-v8a'$`
|
||||||
least honest: unlike the `os.Exit` that came before it, this one leaves
|
assertion in `android-apk.yml` that fails if it comes back). It could
|
||||||
a real crash record with a backtrace.
|
not run on any Android until modernc fixes this — x86 Chromebooks
|
||||||
|
included — and dropping it took the artifact from 27 MB to 15.9 MB.
|
||||||
|
The tombstone was at least honest while it lasted: unlike the
|
||||||
|
`os.Exit` that came before it, it left a real crash record with a
|
||||||
|
backtrace.
|
||||||
|
|
||||||
|
### The emulator still installs it, and it still does not run
|
||||||
|
|
||||||
|
The obvious guess about dropping x86_64 — that `make android-install`
|
||||||
|
would now refuse with `INSTALL_FAILED_NO_MATCHING_ABIS` — is **wrong,
|
||||||
|
and was measured wrong before it was written down.** Google's
|
||||||
|
`google_apis` x86_64 images carry arm64 translation:
|
||||||
|
|
||||||
|
```
|
||||||
|
ro.product.cpu.abilist = x86_64,arm64-v8a
|
||||||
|
```
|
||||||
|
|
||||||
|
So the arm64-only APK installs, the loader maps `lib/arm64/libwails.so`
|
||||||
|
and runs it (the tombstone says `Guest architecture: 'arm64'`). It then
|
||||||
|
dies **before any of our code**, with SIGILL rather than SIGSYS:
|
||||||
|
|
||||||
|
```
|
||||||
|
signal 4 (SIGILL), code -6 (SI_TKILL)
|
||||||
|
#00 pc 00000000015911d0 .../lib/arm64/libwails.so
|
||||||
|
```
|
||||||
|
|
||||||
|
Disassembling that offset names the reason exactly:
|
||||||
|
|
||||||
|
```
|
||||||
|
15911d0: d5380600 mrs x0, ID_AA64ISAR0_EL1
|
||||||
|
```
|
||||||
|
|
||||||
|
That is Go's `internal/cpu` reading the arm64 CPU-feature ID register
|
||||||
|
at runtime init, which the translator does not implement. So it is not
|
||||||
|
"our Go program is unlucky": **no Go binary starts under this
|
||||||
|
translation layer**, and no amount of work on this app changes it.
|
||||||
|
|
||||||
|
The three failures are worth holding side by side, because each looks
|
||||||
|
like the app's fault and none is:
|
||||||
|
|
||||||
|
| build | on x86_64 Android | signal |
|
||||||
|
|---|---|---|
|
||||||
|
| x86_64 | modernc's raw `lstat` vs seccomp | SIGSYS, syscall 6 |
|
||||||
|
| arm64, translated | Go reads `ID_AA64ISAR0_EL1` | SIGILL |
|
||||||
|
| arm64, real device | — | unverified, still |
|
||||||
|
|
||||||
|
**A physical arm64 device remains the only verification path.**
|
||||||
|
|
||||||
### What was fixed to get here
|
### What was fixed to get here
|
||||||
|
|
||||||
@@ -155,12 +212,37 @@ the `indexbuild` tag.
|
|||||||
|
|
||||||
### What is still not done
|
### What is still not done
|
||||||
|
|
||||||
MPRIS is compiled in (`android` implies the `linux` build tag), the
|
The shell is still a desktop shell, and the x86_64 half of the APK is
|
||||||
shell is still a desktop shell, and — the largest one — open-*directory*
|
still dead weight. Everything in plan 016's section A is now built:
|
||||||
dialogs return an error on Android, because the Storage Access Framework
|
storage access, an in-app folder picker (Android's directory dialog
|
||||||
yields tree URIs rather than filesystem paths. This app's first run is
|
returns an error, since the Storage Access Framework yields tree URIs
|
||||||
"choose your music folder" and its library model is filesystem paths, so
|
rather than paths), MPRIS excluded, and a MediaSession with a transport
|
||||||
that is a design question rather than a port.
|
notification and audio focus.
|
||||||
|
|
||||||
|
### Compiling the `android`-tagged Go by hand
|
||||||
|
|
||||||
|
`make lint` and `make test` never see it: their three tag sets are all
|
||||||
|
linux/amd64, so the only thing that compiles `backend/mediacontrols/
|
||||||
|
android.go` is `make android` — a full APK build for a Go type error.
|
||||||
|
The short way round:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
B=$(echo /opt/android-ndk/toolchains/llvm/prebuilt/*/bin)
|
||||||
|
CC=$B/aarch64-linux-android21-clang CXX=$B/aarch64-linux-android21-clang++ \
|
||||||
|
GOOS=android GOARCH=arm64 CGO_ENABLED=1 go build ./backend/...
|
||||||
|
```
|
||||||
|
|
||||||
|
**`CXX` is not optional.** Without it the oboe C++ sources in `oto`
|
||||||
|
compile against the host sysroot and fail on `android/log.h` and
|
||||||
|
`sys/system_properties.h`, which reads like a broken or missing NDK.
|
||||||
|
Restrict it to `./backend/...`: `./...` additionally builds
|
||||||
|
`build/android/gen`, a scaffold shim that only resolves inside the
|
||||||
|
wails task and fails with `undefined: main` on its own.
|
||||||
|
|
||||||
|
A Go method added to a bound service also reaches the frontend unless
|
||||||
|
it says not to — `//wails:ignore` above the func, which `make bindings`
|
||||||
|
then honours. `Player.SetDuck` is driven by OS audio focus and carries
|
||||||
|
one.
|
||||||
|
|
||||||
## The scaffold's own tasks
|
## The scaffold's own tasks
|
||||||
|
|
||||||
@@ -204,3 +286,65 @@ Related, and it will bite once: the launcher activity is
|
|||||||
resolves the leading dot against the *applicationId* and fails with a
|
resolves the leading dot against the *applicationId* and fails with a
|
||||||
class-not-found that reads like a broken build. Always the
|
class-not-found that reads like a broken build. Always the
|
||||||
fully-qualified form.
|
fully-qualified form.
|
||||||
|
|
||||||
|
## What only a device can answer
|
||||||
|
|
||||||
|
The emulator cannot run this app (three separate reasons, none of them
|
||||||
|
ours — see plan 016), so the phone in someone's pocket is a tier, and
|
||||||
|
asking for it is cheap. The first run of it, on 2026-08-17, confirmed
|
||||||
|
the whole of A4 and found two faults **no other tier can see**:
|
||||||
|
|
||||||
|
- **The back gesture.** `MainActivity.onBackPressed` asks
|
||||||
|
`webView.canGoBack()`. Nothing in a desktop shell has a back gesture,
|
||||||
|
so no spec had ever called `page.goBack()` and the app had never
|
||||||
|
pushed a history entry — back quit from any depth. It is a history
|
||||||
|
entry per navigation now, which is also what made it assertable in the
|
||||||
|
browser tier (`e2e/specs/back-navigation.spec.ts`).
|
||||||
|
- **The safe area.** `targetSdk 35` forces edge-to-edge, so the
|
||||||
|
transport and the tab bar sat under the gesture bar. **A browser
|
||||||
|
viewport has no system bars**: `phone-shell.spec.ts` at 390x844 will
|
||||||
|
keep passing on a build the device is clipping 48dp off. Insets are
|
||||||
|
handled in `applyWindowInsets()`.
|
||||||
|
|
||||||
|
So when asking for a device run, ask about what the platform *adds* —
|
||||||
|
system bars, the back gesture, focus and audio interruptions,
|
||||||
|
permission dialogs, the keyboard — not about what the app draws. The
|
||||||
|
drawing is what the other five tiers already cover.
|
||||||
|
|
||||||
|
## Asking the device, not just looking at it
|
||||||
|
|
||||||
|
A real phone can be inspected, and that turns this tier from "reported
|
||||||
|
symptoms" into evidence. Three commands:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make android-screenshot # what the screen shows (.dev/ by default)
|
||||||
|
make android-inspect # forward the WebView's devtools socket
|
||||||
|
make android-eval EXPR='JSON.stringify({vp:[innerWidth,innerHeight]})'
|
||||||
|
```
|
||||||
|
|
||||||
|
Four things about it, each of which costs an hour if met cold:
|
||||||
|
|
||||||
|
- **Only a `debuggable` build has a devtools socket**, and a debug build
|
||||||
|
carries `applicationIdSuffix ".dev"` so it installs **beside** the
|
||||||
|
release app. That matters more than convenience: the two are signed by
|
||||||
|
different certificates, and Android's only remedy for a changed
|
||||||
|
certificate is an uninstall, which takes the user's library with it.
|
||||||
|
Never uninstall to make room for a build.
|
||||||
|
- **Playwright cannot drive it.** `connectOverCDP` calls
|
||||||
|
`Browser.setDownloadBehavior`, a WebView answers "Browser context
|
||||||
|
management is not supported", and the connection dies before the first
|
||||||
|
evaluate. `scripts/android-eval.mjs` is raw CDP over Node's built-in
|
||||||
|
WebSocket for that reason.
|
||||||
|
- **Wireless adb drops when the screen sleeps.** The symptoms are
|
||||||
|
`device offline` mid-session and a `fetch failed` from the eval
|
||||||
|
script. Plug in over USB for anything longer than a couple of probes.
|
||||||
|
- **The socket name carries the pid**, which changes on every launch, so
|
||||||
|
it is resolved rather than remembered.
|
||||||
|
|
||||||
|
**And the reason to bother: the phone is an engine, not a screen.** The
|
||||||
|
first device here renders in **Chrome 113** at 424x439 CSS px. Every
|
||||||
|
other tier runs a current Chromium or WebKit, so a spec that passes at
|
||||||
|
that viewport says nothing about the phone — 113 has no Popover API and
|
||||||
|
no relaxed CSS nesting, and a dropped CSS declaration renders as
|
||||||
|
"present but wrong", which is the hardest failure to read from a
|
||||||
|
picture. Get the version first; it reframes every other symptom.
|
||||||
|
|||||||
@@ -2659,3 +2659,827 @@ it. So the arm64 claim above rests on reading modernc's two code paths,
|
|||||||
not on having run it: verifying the shipped ABI needs an arm64 host, a
|
not on having run it: verifying the shipped ABI needs an arm64 host, a
|
||||||
physical device, or `adb connect` to one. The image was deleted again;
|
physical device, or `adb connect` to one. The image was deleted again;
|
||||||
do not re-download it.
|
do not re-download it.
|
||||||
|
|
||||||
|
## Android media controls need no new JNI and no new dependency (2026-08-16)
|
||||||
|
|
||||||
|
Plan 016's A4 — playback that survives the screen locking — turned out
|
||||||
|
to be reachable entirely through seams that already exist, which is the
|
||||||
|
finding worth keeping. The obvious blocker is that Wails' `androidBridge*`
|
||||||
|
helpers are unexported, so Go cannot call arbitrary Java. It does not
|
||||||
|
need to:
|
||||||
|
|
||||||
|
- **Go → Java** is `application.Android.StartForegroundService(json)`,
|
||||||
|
which *is* exported, and `build/android/` is our tree — so widening
|
||||||
|
the JSON that `WailsBridge.startForegroundService` accepts is a local
|
||||||
|
edit, not a fork of the runtime.
|
||||||
|
- **Java → Go** is `WailsBridge.emitEvent(name, json)` →
|
||||||
|
`nativeEmitEvent` → `app.Event.Emit`, which a Go `app.Event.On`
|
||||||
|
subscriber receives with `Data` as a `map[string]any`.
|
||||||
|
|
||||||
|
So the handler is one JSON document out and one command event back, and
|
||||||
|
`backend/mediacontrols`' existing `Handler`/`Callbacks` interface — written
|
||||||
|
for MPRIS — needed one addition (`OnDuck`) to cover a MediaSession.
|
||||||
|
|
||||||
|
**The Java side needs no androidx.media either.** `MediaSessionCompat`
|
||||||
|
is the documented route, but `android.media.session.MediaSession` and
|
||||||
|
`Notification.MediaStyle` are both API 21 and minSdk here is 21, so the
|
||||||
|
platform API covers it with two `Build.VERSION` branches (the channel,
|
||||||
|
and PendingIntent mutability flags) and no new Gradle dependency.
|
||||||
|
|
||||||
|
Four things measured or reasoned along the way, each of which would
|
||||||
|
have been a bug:
|
||||||
|
|
||||||
|
- **From API 26 the framework ducks the app itself** and sends no
|
||||||
|
`AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK`. So a duck implemented in the
|
||||||
|
player is a *pre-Oreo* path, and `setWillPauseWhenDucked(true)` —
|
||||||
|
which is how you get the callback back — would mean pausing for
|
||||||
|
every notification tone. Implementing both attenuates twice.
|
||||||
|
- **A duck must not touch the user's volume.** `Player.SetDuck` holds
|
||||||
|
the attenuation as a separate offset and re-applies the user's level
|
||||||
|
through `setVolumeLocked`, so it cannot accumulate across repeated
|
||||||
|
ducks and `getUserVolume` — which feeds the event, the persisted
|
||||||
|
state and every relative change — still reports what the user chose.
|
||||||
|
- **From Android 12 a background app may not *start* a foreground
|
||||||
|
service**, but it may keep delivering intents to one already running.
|
||||||
|
Every update after the first is exactly that case (a track change
|
||||||
|
with the screen off), so `WailsBridge` picks `startService` over
|
||||||
|
`startForegroundService` once `WailsForegroundService.running` is set.
|
||||||
|
- **A service started with `startForegroundService` that returns from
|
||||||
|
`onStartCommand` without calling `startForeground` is killed**, so
|
||||||
|
the transport-button intents call it too rather than only the payload
|
||||||
|
path.
|
||||||
|
|
||||||
|
**`make lint` does not see any of this.** Its three passes are the app,
|
||||||
|
`indexbuild` and `dev` tag sets, all on linux/amd64, and `android.go` is
|
||||||
|
behind the `android` build tag — the only thing that compiles it is the
|
||||||
|
cross-compiler in `make android`. That is why the payload keys, the
|
||||||
|
state words and the command names live in `androidpayload.go` *without*
|
||||||
|
a build tag, with a test: it is the half that can be checked on the
|
||||||
|
machine doing the work. A quick manual check of the tagged half is
|
||||||
|
|
||||||
|
```bash
|
||||||
|
B=$(echo /opt/android-ndk/toolchains/llvm/prebuilt/*/bin)
|
||||||
|
CC=$B/aarch64-linux-android21-clang CXX=$B/aarch64-linux-android21-clang++ \
|
||||||
|
GOOS=android GOARCH=arm64 CGO_ENABLED=1 go build ./backend/...
|
||||||
|
```
|
||||||
|
|
||||||
|
— `CXX` matters: without it the oboe C++ sources compile against the
|
||||||
|
host sysroot and fail on `android/log.h`, which reads like a missing NDK.
|
||||||
|
|
||||||
|
**None of it has run.** The APK builds for both ABIs and the Go and Java
|
||||||
|
halves compile; everything above about behaviour is read from the
|
||||||
|
Android documentation and the source. The x86_64 emulator still cannot
|
||||||
|
run this app (modernc `lstat`/seccomp, above) and an arm64 AVD still
|
||||||
|
cannot exist on an x86_64 host, so A4's first real test is a device.
|
||||||
|
|
||||||
|
## Dropping x86_64 cut the APK by 41% (measured 2026-08-16)
|
||||||
|
|
||||||
|
Plan 016's B1, decided: the ABI is gone.
|
||||||
|
|
||||||
|
| | fat (arm64 + x86_64) | arm64 only |
|
||||||
|
|---|---|---|
|
||||||
|
| `bin/yellowjacket.apk` | 27,059,130 B | 15,898,465 B |
|
||||||
|
| `lib/` entries | 2 | 1 |
|
||||||
|
|
||||||
|
It buys nothing to keep. x86_64 Android takes SIGSYS the first time it
|
||||||
|
touches the database (modernc's raw `lstat` against Android's seccomp
|
||||||
|
filter, above), which is *every* x86_64 device — emulators and x86
|
||||||
|
Chromebooks alike — not merely the emulator here.
|
||||||
|
|
||||||
|
Three places had to agree, and the third is the one that would have
|
||||||
|
made this a silent no-op: `abiFilters` in `build/android/app/
|
||||||
|
build.gradle` (what Gradle packages), `android:package` rather than
|
||||||
|
`android:package:fat` in the Makefile (what Go compiles — otherwise the
|
||||||
|
31 MB library is still built and then discarded), and the `native-code`
|
||||||
|
assertion in `android-apk.yml`'s Verify step, which is now
|
||||||
|
`native-code: 'arm64-v8a'$` and fails if a second ABI ever comes back.
|
||||||
|
The anchor is deliberate and was checked against a real artifact:
|
||||||
|
without it the pattern also matches the fat APK's line.
|
||||||
|
|
||||||
|
One consequence for the dev tier was written down before it was
|
||||||
|
checked, and checking it proved it false — see the next entry.
|
||||||
|
|
||||||
|
## arm64 translation runs Go until Go asks the CPU what it is (measured 2026-08-16)
|
||||||
|
|
||||||
|
Predicted, when the x86_64 ABI was dropped: `make android-install`
|
||||||
|
against the emulator would now fail with
|
||||||
|
`INSTALL_FAILED_NO_MATCHING_ABIS`. **Measured: it installs and
|
||||||
|
launches.** Google's `google_apis` x86_64 images carry arm64
|
||||||
|
translation —
|
||||||
|
|
||||||
|
```
|
||||||
|
ro.product.cpu.abilist = x86_64,arm64-v8a
|
||||||
|
```
|
||||||
|
|
||||||
|
— so the loader maps `lib/arm64/libwails.so` and executes it; the
|
||||||
|
tombstone confirms it with `ABI: 'x86_64'` / `Guest architecture:
|
||||||
|
'arm64'`.
|
||||||
|
|
||||||
|
It dies anyway, before a line of our code, and the instruction says
|
||||||
|
exactly why. The fault is at `libwails.so+0x15911d0`:
|
||||||
|
|
||||||
|
```
|
||||||
|
signal 4 (SIGILL), code -6 (SI_TKILL)
|
||||||
|
15911d0: d5380600 mrs x0, ID_AA64ISAR0_EL1
|
||||||
|
```
|
||||||
|
|
||||||
|
That is Go's `internal/cpu` reading the arm64 feature-ID system
|
||||||
|
register during runtime init. The translator does not implement it, so
|
||||||
|
**no Go binary starts under it** — this is not a property of this app
|
||||||
|
and no work here would change it. (`code -6 (SI_TKILL)` also means the
|
||||||
|
signal was re-raised by the process itself: Go's handler caught the
|
||||||
|
SIGILL, printed a traceback to a stdout that goes to `/dev/null`, and
|
||||||
|
re-raised. The invisible-failure rule again.)
|
||||||
|
|
||||||
|
So there are now three distinct ways this app fails on an x86_64
|
||||||
|
Android, none of them a bug in it:
|
||||||
|
|
||||||
|
| build | cause | signal |
|
||||||
|
|---|---|---|
|
||||||
|
| x86_64 | modernc's raw `lstat` vs seccomp | SIGSYS, syscall 6 |
|
||||||
|
| arm64, translated | Go reads `ID_AA64ISAR0_EL1` | SIGILL |
|
||||||
|
| arm64, real device | — | still unverified |
|
||||||
|
|
||||||
|
**A physical arm64 device is still the only verification path**, which
|
||||||
|
is the conclusion the previous session reached by a different route.
|
||||||
|
The value of this entry is that it closes the remaining plausible
|
||||||
|
shortcut, with the instruction that closes it.
|
||||||
|
|
||||||
|
### Two bugs the attempt found in the harness itself
|
||||||
|
|
||||||
|
Both were on `main`, and the first had made the whole tier unusable
|
||||||
|
since the commit that added it.
|
||||||
|
|
||||||
|
**`scripts/android-emulator.sh` did not parse.** A `case` pattern read
|
||||||
|
`*signatures do not match*)`, and `do` is a reserved word: bash fails
|
||||||
|
the parse of the *entire file*, so `make android-emulator`,
|
||||||
|
`android-install`, `android-smoke` and `android-logs` all died with
|
||||||
|
`line 190: syntax error near unexpected token 'do'`. Quoting the inner
|
||||||
|
words fixes it. A shell script that is only run interactively can carry
|
||||||
|
a syntax error indefinitely — `bash -n` in the pre-commit hook would
|
||||||
|
have caught it, and does not exist.
|
||||||
|
|
||||||
|
**A bare `adb` addresses whatever is attached.** With a second emulator
|
||||||
|
present (another project's, or a stale `offline` entry from a previous
|
||||||
|
run), every adb call fails with "more than one device", and
|
||||||
|
`cmd_install` reported that as *"no device — run 'make
|
||||||
|
android-emulator' first"* — directly after that had printed "waiting
|
||||||
|
for boot ok". `pick_device` now resolves `ANDROID_SERIAL` from
|
||||||
|
`ro.boot.qemu.avd_name`, since serials are assigned in boot order and
|
||||||
|
the AVD name is the stable identity. Verified with both emulators
|
||||||
|
running: it selects `yj-test` and installs.
|
||||||
|
|
||||||
|
## The phone shell fits, and what it cost to make it fit (2026-08-16)
|
||||||
|
|
||||||
|
Plan 016 B2, phase 1: the shell below 600px. Measured at 360×780 and
|
||||||
|
390×844 against the real app (`make dev-headless` + Playwright, which
|
||||||
|
is the tier that can answer this — server mode serves the same document
|
||||||
|
an Android WebView renders).
|
||||||
|
|
||||||
|
**What overflowed, and by how much.** The body was 652px wide in a
|
||||||
|
360px viewport before any of this. Walking every element and its shadow
|
||||||
|
roots for a `right` past the viewport named the causes in order:
|
||||||
|
|
||||||
|
| element | width | why |
|
||||||
|
|---|---|---|
|
||||||
|
| `header.top-bar` | 580 | its children's minimums, summed |
|
||||||
|
| `search-bar` | 320 | `.search-container { min-width: 200px }` |
|
||||||
|
| `job-indicator` | 157 | the label, "3 background jobs" |
|
||||||
|
|
||||||
|
A `min-width` in a flex row is a *hard* floor — it does not shrink — and
|
||||||
|
a grid item's implicit minimum is `auto`, i.e. its content. So the
|
||||||
|
header could not get smaller than the sum of what it held, the body grew
|
||||||
|
to the header, and `overflow-x: hidden` would then have hidden a third
|
||||||
|
of the app rather than fitting it. `min-width: 0` on the boxes between
|
||||||
|
the viewport and the content, plus each component standing its own
|
||||||
|
non-essential parts down in its own stylesheet, takes 360 → 360 exactly.
|
||||||
|
At 320px (400% zoom, the width WCAG 1.4.10 names) it is also exact.
|
||||||
|
|
||||||
|
**So an existing spec now asserts the opposite of what it did**, and
|
||||||
|
that is the fix landing rather than the test being weakened.
|
||||||
|
`layout-overflow.spec.ts` used to assert that the 464px of app behind
|
||||||
|
`overflow: hidden` *could be scrolled to* with a wheel gesture, which
|
||||||
|
was the remedy available when the shell had one layout. It reflows now,
|
||||||
|
which is what 1.4.10 asks for; scrolling to the overflow was the
|
||||||
|
concession.
|
||||||
|
|
||||||
|
**And a shared component brings its test handles with it.**
|
||||||
|
`bottom-nav`'s "More" opens the *existing* `<app-sidebar>` in a drawer —
|
||||||
|
the whole point being not to write a second list of destinations — but
|
||||||
|
rendering it unconditionally put a second `data-testid="nav-home"` (and
|
||||||
|
ten siblings) in the DOM. **30 existing specs failed** with "strict mode
|
||||||
|
violation: resolved to 2 elements", on a *desktop* viewport where
|
||||||
|
`bottom-nav` is `display: none` and the drawer can never open. Lazy
|
||||||
|
rendering fixes it; the component test asserts the absence, because the
|
||||||
|
failure is invisible from inside the component and appears in files
|
||||||
|
nobody touched.
|
||||||
|
|
||||||
|
Three smaller things worth keeping:
|
||||||
|
|
||||||
|
- **A new icon name is a runtime failure, not a build one.** `bars` was
|
||||||
|
not in `src/icons/names.txt`, so `offline-icons.spec.ts` caught it —
|
||||||
|
the sweep asserts `window.__yjIconMisses` is empty. `node
|
||||||
|
frontend/scripts/fetch-icons.mjs` re-vendors after adding a line.
|
||||||
|
- **A `wa-drawer` animates, so a test asserts its events**, not its
|
||||||
|
`open` property: setting `open = false` starts a hide that has not
|
||||||
|
finished on the next microtask, and a test reading the property in
|
||||||
|
between sees the state it is leaving.
|
||||||
|
- **`update(el)` in the component tier takes two arguments**
|
||||||
|
(`update(el, {})`), which is only visible from `tsc`, not from a
|
||||||
|
failing test.
|
||||||
|
|
||||||
|
### The local e2e tier was not running the same app CI runs
|
||||||
|
|
||||||
|
`requested-badge.spec.ts` failed two of three tests locally while CI was
|
||||||
|
green, and the reason is worth more than the fix: **`dev-headless.sh`
|
||||||
|
was the only place that did not neutralise `YJ_CORE_INDEX_URL`.**
|
||||||
|
`seed-sandbox.sh` and `ci.yml` both point it at `127.0.0.1:1`; the dev
|
||||||
|
launcher did not, so the app downloaded and built the real ~1M-row
|
||||||
|
Explore catalog into the run's `YJ_HOME`, and a local `make e2e` then
|
||||||
|
ran against a world CI never sees.
|
||||||
|
|
||||||
|
Found by reading the failure screenshot: the spec had searched Explore
|
||||||
|
for its fixture album and the page was full of *real* ones — Real
|
||||||
|
Estate, Arrested Youth, The Yes Album. The staged row was there and
|
||||||
|
invisible among a million others.
|
||||||
|
|
||||||
|
`dev-headless.sh` now defaults the variable to the dead address and
|
||||||
|
takes an explicit one if you want the real catalog for exploring by
|
||||||
|
hand. `make e2e` locally: 97 passed / 3 failed before, 100 passed
|
||||||
|
after.
|
||||||
|
|
||||||
|
The second half of the same problem is that **the backend is one shared
|
||||||
|
process with one database, and specs leave rows in it.**
|
||||||
|
`explore-shelves` staged its catalog only `IfEmpty`, so a single album
|
||||||
|
row left behind by `requested-badge` satisfied that gate, the shelves
|
||||||
|
were drawn from one foreign row, and the artist card the spec clicks did
|
||||||
|
not exist. It fails on the *second* local run and passes on the first,
|
||||||
|
which is the least useful order, and never in CI, where every run gets a
|
||||||
|
fresh `YJ_HOME`.
|
||||||
|
|
||||||
|
"Is the catalog empty" was the wrong question; "are my rows there" is
|
||||||
|
the right one. The staging is unconditional now (`INSERT OR IGNORE`
|
||||||
|
keyed on the MBID) and the assertion moved from *this insert wrote a
|
||||||
|
row* to *every fixture row is present* — which is both idempotent and a
|
||||||
|
stronger check, since an MBID failing `CHECK(length(mbid) = 16)` is
|
||||||
|
silently dropped by OR IGNORE and would otherwise show up as an empty
|
||||||
|
page rather than a failed setup.
|
||||||
|
|
||||||
|
**Verified: the full suite runs twice against the same app, 100 passed
|
||||||
|
both times.** That is the property to keep — a spec tier whose second
|
||||||
|
run differs from its first is a tier that will one day blame the wrong
|
||||||
|
commit.
|
||||||
|
|
||||||
|
## A media query adds no specificity, and dead CSS looks like working CSS (2026-08-16)
|
||||||
|
|
||||||
|
Plan 016 B2 phase 2 shipped the full-screen now-playing view, and
|
||||||
|
checking it with a screenshot found that **phase 1's shell rules had
|
||||||
|
never applied**.
|
||||||
|
|
||||||
|
`index.css` is base rules then component rules, and the phone block had
|
||||||
|
been inserted in the middle — above the plain `.top-bar` and `.title`
|
||||||
|
rules it meant to override. A media query is not a specificity boost,
|
||||||
|
so with equal specificity the *later* declaration wins. Measured at
|
||||||
|
390px before the fix:
|
||||||
|
|
||||||
|
| declared for the phone | actually computed |
|
||||||
|
|---|---|
|
||||||
|
| `padding-left: 0.75em` | 32px (the 2em base) |
|
||||||
|
| `gap: 0.5em` | 16px (base) |
|
||||||
|
| `font-size: 1.1em` | 24px (the 1.5em base) |
|
||||||
|
| `grid-template-columns: minmax(0,1fr) auto auto` | `320px 1fr auto` (base) |
|
||||||
|
|
||||||
|
After moving the block to the end of the file: 12px, 8px, 17.6px, and
|
||||||
|
`154px 187px 33px`.
|
||||||
|
|
||||||
|
**Nothing failed while they were dead**, which is the part worth
|
||||||
|
keeping. The phone spec asserts that the shell does not scroll
|
||||||
|
sideways, and it did not — because the fitting was being done by
|
||||||
|
`min-width: 0` and by each component's *own* media query, which live in
|
||||||
|
their own stylesheets and so had no later rule to lose to. The
|
||||||
|
declarations that did nothing were the cosmetic ones, and no assertion
|
||||||
|
was ever going to see them. A screenshot did, in about ten seconds.
|
||||||
|
|
||||||
|
The file now ends with one phone section, and says why it is last.
|
||||||
|
|
||||||
|
### What the same screenshot found about the view itself
|
||||||
|
|
||||||
|
The bottom bar was still rendering the mini player *underneath* the
|
||||||
|
full-screen view — 4em of a 844px phone spent saying exactly what the
|
||||||
|
view above it says, and invisible to every assertion about either one
|
||||||
|
(both were correct on their own). `index.css` hides `.bottom-bar` while
|
||||||
|
`#main-content[data-active-view="now-playing"]`, through `:has()`
|
||||||
|
rather than a class toggled from `index.ts`: which view is showing is
|
||||||
|
already published as an attribute, and a second expression of the same
|
||||||
|
fact is a second thing to keep in step.
|
||||||
|
|
||||||
|
That took the queue button away with it, since that button lives in the
|
||||||
|
bar — so the view carries its own, toggling the same `open` attribute
|
||||||
|
on the same panel element.
|
||||||
|
|
||||||
|
**And a css`` literal cannot contain a backtick.** A comment reading
|
||||||
|
"the track size is set on the `wa-slider` inside its shadow root"
|
||||||
|
terminates the tagged template, and the failure arrives as
|
||||||
|
`Expected "]" but found "wa"` from the CSS parser, at a line number in
|
||||||
|
the *comment*. `make css-check` exists for this and named it
|
||||||
|
immediately.
|
||||||
|
|
||||||
|
## The index artifact could not be exported, and the reason is a rule this repo already had (2026-08-16)
|
||||||
|
|
||||||
|
`maintain-index` failed on an unrelated push:
|
||||||
|
|
||||||
|
```
|
||||||
|
indexexport: copy rows: SQL logic error: no such column: total_tracks (1)
|
||||||
|
```
|
||||||
|
|
||||||
|
Three minutes in, on the one job that owns the ~205 GB checkpoint and
|
||||||
|
publishes the catalog every user downloads.
|
||||||
|
|
||||||
|
**The cause is the exception that keeps that checkpoint alive.** The
|
||||||
|
index job's `/cache` is a real `YJ_HOME` that survives between runs, so
|
||||||
|
`explore_index` there is classified `Cache` and is deliberately *not*
|
||||||
|
dropped and recreated by `cmd/indexbuild`'s schema repair
|
||||||
|
(`staleschema.go`). A column added to the schema afterwards is
|
||||||
|
therefore simply absent from that database — and `total_tracks` was
|
||||||
|
added by the album-completeness work. The exporter selected it anyway.
|
||||||
|
|
||||||
|
**The fix is the rule the importer already follows.**
|
||||||
|
`artifactHasTotals()` exists precisely because "adding a column to the
|
||||||
|
importer's SELECT is how you break every artifact already published";
|
||||||
|
the mirror image — *reading* an index older than the binary — had no
|
||||||
|
such guard. `sourceColumns()` asks
|
||||||
|
`pragma_table_info('explore_index', 'main')` and selects a literal `0`
|
||||||
|
when the column is not there, which is what the column already means by
|
||||||
|
"the catalog does not say" and what the app already renders as unknown
|
||||||
|
rather than as incomplete. The destination keeps every column, so an
|
||||||
|
importer needs no second shape.
|
||||||
|
|
||||||
|
So the pattern generalises, and is worth stating once: **any query that
|
||||||
|
crosses a version boundary in either direction asks the schema rather
|
||||||
|
than trusting it.** There are now three of these — `artifactStoresText`
|
||||||
|
(encoding), `artifactHasTotals` (import), `sourceColumns` (export).
|
||||||
|
|
||||||
|
Two things about the test are worth keeping.
|
||||||
|
|
||||||
|
It reproduces the failure **symptom first**: with the fix removed it
|
||||||
|
fails with the CI message verbatim, `copy rows: SQL logic error: no
|
||||||
|
such column: total_tracks (1)`. That was checked, not assumed.
|
||||||
|
|
||||||
|
And its first version silently proved nothing. `oldColumns` was
|
||||||
|
`strings.Replace(catalogColumns, "total_tracks, ", "", 1)` — which
|
||||||
|
matches *nothing*, because the list is formatted across lines and the
|
||||||
|
name is followed by a newline rather than a space. So the "old" index
|
||||||
|
had every current column, the probe correctly said so, and the only
|
||||||
|
reason this was caught is that the assertion about the probe ran before
|
||||||
|
the assertion about the export. A fixture built by string surgery on a
|
||||||
|
formatted constant needs to be whitespace-independent; it filters the
|
||||||
|
list now.
|
||||||
|
|
||||||
|
## Long-press is one document listener, and the header row is a row (2026-08-17)
|
||||||
|
|
||||||
|
Plan 016 B2 phase 3. A phone has no right-click, and every context menu
|
||||||
|
in this app opens from a `contextmenu` event — six components' worth,
|
||||||
|
bound three different ways (delegated on a virtualizer, per row, per
|
||||||
|
card). `frontend/src/utils/long-press.ts` is one document-capture
|
||||||
|
listener installed once from `index.ts`: a touch that holds still for
|
||||||
|
500 ms dispatches a synthetic `contextmenu` at the touch point, and
|
||||||
|
**every existing handler runs unchanged**. No component opted in, and
|
||||||
|
none can forget to.
|
||||||
|
|
||||||
|
Four things it has to get right, and each is a way the obvious version
|
||||||
|
fails:
|
||||||
|
|
||||||
|
- **The target is `composedPath()[0]`, not `elementFromPoint`**, which
|
||||||
|
stops at the outermost shadow host. Every menu here is bound inside
|
||||||
|
one, so a host-targeted event reaches a delegated listener and no
|
||||||
|
per-row one.
|
||||||
|
- **A browser that fires its own must win.** Chromium already dispatches
|
||||||
|
`contextmenu` on long-press; WebKit and the WebView vary. One arriving
|
||||||
|
during the press cancels ours; one arriving after ours is swallowed at
|
||||||
|
document capture.
|
||||||
|
- **Ours is told from theirs by identity** (a `WeakSet`), not by
|
||||||
|
`isTrusted`. `isTrusted` would work in the app and is untestable — no
|
||||||
|
test can dispatch a trusted event — so the suppression path would have
|
||||||
|
been the one thing with no coverage.
|
||||||
|
- **The click ending the gesture is swallowed**, keyed on the gesture
|
||||||
|
(cleared by the next `pointerdown`) rather than a time window, or a
|
||||||
|
quick tap on the menu that just opened is eaten too.
|
||||||
|
|
||||||
|
**What cost the time was the assertion, not the code.** The e2e spec
|
||||||
|
pressed `[role="row"]` — which is the *column header*, and it is the
|
||||||
|
first one. The gesture fired correctly, the header correctly ignored it,
|
||||||
|
and the failure looked exactly like a menu that would not open. Found by
|
||||||
|
probing the running app (`playwright-cli eval`, dispatching the same
|
||||||
|
pointer events and logging what saw the `contextmenu`), which showed the
|
||||||
|
event reaching the row's own listener with no menu behind it — i.e. the
|
||||||
|
handler was refusing it, not missing it. `.track-row` is the selector.
|
||||||
|
|
||||||
|
Verified by execution: 8 component tests (real browser, real shadow
|
||||||
|
boundary, real timings) and 2 e2e specs against the running app, twice
|
||||||
|
in a row. Not verified: any of it under a real finger on a real
|
||||||
|
WebView — the pointer events are dispatched, because neither Desktop
|
||||||
|
Chrome nor Desktop Safari has touch and there is no device tier.
|
||||||
|
|
||||||
|
## The first device run: A4 works, and two things only a phone could say (2026-08-17)
|
||||||
|
|
||||||
|
The published v1.5.0 APK, on a real phone, owner-reported. **This is the
|
||||||
|
first runtime evidence any of the Android work has ever had** — A4
|
||||||
|
shipped entirely reasoned from source.
|
||||||
|
|
||||||
|
**What holds.** Playback survives the screen locking. The MediaSession
|
||||||
|
notification appears in the status pane *with album art* — which
|
||||||
|
answers, in one observation, four of the open questions from plan 016:
|
||||||
|
the foreground service starts, POST_NOTIFICATIONS was granted and the
|
||||||
|
notification is visible, the session is picked up, and **cover art
|
||||||
|
decoded from a `MANAGE_EXTERNAL_STORAGE` path by a service is
|
||||||
|
readable**. The last was the one nobody could argue from documentation.
|
||||||
|
|
||||||
|
**Two bugs, and neither is visible from any tier we have.**
|
||||||
|
|
||||||
|
*Back did not navigate back.* The scaffold's
|
||||||
|
`MainActivity.onBackPressed` asks `webView.canGoBack()` and finishes the
|
||||||
|
activity otherwise — and this app had never touched `history`, so that
|
||||||
|
was false at every depth and back quit from anywhere. The fix is in the
|
||||||
|
frontend, not in Java: a navigation is a `history` entry now
|
||||||
|
(`recordNavigation` in `index.ts`, same URL, the destination in the
|
||||||
|
entry's state) and `popstate` replays it with `_isBack`. The Java half
|
||||||
|
needs no change, because the mechanism it already uses is the one we
|
||||||
|
were failing to feed.
|
||||||
|
|
||||||
|
Two rules keep it honest. The **first** navigation replaces the launch
|
||||||
|
entry rather than pushing one, or every launch costs a back press before
|
||||||
|
the app will close. And the in-app back buttons go through
|
||||||
|
`history.back()` rather than popping a stack of their own — `navStack`
|
||||||
|
is **deleted**, not kept alongside, because two stacks is exactly how
|
||||||
|
the detail view's own button and the phone's gesture come to disagree
|
||||||
|
about how far back one press goes. `back-navigation.spec.ts` pins that
|
||||||
|
invariant.
|
||||||
|
|
||||||
|
*The transport was off screen.* **`targetSdk 35` is Android 15, which
|
||||||
|
lays every app out edge-to-edge**, ignores the deprecated
|
||||||
|
`statusBarColor`/`navigationBarColor` the theme still sets, and hands
|
||||||
|
the app a window the size of the screen. The WebView is `match_parent`,
|
||||||
|
so the page's bottom band — the transport, and on a phone the tab bar —
|
||||||
|
was drawn underneath the gesture bar. `applyWindowInsets()` pads the
|
||||||
|
container by `systemBars | displayCutout | ime` and returns the insets
|
||||||
|
rather than consuming them. The window background goes black to match
|
||||||
|
the app's own ramp, or the padding shows as a blue-grey band.
|
||||||
|
|
||||||
|
**Neither is findable in the browser tier, and that is the lesson worth
|
||||||
|
keeping**: a viewport has no system bars, so `phone-shell.spec.ts` at
|
||||||
|
390x844 renders a shell that fits perfectly while the device cuts 48dp
|
||||||
|
off the bottom — and `page.goBack()` was never called because nothing in
|
||||||
|
a desktop shell has a back gesture. The Android tier's own note says
|
||||||
|
failure there is invisible; this is the milder version, where the app
|
||||||
|
works and is simply wrong in ways only the platform can show you.
|
||||||
|
|
||||||
|
Verified by execution: the APK builds with the Java change; 3 e2e specs
|
||||||
|
cover the history behaviour, on Chromium locally and WebKit in CI.
|
||||||
|
Not verified: the insets themselves, which need the next APK on the
|
||||||
|
owner's phone. What to look for is one thing — the transport and the tab
|
||||||
|
bar clear of the gesture bar, and the header clear of the status bar.
|
||||||
|
|
||||||
|
## The phone is a Chrome 113 WebView, and that reframes everything (2026-08-17)
|
||||||
|
|
||||||
|
The device is reachable over adb now, so the tier can be *asked* rather
|
||||||
|
than reported on. `make android-inspect` + `make android-eval` are that:
|
||||||
|
a debug build (`applicationIdSuffix ".dev"`, so it installs **beside**
|
||||||
|
the release app rather than needing the uninstall that would take the
|
||||||
|
library with it) opens `webview_devtools_remote_<pid>`, and raw CDP over
|
||||||
|
Node's built-in WebSocket evaluates in the real page. **Playwright
|
||||||
|
cannot do this** — `connectOverCDP` calls `Browser.setDownloadBehavior`
|
||||||
|
and a WebView answers "Browser context management is not supported",
|
||||||
|
killing the connection before the first evaluate.
|
||||||
|
|
||||||
|
Measured on the device (Light Phone III, TLP301):
|
||||||
|
|
||||||
|
| fact | value |
|
||||||
|
| --- | --- |
|
||||||
|
| Android | 14, SDK 34 |
|
||||||
|
| screen | 1080x1240, density 408 |
|
||||||
|
| WebView viewport | **424 x 439 CSS px**, DPR 2.55 |
|
||||||
|
| WebView engine | **Chrome 113.0.5672.136** (mid-2023) |
|
||||||
|
|
||||||
|
**The first correction: the insets commit does not explain the report.**
|
||||||
|
Edge-to-edge is forced for apps *running on* Android 15, and this phone
|
||||||
|
is Android 14 — the screenshot shows the app correctly inset, with the
|
||||||
|
status bar and the gesture bar outside it. `applyWindowInsets()` is
|
||||||
|
right and stays (the next phone, or one OS update, is Android 15), but
|
||||||
|
it is **pre-emptive, not the fix for "the controls are off screen"**.
|
||||||
|
That was an inference from a version number, and the device disagreed.
|
||||||
|
|
||||||
|
**The second correction: the black `fill` proves nothing.** A wa-icon on
|
||||||
|
the device has the right `color` (#ffd43b) and an `<svg>` in its shadow
|
||||||
|
root, and `getComputedStyle(svg).fill` is black — but that is the *svg
|
||||||
|
root*, and every vendored Font Awesome path carries
|
||||||
|
`fill="currentColor"` itself, so the root's fill is irrelevant. Measuring
|
||||||
|
the wrong node produced a diagnosis-shaped result. `__yjIconMisses` is
|
||||||
|
empty, so no name is unbundled either. Why the icons do not appear in the
|
||||||
|
screenshot is **still open**.
|
||||||
|
|
||||||
|
**What the engine version does explain, and what to check next.**
|
||||||
|
Chrome 113 has `:has()`, `color-mix()` and `dialog.showModal()`, and
|
||||||
|
lacks three things this app's dependencies use:
|
||||||
|
|
||||||
|
- **Relaxed CSS nesting** (Chrome 120): a nested rule starting with a
|
||||||
|
bare element selector is dropped. `.x { svg { ... } }` parses to
|
||||||
|
nothing; `.x { & svg { ... } }` parses. Any Web Awesome or app
|
||||||
|
stylesheet written the modern way silently loses declarations here,
|
||||||
|
and dropped declarations are exactly the failure that looks like
|
||||||
|
"rendered but wrong".
|
||||||
|
- **The Popover API** (Chrome 114). Web Awesome's popup calls
|
||||||
|
`showPopover?.()` — optional, so nothing throws — but also sets
|
||||||
|
`popover="manual"`, which on 113 is an unknown attribute doing
|
||||||
|
nothing. Every context menu, dropdown and the whole menu keyboard
|
||||||
|
model rides on that, so it is the first thing to test with a library
|
||||||
|
present.
|
||||||
|
- `light-dark()` and relative colour syntax (`rgb(from ...)`).
|
||||||
|
|
||||||
|
**The lesson for the tier: a device is an engine, not just a screen.**
|
||||||
|
Every browser tier here runs a current Chromium or WebKit, and the phone
|
||||||
|
that will actually run this app is two years behind — so "it renders at
|
||||||
|
424x439 in Chromium" (checked, the transport is on screen) says nothing
|
||||||
|
about whether it renders on the phone. The e2e tier cannot be fixed by
|
||||||
|
resizing; the missing signal is version, and CDP against the device is
|
||||||
|
the only place to get it.
|
||||||
|
|
||||||
|
Verified by execution: every number in the table, the four feature
|
||||||
|
probes, and that the hardware back button no longer kills the app (the
|
||||||
|
`.dev` build carries the history fix; pid survived a BACK press).
|
||||||
|
Unverified: what happened to the icons and the transport controls, which
|
||||||
|
is where this resumes.
|
||||||
|
|
||||||
|
## What the device actually said, with both builds side by side (2026-08-17)
|
||||||
|
|
||||||
|
The phone inspectable and awake, the same Light Phone III running two
|
||||||
|
builds of this app in turn. This closes both questions the previous entry
|
||||||
|
left open, and **neither answer was the one the symptom suggested**.
|
||||||
|
|
||||||
|
**"The playback controls are off screen" was true, literal, and already
|
||||||
|
fixed.** The installed build is from B2 **phase 1** — it carries
|
||||||
|
`bottom-nav` and no `now-playing-view`, which dates it between 57bfbdf
|
||||||
|
and 1b05dde. Settled (30 s after launch, not 6), its player bar shows
|
||||||
|
art, title, favourite, shuffle, prev — and stops. Play/pause, next,
|
||||||
|
repeat and queue are past the right edge, because at 424 px the bar was
|
||||||
|
still carrying the seek bar and volume that **phase 2 moved into
|
||||||
|
`now-playing-view`**. On the current build, on the same phone and the
|
||||||
|
same engine, `document.body.scrollWidth` equals `clientWidth` (424) and
|
||||||
|
`player-controls` measures 200..380 inside 424. So the fix was already
|
||||||
|
on main, unreleased, and the device is what proved it rather than
|
||||||
|
argued it.
|
||||||
|
|
||||||
|
**"No icons" was an artefact of my own screenshot.** A `wa-icon` on the
|
||||||
|
device has `path` computed fill `rgb(255,212,59)` and paints; the first
|
||||||
|
capture was six seconds after a cold start, before the icon fetches had
|
||||||
|
landed. Two corrections in two entries from the same misreading: measure
|
||||||
|
the node that paints, and let the app settle before believing a picture.
|
||||||
|
|
||||||
|
**Chrome 113's missing Popover API does not break the menus.** This was
|
||||||
|
the leading worry and it is unfounded: a long-press on a row opens the
|
||||||
|
real panel at (212,145), 162x193, `visibility: visible`, seven
|
||||||
|
`role=menuitem`s, all seven inside the panel and clear of the player bar
|
||||||
|
— confirmed by screenshot as well as by measurement. Web Awesome's
|
||||||
|
`showPopover?.()` is an optional call and `wa-popup` positions itself,
|
||||||
|
so the attribute being inert costs nothing. **Long-press itself works on
|
||||||
|
real hardware**, over a real 1,744-track library, which is the phase 3
|
||||||
|
verification the browser tier could only approximate.
|
||||||
|
|
||||||
|
**The one genuine fault the device adds is phase 4's.** `track-list` at
|
||||||
|
424 px computes `--grid-cols: 24px 102px 101px 101px 80px` — which fits
|
||||||
|
the host exactly, so nothing overflows — but "Duration" does not fit in
|
||||||
|
80 px and neither does most content. The columns are not too wide; there
|
||||||
|
are simply too many of them for a phone, which is what phase 4 already
|
||||||
|
says. It is now a measurement rather than a prediction.
|
||||||
|
|
||||||
|
Two operational notes. The debug sibling scanned the phone's real music
|
||||||
|
and its data directory is **414 MB**, so it is worth uninstalling when
|
||||||
|
done (`adb uninstall app.yellowjacket.dev` — the sibling id is exactly
|
||||||
|
what makes that safe). And `am start` does not reliably take focus while
|
||||||
|
another app is foreground: check `topResumedActivity` before trusting a
|
||||||
|
screenshot, or you will read someone else's app.
|
||||||
|
|
||||||
|
## The phone track list, and the bug a viewport could not have found (2026-08-17)
|
||||||
|
|
||||||
|
B2 phase 4. A phone draws `titleArtist` — the title with the artist
|
||||||
|
under it — plus the duration, and drops the column headers and the
|
||||||
|
resize handles. It is a **column set, not a second row template**: the
|
||||||
|
row, its delegated events, the selection semantics, the playing marker
|
||||||
|
and the virtualizer never learn that anything changed, because from
|
||||||
|
their side only the number of columns did.
|
||||||
|
|
||||||
|
Three rules, each one a way it breaks otherwise. The row height is in
|
||||||
|
two places (`PHONE_ROW_HEIGHT` and the CSS) and they must agree, since
|
||||||
|
the virtualizer positions rows from that number. What is *drawn* and
|
||||||
|
what can be *sorted* are separate questions — the sort list is built
|
||||||
|
from `configuredColumns`, or a phone with no headers could sort by
|
||||||
|
nothing but title and duration. And a phone's widths are neither loaded
|
||||||
|
nor saved.
|
||||||
|
|
||||||
|
**That last one is the finding, and it came from the device.** With the
|
||||||
|
arrangement passing five component tests and five e2e specs at
|
||||||
|
424x439, the phone showed `24px 148px 236px`: the duration column with
|
||||||
|
55% of the row. `loadColumnWidths` is keyed by column *id* and fills a
|
||||||
|
gap with `MIN_COLUMN_WIDTH`, so the stacked column — which nothing can
|
||||||
|
ever have saved a width for, there being no handles to drag — came out
|
||||||
|
at the minimum while `trackLength` inherited a width saved for a
|
||||||
|
four-column desktop row. The mirror image is worse and was never
|
||||||
|
reachable from a phone at all: `saveColumnWidths` would have written the
|
||||||
|
computed phone widths back under the same ids, replacing the width the
|
||||||
|
user dragged on a desktop.
|
||||||
|
|
||||||
|
**Why every browser test missed it.** The specs assert the *shape* — how
|
||||||
|
many grid tracks, no header, no overflow, the title's share of the row —
|
||||||
|
and the width bug depends on what is in `localStorage` for a *different*
|
||||||
|
column set. dev-headless's seed happened to hold widths that split the
|
||||||
|
other way, so the same assertion passed in the browser and failed on the
|
||||||
|
phone. The unit test now carries the desktop map as a fixture, which is
|
||||||
|
the reproduction the browser needed to have.
|
||||||
|
|
||||||
|
**Confirmed on the phone afterwards**, with the fix installed:
|
||||||
|
`24px 304px 80px`, 52 px rows, no header row, the title 298 px and not
|
||||||
|
truncated, `body.scrollWidth == clientWidth`. The same numbers the
|
||||||
|
browser gives at that viewport, which is the point of having measured
|
||||||
|
both.
|
||||||
|
|
||||||
|
Two tooling notes worth keeping. `playwright-cli` holds its page across
|
||||||
|
a `make dev-headless` restart, so a probe after a rebuild can be
|
||||||
|
answering for the *old* bundle — it reported the desktop layout at 424 px
|
||||||
|
until the page was reopened. And wireless adb dropped twice more mid-
|
||||||
|
session when the screen slept; USB for anything longer than a few
|
||||||
|
probes.
|
||||||
|
|
||||||
|
## The catalog download now asks about the connection (2026-08-17)
|
||||||
|
|
||||||
|
Plan 016 B4. ~0.6 GB had no network awareness at all; it is skipped on a
|
||||||
|
cellular connection unless the user says otherwise
|
||||||
|
(`AllowMeteredCatalogDownload`, default false, toggle in Settings' Search
|
||||||
|
Index section).
|
||||||
|
|
||||||
|
**The shape is dictated by the cgo rule, not by taste.** `explore` is
|
||||||
|
imported by `cmd/indexbuild`, which builds with `CGO_ENABLED=0` and must
|
||||||
|
not link Wails, so `netpolicy.go` holds the policy and the JSON parsing —
|
||||||
|
tested on every platform — while the one platform call is a closure
|
||||||
|
injected from `app.go`, where naming `application` is already legitimate.
|
||||||
|
|
||||||
|
Four things measured or corrected in the doing:
|
||||||
|
|
||||||
|
- **The portable name is `application.Mobile`, not `application.Android`**
|
||||||
|
(which the plan and `CLAUDE.md` both named). `Android` exists only
|
||||||
|
under the `android` build tag; `Mobile`'s desktop implementation is a
|
||||||
|
stub whose `NetworkJSON()` returns `""`.
|
||||||
|
- **The runtime reports no metered flag.** `{"connected":bool,
|
||||||
|
"type":"wifi|cellular|ethernet|none"}` is all there is, so cellular is
|
||||||
|
the signal and a metered *Wi-Fi* — a phone hotspot, a hotel — cannot be
|
||||||
|
detected. Android itself knows (`NET_CAPABILITY_NOT_METERED`) and the
|
||||||
|
runtime does not pass it on. Documented gap, not an oversight.
|
||||||
|
- **An unknown answer must not read as metered.** Every desktop answers
|
||||||
|
`""`, so the obvious defensive default would have disabled the catalog
|
||||||
|
download for every desktop user in the world.
|
||||||
|
- **The gate belongs before the first status write.** Declining is a
|
||||||
|
no-op — no job in the indicator, no error tier to dismiss — which is
|
||||||
|
what makes the refusal safe to have on by default.
|
||||||
|
|
||||||
|
## The stale-shape repair dropped the CI catalog (2026-08-17)
|
||||||
|
|
||||||
|
Not our change, but it is the operational state everything else now runs
|
||||||
|
in, and the restore condition needs to be written down somewhere that is
|
||||||
|
not a commit message.
|
||||||
|
|
||||||
|
`fix(database): retire a table whose shape the schema moved past` added
|
||||||
|
`staleshape.go`: before `applySchema`, drop any non-Authored table whose
|
||||||
|
live shape disagrees with the schema. That is the right rule for an
|
||||||
|
install — a client's catalog is *downloaded*, so a stale one costs a
|
||||||
|
minute of re-fetching the artifact, and keeping it costs every Explore
|
||||||
|
read.
|
||||||
|
|
||||||
|
It runs inside `database.NewDB`, which `cmd/indexbuild` also calls. On
|
||||||
|
the first run after it landed, 19 seconds in:
|
||||||
|
|
||||||
|
```
|
||||||
|
16:15:51 retiring a table ... table=explore_index
|
||||||
|
reason="column entity_type is TEXT, schema declares INTEGER"
|
||||||
|
16:16:05 index maintenance mode=build reason="no completed import yet"
|
||||||
|
lastImported=never baselineSeries=0
|
||||||
|
```
|
||||||
|
|
||||||
|
**The premise was false for the one database where it was expensive.**
|
||||||
|
That catalog is not stale; it is deliberately kept in the older text
|
||||||
|
encoding, which `artifactStoresText` and `sourceColumns` exist to
|
||||||
|
tolerate — so it would have been judged stale and dropped on *every*
|
||||||
|
run. And `retireLibraryTables`, in the same package, already documents
|
||||||
|
the opposite rule for this database: drop everything the datamap does
|
||||||
|
**not** call Cache.
|
||||||
|
|
||||||
|
`fix(database): never retire the catalog the index build derives` makes
|
||||||
|
the policy a build tag (`retireStaleCache`, false under `indexbuild`),
|
||||||
|
which is how this project already separates the index tools. It prevents
|
||||||
|
recurrence and cannot undo the drop: that volume was the only copy.
|
||||||
|
|
||||||
|
**What it cost, and the shape of the cost.** A full re-import from the
|
||||||
|
MetaBrainz dumps, resumed across runs from a checkpoint, at a rate that
|
||||||
|
swung between 2 and 15 MB/s. The job runs on **every push to main** with
|
||||||
|
a 3 h budget on a runner of capacity 1 — so until the import completes,
|
||||||
|
every push books three hours and ordinary CI queues behind it. That is
|
||||||
|
the real damage: not one lost job, but a repeating one.
|
||||||
|
|
||||||
|
So the `push:` trigger in `index-artifact.yml` is **commented out**
|
||||||
|
until a run reports `complete=true`; the weekly cron and
|
||||||
|
`workflow_dispatch` still resume the build, which is all it needs.
|
||||||
|
Restoring those two lines is the whole revert.
|
||||||
|
|
||||||
|
Three things worth keeping from it:
|
||||||
|
|
||||||
|
- **A repair belongs where its assumptions hold.** `NewDB` is the one
|
||||||
|
chokepoint every binary in this project shares, including the one
|
||||||
|
whose database cannot be re-derived cheaply. Anything destructive
|
||||||
|
there needs to ask which binary it is in — the build tag was available
|
||||||
|
and is what the fix used.
|
||||||
|
- **The only copy of a 205 GB derived asset is one Docker volume.**
|
||||||
|
There is no snapshot, so the restore time is "however long
|
||||||
|
MetaBrainz takes today". A periodic copy would turn this class of
|
||||||
|
incident into twenty minutes.
|
||||||
|
- **The fix's residual trade is now the thing to watch**: with Cache
|
||||||
|
tables never retired under `indexbuild`, a future `explore_index`
|
||||||
|
column fails that job loudly at build time instead of silently
|
||||||
|
rebuilding. That is the right default, and it means the next schema
|
||||||
|
change touching `explore_index` needs a deliberate plan for this one
|
||||||
|
database rather than none.
|
||||||
|
|
||||||
|
## Two guards for the index cache, and what each one is worth (2026-08-17)
|
||||||
|
|
||||||
|
Both come out of the incident above, and they protect different halves
|
||||||
|
of it.
|
||||||
|
|
||||||
|
**`TestNoCacheTableIsRetiredHere` asserts the outcome, not the
|
||||||
|
mechanism.** The test that shipped with the fix pins one table in one
|
||||||
|
wrong shape, which is the failure that happened; what actually cost the
|
||||||
|
rebuild was a destructive repair added at `database.NewDB` — the
|
||||||
|
chokepoint every binary here shares — without asking which binary it was
|
||||||
|
in. The next one will have a different name and a different reason. So
|
||||||
|
this puts *every* `datamap` Cache table into a shape the schema has
|
||||||
|
moved past, opens the database the way `cmd/indexbuild` does, and
|
||||||
|
requires all of them to still be there.
|
||||||
|
|
||||||
|
Three things it got right by being written this way. The table list is
|
||||||
|
`datamap.ByKind(Cache)`, so the two credit tables added the same day
|
||||||
|
were covered without anyone adding them — flipping the policy back fails
|
||||||
|
on **five** tables including `artist_credit_part` and
|
||||||
|
`artist_credit_ref`, where the single-table test fails on one. It
|
||||||
|
asserts rows survive as well as the table, because SQLite does an
|
||||||
|
implicit DELETE before a DROP and a repair that recreated the table
|
||||||
|
would otherwise look identical. And it *accepts* an error from `NewDB`,
|
||||||
|
because that is the trade the fix documents: loud failure instead of a
|
||||||
|
silent day of downloading.
|
||||||
|
|
||||||
|
**`scripts/index-cache-snapshot.sh` covers the half no test can.** The
|
||||||
|
volume held the only copy of a catalog whose rebuild is hours of someone
|
||||||
|
else's bandwidth. `VACUUM INTO` rather than `cp`, because a byte copy of
|
||||||
|
a live SQLite file is a corrupt file of plausible size; the staging
|
||||||
|
directory is deliberately not copied, since a build resumes without it;
|
||||||
|
and the snapshot is reopened and asked for its catalog row count before
|
||||||
|
any rotation happens. Both failure paths were exercised rather than
|
||||||
|
argued: a corrupt source and an empty catalog each exit non-zero, delete
|
||||||
|
their own output, and leave the previous snapshots in place.
|
||||||
|
|
||||||
|
`docs/index-cache.md` is the restore procedure, and the number that
|
||||||
|
makes it worth having: a restored snapshot resolves to `refresh` and
|
||||||
|
folds in the incremental listens since — minutes, against the 3–23 h a
|
||||||
|
rebuild was estimating.
|
||||||
|
|
||||||
|
## A green release pipeline can ship an empty changelog (2026-08-18)
|
||||||
|
|
||||||
|
`conventional-changelog-conventionalcommits@10` is silently incompatible
|
||||||
|
with the writer `@semantic-release/release-notes-generator@14` depends on
|
||||||
|
(`conventional-changelog-writer@^8`). Every release note renders as a bare
|
||||||
|
`## 0.0.1 (date)` heading with **no sections and no commits under it**, no
|
||||||
|
step fails, and the release ships with an empty body.
|
||||||
|
|
||||||
|
It is pinned to `9` in `.gitea/workflows/release.yml` and in
|
||||||
|
`make release-dry`, which must stay identical. **Check the rendered notes,
|
||||||
|
never the exit code** — this is invisible to every tick in the pipeline.
|
||||||
|
|
||||||
|
## semantic-release needs push rights to the branch even when it never pushes to it (2026-08-18)
|
||||||
|
|
||||||
|
Core runs `git push --dry-run HEAD:<branch>` as a permission check, before
|
||||||
|
and independently of any plugin. With `@semantic-release/git` removed
|
||||||
|
nothing ever pushes to `main`, and the check still runs.
|
||||||
|
|
||||||
|
Two things this looked like and was not:
|
||||||
|
|
||||||
|
- **Not branch protection.** A `--dry-run` push does not reach the
|
||||||
|
pre-receive hook: pushing one to protected `main` with a write-scoped
|
||||||
|
token succeeds. So `main`'s `enable_push: false` is not what fails here.
|
||||||
|
- **A flat `403 Forbidden`, not Gitea's protection message.** That is the
|
||||||
|
tell. `PACKAGE_TOKEN` had package-write and repo-*read* — enough to
|
||||||
|
clone a private repo, so every other workflow was fine — and needed
|
||||||
|
`write:repository`.
|
||||||
|
|
||||||
|
## A tag-triggered workflow runs the workflow file at the *tagged* commit (2026-08-18)
|
||||||
|
|
||||||
|
Not the one on `main`. Moving `v0.0.0` onto a pre-merge commit ran that
|
||||||
|
commit's version of `homebrew-formula.yml`, which predated the `v0.0.0`
|
||||||
|
skip guard added in the same plan, and it pushed a `0.0.0` formula to the
|
||||||
|
public tap.
|
||||||
|
|
||||||
|
A guard added today does not protect a tag that points at yesterday. When
|
||||||
|
re-pointing a tag, check what the workflows looked like *there*.
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ reviews. Nothing was changed.
|
|||||||
|
|
||||||
Findings below are numbered `H-n` (hands-on) and cross-reference the
|
Findings below are numbered `H-n` (hands-on) and cross-reference the
|
||||||
static reports where they overlap. The reconciliation plan built from
|
static reports where they overlap. The reconciliation plan built from
|
||||||
all four files is `.planning/plans/pending/007-ui-reconciliation.md`.
|
all four files is `.planning/plans/completed/007-ui-reconciliation.md`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
+2
@@ -1,5 +1,7 @@
|
|||||||
# 012 — What we ask the network for, and what we already had
|
# 012 — What we ask the network for, and what we already had
|
||||||
|
|
||||||
|
> **Completed.** Findings 1, 2 and 4 shipped. Finding 3 — the bound-but-uncalled methods — is now **#86**.
|
||||||
|
|
||||||
**Status:** all four findings fixed. Lint (3 configs), Go tests (3
|
**Status:** all four findings fixed. Lint (3 configs), Go tests (3
|
||||||
configs), `tsc` and 752 Vitest tests pass; **not driven against the
|
configs), `tsc` and 752 Vitest tests pass; **not driven against the
|
||||||
real app**, so the numbers below are read off the code, not measured.
|
real app**, so the numbers below are read off the code, not measured.
|
||||||
+2
@@ -1,5 +1,7 @@
|
|||||||
# 015 — Android release pipeline
|
# 015 — Android release pipeline
|
||||||
|
|
||||||
|
> **Completed.** The pipeline ships a signed APK from CI on every `v*` tag; `docs/android-release.md` is its operating document.
|
||||||
|
|
||||||
Ship an Android APK from CI on every version tag, published to the Gitea
|
Ship an Android APK from CI on every version tag, published to the Gitea
|
||||||
generic package registry so Obtainium can poll a plain URL.
|
generic package registry so Obtainium can poll a plain URL.
|
||||||
|
|
||||||
@@ -0,0 +1,339 @@
|
|||||||
|
# 015 — Multi-artist credits, navigable
|
||||||
|
|
||||||
|
> **Completed.** Phases 1, 2 and 4 shipped. Running the ingest against the real dump and publishing an artifact that carries credits is **#88**; Phase 3 (`file_artists`) is **#89**, blocked on it.
|
||||||
|
|
||||||
|
## The problem
|
||||||
|
|
||||||
|
A track credited to more than one artist has exactly one navigable
|
||||||
|
artist in this app, and the others are punctuation.
|
||||||
|
|
||||||
|
`audio_files` carries `artist_credit` (the credit as tagged, for
|
||||||
|
display) and `artist_id` (one artist, for grouping and browsing).
|
||||||
|
`primaryArtist()` (`backend/library/artistcredit.go:53`) resolves that
|
||||||
|
one artist by *string-parsing* the credit: it strips a " feat. "
|
||||||
|
clause, and deliberately does not split on `&`, `x`, `with` or `,`
|
||||||
|
because those appear inside real artist names. So "Lana Del Rey ft.
|
||||||
|
Sean Lennon" stores Lana Del Rey and discards Sean Lennon entirely,
|
||||||
|
and "Alina Baraz & Galimatias" stores one artist whose name is the
|
||||||
|
whole credit.
|
||||||
|
|
||||||
|
### What the measurement says
|
||||||
|
|
||||||
|
Measured 2026-08-16 against a real 26,069-file library (19,840 mp3,
|
||||||
|
6,229 flac; 57 unreadable, m4a/ogg not examined), plus an 80+80
|
||||||
|
MusicBrainz `inc=artist-credits` sample.
|
||||||
|
|
||||||
|
- **13%** of a random sample of the library's recordings have more
|
||||||
|
than one credited artist in MusicBrainz (10 of 79 resolved).
|
||||||
|
Extrapolates to ~3,250 of the 24,989 files carrying a recording
|
||||||
|
MBID.
|
||||||
|
- **0.86%** of files (224) carry any structured multi-artist signal in
|
||||||
|
their own tags. mp3 carries **zero** files with multiple
|
||||||
|
`MUSICBRAINZ_ARTISTID` values across 19,840 files; flac has 87.
|
||||||
|
- **1,286** files say "feat." in `ARTIST`; **1,159 of them (90%)**
|
||||||
|
have nothing structured behind it. A sample of 80 such files was
|
||||||
|
multi-artist in MB **80 of 80 times**.
|
||||||
|
|
||||||
|
CLAUDE.md currently justifies plan 013's removal of `artist_credit` /
|
||||||
|
`artist_credit_artist` with "3 credits of 2,823 listed more than one
|
||||||
|
artist". That figure measured **our own writer**, not the library:
|
||||||
|
`cachedLinkArtist` was called exactly once per credit
|
||||||
|
(`e7748f1^:backend/library/library.go:1842`), so a collaboration could
|
||||||
|
never have been recorded, and the three were resolution collisions on
|
||||||
|
shared credit text. Dropping the join table was still correct — it only
|
||||||
|
ever held one row, so it was pure join cost — but the stated evidence
|
||||||
|
does not support "multi-artist is rare". Correcting that claim is part
|
||||||
|
of this plan.
|
||||||
|
|
||||||
|
### Why the tags cannot answer it
|
||||||
|
|
||||||
|
Deriving the decomposition locally, with no network, works **79% of the
|
||||||
|
time** (169 of 215 files with a multi-value `ARTISTS` tag: mp3 69/105,
|
||||||
|
flac 100/110), and the failures are systematic rather than random:
|
||||||
|
|
||||||
|
```
|
||||||
|
ARTIST = '2Pac feat. Snoop Dogg, Nate Dogg, Hussein Fatal & Yaki Kadafi'
|
||||||
|
ARTISTS = ['2Pac', 'Snoop Doggy Dogg', 'Nate Dogg', 'Fatal', 'Yaki Kadafi']
|
||||||
|
```
|
||||||
|
|
||||||
|
`ARTISTS` holds **canonical** artist names; `ARTIST` holds
|
||||||
|
**as-credited** names. Locating one inside the other fails on
|
||||||
|
"Snoop Doggy Dogg" vs "Snoop Dogg", on "Fatal" vs "Hussein Fatal", and
|
||||||
|
on Unicode (`Michel'le` vs `Michel’le`, `K-Ci` vs `K‐Ci` — U+2010, not
|
||||||
|
a hyphen). That distinction is precisely what a join phrase encodes,
|
||||||
|
and it is why this cannot be a tag-parsing feature.
|
||||||
|
|
||||||
|
Two format details that will mislead anyone re-running the probe:
|
||||||
|
Picard writes `ARTISTS` **slash-joined into one TXXX frame** on mp3 and
|
||||||
|
as **true repeated Vorbis keys** on flac, so a probe splitting only on
|
||||||
|
NUL undercounts mp3 to zero.
|
||||||
|
|
||||||
|
## The shape
|
||||||
|
|
||||||
|
MusicBrainz models a credit as ordered parts, and the credit *string*
|
||||||
|
is derived from them — `artist_credit.name` is a cached render, nothing
|
||||||
|
more. Each participant is `(position, artist, name, join_phrase)`,
|
||||||
|
where `artist` is the MBID (canonical, what you navigate to) and `name`
|
||||||
|
is the credited spelling (what you display).
|
||||||
|
|
||||||
|
**Join phrases are assembly instructions, not disassembly
|
||||||
|
instructions.** Rendering is a concatenation, never a search:
|
||||||
|
|
||||||
|
```
|
||||||
|
for each (position, artist_mbid, credited_name, join_phrase):
|
||||||
|
emit link(credited_name -> artist_mbid)
|
||||||
|
emit text(join_phrase)
|
||||||
|
```
|
||||||
|
|
||||||
|
The link positions are known **by construction**. This is load-bearing:
|
||||||
|
if we instead located each `credited_name` inside the stored
|
||||||
|
`artist_credit` text, we would reintroduce the mismatch above — the
|
||||||
|
stored string may have come from the tags while the parts come from the
|
||||||
|
catalog, and those **disagree for ~1 in 3 multi-artist files** (61 of
|
||||||
|
90 sampled credits rendered exactly equal to the tag string).
|
||||||
|
Divergences seen: `'Skrillex feat. Swae Lee'` tagged vs
|
||||||
|
`'Skrillex & Swae Lee'` in MB; `'STRFKR'` vs `'Starfucker'`;
|
||||||
|
`'Zedd feat. Hayley Williams'` vs `'... of Paramore'`. Either MB was
|
||||||
|
edited after tagging or Picard versions differ; either way the search
|
||||||
|
would miss or match the wrong span.
|
||||||
|
|
||||||
|
So `audio_files.artist_credit` stops being the source of truth and
|
||||||
|
becomes the **fallback**, used only where there are no parts.
|
||||||
|
|
||||||
|
## Where the data comes from
|
||||||
|
|
||||||
|
The catalog carries the decomposition; no user ever makes a
|
||||||
|
per-recording call. Two sources were ruled out first, both cheaply:
|
||||||
|
|
||||||
|
- **The canonical dump — which is what CI already pulls
|
||||||
|
(`dumpimport.go:84-85`) — does not have it.**
|
||||||
|
`canonical_musicbrainz_data.csv` gives `artist_mbids` (ordered list)
|
||||||
|
and `artist_credit_name`, but that last column is the *rendered*
|
||||||
|
string. Splitting it on CI needs the as-credited names, so CI would
|
||||||
|
fail exactly the way a local parse does.
|
||||||
|
- **The JSON dumps do not cover the catalog.**
|
||||||
|
`json-dumps/recording.tar.xz` is 31 MB / 368 MB uncompressed and
|
||||||
|
holds **153,691 recordings**, not ~35M. Measured against the test
|
||||||
|
library's 24,885 recording MBIDs: **0.00% overlap, zero rows**. It is
|
||||||
|
some other subset and is not usable.
|
||||||
|
|
||||||
|
That leaves the core dump, **`mbdump.tar.bz2`** (7.1 GB compressed at
|
||||||
|
the 20260815 export), from
|
||||||
|
`https://data.metabrainz.org/pub/musicbrainz/data/fullexport/`. Four
|
||||||
|
members are needed:
|
||||||
|
|
||||||
|
| member | why | approx rows |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `mbdump/artist_credit_name` | `(artist_credit, position, artist, name, join_phrase)` — the payload | ~4M |
|
||||||
|
| `mbdump/artist` | `id -> gid`, since the above references artist *row ids* | ~2.6M |
|
||||||
|
| `mbdump/recording` | `gid -> artist_credit`, to key credits by recording MBID | ~35M |
|
||||||
|
| `mbdump/release_group` | same, for album credits | ~2M |
|
||||||
|
|
||||||
|
### Coverage is not a concern
|
||||||
|
|
||||||
|
Of 24,885 distinct recording MBIDs in the test library, **24,808
|
||||||
|
(99.7%)** already have an `explore_index` recording row, measured
|
||||||
|
against a database at 2,052,200 rows — i.e. shipped-artifact coverage,
|
||||||
|
not a local build's. The popularity filter does not strand the long
|
||||||
|
tail here.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
- **Phase 1 — done.** `backend/explore/dumpcredits.go` +
|
||||||
|
`dumpcreditswrite.go`, wired into `dumpimport.go`'s `run` behind its
|
||||||
|
own `credits_import_done` marker.
|
||||||
|
- **Phase 2 — done.** `cmd/indexexport` writes the two tables;
|
||||||
|
`artifactimport.go` reads them behind `artifactHasCredits()`.
|
||||||
|
- **Phase 4 — done, and it does not need Phase 3.** `explore.GetCredits`
|
||||||
|
reads the catalog tables keyed on the *recording* MBID, which both
|
||||||
|
sides of the app already carry — a catalog row has one and so does a
|
||||||
|
local file (`library.Track.RecordingMBID`). So one binding serves the
|
||||||
|
Explore pages and the library's own lists, and all ten artist-link
|
||||||
|
call sites render credits today without a local table.
|
||||||
|
- **Phase 3 (`file_artists`) — not started, and now an
|
||||||
|
offline-resilience task rather than a prerequisite.** The table is
|
||||||
|
deliberately *not* declared yet: nothing writes or reads it, and a
|
||||||
|
schema file plus a datamap note describing behaviour that does not
|
||||||
|
exist is a claim the code cannot back. Its remaining
|
||||||
|
value is that credits currently vanish when the catalog is absent or
|
||||||
|
still downloading, which is precisely the `no-index` state
|
||||||
|
`ShelfPage.State` exists to describe. Materialising into
|
||||||
|
`file_artists` is what makes a library stand on its own.
|
||||||
|
|
||||||
|
**Nothing renders yet in practice**, because no published artifact
|
||||||
|
carries credit tables — every credit falls back to its single link
|
||||||
|
until an index build with Phase 1 runs and is exported.
|
||||||
|
|
||||||
|
**Column layouts are verified against the real 20260815 export**, not
|
||||||
|
taken from the schema docs — `artist(id, gid, …)`,
|
||||||
|
`artist_credit(id, name, artist_count, …)`,
|
||||||
|
`artist_credit_name(credit, position, artist, name, join_phrase)` and
|
||||||
|
`recording(id, gid, name, artist_credit, …)` were each read out of the
|
||||||
|
dump. `release_group` shares `recording`'s first four columns and is
|
||||||
|
the one layout still taken on trust; `ErrDumpShape` turns a wrong guess
|
||||||
|
into a loud failure rather than a quietly wrong catalog.
|
||||||
|
|
||||||
|
**Still unrun: the ingest against the real 7.1 GB dump.** Everything is
|
||||||
|
covered by tests over a synthetic tar, which cannot catch a surprise in
|
||||||
|
the other ~35M rows.
|
||||||
|
|
||||||
|
### Phase 1 — Ingest credits on CI
|
||||||
|
|
||||||
|
New dump stage in `cmd/indexbuild`, behind the `indexbuild` tag with
|
||||||
|
the rest of `dumpimport.go`'s stages.
|
||||||
|
|
||||||
|
**Constraint from `b98840e`:** `cmd/indexbuild` is built
|
||||||
|
`CGO_ENABLED=0` in a plain `golang` container and must not reach the
|
||||||
|
Wails `application` package — `TestIndexToolsDoNotImportWails` walks
|
||||||
|
`go list -deps -tags indexbuild`. Nothing here should need it, but a
|
||||||
|
new `ServiceStartup` hook on a package this imports is how it comes
|
||||||
|
back. Go's `compress/bzip2` is pure Go and decompress-only, which is
|
||||||
|
all this needs.
|
||||||
|
|
||||||
|
**Measured, 20260815 export.** Tar members are **alphabetical**, and
|
||||||
|
that is favourable: `artist` (435 MB), `artist_credit` (414 MB) and
|
||||||
|
`artist_credit_name` (237 MB) all fall inside the first ~900 MB
|
||||||
|
compressed, while `recording` and `release_group` come later. So the
|
||||||
|
maps are complete before the rows that consume them arrive, and no
|
||||||
|
recording data is ever buffered.
|
||||||
|
|
||||||
|
Pure-Go `compress/bzip2` decompresses at **26 MB/s uncompressed /
|
||||||
|
8.7 MB/s compressed** (measured on a 250 MB prefix, 3.01x ratio) —
|
||||||
|
**~13.7 min** for the whole file single-threaded, and less because the
|
||||||
|
stream can stop after `release_group` rather than reading the
|
||||||
|
`series`/`tag`/`track`/`url`/`work` tail. The 2 MB/s origin throttle
|
||||||
|
dominates, as it already does for every other dump here.
|
||||||
|
|
||||||
|
Do not, however, *depend* on the ordering: assert it and fall back to
|
||||||
|
buffering if a future export reorders, rather than silently emitting
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
- `artist` -> `map[int32]uuid16` (~2.6M x ~20 B = ~60 MB)
|
||||||
|
- `artist_credit_name` -> `map[int32][]creditPart` (~4M x ~40 B =
|
||||||
|
~200 MB)
|
||||||
|
- `recording` / `release_group` -> emit `gid -> credit_id` **only for
|
||||||
|
MBIDs already in `explore_index`** (the kept set is ~1.4M x 16 B =
|
||||||
|
~22 MB), which is what keeps 35M rows from being held
|
||||||
|
|
||||||
|
Peak ~300 MB, one sequential pass.
|
||||||
|
|
||||||
|
**Only multi-artist credits are stored.** A single-artist credit is
|
||||||
|
`(name, "")` and is already fully described by `explore_index`'s
|
||||||
|
`artist_name` / `artist_mbid`; storing it would triple the table for
|
||||||
|
nothing. Post-filter after loading, once the row count per credit is
|
||||||
|
known.
|
||||||
|
|
||||||
|
New tables (and `datamap` entries, or `TestCatalogCoversSchema` fails
|
||||||
|
the build — both are `Cache`, matching `explore_index`):
|
||||||
|
|
||||||
|
```
|
||||||
|
artist_credit_part(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
```
|
||||||
|
|
||||||
|
with `explore_index.artist_credit_id` as the link. Credits are
|
||||||
|
**shared** — an album's twelve tracks by one artist share one credit
|
||||||
|
row — which is the opposite of 013's local verdict, and correctly so:
|
||||||
|
1:1 in a local library, genuinely many-to-one at 2M-row catalog scale.
|
||||||
|
|
||||||
|
### Phase 2 — Ship them in the artifact
|
||||||
|
|
||||||
|
`cmd/indexexport` currently creates exactly two tables in the artifact
|
||||||
|
(`explore_index`, `artifact_meta`, at `cmd/indexexport/*.go:147,170`),
|
||||||
|
so this is a structural addition, not a column.
|
||||||
|
|
||||||
|
Estimated size: ~13% of 1.4M recordings, deduplicated by shared credit,
|
||||||
|
at ~2.3 parts each — order 400k rows, ~18 MB uncompressed. Against a
|
||||||
|
~0.6 GB install that is acceptable; it must be measured rather than
|
||||||
|
assumed before merge.
|
||||||
|
|
||||||
|
`artifactimport.go` must read it **only if present**, on the writer
|
||||||
|
handle where `core` is attached — the `artifactHasTotals()` /
|
||||||
|
`artifactStoresText()` pattern (`artifactimport.go:145-175`), one step
|
||||||
|
up from a column to a table. An artifact published before this exists
|
||||||
|
is still a perfectly good catalog and must import as one that declines
|
||||||
|
to answer. Adding this to the importer's SELECT list without the probe
|
||||||
|
is how every already-published artifact starts failing.
|
||||||
|
|
||||||
|
`artifactCatalogColumns` gains `artist_credit_id`; it is kept in sync
|
||||||
|
with the exporter by `TestArtifactColumnsMatchExporter`.
|
||||||
|
|
||||||
|
### Phase 3 — Materialize locally
|
||||||
|
|
||||||
|
```
|
||||||
|
file_artists(audio_file_id, position, artist_id, credited_name, join_phrase)
|
||||||
|
```
|
||||||
|
|
||||||
|
`credited_name` is stored **per row**, not looked up from
|
||||||
|
`artists.name` — that is the Snoop-Doggy-Dogg distinction, and it is
|
||||||
|
the whole point.
|
||||||
|
|
||||||
|
Filled at scan/import time by joining `audio_files.recording_mbid`
|
||||||
|
against the catalog. **Materialized rather than resolved live**,
|
||||||
|
because the catalog is a downloaded artifact that can be absent or
|
||||||
|
still arriving — that is why `ShelfPage.State` has a `no-index` value —
|
||||||
|
and a library whose track rows lose their artists when the catalog is
|
||||||
|
missing is worse than today.
|
||||||
|
|
||||||
|
That implies a backfill for the case where the catalog arrives *after*
|
||||||
|
the library was scanned. It registers with `jobs` (progress, cancel)
|
||||||
|
like every other long pass, and takes a **distinct kind** from
|
||||||
|
`index-build`, since `job-controls.ts` keys its "you will discard hours
|
||||||
|
of downloading" confirmation on that kind.
|
||||||
|
|
||||||
|
`artists` gains rows for guests who own no files. **This changes what
|
||||||
|
the artists grid shows** and is an open question below.
|
||||||
|
|
||||||
|
### Phase 4 — Render
|
||||||
|
|
||||||
|
`utils/explore-link.ts` gains a credit-rendering entry point taking
|
||||||
|
ordered parts and returning a `TemplateResult`. Every row and detail
|
||||||
|
view already renders artist names through it, so they inherit
|
||||||
|
multi-artist links without individually knowing credits exist — the
|
||||||
|
property that made centralising it worthwhile.
|
||||||
|
|
||||||
|
Its existing fallback philosophy already covers the no-parts case: "a
|
||||||
|
list where some rows are clickable and others silently are not reads as
|
||||||
|
a bug, not as a statement about metadata." Where there are no parts
|
||||||
|
(no recording MBID, or no catalog row — ~4% of the test library) render
|
||||||
|
today's behaviour: the flat `artist_credit` string with one link to the
|
||||||
|
primary artist. **Do not split the string there.** There is genuinely
|
||||||
|
no information to split on, and that is the one place the temptation
|
||||||
|
returns.
|
||||||
|
|
||||||
|
`primaryArtist()` stays exactly as it is. It remains the fallback and
|
||||||
|
is still what `artist_id` means.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
1. **Catalog credit vs tagged credit, when they disagree** (~1 in 3
|
||||||
|
multi-artist files). Rendering the catalog's decomposition is what
|
||||||
|
makes names navigable; preserving the file's is what makes the app
|
||||||
|
reflect the user's files. Leaning toward: render the catalog
|
||||||
|
decomposition, keep `artist_credit` as the fallback string. Wants a
|
||||||
|
deliberate decision, not an accident.
|
||||||
|
2. **Do guest artists appear in the artists grid?** Phase 3 creates
|
||||||
|
`artists` rows for people who own no files. The grid currently means
|
||||||
|
"artists in your library" and joins `audio_files`. A guest on one
|
||||||
|
track is arguably in the library and arguably not. Whichever way,
|
||||||
|
the ownership question stays "is there a file" — that rule does not
|
||||||
|
bend.
|
||||||
|
3. **`release_group` credits** are ingested in the same pass for
|
||||||
|
nearly nothing, but album-artist rendering is a separate surface.
|
||||||
|
Ship the data in phase 1, render in a follow-up rather than widening
|
||||||
|
phase 4.
|
||||||
|
4. **Our own `tagwriter`** does not write `ARTISTS` or multiple
|
||||||
|
`MUSICBRAINZ_ARTISTID` frames, so autotagging a folder degrades the
|
||||||
|
very field this rests on — the same shape as the existing
|
||||||
|
track-totals note. Out of scope here; worth recording.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Coverage: re-run the library probe and assert `file_artists` is
|
||||||
|
populated for ~13% of files, not ~0.9%.
|
||||||
|
- `TestCatalogCoversSchema` / `TestLifetimesMatchSchema` for the new
|
||||||
|
tables.
|
||||||
|
- `TestIndexToolsDoNotImportWails` still passes with the new stage.
|
||||||
|
- An artifact **without** the credits table imports cleanly (the
|
||||||
|
`artifactHasTotals` regression shape).
|
||||||
|
- Round-trip: a known multi-artist recording renders each name as a
|
||||||
|
separate link with the correct join phrases between them.
|
||||||
@@ -0,0 +1,412 @@
|
|||||||
|
# 016 — What Android parity would actually take
|
||||||
|
|
||||||
|
> **Completed.** Sections A, B1, B2 and B4 shipped. B3, writing tags on the device, is now **#87**; the device-found UI faults are #51–#72, sequenced by #73.
|
||||||
|
|
||||||
|
> **Status: all of section A is done.** A1–A3 landed with "let the app
|
||||||
|
> reach the user's music"; A4 (MediaSession, transport notification,
|
||||||
|
> audio focus) landed with "survive the screen locking". The direction
|
||||||
|
> taken is **option 1, the full librarian**: `MANAGE_EXTERNAL_STORAGE`
|
||||||
|
> plus an in-app folder browser, which keeps the path-keyed model
|
||||||
|
> intact. B1/B2 remain, both awaiting a decision rather than work. The
|
||||||
|
> sections below are kept as written, because they are the argument the
|
||||||
|
> decision rests on — see "What is left" at the end for the current
|
||||||
|
> state.
|
||||||
|
|
||||||
|
Plan 015 shipped a *pipeline*: the app cross-compiles, is signed and
|
||||||
|
versioned, and publishes from CI. This is the assessment of what stands
|
||||||
|
between that and an Android app worth installing.
|
||||||
|
|
||||||
|
**The headline: parity is the wrong target, and choosing it would be
|
||||||
|
the expensive mistake.** Four of the blockers below are not porting work
|
||||||
|
— they are the Android platform declining to support the model this app
|
||||||
|
is built on. The decision to make first is in "The fork in the road" at
|
||||||
|
the end; everything before it is evidence for that decision.
|
||||||
|
|
||||||
|
Severity is what the app *does* today, verified against the source and
|
||||||
|
the generated manifest, not guessed.
|
||||||
|
|
||||||
|
## A. It cannot work at all until these are fixed
|
||||||
|
|
||||||
|
### A1. The app can read no music. (deepest)
|
||||||
|
|
||||||
|
`build/android/app/src/main/AndroidManifest.xml` requests INTERNET,
|
||||||
|
VIBRATE, ACCESS_NETWORK_STATE, USE_BIOMETRIC, POST_NOTIFICATIONS, the
|
||||||
|
two location permissions, CAMERA and the two FOREGROUND_SERVICE ones.
|
||||||
|
**There is no storage or media permission of any kind.** At
|
||||||
|
`targetSdk 35` that means the app can see its own private directory and
|
||||||
|
nothing else.
|
||||||
|
|
||||||
|
Adding `READ_MEDIA_AUDIO` is necessary and *not sufficient*, because it
|
||||||
|
grants access through **MediaStore**, not through the filesystem. This
|
||||||
|
app's entire model is absolute paths: `audio_files.file_path` is the
|
||||||
|
primary key of ownership, `AddLibrary(path)` takes a directory, the
|
||||||
|
scanner walks it with `os.ReadDir`, and every one of
|
||||||
|
`GetFilePathsByAlbums` / `ByGenres` / `ByRecordingMBIDs` exists to hand
|
||||||
|
paths to the player. Scoped storage does not offer a stable directory
|
||||||
|
to walk.
|
||||||
|
|
||||||
|
The honest options are three, and they are not close in cost:
|
||||||
|
|
||||||
|
- **MediaStore as the library source.** Query the content resolver,
|
||||||
|
keep MediaStore IDs (or content URIs) beside or instead of paths, and
|
||||||
|
open audio through a `ContentResolver` file descriptor. This is the
|
||||||
|
Android-native answer and it touches the schema, the scanner, the
|
||||||
|
player's file opening and every path-keyed query.
|
||||||
|
- **`MANAGE_EXTERNAL_STORAGE`.** Keeps the path model intact and is
|
||||||
|
effectively barred from Google Play except for genuine file managers.
|
||||||
|
Viable *only* because we distribute through Obtainium — which is a
|
||||||
|
real point in its favour here, and worth stating plainly rather than
|
||||||
|
dismissing.
|
||||||
|
- **App-private storage only**, i.e. the user copies music into the
|
||||||
|
app's sandbox. Trivial to build, and nobody wants it.
|
||||||
|
|
||||||
|
### A2. The first-run flow cannot complete.
|
||||||
|
|
||||||
|
`first-run-wizard.ts` calls `DirectoryPicker()`, which is
|
||||||
|
`frontendutil.DirectoryPicker` → `app.Dialog.OpenFile().
|
||||||
|
CanChooseDirectories(true)`. Wails' own `ANDROID.md` lists open-directory
|
||||||
|
dialogs as **"❌ Returns an error — SAF yields tree URIs, not filesystem
|
||||||
|
paths"**. So the one action the wizard exists to perform fails, and
|
||||||
|
`<first-run-wizard>` intercepts all pointer events until a library
|
||||||
|
exists — so the app is not merely empty, it is inert.
|
||||||
|
|
||||||
|
Whatever A1 resolves to decides this: a MediaStore library needs no
|
||||||
|
picker at all, and a SAF tree needs the picker to return a URI the
|
||||||
|
backend can use.
|
||||||
|
|
||||||
|
### A3. MPRIS is compiled into the Android build.
|
||||||
|
|
||||||
|
`mpris_linux.go` is `//go:build linux`, and **`android` implies
|
||||||
|
`linux`** (documented, and the reason it is in the APK). It will look
|
||||||
|
for a session bus that does not exist. It needs `//go:build linux &&
|
||||||
|
!android`, and its Android counterpart is A4.
|
||||||
|
|
||||||
|
This one is cheap and should be done regardless — it is a two-character
|
||||||
|
build-tag change plus whatever `mediacontrols.New` returns instead.
|
||||||
|
|
||||||
|
### A4. Playback will be killed the moment the screen locks.
|
||||||
|
|
||||||
|
The scaffold's `WailsForegroundService` is typed **`dataSync`**
|
||||||
|
(`foregroundServiceType="dataSync"`, `FOREGROUND_SERVICE_TYPE_DATA_SYNC`),
|
||||||
|
and the manifest requests `FOREGROUND_SERVICE_DATA_SYNC`. A music player
|
||||||
|
needs `mediaPlayback` and `FOREGROUND_SERVICE_MEDIA_PLAYBACK`, plus a
|
||||||
|
`MediaSession` for lock-screen and notification transport controls,
|
||||||
|
plus **audio focus** — pause on a phone call, duck for a notification,
|
||||||
|
pause on headphone unplug. None of that exists today. `oto` will happily
|
||||||
|
keep writing to a stream nobody can hear.
|
||||||
|
|
||||||
|
This is the difference between "an app that plays audio" and "a music
|
||||||
|
player", and it is Java-side work in the scaffold plus a Go-side bridge.
|
||||||
|
|
||||||
|
## B. It works, but wrongly
|
||||||
|
|
||||||
|
### B1. The x86_64 half of the APK cannot run on any Android.
|
||||||
|
|
||||||
|
Established in plan 015: `modernc.org/libc`'s `Xlstat64` issues a raw
|
||||||
|
`lstat` on linux/amd64, which Android's seccomp forbids, so the process
|
||||||
|
takes `SIGSYS` the first time it touches the database. arm64 is
|
||||||
|
structurally unaffected (no `lstat` syscall exists; it routes through
|
||||||
|
`fstatat`).
|
||||||
|
|
||||||
|
So ~31 MB of the artifact is dead weight on *every* Android device,
|
||||||
|
including x86 Chromebooks. Options: drop `x86_64` from `abiFilters`
|
||||||
|
(smaller APK, no emulator target — which does not work anyway), or
|
||||||
|
carry it against a future modernc fix. **Dropping it is the honest
|
||||||
|
default**; it is also the only item in this plan that is a five-minute
|
||||||
|
change.
|
||||||
|
|
||||||
|
### B2. The UI is a desktop shell.
|
||||||
|
|
||||||
|
`MinWidth`/`MinHeight` are 800×600 and were *measured* — below ~780 the
|
||||||
|
header subtitle wraps the title out of its bar. A phone is ~360–430 CSS
|
||||||
|
px wide. The sidebar collapses to icons below 900px, which is a
|
||||||
|
laptop-sized breakpoint, not a phone one. Beyond width: the app is built
|
||||||
|
on hover (the marquee's `hover` mode, tooltips), right-click context
|
||||||
|
menus, a keyboard shortcut layer with its own overlay and settings page,
|
||||||
|
multi-select with ctrl/shift, and a resizable-column track list. None of
|
||||||
|
those are gestures.
|
||||||
|
|
||||||
|
This is not a stylesheet pass. It is a second front end for the views
|
||||||
|
worth having on a phone, sharing the stores and bindings — which the
|
||||||
|
architecture supports, since a view is already a lazily-loaded chunk
|
||||||
|
behind `VIEW_LOADERS`.
|
||||||
|
|
||||||
|
### B3. Tag writing cannot reach the user's files.
|
||||||
|
|
||||||
|
`tagwriter` rewrites tags in place, and autotag's whole purpose is
|
||||||
|
applying them to a folder. Under scoped storage that is impossible
|
||||||
|
outside the sandbox without a SAF write grant per tree. If A1 lands on
|
||||||
|
MediaStore, in-place tag writing needs `MediaStore` write requests and
|
||||||
|
user confirmation per file on Android 11+.
|
||||||
|
|
||||||
|
Autotagging is arguably a desktop-only feature and saying so is a
|
||||||
|
legitimate answer.
|
||||||
|
|
||||||
|
### B4. The Explore catalog is a ~0.6 GB download into app-private storage.
|
||||||
|
|
||||||
|
It works — but with no awareness of a metered connection and no
|
||||||
|
accounting for a device where that is a meaningful fraction of free
|
||||||
|
space. At minimum it needs to be opt-in on mobile and to refuse a
|
||||||
|
metered network by default. `Android.NetworkJSON()` reports
|
||||||
|
`{connected,type}`, so the signal is available.
|
||||||
|
|
||||||
|
## C. Inert, and fine
|
||||||
|
|
||||||
|
Window geometry, menus and the system tray are documented no-ops on
|
||||||
|
mobile. The keyboard shortcut layer is harmless but its Settings page
|
||||||
|
is dead weight. `profiling` is already compiled out of production
|
||||||
|
builds. These cost nothing and need no work.
|
||||||
|
|
||||||
|
## D. Unknown until it runs on a device
|
||||||
|
|
||||||
|
**Nothing in section A or B has been observed on Android**, because the
|
||||||
|
x86_64 emulator cannot run the app (B1) and emulator 37 refuses arm64
|
||||||
|
images on an x86_64 host. Everything above is read from the source, the
|
||||||
|
generated manifest and Wails' own documentation. The first real device
|
||||||
|
run will find things this list does not have, and the most likely
|
||||||
|
places are audio latency and buffering under `oto`/oboe, and SQLite
|
||||||
|
behaviour on app-private storage.
|
||||||
|
|
||||||
|
## The fork in the road
|
||||||
|
|
||||||
|
The four blockers in section A are all the same question wearing
|
||||||
|
different clothes: **is the Android app a librarian, or a player?**
|
||||||
|
|
||||||
|
YellowJacket on the desktop is a *librarian*. It scans folders,
|
||||||
|
deduplicates covers, detects duplicate tracks, reconciles against
|
||||||
|
MusicBrainz, rewrites tags on disk, and manages downloads. That model
|
||||||
|
rests on owning a filesystem, which is precisely what Android declines
|
||||||
|
to give.
|
||||||
|
|
||||||
|
Three coherent products, and only the first is "parity":
|
||||||
|
|
||||||
|
1. **Full librarian on Android.** Requires `MANAGE_EXTERNAL_STORAGE`
|
||||||
|
(Obtainium-only distribution, which we already have), a phone UI for
|
||||||
|
every view, and media-session playback. Largest scope by far; the
|
||||||
|
result is an app almost nobody has asked for on a phone.
|
||||||
|
2. **A player for music already on the phone.** MediaStore as the
|
||||||
|
source, no scanner, no autotag, no downloads; the library, queue,
|
||||||
|
playlists, favourites and Explore-as-browsing all still make sense.
|
||||||
|
This is a genuinely good Android app and it is *not* parity — it is
|
||||||
|
a subset with a different data source.
|
||||||
|
3. **A companion to the desktop app.** The phone browses and controls
|
||||||
|
the desktop's library over the network, or syncs a subset. Smallest
|
||||||
|
Android surface, and it leans on the thing that already works.
|
||||||
|
|
||||||
|
**Option 2 is the recommendation** if the goal is an app people use;
|
||||||
|
option 3 if the goal is the least work for the most value. Option 1 is
|
||||||
|
the only one that answers "feature parity" literally, and it is the one
|
||||||
|
worth arguing hardest against.
|
||||||
|
|
||||||
|
> **Decided:** option 1's *data model* (the librarian keeps its
|
||||||
|
> filesystem and its scanner — A1 shipped that) with option 2's
|
||||||
|
> *surface*. The phone is a player over the library this app already
|
||||||
|
> builds; it does not get every view. The list is below.
|
||||||
|
|
||||||
|
## The phone gets a subset (decided)
|
||||||
|
|
||||||
|
B2 is not a stylesheet pass and not a second front end either. A view
|
||||||
|
is already a lazily-loaded chunk behind `VIEW_LOADERS` /
|
||||||
|
`DETAIL_LOADERS` in `index.ts`, and the stores and bindings are shared,
|
||||||
|
so the phone build is **a different loader table and a different
|
||||||
|
chrome**, over the same stores.
|
||||||
|
|
||||||
|
**In**, because each is something a person does with a phone in their
|
||||||
|
hand:
|
||||||
|
|
||||||
|
- **Home** — the shelves are already a phone-shaped surface.
|
||||||
|
- **Library browse** — albums, artists, genres. The grids are already
|
||||||
|
virtualized and card-shaped.
|
||||||
|
- **Now playing** — which on a phone is a *view*, not a 4em bar.
|
||||||
|
- **The queue.**
|
||||||
|
- **Search** — the header box, scoped as it already is.
|
||||||
|
- **Playlists**, including smart ones, as lists to play rather than to
|
||||||
|
edit.
|
||||||
|
|
||||||
|
**Out**, and each for a reason rather than by omission:
|
||||||
|
|
||||||
|
- **Autotag** — the review UI is a wide table and the action rewrites
|
||||||
|
files on disk; B3 has not been verified even as *possible* yet.
|
||||||
|
- **Downloads** — two tab panels of client configuration.
|
||||||
|
- **Explore** — the catalog is a ~0.6 GB download (B4); browsing it is
|
||||||
|
the last thing to earn a phone's storage.
|
||||||
|
- **Settings** — not the page. The phone needs a handful of settings
|
||||||
|
(theme, the library folder, playback) and not the 93 controls the
|
||||||
|
desktop page carries.
|
||||||
|
- **Jobs**, **shortcuts overlay**, **column configuration** — a phone
|
||||||
|
has no keyboard and no resizable columns, and the jobs indicator is
|
||||||
|
enough.
|
||||||
|
|
||||||
|
What the shell has to lose, from the audit at the top of this section:
|
||||||
|
the 800×600 minimum, the 11-item sidebar (a phone wants a bottom tab
|
||||||
|
bar over the five things above), hover as a route to anything,
|
||||||
|
right-click as the only route to a context menu (long-press is the
|
||||||
|
gesture), and ctrl/shift multi-select.
|
||||||
|
|
||||||
|
One rule for the work: **no view forks.** A phone layout that copies a
|
||||||
|
view's template is two templates to fix every bug in. Where a view
|
||||||
|
cannot serve both, the split belongs at the chunk boundary that already
|
||||||
|
exists.
|
||||||
|
|
||||||
|
Phase 1 followed that rule and found its cost: reusing `<app-sidebar>`
|
||||||
|
inside the drawer means reusing its `data-testid`s too, and a second
|
||||||
|
copy standing by in the DOM broke 30 specs that had nothing to do with
|
||||||
|
the phone. The rule holds — a second list of destinations would be
|
||||||
|
worse — but a shared component must be rendered only when it is wanted,
|
||||||
|
and the guard belongs in a test that names the reason.
|
||||||
|
|
||||||
|
## What is worth doing regardless of that decision
|
||||||
|
|
||||||
|
Cheap, independently useful, and each unblocks measurement:
|
||||||
|
|
||||||
|
1. **Drop `x86_64` from `abiFilters`** (B1) — or keep it and document
|
||||||
|
why. Five minutes.
|
||||||
|
2. **`//go:build linux && !android` on `mpris_linux.go`** (A3), so the
|
||||||
|
Android build stops carrying a D-Bus client. Small.
|
||||||
|
3. **A device smoke run**, which needs someone's phone and the published
|
||||||
|
APK. Everything in D depends on it, and it is the single highest
|
||||||
|
information-per-minute action available.
|
||||||
|
4. **Make the first-run wizard fail legibly** rather than inertly (A2)
|
||||||
|
— the picker's error already routes through `describeError`, but the
|
||||||
|
wizard still blocks pointer events, so an Android user sees a dead
|
||||||
|
screen rather than a sentence. Even under option 3 this is the right
|
||||||
|
behaviour.
|
||||||
|
|
||||||
|
|
||||||
|
## What is left (updated after A4)
|
||||||
|
|
||||||
|
**A4 is done.** `backend/mediacontrols/android.go` is a `Handler`
|
||||||
|
beside the MPRIS one, and the Java half is
|
||||||
|
`WailsForegroundService.java`: a `MediaSession`, a `MediaStyle`
|
||||||
|
transport notification and audio focus. It needed no new JNI and no new
|
||||||
|
Gradle dependency — `application.Android.StartForegroundService(json)`
|
||||||
|
going out, `WailsBridge.emitEvent` → the application event bus coming
|
||||||
|
back, and the platform `android.media.session` API rather than
|
||||||
|
androidx.media, which minSdk 21 makes available anyway.
|
||||||
|
|
||||||
|
Four decisions in it are worth keeping:
|
||||||
|
|
||||||
|
- **Ducking is a player concept, not a volume change.**
|
||||||
|
`Player.SetDuck` re-applies the *user's* level with an attenuation
|
||||||
|
offset, so `getUserVolume` still reports what the user chose and
|
||||||
|
nothing is persisted or emitted. A duck that wrote through to the
|
||||||
|
volume would let one notification tone permanently turn the music
|
||||||
|
down.
|
||||||
|
- **The duck path is pre-Oreo only.** From API 26 the framework ducks
|
||||||
|
the app itself and sends no `CAN_DUCK` focus change, so asking to be
|
||||||
|
told instead (`setWillPauseWhenDucked`) would mean pausing for every
|
||||||
|
notification tone, and doing both would attenuate twice.
|
||||||
|
- **An unchanged payload is not an event here either.** Every push
|
||||||
|
crosses JNI and re-delivers an Intent, and the player pushes state on
|
||||||
|
several paths that can agree.
|
||||||
|
- **After the first start, updates use `startService`.** From Android
|
||||||
|
12 an app in the background may not *start* a foreground service, but
|
||||||
|
it may keep delivering intents to one it already has — which is every
|
||||||
|
track change with the screen off.
|
||||||
|
|
||||||
|
The contract with Java — the payload keys, the state words, the command
|
||||||
|
names — is in `androidpayload.go`, deliberately *without* the `android`
|
||||||
|
build tag, so `go test` exercises it on every platform. Everything left
|
||||||
|
in `android.go` is untested by construction: it compiles only under a
|
||||||
|
cross-compiler and runs only on a phone.
|
||||||
|
|
||||||
|
**B1 is done: x86_64 is dropped.** 27.1 MB → 15.9 MB, measured. Three
|
||||||
|
places had to agree — `abiFilters`, the Makefile's `android:package`
|
||||||
|
(or Go still compiles a library Gradle then discards) and the
|
||||||
|
`native-code: 'arm64-v8a'$` assertion in `android-apk.yml`, whose
|
||||||
|
anchor is what stops it also matching the fat APK's line. Adding the
|
||||||
|
ABI back, if modernc ever fixes `Xlstat64`, is those same three edits.
|
||||||
|
|
||||||
|
**B2, the desktop shell.** Scope decided (below); **all four phases are
|
||||||
|
done.**
|
||||||
|
|
||||||
|
- *Phase 1, the shell.* Below 600px the sidebar column is gone,
|
||||||
|
`<bottom-nav>` is the primary navigation, and the shell fits 320px
|
||||||
|
exactly — measured, from 652px in a 360px viewport before.
|
||||||
|
- *Phase 2, the full-screen now-playing view.* Where phase 1's seek bar
|
||||||
|
and volume went. A detail view, so Back pops the nav stack; it
|
||||||
|
composes the real transport components rather than copying them; and
|
||||||
|
it hides the bottom bar while it is up, so it carries its own queue
|
||||||
|
button.
|
||||||
|
- *Phase 3, long-press.* `utils/long-press.ts`: one document-capture
|
||||||
|
listener, installed once from `index.ts`, which turns a 500 ms
|
||||||
|
stationary touch into a synthetic `contextmenu` at the touch point.
|
||||||
|
Every menu in the app opens from that event, so all six components
|
||||||
|
gained the gesture without one of them changing — which is the same
|
||||||
|
argument `ContextMenuController` rests on, one layer lower. The
|
||||||
|
details that are not obvious are in `NOTES.md` (2026-08-17); the one
|
||||||
|
worth repeating is that ours is told from the browser's own
|
||||||
|
long-press event by **identity**, not `isTrusted`, because a test
|
||||||
|
cannot dispatch a trusted event and that path would otherwise be the
|
||||||
|
only uncovered one.
|
||||||
|
|
||||||
|
- *Phase 4, the track list.* A phone draws `titleArtist` (title over
|
||||||
|
artist) plus the duration, and drops the column headers and the resize
|
||||||
|
handles — a column set rather than a second row template, so the row
|
||||||
|
and everything delegated on it is unchanged. Verified at the device's
|
||||||
|
own 424x439: `24px 304px 80px`, 52 px rows, no truncation, no
|
||||||
|
overflow. The device also found the bug in it, which no browser
|
||||||
|
viewport would have: saved *desktop* column widths reached the phone
|
||||||
|
through an id-keyed store and gave the duration column 55% of the row.
|
||||||
|
|
||||||
|
**B2 and B4 are complete.** B4 is `backend/explore/netpolicy.go`: the
|
||||||
|
catalog download is skipped on a cellular connection unless
|
||||||
|
`AllowMeteredCatalogDownload` is on, with the toggle in Settings' Search
|
||||||
|
Index section. The policy and the JSON parsing are in `explore` (tested
|
||||||
|
on every platform) and only the platform call is injected from `app.go`,
|
||||||
|
because `cmd/indexbuild` imports `explore` and must not link Wails. Two
|
||||||
|
things the plan got slightly wrong: the portable API is
|
||||||
|
`application.Mobile.NetworkJSON()` rather than `Android`'s, and it
|
||||||
|
reports no metered flag — so cellular is the signal and a metered Wi-Fi
|
||||||
|
cannot be seen.
|
||||||
|
|
||||||
|
What is left in this plan is B3 (tag writing, which needs a device) and
|
||||||
|
the standing question of the Light Phone's Chrome 113 — which so far has
|
||||||
|
cost nothing: menus, dialogs and long-press all work on it.
|
||||||
|
|
||||||
|
**B3/B4** are unchanged, and B3 is now *possible* where it was not:
|
||||||
|
with all-files access, `tagwriter` can write in place.
|
||||||
|
|
||||||
|
### What the first device run answered (2026-08-17)
|
||||||
|
|
||||||
|
A4 **works**: playback survives the screen locking, and the transport
|
||||||
|
notification appears with cover art — which also settles the service's
|
||||||
|
access to a `MANAGE_EXTERNAL_STORAGE` path, the permission grant and
|
||||||
|
the lock-screen session in one observation. Everything below in "what
|
||||||
|
none of section A answered" was written before this and is now answered
|
||||||
|
except the OEM permission-flow variance.
|
||||||
|
|
||||||
|
It also found two faults no browser tier can see, both fixed and both
|
||||||
|
awaiting the next APK for confirmation (`NOTES.md`, same date):
|
||||||
|
|
||||||
|
- **Back quit the app from any depth.** The scaffold asks
|
||||||
|
`webView.canGoBack()`; the frontend had never used `history`. A
|
||||||
|
navigation is a history entry now, and `navStack` is gone rather than
|
||||||
|
kept beside it.
|
||||||
|
- **The transport was under the gesture bar** — or so the version
|
||||||
|
number said. `applyWindowInsets()` in `MainActivity` is right and
|
||||||
|
stays, but the phone is **Android 14**, where the system still insets
|
||||||
|
the window: the fix is pre-emptive and the symptom has another cause.
|
||||||
|
Still open, along with icons that do not appear at all. The phone's
|
||||||
|
WebView is **Chrome 113**, which is the lead (no Popover API, no
|
||||||
|
relaxed CSS nesting), and `make android-inspect` / `android-eval` are
|
||||||
|
how it gets asked.
|
||||||
|
|
||||||
|
The standing item is unchanged in kind: **B3 (tag writing) and the
|
||||||
|
permission flow still need a device**, and so does confirming these two.
|
||||||
|
|
||||||
|
### What none of section A answered
|
||||||
|
|
||||||
|
Nothing here has been observed on a device. The permission flow in
|
||||||
|
particular is the kind of thing that behaves differently across OEM
|
||||||
|
builds — `ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION` is
|
||||||
|
implemented inconsistently, which is why there is a fallback to the
|
||||||
|
global list, and neither path has been exercised.
|
||||||
|
|
||||||
|
A4 adds its own list of things only a device can answer, and they are
|
||||||
|
the likely first failures: whether the notification appears at all
|
||||||
|
(POST_NOTIFICATIONS is requested from `startForegroundService`, so a
|
||||||
|
user who declines gets a service with an invisible notification),
|
||||||
|
whether audio focus arrives while `oto`/oboe holds the output, whether
|
||||||
|
the lock screen picks up the session, and whether cover art decoded
|
||||||
|
from a `MANAGE_EXTERNAL_STORAGE` path is readable by the service.
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# 017 — Releases that happen by themselves
|
||||||
|
|
||||||
|
**Shipped as `v0.0.1`.** A merge to `main` now reads the Conventional
|
||||||
|
Commits since the last tag, cuts the tag and the Gitea release whose body
|
||||||
|
is the generated changelog, and the four publishing workflows build that
|
||||||
|
tag and attach their artifacts. Nothing is released by hand.
|
||||||
|
|
||||||
|
## What it looks like now
|
||||||
|
|
||||||
|
`release.yml` on push to `main` → semantic-release → tag → four `v*`
|
||||||
|
workflows in parallel (serialised in practice by the capacity-1 runner):
|
||||||
|
|
||||||
|
| workflow | publishes | attaches |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `arch-package` | pacman registry | `…-x86_64.pkg.tar.zst` |
|
||||||
|
| `android-apk` | generic registry (Obtainium) | `…-android-arm64.apk` |
|
||||||
|
| `desktop-assets` | — | `…-linux-amd64.tar.gz` |
|
||||||
|
| `homebrew-formula` | the public tap | — (builds from source) |
|
||||||
|
|
||||||
|
Verified on the real thing: all five green, three assets on the release,
|
||||||
|
the tap at `0.0.1`, and the Obtainium `latest` URL serving 200.
|
||||||
|
|
||||||
|
## The five decisions, and what they cost
|
||||||
|
|
||||||
|
1. **semantic-release, not a shell script.** The first draft of this plan
|
||||||
|
proposed hand-rolling it and the argument did not survive checking:
|
||||||
|
`@semantic-release/exec` is first-party and current, and the
|
||||||
|
Gitea-shaped part is one `curl`. What I would have hand-rolled —
|
||||||
|
commit parsing, semver ordering, note rendering — is the part with the
|
||||||
|
edge cases and none of it is Gitea-shaped.
|
||||||
|
2. **`@saithodev/semantic-release-gitea` is a dead end** and was offered
|
||||||
|
before it was checked: last published 2022, `got@10`, and no peer
|
||||||
|
dependency on semantic-release at all.
|
||||||
|
3. **No `@semantic-release/git`.** `main` is protected, so a changelog
|
||||||
|
commit-back is rejected by the pre-receive hook — and would be
|
||||||
|
rejected *after* the tag was pushed, leaving a tagged release the run
|
||||||
|
reports as failed. The release page is the changelog;
|
||||||
|
`.release-notes.md` is a gitignored carrier and `CHANGELOG.md` is a
|
||||||
|
signpost.
|
||||||
|
4. **Versions restart at `0.0.1`**, a downgrade on every channel. No
|
||||||
|
`epoch`, no `versionCode` offset: both are permanent, a reinstall is
|
||||||
|
once. Documented in `packaging/homebrew/README.md` and
|
||||||
|
`docs/android-release.md`.
|
||||||
|
5. **No macOS and no Windows.** `GOOS=darwin CGO_ENABLED=0` fails at
|
||||||
|
`wails/v3/pkg/mac` and there is no macOS runner, so Homebrew-from-source
|
||||||
|
stays that channel. Windows cross-compiles in ~2.5 s and is withheld
|
||||||
|
because no build of it has ever been *run*.
|
||||||
|
|
||||||
|
## Four things that only showed up by running it
|
||||||
|
|
||||||
|
- **`conventional-changelog-conventionalcommits@10` renders empty
|
||||||
|
notes.** Silently: right version, right tag, every step green, and a
|
||||||
|
release body that is a bare `## 0.0.1 (date)` heading with nothing
|
||||||
|
beneath it. Held at `9`, in `release.yml` and `make release-dry`, with
|
||||||
|
the reason beside both. **Check the rendered notes, never the exit
|
||||||
|
code.**
|
||||||
|
- **semantic-release core dry-run-pushes to the release branch** as a
|
||||||
|
permission check, independently of any plugin. `PACKAGE_TOKEN` had
|
||||||
|
package-write and repo-*read* — enough to clone, not enough for this —
|
||||||
|
and it failed with a flat `403 Forbidden` that reads exactly like
|
||||||
|
branch protection. It is not: a `--dry-run` push never reaches the
|
||||||
|
pre-receive hook, which a one-line experiment settled. The token needed
|
||||||
|
`write:repository`.
|
||||||
|
- **The floor tag must go on `HEAD^`, not `HEAD`.** Seeded on the merge
|
||||||
|
commit itself it leaves nothing between the floor and HEAD, and
|
||||||
|
semantic-release correctly reports there is nothing to release. The
|
||||||
|
first run did exactly that and cut nothing.
|
||||||
|
- **A tag-triggered workflow runs from the tagged commit's tree.**
|
||||||
|
Moving `v0.0.0` back to `6fb7b5e` ran the *pre-merge* homebrew
|
||||||
|
workflow, which predates the `v0.0.0` skip guard, and pushed a `0.0.0`
|
||||||
|
formula to the public tap. Self-corrected at `0.0.1`. The corollary is
|
||||||
|
general: a guard added today does not protect a tag pointing at
|
||||||
|
yesterday.
|
||||||
|
|
||||||
|
## Two mechanisms confirmed, having been assumptions
|
||||||
|
|
||||||
|
- **A tag pushed with a user PAT does start the `v*` workflows**; one
|
||||||
|
pushed with the Actions token does not (go-gitea#33123). Both halves
|
||||||
|
are load-bearing and both were observed: the floor seed triggered
|
||||||
|
nothing, and the release tag triggered all four.
|
||||||
|
- **Tags are not protected** on this repo, only `main` — which is what
|
||||||
|
lets semantic-release tag at all.
|
||||||
|
|
||||||
|
## Left behind deliberately
|
||||||
|
|
||||||
|
`v0.0.0` stays on `origin` as the floor. It carries no release, and all
|
||||||
|
four publishers skip it by name.
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
# Autotag (v1.3) — MusicBrainz Autotagger
|
# Autotag (v1.3) — MusicBrainz Autotagger
|
||||||
|
|
||||||
|
> **Historical record.** Phases 008–010 shipped, and the scoring engine was subsequently overhauled (`recommend.go`, `rank.go`, `mixedbag.go`), which makes the 011/012 sections below stale in their details. What is actually left is **#90** (auto-accept and entry points) and **#91** (settings, and a way back from the dismissed file-write warning).
|
||||||
|
|
||||||
The MusicBrainz autotagger, collectively **v1.3**. Builds on the explore-browser API client + cache foundation. Five sequential phases (008–012), each depending on the prior one.
|
The MusicBrainz autotagger, collectively **v1.3**. Builds on the explore-browser API client + cache foundation. Five sequential phases (008–012), each depending on the prior one.
|
||||||
|
|
||||||
| Phase | Title | Status |
|
| Phase | Title | Status |
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
# 010 — Owned albums, offline
|
|
||||||
|
|
||||||
**Status:** not started — and **much smaller than when it was written**
|
|
||||||
**Branch:** none yet
|
|
||||||
**Created:** 2026-08-13
|
|
||||||
**Depends on:** nothing
|
|
||||||
**Related:** the `AlbumReleasesFailed` fix that prompted it, and the
|
|
||||||
tag-derived completeness that landed after it (same session)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## What already shipped, and what it leaves
|
|
||||||
|
|
||||||
The common case is solved without this plan. `GetAlbumCompleteness`
|
|
||||||
reads the "5/12" denominator off the files' own tags — persisted to
|
|
||||||
`release_group_recordings.total_tracks`, having been extracted at every
|
|
||||||
scan since forever and discarded — and an album that is **MBID-matched
|
|
||||||
and complete** now opens with **no catalog call at all**. Identity from
|
|
||||||
the MBID, tracklist from the tags; those were the two things the browse
|
|
||||||
was being spent on.
|
|
||||||
|
|
||||||
So the set this plan still has to serve is not "albums you own a track
|
|
||||||
of". It is:
|
|
||||||
|
|
||||||
- albums that are genuinely **incomplete** (the catalog is the only way
|
|
||||||
to say *which* tracks are missing — tags give the count, not the
|
|
||||||
names), and
|
|
||||||
- albums whose tags **never declared a total**, where completeness is
|
|
||||||
unknowable locally and the catalog is the only source.
|
|
||||||
|
|
||||||
On a well-tagged library that is a small minority, which changes the
|
|
||||||
economics below considerably: the run is shorter, and the rate limiter
|
|
||||||
contention that dominates this design is proportionally less severe.
|
|
||||||
Re-measure before building — the answer may now be "the prefetch is
|
|
||||||
enough".
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## The problem
|
|
||||||
|
|
||||||
Opening an album detail page for an album **you already own** hits
|
|
||||||
MusicBrainz. Every time it is not in the response cache, which for most
|
|
||||||
of a library is every time, because nothing warms that cache except a
|
|
||||||
capped prefetch on the artist page.
|
|
||||||
|
|
||||||
The user's framing: *this is a classic example of an album we should
|
|
||||||
have had locally.*
|
|
||||||
|
|
||||||
## Why we do not have it, despite the discography backfill
|
|
||||||
|
|
||||||
`BackfillLibraryDiscographies` / `EnsureArtistDiscography`
|
|
||||||
(`backend/explore/searchindex.go:301`, `:397`) do less than the name
|
|
||||||
suggests. Per artist, `indexOneArtist` fetches:
|
|
||||||
|
|
||||||
- `fetchTopReleaseGroups` — capped at `indexMaxRGs` (50)
|
|
||||||
- `fetchTopRecordings` — capped at `indexMaxRecs` (200)
|
|
||||||
|
|
||||||
and writes them as **flat `explore_index` rows**. There is no release
|
|
||||||
group → tracklist relation anywhere in the index, and no release-level
|
|
||||||
rows at all. `explore_index` recordings carry `caa_release_mbid` and
|
|
||||||
`release_name`, which name the release used for cover art — not a
|
|
||||||
tracklist.
|
|
||||||
|
|
||||||
So "we have full discographies for library artists" means *we know
|
|
||||||
which albums the artist made, offline*. It has never meant we know
|
|
||||||
what is on any of them.
|
|
||||||
|
|
||||||
The only store of release-level catalog data in the app is `http_cache`
|
|
||||||
under `mb:browse:releases:<rg>` (90-day TTL, `musicbrainz.go:27`),
|
|
||||||
populated **only** by a live `BrowseReleases` with
|
|
||||||
`Includes: ["recordings", "media"]` at `MaxLimit` — the most expensive
|
|
||||||
call the app makes to MusicBrainz. It is warmed by exactly one thing:
|
|
||||||
`PrefetchReleases` (`explore.go:746`), capped at 8, called only when an
|
|
||||||
artist page renders.
|
|
||||||
|
|
||||||
An album opened from the library grid therefore always browses live.
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
**A post-scan backfill that warms the release cache for release groups
|
|
||||||
that are owned but not known-complete** — bounded, resumable, and
|
|
||||||
shaped exactly like `BackfillLibraryDiscographies`, which is the proven
|
|
||||||
pattern for this in the codebase.
|
|
||||||
|
|
||||||
The scoping rule is the user's and it is the right one: not "every
|
|
||||||
album by every artist in the library" (50 release groups per artist,
|
|
||||||
mostly never opened) but albums with owned tracks — narrowed further,
|
|
||||||
now, to the ones a local answer cannot already cover. The query gains
|
|
||||||
one clause: skip release groups whose `GetAlbumCompleteness` reports
|
|
||||||
`complete`.
|
|
||||||
|
|
||||||
Sketch:
|
|
||||||
|
|
||||||
1. A query for release groups with ≥1 owned track and no warm release
|
|
||||||
cache entry. `release_groups.mbid` is the key; the owned-track join
|
|
||||||
is `audio_files → recordings → release_group_recordings`, the same
|
|
||||||
shape `unenrichedLibraryArtistMBIDs` already uses one table over.
|
|
||||||
2. Order by owned-track count descending, so the albums the user has
|
|
||||||
most of are warmed first — same reasoning as the discography
|
|
||||||
backfill's ordering, same benefit if a run is cut short.
|
|
||||||
3. Run through `releasesSF`, so it never double-fetches a release group
|
|
||||||
an interactive open is already handling.
|
|
||||||
4. Bound a run (`discogBackfillMaxPerRun` has a value to copy) and make
|
|
||||||
it resumable: the resume marker is the response cache itself —
|
|
||||||
`BrowseReleasesCached` already answers "is this one done", so unlike
|
|
||||||
the discography path this needs **no new flag column**.
|
|
||||||
5. Trigger it where `BackfillLibraryDiscographies` is triggered, and
|
|
||||||
register it with `jobs` so it has progress, pause and cancel like
|
|
||||||
every other long-running operation.
|
|
||||||
|
|
||||||
### The rate limiter is the whole design constraint
|
|
||||||
|
|
||||||
> **Update (2026-08-13): the priority half is built, and the sentence
|
|
||||||
> below is wrong on a detail.** `e.mb` runs on `mbSearchLimiter`
|
|
||||||
> (`NewRateLimiterBurst(3, 1)`); the 1 req/s `NewRateLimiter()` cited
|
|
||||||
> here is the *artist image* limiter. Both are shared and both were
|
|
||||||
> FIFO. `RateLimiter.WithBackgroundLane` + `WithBackgroundPriority(ctx)`
|
|
||||||
> now make a marked caller yield to interactive work and pace at 1/s,
|
|
||||||
> and `jobs.KindCatalogEnrich` + `startBackfillJob` give the existing
|
|
||||||
> backfills progress and cancel. **"Do not start until the priority
|
|
||||||
> question has an answer" is satisfied** — mark this backfill's context
|
|
||||||
> and register it the way `BackfillLibraryDiscographies` now is.
|
|
||||||
> `PrefetchReleases`' cap of 8 is still unrevisited.
|
|
||||||
|
|
||||||
One shared `NewRateLimiter()` at 1 req/s (`explore.go:84`) serves this,
|
|
||||||
`PrefetchReleases`, and every interactive browse. A backfill over a
|
|
||||||
few thousand owned albums is *hours* of wall clock at that rate — which
|
|
||||||
is fine for a background job, and not fine if it starves the album page
|
|
||||||
the user is looking at right now.
|
|
||||||
|
|
||||||
That is the real work in this plan, and it is not the query:
|
|
||||||
|
|
||||||
- Interactive browses need to **jump the queue**. Today they cannot;
|
|
||||||
there is one limiter and it is FIFO.
|
|
||||||
- `PrefetchReleases`' cap of 8 was sized when nothing else competed for
|
|
||||||
the limiter. Revisit it in the same change.
|
|
||||||
- The 60 s fallback the `AlbumReleasesFailed` fix installed is sized
|
|
||||||
for today's contention. If a backfill can queue behind it, that
|
|
||||||
number is wrong again — which is an argument for priority, not for a
|
|
||||||
bigger number.
|
|
||||||
|
|
||||||
Do not start the query until the priority question has an answer.
|
|
||||||
|
|
||||||
## The alternative that was considered and rejected
|
|
||||||
|
|
||||||
**Project release-group tracklists in the dump build and ship them in
|
|
||||||
the artifact.** The data is there: `canonical_musicbrainz_data.csv`
|
|
||||||
carries `release_mbid` *and* `recording_mbid`
|
|
||||||
(`dumpcatalog.go:520`), and `release_to_rg` already maps release →
|
|
||||||
release group. It is derivable from bytes the index build already
|
|
||||||
streams, with no new API surface at all, and it would work offline on
|
|
||||||
first launch with no per-user backfill.
|
|
||||||
|
|
||||||
It is rejected **for this plan** because the artifact is built
|
|
||||||
centrally and is byte-identical for every user, so "albums the user
|
|
||||||
owns a track of" cannot be a filter on it. Shipping tracklists for the
|
|
||||||
whole catalog means per-recording rows against a ~900 MB artifact
|
|
||||||
budget (~426 B/row measured), and gating on a popularity floor means it
|
|
||||||
is absent for exactly the obscure albums a local backfill would have
|
|
||||||
covered.
|
|
||||||
|
|
||||||
Worse than absent, in fact — and this is the argument that actually
|
|
||||||
kills it. The floor is not one number over artists; it is a **per
|
|
||||||
artist track budget** (`dumpcatalog.go:58-89`): 50 tracks for a tier-A
|
|
||||||
artist, 25 for tier B, 12 for tier C. A projected tracklist would
|
|
||||||
therefore be *whichever* of an album's tracks survived that budget,
|
|
||||||
with nothing marking the rest as absent — so the album page would count
|
|
||||||
owned against a truncated denominator and render "Play 7 of 9" for a
|
|
||||||
twelve-track album. That is a confident lie, where the honest states
|
|
||||||
this plan's alternative produces (complete / incomplete / unknown) are
|
|
||||||
at worst silent.
|
|
||||||
|
|
||||||
Note that `markLibraryArtists` (`dumpcatalog.go:246`) already grants
|
|
||||||
every library artist full coverage — 500 tracks, 100 release groups —
|
|
||||||
by reading the local library, so the per-user tailoring this option
|
|
||||||
supposedly cannot have does exist in code. It is a no-op in the CI
|
|
||||||
build (empty library), and reaching it means a **local** dump build:
|
|
||||||
the ~205 GB, half-a-day download the entire artifact design exists to
|
|
||||||
avoid. Whoever finds that function next should read this paragraph
|
|
||||||
before getting excited about it.
|
|
||||||
|
|
||||||
Worth revisiting if the artifact ever gains per-user tailoring, or if a
|
|
||||||
measurement shows the row count is smaller than feared. Note it also
|
|
||||||
yields the *canonical* tracklist rather than MusicBrainz's full version
|
|
||||||
list, so the versions dropdown would still browse live when opened.
|
|
||||||
|
|
||||||
## Done when
|
|
||||||
|
|
||||||
- Opening an owned album that has never been opened before renders its
|
|
||||||
catalog tracklist with no network call, after one backfill run.
|
|
||||||
- An interactive browse issued while the backfill is running is not
|
|
||||||
delayed by it.
|
|
||||||
- The backfill appears in the jobs indicator, and can be paused and
|
|
||||||
cancelled there.
|
|
||||||
- A second run after a completed one does approximately nothing.
|
|
||||||
@@ -1,204 +0,0 @@
|
|||||||
# 016 — What Android parity would actually take
|
|
||||||
|
|
||||||
Plan 015 shipped a *pipeline*: the app cross-compiles, is signed and
|
|
||||||
versioned, and publishes from CI. This is the assessment of what stands
|
|
||||||
between that and an Android app worth installing.
|
|
||||||
|
|
||||||
**The headline: parity is the wrong target, and choosing it would be
|
|
||||||
the expensive mistake.** Four of the blockers below are not porting work
|
|
||||||
— they are the Android platform declining to support the model this app
|
|
||||||
is built on. The decision to make first is in "The fork in the road" at
|
|
||||||
the end; everything before it is evidence for that decision.
|
|
||||||
|
|
||||||
Severity is what the app *does* today, verified against the source and
|
|
||||||
the generated manifest, not guessed.
|
|
||||||
|
|
||||||
## A. It cannot work at all until these are fixed
|
|
||||||
|
|
||||||
### A1. The app can read no music. (deepest)
|
|
||||||
|
|
||||||
`build/android/app/src/main/AndroidManifest.xml` requests INTERNET,
|
|
||||||
VIBRATE, ACCESS_NETWORK_STATE, USE_BIOMETRIC, POST_NOTIFICATIONS, the
|
|
||||||
two location permissions, CAMERA and the two FOREGROUND_SERVICE ones.
|
|
||||||
**There is no storage or media permission of any kind.** At
|
|
||||||
`targetSdk 35` that means the app can see its own private directory and
|
|
||||||
nothing else.
|
|
||||||
|
|
||||||
Adding `READ_MEDIA_AUDIO` is necessary and *not sufficient*, because it
|
|
||||||
grants access through **MediaStore**, not through the filesystem. This
|
|
||||||
app's entire model is absolute paths: `audio_files.file_path` is the
|
|
||||||
primary key of ownership, `AddLibrary(path)` takes a directory, the
|
|
||||||
scanner walks it with `os.ReadDir`, and every one of
|
|
||||||
`GetFilePathsByAlbums` / `ByGenres` / `ByRecordingMBIDs` exists to hand
|
|
||||||
paths to the player. Scoped storage does not offer a stable directory
|
|
||||||
to walk.
|
|
||||||
|
|
||||||
The honest options are three, and they are not close in cost:
|
|
||||||
|
|
||||||
- **MediaStore as the library source.** Query the content resolver,
|
|
||||||
keep MediaStore IDs (or content URIs) beside or instead of paths, and
|
|
||||||
open audio through a `ContentResolver` file descriptor. This is the
|
|
||||||
Android-native answer and it touches the schema, the scanner, the
|
|
||||||
player's file opening and every path-keyed query.
|
|
||||||
- **`MANAGE_EXTERNAL_STORAGE`.** Keeps the path model intact and is
|
|
||||||
effectively barred from Google Play except for genuine file managers.
|
|
||||||
Viable *only* because we distribute through Obtainium — which is a
|
|
||||||
real point in its favour here, and worth stating plainly rather than
|
|
||||||
dismissing.
|
|
||||||
- **App-private storage only**, i.e. the user copies music into the
|
|
||||||
app's sandbox. Trivial to build, and nobody wants it.
|
|
||||||
|
|
||||||
### A2. The first-run flow cannot complete.
|
|
||||||
|
|
||||||
`first-run-wizard.ts` calls `DirectoryPicker()`, which is
|
|
||||||
`frontendutil.DirectoryPicker` → `app.Dialog.OpenFile().
|
|
||||||
CanChooseDirectories(true)`. Wails' own `ANDROID.md` lists open-directory
|
|
||||||
dialogs as **"❌ Returns an error — SAF yields tree URIs, not filesystem
|
|
||||||
paths"**. So the one action the wizard exists to perform fails, and
|
|
||||||
`<first-run-wizard>` intercepts all pointer events until a library
|
|
||||||
exists — so the app is not merely empty, it is inert.
|
|
||||||
|
|
||||||
Whatever A1 resolves to decides this: a MediaStore library needs no
|
|
||||||
picker at all, and a SAF tree needs the picker to return a URI the
|
|
||||||
backend can use.
|
|
||||||
|
|
||||||
### A3. MPRIS is compiled into the Android build.
|
|
||||||
|
|
||||||
`mpris_linux.go` is `//go:build linux`, and **`android` implies
|
|
||||||
`linux`** (documented, and the reason it is in the APK). It will look
|
|
||||||
for a session bus that does not exist. It needs `//go:build linux &&
|
|
||||||
!android`, and its Android counterpart is A4.
|
|
||||||
|
|
||||||
This one is cheap and should be done regardless — it is a two-character
|
|
||||||
build-tag change plus whatever `mediacontrols.New` returns instead.
|
|
||||||
|
|
||||||
### A4. Playback will be killed the moment the screen locks.
|
|
||||||
|
|
||||||
The scaffold's `WailsForegroundService` is typed **`dataSync`**
|
|
||||||
(`foregroundServiceType="dataSync"`, `FOREGROUND_SERVICE_TYPE_DATA_SYNC`),
|
|
||||||
and the manifest requests `FOREGROUND_SERVICE_DATA_SYNC`. A music player
|
|
||||||
needs `mediaPlayback` and `FOREGROUND_SERVICE_MEDIA_PLAYBACK`, plus a
|
|
||||||
`MediaSession` for lock-screen and notification transport controls,
|
|
||||||
plus **audio focus** — pause on a phone call, duck for a notification,
|
|
||||||
pause on headphone unplug. None of that exists today. `oto` will happily
|
|
||||||
keep writing to a stream nobody can hear.
|
|
||||||
|
|
||||||
This is the difference between "an app that plays audio" and "a music
|
|
||||||
player", and it is Java-side work in the scaffold plus a Go-side bridge.
|
|
||||||
|
|
||||||
## B. It works, but wrongly
|
|
||||||
|
|
||||||
### B1. The x86_64 half of the APK cannot run on any Android.
|
|
||||||
|
|
||||||
Established in plan 015: `modernc.org/libc`'s `Xlstat64` issues a raw
|
|
||||||
`lstat` on linux/amd64, which Android's seccomp forbids, so the process
|
|
||||||
takes `SIGSYS` the first time it touches the database. arm64 is
|
|
||||||
structurally unaffected (no `lstat` syscall exists; it routes through
|
|
||||||
`fstatat`).
|
|
||||||
|
|
||||||
So ~31 MB of the artifact is dead weight on *every* Android device,
|
|
||||||
including x86 Chromebooks. Options: drop `x86_64` from `abiFilters`
|
|
||||||
(smaller APK, no emulator target — which does not work anyway), or
|
|
||||||
carry it against a future modernc fix. **Dropping it is the honest
|
|
||||||
default**; it is also the only item in this plan that is a five-minute
|
|
||||||
change.
|
|
||||||
|
|
||||||
### B2. The UI is a desktop shell.
|
|
||||||
|
|
||||||
`MinWidth`/`MinHeight` are 800×600 and were *measured* — below ~780 the
|
|
||||||
header subtitle wraps the title out of its bar. A phone is ~360–430 CSS
|
|
||||||
px wide. The sidebar collapses to icons below 900px, which is a
|
|
||||||
laptop-sized breakpoint, not a phone one. Beyond width: the app is built
|
|
||||||
on hover (the marquee's `hover` mode, tooltips), right-click context
|
|
||||||
menus, a keyboard shortcut layer with its own overlay and settings page,
|
|
||||||
multi-select with ctrl/shift, and a resizable-column track list. None of
|
|
||||||
those are gestures.
|
|
||||||
|
|
||||||
This is not a stylesheet pass. It is a second front end for the views
|
|
||||||
worth having on a phone, sharing the stores and bindings — which the
|
|
||||||
architecture supports, since a view is already a lazily-loaded chunk
|
|
||||||
behind `VIEW_LOADERS`.
|
|
||||||
|
|
||||||
### B3. Tag writing cannot reach the user's files.
|
|
||||||
|
|
||||||
`tagwriter` rewrites tags in place, and autotag's whole purpose is
|
|
||||||
applying them to a folder. Under scoped storage that is impossible
|
|
||||||
outside the sandbox without a SAF write grant per tree. If A1 lands on
|
|
||||||
MediaStore, in-place tag writing needs `MediaStore` write requests and
|
|
||||||
user confirmation per file on Android 11+.
|
|
||||||
|
|
||||||
Autotagging is arguably a desktop-only feature and saying so is a
|
|
||||||
legitimate answer.
|
|
||||||
|
|
||||||
### B4. The Explore catalog is a ~0.6 GB download into app-private storage.
|
|
||||||
|
|
||||||
It works — but with no awareness of a metered connection and no
|
|
||||||
accounting for a device where that is a meaningful fraction of free
|
|
||||||
space. At minimum it needs to be opt-in on mobile and to refuse a
|
|
||||||
metered network by default. `Android.NetworkJSON()` reports
|
|
||||||
`{connected,type}`, so the signal is available.
|
|
||||||
|
|
||||||
## C. Inert, and fine
|
|
||||||
|
|
||||||
Window geometry, menus and the system tray are documented no-ops on
|
|
||||||
mobile. The keyboard shortcut layer is harmless but its Settings page
|
|
||||||
is dead weight. `profiling` is already compiled out of production
|
|
||||||
builds. These cost nothing and need no work.
|
|
||||||
|
|
||||||
## D. Unknown until it runs on a device
|
|
||||||
|
|
||||||
**Nothing in section A or B has been observed on Android**, because the
|
|
||||||
x86_64 emulator cannot run the app (B1) and emulator 37 refuses arm64
|
|
||||||
images on an x86_64 host. Everything above is read from the source, the
|
|
||||||
generated manifest and Wails' own documentation. The first real device
|
|
||||||
run will find things this list does not have, and the most likely
|
|
||||||
places are audio latency and buffering under `oto`/oboe, and SQLite
|
|
||||||
behaviour on app-private storage.
|
|
||||||
|
|
||||||
## The fork in the road
|
|
||||||
|
|
||||||
The four blockers in section A are all the same question wearing
|
|
||||||
different clothes: **is the Android app a librarian, or a player?**
|
|
||||||
|
|
||||||
YellowJacket on the desktop is a *librarian*. It scans folders,
|
|
||||||
deduplicates covers, detects duplicate tracks, reconciles against
|
|
||||||
MusicBrainz, rewrites tags on disk, and manages downloads. That model
|
|
||||||
rests on owning a filesystem, which is precisely what Android declines
|
|
||||||
to give.
|
|
||||||
|
|
||||||
Three coherent products, and only the first is "parity":
|
|
||||||
|
|
||||||
1. **Full librarian on Android.** Requires `MANAGE_EXTERNAL_STORAGE`
|
|
||||||
(Obtainium-only distribution, which we already have), a phone UI for
|
|
||||||
every view, and media-session playback. Largest scope by far; the
|
|
||||||
result is an app almost nobody has asked for on a phone.
|
|
||||||
2. **A player for music already on the phone.** MediaStore as the
|
|
||||||
source, no scanner, no autotag, no downloads; the library, queue,
|
|
||||||
playlists, favourites and Explore-as-browsing all still make sense.
|
|
||||||
This is a genuinely good Android app and it is *not* parity — it is
|
|
||||||
a subset with a different data source.
|
|
||||||
3. **A companion to the desktop app.** The phone browses and controls
|
|
||||||
the desktop's library over the network, or syncs a subset. Smallest
|
|
||||||
Android surface, and it leans on the thing that already works.
|
|
||||||
|
|
||||||
**Option 2 is the recommendation** if the goal is an app people use;
|
|
||||||
option 3 if the goal is the least work for the most value. Option 1 is
|
|
||||||
the only one that answers "feature parity" literally, and it is the one
|
|
||||||
worth arguing hardest against.
|
|
||||||
|
|
||||||
## What is worth doing regardless of that decision
|
|
||||||
|
|
||||||
Cheap, independently useful, and each unblocks measurement:
|
|
||||||
|
|
||||||
1. **Drop `x86_64` from `abiFilters`** (B1) — or keep it and document
|
|
||||||
why. Five minutes.
|
|
||||||
2. **`//go:build linux && !android` on `mpris_linux.go`** (A3), so the
|
|
||||||
Android build stops carrying a D-Bus client. Small.
|
|
||||||
3. **A device smoke run**, which needs someone's phone and the published
|
|
||||||
APK. Everything in D depends on it, and it is the single highest
|
|
||||||
information-per-minute action available.
|
|
||||||
4. **Make the first-run wizard fail legibly** rather than inertly (A2)
|
|
||||||
— the picker's error already routes through `describeError`, but the
|
|
||||||
wizard still blocks pointer events, so an Android user sees a dead
|
|
||||||
screen rather than a sentence. Even under option 3 this is the right
|
|
||||||
behaviour.
|
|
||||||
+52
-16
@@ -1,6 +1,20 @@
|
|||||||
# semantic-release configuration
|
# semantic-release configuration.
|
||||||
# Runs on main branch pushes to auto-determine version from conventional commits.
|
#
|
||||||
# Creates a git tag + GitHub Release draft; a separate workflow builds binaries.
|
# Runs on pushes to main from .gitea/workflows/release.yml: determine the
|
||||||
|
# version from the Conventional Commits since the last tag, write the
|
||||||
|
# changelog, commit it, push the tag, and create the Gitea release.
|
||||||
|
#
|
||||||
|
# **There is no `@semantic-release/github` plugin here and there must not
|
||||||
|
# be.** Gitea's API is `/api/v1` and is not GitHub's surface. The Gitea
|
||||||
|
# community plugin (@saithodev/semantic-release-gitea) was considered and
|
||||||
|
# rejected: last published 2022, depends on got@10, and declares no peer
|
||||||
|
# dependency on semantic-release at all — i.e. untested against anything
|
||||||
|
# since v19, against a core now at v25. `exec` is first-party, current,
|
||||||
|
# and the Gitea-shaped part is one curl.
|
||||||
|
#
|
||||||
|
# The type list below is the one scripts/commit-check.sh enforces the
|
||||||
|
# grammar for — keep the two in step, or semantic-release will silently
|
||||||
|
# decline to release something the commit hook accepted.
|
||||||
branches:
|
branches:
|
||||||
- main
|
- main
|
||||||
|
|
||||||
@@ -63,19 +77,41 @@ plugins:
|
|||||||
section: Build
|
section: Build
|
||||||
hidden: true
|
hidden: true
|
||||||
|
|
||||||
# Write CHANGELOG.md.
|
# Render the notes to a file.
|
||||||
|
#
|
||||||
|
# **This plugin is here to carry the notes, not to maintain a document.**
|
||||||
|
# It is how they reach the Gitea API *without being interpolated into a
|
||||||
|
# shell command*: release notes are rendered commit messages — arbitrary
|
||||||
|
# text carrying backticks, quotes and `$` — so templating
|
||||||
|
# ${nextRelease.notes} into `publishCmd` would be a shell injection with
|
||||||
|
# the commit log as its input. scripts/gitea-release.sh reads the top
|
||||||
|
# section of this file instead, and the only thing interpolated below is
|
||||||
|
# a semver string.
|
||||||
|
#
|
||||||
|
# The target is a gitignored build artifact rather than CHANGELOG.md,
|
||||||
|
# because nothing commits it back — see below.
|
||||||
- - "@semantic-release/changelog"
|
- - "@semantic-release/changelog"
|
||||||
- changelogFile: CHANGELOG.md
|
- changelogFile: .release-notes.md
|
||||||
|
changelogTitle: "# Release notes"
|
||||||
|
|
||||||
# Commit the changelog back to the repo.
|
# Create the Gitea release, whose body is that section.
|
||||||
- - "@semantic-release/git"
|
# `publish` runs after `prepare`, so the tag already exists by here.
|
||||||
- assets:
|
- - "@semantic-release/exec"
|
||||||
- CHANGELOG.md
|
- publishCmd: "./scripts/gitea-release.sh ${nextRelease.version}"
|
||||||
message: "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
|
|
||||||
|
# **There is deliberately no @semantic-release/git here.**
|
||||||
|
#
|
||||||
|
# `main` is a protected branch with `enable_push: false` and an empty
|
||||||
|
# push whitelist, so a changelog commit-back would be rejected by the
|
||||||
|
# pre-receive hook — *after* the tag had already been pushed, leaving a
|
||||||
|
# tagged release the run then reported as failed. The alternative was to
|
||||||
|
# whitelist the CI user, which weakens a protection someone set on
|
||||||
|
# purpose and lets a bot push to main without passing the checks every
|
||||||
|
# human PR has to.
|
||||||
|
#
|
||||||
|
# So the release page is the changelog. Tags are not protected, so the
|
||||||
|
# tag push semantic-release does itself is unaffected. CHANGELOG.md in
|
||||||
|
# the repo is a signpost to the releases page and is not written by any
|
||||||
|
# of this; a file that claimed to be a changelog and silently stopped
|
||||||
|
# updating would be worse than no file at all.
|
||||||
|
|
||||||
# Create the GitHub Release (draft, so the build workflow can attach binaries).
|
|
||||||
- - "@semantic-release/github"
|
|
||||||
- draft: true
|
|
||||||
successComment: false
|
|
||||||
failComment: false
|
|
||||||
releasedLabels: false
|
|
||||||
|
|||||||
+16
-372
@@ -1,377 +1,21 @@
|
|||||||
## [1.3.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.3...v1.3.0) (2026-03-20)
|
# Changelog
|
||||||
|
|
||||||
### Features
|
The changelog is the releases page:
|
||||||
|
|
||||||
* **09-01:** add scan control events and cancelled metrics field ([c695024](https://github.com/onion-4-dinner/yellowjacket/commit/c695024241a7513b8fedb3fbf7ff364d0515b392))
|
<https://git.ljones.me/yonlu/yellowjacket/releases>
|
||||||
* **09-01:** add scan control fields and per-scan cancellable context ([cf22e52](https://github.com/onion-4-dinner/yellowjacket/commit/cf22e52a64850a80b9fcc63c21d81313e6bd56ab))
|
|
||||||
* **09-02:** add frontend keyboard shortcut service, store, and controller ([40d4815](https://github.com/onion-4-dinner/yellowjacket/commit/40d48151dd798b57eed9f54a572ae4735356d09e))
|
|
||||||
* **09-02:** add shortcuts config package with default bindings and Wails persistence ([6285ca9](https://github.com/onion-4-dinner/yellowjacket/commit/6285ca9dc4e6f211197e377d01c485b1ef65c300))
|
|
||||||
* **09-03:** add scan control UI with pause/resume/cancel and confirmation dialog ([3914369](https://github.com/onion-4-dinner/yellowjacket/commit/391436927c826f2f17a4523be7829aefc04a6b12))
|
|
||||||
* **09-04:** add keyboard shortcuts section to config page with conflict detection ([0451fb3](https://github.com/onion-4-dinner/yellowjacket/commit/0451fb38805ff2c27e43deb152daa892e733d2db))
|
|
||||||
* **10-01:** implement migration 6 and pre-migration backup ([1179f56](https://github.com/onion-4-dinner/yellowjacket/commit/1179f56c3680112692e71e8dc7ce946446fa8a8a))
|
|
||||||
* **10-01:** update SQL schema files for multi-library fresh installs ([535855b](https://github.com/onion-4-dinner/yellowjacket/commit/535855b383a457dd2be3298b4361313bef22b39d))
|
|
||||||
* **10-02:** add migration 6 integration tests and NewTestDBWithLibrary helper ([bc15189](https://github.com/onion-4-dinner/yellowjacket/commit/bc151891b50e59e41da2e00dbfafbecaad11b4ac))
|
|
||||||
* **10-02:** add sqlc queries for libraries and update playlist queries for phantom support ([02548dd](https://github.com/onion-4-dinner/yellowjacket/commit/02548dd55e59b28f3d6c8d9614f209140c979250))
|
|
||||||
* **11-01:** per-library scan pipeline with queue coordinator ([943db1c](https://github.com/onion-4-dinner/yellowjacket/commit/943db1cf274bdf59daf28ab6c20f78ef5ef53105))
|
|
||||||
* **11-02:** update config-page with per-library progress display and queue-aware cancel dialog ([d01591d](https://github.com/onion-4-dinner/yellowjacket/commit/d01591d6cc054a63b832c05a3164a72fdcaba342))
|
|
||||||
* **11-02:** update library-manager with per-library progress and Scan All button ([d61f122](https://github.com/onion-4-dinner/yellowjacket/commit/d61f122b567e8ac2b30fa96c637cbebc14493c89))
|
|
||||||
* **12-01:** add queue compaction method and wire removal hooks ([5995dfd](https://github.com/onion-4-dinner/yellowjacket/commit/5995dfd01d61cd4d2c0749eeeee2a1f93b739d68))
|
|
||||||
* **12-01:** implement library CRUD methods and orphan cleanup pipeline ([bd44f83](https://github.com/onion-4-dinner/yellowjacket/commit/bd44f8306c9129b9420ad81938bcf8105a1cb55a))
|
|
||||||
* **12-02:** make config sections collapsible with chevron dropdown ([12c6782](https://github.com/onion-4-dinner/yellowjacket/commit/12c678284c7582bd85cd52722f4d405b0bd0e20f))
|
|
||||||
* **12-02:** remove Libraries sidebar nav item and view routing ([e199712](https://github.com/onion-4-dinner/yellowjacket/commit/e199712a56e1cb3c0fc43d3340abb892a6f5fa7b))
|
|
||||||
* **12-02:** replace config-page library section with full library management UI ([ffc5d96](https://github.com/onion-4-dinner/yellowjacket/commit/ffc5d9639cf7c916a4f846590ae0d67cf13afe27))
|
|
||||||
* **12-02:** selectable library list with checkbox scan targeting ([13a42ae](https://github.com/onion-4-dinner/yellowjacket/commit/13a42aea2287d7ed0ec9ff9856f52c1fa7767338))
|
|
||||||
* **12-02:** show scan progress bar inline in library list entry ([df824c6](https://github.com/onion-4-dinner/yellowjacket/commit/df824c6989e92b2aefaa1ddf05b131ee319612d8))
|
|
||||||
* **13-01:** add library-filtered Go query methods and FTS search ([5f7de50](https://github.com/onion-4-dinner/yellowjacket/commit/5f7de5060a5bc557b96203267de694ef366ed507))
|
|
||||||
* **13-01:** add library-filtered sqlc queries for all browse views ([5cc58ce](https://github.com/onion-4-dinner/yellowjacket/commit/5cc58ce66ab70d8d5a570df5067f79ae2201037e))
|
|
||||||
* **13-02:** add library filter dropdown and wire all views to respect active filter ([42b8cf9](https://github.com/onion-4-dinner/yellowjacket/commit/42b8cf9f52133499ffcd7363bd39dd0c1069e091))
|
|
||||||
* **15-01:** migrate FTS5 search_index to contentless_delete=1 ([cb5155b](https://github.com/onion-4-dinner/yellowjacket/commit/cb5155b8906357ff77c5c579d57d02cf2eec6abe))
|
|
||||||
* **15-02:** create backend/fileutil package with AtomicWrite ([4d64b5d](https://github.com/onion-4-dinner/yellowjacket/commit/4d64b5dcfe43951e8ec63383bbf72c99107c63c4))
|
|
||||||
* **16-01:** add selectAll() to SelectionController and dispatch shortcut:select-all event ([f567762](https://github.com/onion-4-dinner/yellowjacket/commit/f5677628ef283b67370630b564f23178e43da3d2))
|
|
||||||
* **16-01:** wire shortcut:select-all listener in track-list, queue-panel, and playlist-view ([906ea28](https://github.com/onion-4-dinner/yellowjacket/commit/906ea28751ce9f96fdeeb9410ab5f6518f09fcb9))
|
|
||||||
* **16-02:** add go-flac dependencies and implement FLAC tag writer ([3642cbe](https://github.com/onion-4-dinner/yellowjacket/commit/3642cbe0d58f8912a786a4fc5380c40403add94a))
|
|
||||||
* **16-03:** implement DB sync module for tag write pipeline ([2966079](https://github.com/onion-4-dinner/yellowjacket/commit/2966079625cd42412411429af02184d015526e9b))
|
|
||||||
* **16-03:** WriteTrackTags pipeline with player safety, scan mutex, events, and app wiring ([64322f9](https://github.com/onion-4-dinner/yellowjacket/commit/64322f93538515d5a3e486dc14691b9c9dcf6f66))
|
|
||||||
* **17-01:** add TrackMetadataChanged handler and remove selection gate on Track Details ([fc5cf70](https://github.com/onion-4-dinner/yellowjacket/commit/fc5cf70e4c1be3d3f1545c140db5202601a08109))
|
|
||||||
* **17-01:** add WriteTrackTagsByPath and ImageFilePicker backend methods ([4235b4a](https://github.com/onion-4-dinner/yellowjacket/commit/4235b4a4d555882ce86628a88dd4e4eeee2c9097))
|
|
||||||
* **17-02:** implement save flow, cover art editing, and error handling ([265a9ea](https://github.com/onion-4-dinner/yellowjacket/commit/265a9ea8ceba893f956a03546e9ac4189adc7716))
|
|
||||||
* **18-01:** add BatchWriteProgress event constant ([3dba0e1](https://github.com/onion-4-dinner/yellowjacket/commit/3dba0e143c091327d305d39d2fa7a687ec47e172))
|
|
||||||
* **18-01:** add BatchWriteTrackTags with progress, cancellation, and partial failure ([f557ffd](https://github.com/onion-4-dinner/yellowjacket/commit/f557ffd652179b7cf8f8ff4a06824f30edf08007))
|
|
||||||
* **18-02:** add batch edit mode to track-details component ([6dab32b](https://github.com/onion-4-dinner/yellowjacket/commit/6dab32b36b497d54e8645e969aa79737ad3523ab))
|
|
||||||
* **18-02:** wire batch track-details to all 4 view context menus ([656985a](https://github.com/onion-4-dinner/yellowjacket/commit/656985add92663440baebb871f8cd6d5723117fd))
|
|
||||||
* **19-01:** implement WAV RIFF parser/writer and writeWavTags ([e6610ff](https://github.com/onion-4-dinner/yellowjacket/commit/e6610ff15e041213b6898ad48ff63b7060b312e7))
|
|
||||||
* **20-01:** implement OGG Vorbis tag writer with custom page parser and CRC32 ([5e98c03](https://github.com/onion-4-dinner/yellowjacket/commit/5e98c036342b9e174abdc6d00db21c2e2901f18b))
|
|
||||||
* **quick-17:** create playlist-details subpage component ([dc5c7d6](https://github.com/onion-4-dinner/yellowjacket/commit/dc5c7d6ca6cfbfac15546c048f1b33aaf47209c6))
|
|
||||||
* **quick-18:** replace track-info with multi-column grid layout in playlist-details ([ce23177](https://github.com/onion-4-dinner/yellowjacket/commit/ce2317722870f932792dc6456a63235ff4611466))
|
|
||||||
|
|
||||||
### Bug Fixes
|
Every release there is generated from the Conventional Commits it
|
||||||
|
contains, by `.gitea/workflows/release.yml` on merge to `main`. Each one
|
||||||
|
carries its notes as its body, grouped by change type, with a link to the
|
||||||
|
commit behind every line.
|
||||||
|
|
||||||
* **09-05:** emit VolumeChanged event and persist state in ChangeVolume and MuteToggle ([bb3fd20](https://github.com/onion-4-dinner/yellowjacket/commit/bb3fd204f0895f357a14479b40754f397aae74c4))
|
**This file is not generated and is not a copy of that.** `main` is a
|
||||||
* **10-01:** move library_id index to migration 6 to fix existing DB startup ([75b2a34](https://github.com/onion-4-dinner/yellowjacket/commit/75b2a349ebd6fada5cbc92bfae9854cc2cd53c63))
|
protected branch, so nothing pushes a changelog commit back to it — and a
|
||||||
* **12-02:** claim orphaned tracks when adding library with matching path ([f60b6b5](https://github.com/onion-4-dinner/yellowjacket/commit/f60b6b525546ef77a3329fe92f03f336b7435a0e))
|
file that claimed to be a changelog while silently never updating would
|
||||||
* **12-02:** count failed saves as skipped so scan progress bar advances ([b36e472](https://github.com/onion-4-dinner/yellowjacket/commit/b36e472212957ff089f4f5d35f3978a754e23502))
|
be worse than no file at all. `make release-dry` prints what the next
|
||||||
* **12-02:** delete artist_credit_artist before artist_credit in removal pipeline ([890284d](https://github.com/onion-4-dinner/yellowjacket/commit/890284ddb1d0fb95e423bddf27b40fb0db2d11e5))
|
merge would release.
|
||||||
* **12-02:** dismiss inline rename on click outside ([9272b06](https://github.com/onion-4-dinner/yellowjacket/commit/9272b060bf98118e37f19a8c0834034691bfe6a2))
|
|
||||||
* **12-02:** downgrade per-file save error to Debug, add warning count to scan summary ([cf18c39](https://github.com/onion-4-dinner/yellowjacket/commit/cf18c39dbd849d60218228cf1d2285ab2071e788))
|
|
||||||
* **12-02:** invalidate library store cache on LibraryRemoved event ([b093fbb](https://github.com/onion-4-dinner/yellowjacket/commit/b093fbb10a24054c4ef62b0bd13f28d9bfe6f121))
|
|
||||||
* **12-02:** keep Add Library button visible during scan ([649e516](https://github.com/onion-4-dinner/yellowjacket/commit/649e516aa30090665e9f10e89c1ccce378e36b96))
|
|
||||||
* **12-02:** move Add Library button inline with scan buttons ([771345d](https://github.com/onion-4-dinner/yellowjacket/commit/771345dd9d3870b3a907e1cce09c7456ab7ccd85))
|
|
||||||
* **12-02:** move scan buttons above library list, default to none selected ([ba3f840](https://github.com/onion-4-dinner/yellowjacket/commit/ba3f840a28fe2c6ca40c558305814d29c233d6e0))
|
|
||||||
* **12-02:** refresh library track counts after scan completes ([1f872aa](https://github.com/onion-4-dinner/yellowjacket/commit/1f872aa005a9405d9bc1f64a4b1dd2f1f1d4a16c))
|
|
||||||
* **12-02:** reorder orphan cleanup to delete FK children before recordings ([1d735c3](https://github.com/onion-4-dinner/yellowjacket/commit/1d735c3a5f5a78996d6ddbe5c787adf040fe2f21))
|
|
||||||
* **12-02:** replace removed Scan() import with ScanAllLibraries() ([0559822](https://github.com/onion-4-dinner/yellowjacket/commit/05598224e4d5532d2e2a3a7e5d3b5411240b1024))
|
|
||||||
* **12-02:** resolve phantom tracks caused by empty library root after TOML cleanup ([717e249](https://github.com/onion-4-dinner/yellowjacket/commit/717e249c368fd1cc8d5c8f945c352175708691cf))
|
|
||||||
* **12-02:** serialize ScanWarning.Err as string instead of error interface ([ac8cbb3](https://github.com/onion-4-dinner/yellowjacket/commit/ac8cbb3296bd561a305627668c211dce7209df25))
|
|
||||||
* **12-02:** soft scan claims orphaned library_id=0 tracks on startup ([1ad099a](https://github.com/onion-4-dinner/yellowjacket/commit/1ad099a9d35fc722475e238d3443fd5473566acd))
|
|
||||||
* **12-02:** soft scan on launch — only scan libraries with changed file counts ([92c4d23](https://github.com/onion-4-dinner/yellowjacket/commit/92c4d23a9a1e545fab497816ee3dce43a181cded))
|
|
||||||
* **12-02:** wait for scan to stop before library removal, surface errors in UI ([cf00498](https://github.com/onion-4-dinner/yellowjacket/commit/cf004986c95732d00208e83467267904ea3f2ef6))
|
|
||||||
* **13-02:** auto-resolve phantom playlist tracks after library scan ([93262b9](https://github.com/onion-4-dinner/yellowjacket/commit/93262b9ae0f737d2893839ac585776207b3b44b6))
|
|
||||||
* **13-02:** defer virtualizer event delegation until element exists ([f05d2bb](https://github.com/onion-4-dinner/yellowjacket/commit/f05d2bb603f5ea827164466fd0795a6c6e662529))
|
|
||||||
* **13-02:** resolve phantom playlist tracks using M3U8 paths after scan ([9f595b7](https://github.com/onion-4-dinner/yellowjacket/commit/9f595b7ac10c2191b5469004901cbbc1331c1abb))
|
|
||||||
* **14-01:** downgrade main-panel from contain:strict to layout+style+paint ([4b7d35d](https://github.com/onion-4-dinner/yellowjacket/commit/4b7d35d7ec4c8b14453a8f8250cd154b8c4c2537))
|
|
||||||
* **14-perf:** fix scroll jumping and input latency ([3b2e189](https://github.com/onion-4-dinner/yellowjacket/commit/3b2e189e7d0e6d00393d087565190fd307774257))
|
|
||||||
* **17-02:** fix cover art replace and remove ([d7c2965](https://github.com/onion-4-dinner/yellowjacket/commit/d7c2965752ae0ac9009d00f2431d5919a24558b7))
|
|
||||||
* **17-02:** handle float64 numeric values from Wails JSON deserialization ([900db2e](https://github.com/onion-4-dinner/yellowjacket/commit/900db2e56cca254873a3a5a7a384008feac4211b))
|
|
||||||
* **17-02:** refresh cover art URLs after save ([8cd4914](https://github.com/onion-4-dinner/yellowjacket/commit/8cd4914842f61c0c6b49e0216c7816e201a3c94a))
|
|
||||||
* **17-02:** refresh track-details dialog data after successful save ([ffcdc41](https://github.com/onion-4-dinner/yellowjacket/commit/ffcdc41b0d4fad8ed428dbaa55f6cdd38c096822))
|
|
||||||
* **18-02:** add field labels above title/artist/album inputs in batch edit mode ([9df2d67](https://github.com/onion-4-dinner/yellowjacket/commit/9df2d6764a0b0566dda33cff675debea4a61dea8))
|
|
||||||
* **18-02:** add field labels to all track-details states (single/batch, read/edit) ([d430ad8](https://github.com/onion-4-dinner/yellowjacket/commit/d430ad884bfd38bea93389d8be730ff00388a7be))
|
|
||||||
* **19-01:** add album_artist TPE2 mapping to applyTextChanges ([8f4c4a0](https://github.com/onion-4-dinner/yellowjacket/commit/8f4c4a0c2b14eeeaeccb972a40addb11f3d65437))
|
|
||||||
* preserve scroll position in cached grid views ([54df917](https://github.com/onion-4-dinner/yellowjacket/commit/54df917ffdd69c4f7ffaeccf2d161261ca80d84e))
|
|
||||||
* **queue-panel:** set flow layout _itemSize to match actual track item height ([288d9de](https://github.com/onion-4-dinner/yellowjacket/commit/288d9deae22d437fcd7857b368827db7b62c24f6))
|
|
||||||
* **queue-panel:** suppress virtualizer scroll corrections during scrollbar drag ([0bd8cef](https://github.com/onion-4-dinner/yellowjacket/commit/0bd8cefa00dcae2f8bd9579de2aefd58e0a9e6c9))
|
|
||||||
* **quick-19:** multi-root path resolution for playlist M3U8 tracks ([9144ded](https://github.com/onion-4-dinner/yellowjacket/commit/9144dedc2742925dc252d491763b4f2929238d0e))
|
|
||||||
* **S21/T01:** fix all lint warnings and upgrade wsl to wsl_v5 ([f16157a](https://github.com/onion-4-dinner/yellowjacket/commit/f16157a2134cbeb1787ff851d4875d77f2f3f86b))
|
|
||||||
|
|
||||||
### Performance
|
History before `v0.0.1` is in `git log`. The versions before it were cut
|
||||||
|
by hand and are not on the releases page; the entries this file used to
|
||||||
* **12-02:** increase scan batch size from 50 to 300 ([21ea71e](https://github.com/onion-4-dinner/yellowjacket/commit/21ea71e2575d76258bd81d89ab8ac883aa3bed36))
|
hold were generated against a GitHub remote this project no longer has,
|
||||||
* **12-02:** skip FTS5 rebuild during library removal ([30f4461](https://github.com/onion-4-dinner/yellowjacket/commit/30f4461e6957e20d3dc607fa0886a75b5c21b3cf))
|
and every link in them was dead.
|
||||||
* **14-01:** add CSS containment to app shell layout boundaries ([efa06f7](https://github.com/onion-4-dinner/yellowjacket/commit/efa06f7edf1e4acdc3d8865cad264403257ae40d))
|
|
||||||
* **14-01:** add GPU promotion and containment to all scroll containers ([ac8a52e](https://github.com/onion-4-dinner/yellowjacket/commit/ac8a52e110f9f8ebdc3433b60594370352126a18))
|
|
||||||
* **14-02:** replace innerHTML navigation with view caching system ([ad91043](https://github.com/onion-4-dinner/yellowjacket/commit/ad9104374a628342e0ea30cf409ff43de2c2f86e))
|
|
||||||
* **14-03:** add notification batching to queue store and granular change tracking to library store ([d0c05dc](https://github.com/onion-4-dinner/yellowjacket/commit/d0c05dc1d43a4fe12cc07f3cff25375b08a74ba0))
|
|
||||||
* **14-03:** eliminate per-item closure allocation in scroll render paths ([2f7ed70](https://github.com/onion-4-dinner/yellowjacket/commit/2f7ed7030425ed0ebb7a1a186917a79a7b26b850))
|
|
||||||
* **14-04:** RAF-throttle scroll position saves and add overflow-anchor to queue panel ([6ca0b3c](https://github.com/onion-4-dinner/yellowjacket/commit/6ca0b3c5a84769af064ebe45a6eaac014d1a270a))
|
|
||||||
* auto-detect NVIDIA+Wayland for DMABuf workaround ([915591a](https://github.com/onion-4-dinner/yellowjacket/commit/915591aea962beb60da2e96ac0f57307f646f675))
|
|
||||||
* inline SVGs, memoize grid slices, batch store notifications ([a4eac39](https://github.com/onion-4-dinner/yellowjacket/commit/a4eac394cebefd29d0ebcb4b1e331444dcb8fbaf))
|
|
||||||
* reduce software rendering overhead for NVIDIA+Wayland ([199c910](https://github.com/onion-4-dinner/yellowjacket/commit/199c91013fd806f6aefce49357df8a32b46faaa0))
|
|
||||||
|
|
||||||
### Refactoring
|
|
||||||
|
|
||||||
* **quick-17:** simplify playlist-view to navigate instead of expand ([955cd68](https://github.com/onion-4-dinner/yellowjacket/commit/955cd68be2dbf7a9071ef1c93084d687b59b6bd7))
|
|
||||||
|
|
||||||
## [1.2.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.1...v1.2.2) (2026-03-06)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* recover from go-mp3 seek panic on startup ([#86](https://github.com/onion-4-dinner/yellowjacket/issues/86)) ([2f9d9f8](https://github.com/onion-4-dinner/yellowjacket/commit/2f9d9f8508b90b6188fe894c282c5b8e330e8046))
|
|
||||||
|
|
||||||
## [1.2.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.0...v1.2.1) (2026-03-06)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **deps:** pin go-webview2 to v1.0.21 for Wails v2 compat ([25f0fe8](https://github.com/onion-4-dinner/yellowjacket/commit/25f0fe81560eeff36a0b2beb52ce1bdf13d5e122))
|
|
||||||
|
|
||||||
## [1.2.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.3...v1.2.0) (2026-03-06)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **02-02:** add ScanWarning type and reclassify scan errors as warnings ([e6866de](https://github.com/onion-4-dinner/yellowjacket/commit/e6866ded9dc0ea30ff942cd31b6c5ea3269e9584))
|
|
||||||
* **03-01:** create NewTestDB helper for in-memory SQLite test databases ([bae9d70](https://github.com/onion-4-dinner/yellowjacket/commit/bae9d70d23157ef4e79e60dd713d9a02ab63790b))
|
|
||||||
* **03-01:** extract shared applyPRAGMAs and add production PRAGMAs to NewDB ([d348815](https://github.com/onion-4-dinner/yellowjacket/commit/d34881530adda7fb75be84737798da46d17bfa8c))
|
|
||||||
* **06-01:** create track_metadata VIEW schema and migration 4 ([9c7e5a9](https://github.com/onion-4-dinner/yellowjacket/commit/9c7e5a96344a81bf132de487b4763f1dc3ff6df9))
|
|
||||||
* **06-02:** create Go→TypeScript event constant codegen tool ([3e9edd0](https://github.com/onion-4-dinner/yellowjacket/commit/3e9edd05e87395499ac24e456640d1f6d9b97f04))
|
|
||||||
* **06-03:** migrate lookupChunk to sqlc-generated LookupTrackMetaByPaths query ([2221a68](https://github.com/onion-4-dinner/yellowjacket/commit/2221a68459850a837c996c6e6d2bc95d41b20fb3))
|
|
||||||
* **08-01:** define design token CSS custom properties for icon sizes and type scale ([1444a66](https://github.com/onion-4-dinner/yellowjacket/commit/1444a66bb201ce5fdf16552a32bcd281089c64ed))
|
|
||||||
* **08-04:** apply design tokens to cover-grid, track-list, queue-panel, and detail components ([1303422](https://github.com/onion-4-dinner/yellowjacket/commit/1303422e69c27d528363900b3ca5287a48cc9f8e))
|
|
||||||
* **08-04:** convert sidebar em-based spacing to px and apply icon/type tokens ([aed90d7](https://github.com/onion-4-dinner/yellowjacket/commit/aed90d7b1710d0c5cece2e4956c0a6ce77b9a999))
|
|
||||||
* add scan progress bar with phase indicator ([a28b4d1](https://github.com/onion-4-dinner/yellowjacket/commit/a28b4d1e0673658824750d4c702359321dc9a78e))
|
|
||||||
* **quick-001:** add multi-file picker and batch import support ([c34e4ad](https://github.com/onion-4-dinner/yellowjacket/commit/c34e4ad029c119bff8f70a07ccc6bca58b11ea3c))
|
|
||||||
* **quick-001:** regenerate bindings and update frontend for multi-import ([2a542bf](https://github.com/onion-4-dinner/yellowjacket/commit/2a542bf3bcdc7772edb1aceb41f488774494f656))
|
|
||||||
* **quick-002:** add CountPlaylistsByName SQL query and regenerate sqlc ([04b2088](https://github.com/onion-4-dinner/yellowjacket/commit/04b2088b28b84a4d4df25b23d97112c5a955dff1))
|
|
||||||
* **quick-002:** add uniquePlaylistName helper and wire into ImportPlaylist ([8ba8bbe](https://github.com/onion-4-dinner/yellowjacket/commit/8ba8bbe7bed2ecff97613ebaa42a49a662050353))
|
|
||||||
* **quick-006:** remove list icon from playlists, add favorites icon to default ([3c19766](https://github.com/onion-4-dinner/yellowjacket/commit/3c19766fd0885d4171cf9929db6d69a3d5c1a3ff))
|
|
||||||
* **quick-11:** add configurable log level via YJ_LOG_LEVEL env var ([55b4902](https://github.com/onion-4-dinner/yellowjacket/commit/55b4902fac7b7f2c04ad5efac398ecedc5fedc2f))
|
|
||||||
* **quick-11:** add make dev-debug target for verbose logging ([c45bca4](https://github.com/onion-4-dinner/yellowjacket/commit/c45bca411ba1d4f32deea6027acf91237173dd15))
|
|
||||||
* **quick-12:** add favorite icon to album dropdown track rows ([12a0bbc](https://github.com/onion-4-dinner/yellowjacket/commit/12a0bbc89c19128485d597a61bd16bd0786450ad))
|
|
||||||
* **quick-15:** add BufferedStreamer with goroutine read-ahead ([85b23ac](https://github.com/onion-4-dinner/yellowjacket/commit/85b23acb24a048d2f7b85808e477bb991ae124e6))
|
|
||||||
* **quick-15:** insert BufferedStreamer into player pipeline and increase speaker buffer ([8a0b16a](https://github.com/onion-4-dinner/yellowjacket/commit/8a0b16a4ec08a95bfd3834c8216e21dce854432d))
|
|
||||||
* **quick-3:** add playlist-level multi-select state and selection handling ([e13151f](https://github.com/onion-4-dinner/yellowjacket/commit/e13151ffa5dc86e41ce242421679d65a740c3af0))
|
|
||||||
* **quick-3:** wire playlist context menu for batch delete of selected playlists ([c92ced2](https://github.com/onion-4-dinner/yellowjacket/commit/c92ced2c74e72bfc123c880c047462dc969cde34))
|
|
||||||
* **quick-4:** add 'Set as Default Playlist' context menu option ([9971b63](https://github.com/onion-4-dinner/yellowjacket/commit/9971b635b81fe3f8621c80a6664eccb3e1fc4bb8))
|
|
||||||
* **quick-5:** add CreatedAt/UpdatedAt to playlist Summary struct ([bdaff47](https://github.com/onion-4-dinner/yellowjacket/commit/bdaff478e802ee5c0745327c52dd9b190fcfef7d))
|
|
||||||
* **quick-5:** add sort dropdown UI and client-side sorting to playlist view ([5c07485](https://github.com/onion-4-dinner/yellowjacket/commit/5c074855351f1363cc7918837a78bbd3c0b7ebf5))
|
|
||||||
* **quick-7:** add PinDefault config field with backend getter/setter ([6e123bd](https://github.com/onion-4-dinner/yellowjacket/commit/6e123bd47f55e6d565f20bf7f19950e65f80787f))
|
|
||||||
* **quick-7:** wire frontend pin-default-playlist feature end-to-end ([e6378e1](https://github.com/onion-4-dinner/yellowjacket/commit/e6378e1f0d3b0f2a7604b8ef6097dba9050cdd16))
|
|
||||||
* **quick-8:** add FindDuplicateTracksInPlaylist backend method ([83de934](https://github.com/onion-4-dinner/yellowjacket/commit/83de934c39ca7d850a8b5925c90e6d0b3fe0a487))
|
|
||||||
* **quick-8:** create duplicate-tracks-dialog component ([9f3ba2b](https://github.com/onion-4-dinner/yellowjacket/commit/9f3ba2b9d474fa30dcb4934b01d4650e0d0d3cba))
|
|
||||||
* **quick-8:** wire duplicate detection into playlist-picker and playlist-view ([917a79a](https://github.com/onion-4-dinner/yellowjacket/commit/917a79a8d6e30dddd2170323bb26692386794872))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **01-01:** add mutex protection to Queue, Library, and Playlist SetContext methods ([daaa6b7](https://github.com/onion-4-dinner/yellowjacket/commit/daaa6b7f9779385979fe9dddae4e7bb388b3e5fb))
|
|
||||||
* **01-01:** collapse Player.SetContext double-lock into single acquisition ([3abaeba](https://github.com/onion-4-dinner/yellowjacket/commit/3abaeba3afb0f4d0edb81e26ca55b31bf59990ac))
|
|
||||||
* **02-01:** eliminate package-level startupErr and fix config file permissions ([2a86408](https://github.com/onion-4-dinner/yellowjacket/commit/2a864082017e489ffa086c136f1002277a77a7c4))
|
|
||||||
* **02-01:** log MPRIS callback errors instead of discarding them ([0860b2f](https://github.com/onion-4-dinner/yellowjacket/commit/0860b2fd4b2250da1eeb80c21f14fdf341697501))
|
|
||||||
* **08-02:** revert repeat() inside lit-virtualizer, restore .renderItem + .keyFunction ([72ef719](https://github.com/onion-4-dinner/yellowjacket/commit/72ef719ba70eeca0fa4bae47df092706f6fbaeed))
|
|
||||||
* drop+recreate contentless FTS5 index instead of DELETE ([8e9a616](https://github.com/onion-4-dinner/yellowjacket/commit/8e9a61603779eacbee7013b9bc760b315baf782a))
|
|
||||||
* **frontend:** reposition search indicator into toolbar and fix album cover art lookup ([a29137b](https://github.com/onion-4-dinner/yellowjacket/commit/a29137b2ba4c6b33ce9a5f868cbd6013e0e3b116))
|
|
||||||
* include full track metadata in GetAudioFilesByReleaseGroup query ([97f256d](https://github.com/onion-4-dinner/yellowjacket/commit/97f256d67f463d752f7adc5b400c4bf34eae1df1))
|
|
||||||
* **quick-10:** add migration 5 and fix entity cache for composite album key ([d43ba7b](https://github.com/onion-4-dinner/yellowjacket/commit/d43ba7bd0c7ace2a9ed71990a19498f8e9f90751))
|
|
||||||
* **quick-10:** update release_groups schema and queries for composite uniqueness ([999ab96](https://github.com/onion-4-dinner/yellowjacket/commit/999ab967beb9107a3f30ba287acbffad22f0b0de))
|
|
||||||
* **quick-13:** resolve lint issues in main source files ([e1a95e6](https://github.com/onion-4-dinner/yellowjacket/commit/e1a95e65a9f0f436b2e2d92befa9c881b6e8e430))
|
|
||||||
* **quick-14:** add roll-back-on-failure to queue index advancement ([2820de2](https://github.com/onion-4-dinner/yellowjacket/commit/2820de2510560fcd6d1015c18542d5ac30468247))
|
|
||||||
* **quick-9:** set fixed height on queue track items for stable virtualizer scroll ([ebde5e5](https://github.com/onion-4-dinner/yellowjacket/commit/ebde5e5a8bc4da8f40bef8f171c7ed86c213a336))
|
|
||||||
|
|
||||||
### Performance
|
|
||||||
|
|
||||||
* **07-01:** add incremental persistence helpers for queue mutations ([cdd17db](https://github.com/onion-4-dinner/yellowjacket/commit/cdd17db27509908514c21517631306655a2b3bd7))
|
|
||||||
* **07-01:** eliminate redundant lookups in SetQueue Phase 2 ([ced58fe](https://github.com/onion-4-dinner/yellowjacket/commit/ced58fe6a93d6f220137562b8ff09ffc33c69266))
|
|
||||||
* **07-02:** defer eagerFetch to after DOM ready for instant app shell ([cd98ad6](https://github.com/onion-4-dinner/yellowjacket/commit/cd98ad6dc8c2e4e6e0f01a48099b0c0511bf5a98))
|
|
||||||
* **08-01:** add queueMicrotask coalescing to library store and debounce search input ([3bf66ed](https://github.com/onion-4-dinner/yellowjacket/commit/3bf66ed125ed55bfbde95b0bc973710c2f2243b8))
|
|
||||||
* **08-02:** migrate cover-grid, artists-view, and genres-view virtualizers to repeat() directive ([1c3514d](https://github.com/onion-4-dinner/yellowjacket/commit/1c3514da1d0491b9758d7a6f9f72d59ef78fc8ed))
|
|
||||||
* **08-02:** migrate track-list and queue-panel virtualizers to repeat() directive ([d2d7d8c](https://github.com/onion-4-dinner/yellowjacket/commit/d2d7d8c6ce22923772cae4858b02804d15f74bb7))
|
|
||||||
* **08-03:** optimize column rendering and apply classMap to queue-panel renderTrackItem ([62f41c2](https://github.com/onion-4-dinner/yellowjacket/commit/62f41c24910632b270f9f5765e20e48db4b95ec9))
|
|
||||||
* **08-03:** replace class string construction with classMap directive in renderTrackRow ([ad21027](https://github.com/onion-4-dinner/yellowjacket/commit/ad210278fc20729dc76390e6bba9bff050549046))
|
|
||||||
|
|
||||||
### Refactoring
|
|
||||||
|
|
||||||
* **06-01:** consolidate search queries to use track_metadata VIEW ([9159b40](https://github.com/onion-4-dinner/yellowjacket/commit/9159b409dcd2afaa7dcc97bf5b0694edf85f06a4))
|
|
||||||
* **quick-14:** make playOrLoadCurrentTrack and playCurrentTrack return bool ([6eeddda](https://github.com/onion-4-dinner/yellowjacket/commit/6eeddda97669258cc5b7ba175a3c98d598a2871f))
|
|
||||||
|
|
||||||
## [1.1.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.2...v1.1.3) (2026-02-21)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* add typescript as explicit devDependency and auto-install frontend deps in setup ([#70](https://github.com/onion-4-dinner/yellowjacket/issues/70)) ([7316587](https://github.com/onion-4-dinner/yellowjacket/commit/73165877fa79656ab9bc6f60bd8e9e52d6be206c))
|
|
||||||
* use local tsc binary in pre-commit hook to avoid PATH issues ([#71](https://github.com/onion-4-dinner/yellowjacket/issues/71)) ([6079e55](https://github.com/onion-4-dinner/yellowjacket/commit/6079e558ff913d38c7f1c4aeb52cc09474c4ed20))
|
|
||||||
|
|
||||||
## [1.1.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.1...v1.1.2) (2026-02-15)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* r2 upload ([#69](https://github.com/onion-4-dinner/yellowjacket/issues/69)) ([0252466](https://github.com/onion-4-dinner/yellowjacket/commit/0252466f615b4e2fd9694790c6d311a9eac1ccf2))
|
|
||||||
|
|
||||||
## [1.1.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.0...v1.1.1) (2026-02-15)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** remove build-check job from CI workflow ([#66](https://github.com/onion-4-dinner/yellowjacket/issues/66)) ([42d3f45](https://github.com/onion-4-dinner/yellowjacket/commit/42d3f45d85afa694e9545997af3ff4ac814ad021))
|
|
||||||
|
|
||||||
## [1.1.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.3...v1.1.0) (2026-02-15)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** upload release artifacts to Cloudflare R2 ([#65](https://github.com/onion-4-dinner/yellowjacket/issues/65)) ([8985084](https://github.com/onion-4-dinner/yellowjacket/commit/89850848cbf7783e5c85348ff18f7cd11d60231a))
|
|
||||||
|
|
||||||
## [1.0.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.2...v1.0.3) (2026-02-15)
|
|
||||||
|
|
||||||
### ⚠ BREAKING CHANGES
|
|
||||||
|
|
||||||
* **deps:** update module github.com/evilmartians/lefthook to v2 (#61)
|
|
||||||
* **deps:** update actions/checkout action to v6 (#45)
|
|
||||||
* **deps:** update dependency vite to v7 (#53)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* resolve all lint errors and make linting a required CI check ([#62](https://github.com/onion-4-dinner/yellowjacket/issues/62)) ([30b2480](https://github.com/onion-4-dinner/yellowjacket/commit/30b2480df49f57878b0e8c923da6ad8d6fe99416))
|
|
||||||
* virtual list and cover grid ([#63](https://github.com/onion-4-dinner/yellowjacket/issues/63)) ([7579a76](https://github.com/onion-4-dinner/yellowjacket/commit/7579a768be84225ed46db4e7a90781f3e30e2953))
|
|
||||||
|
|
||||||
### Miscellaneous
|
|
||||||
|
|
||||||
* **deps:** update actions/checkout action to v6 ([#45](https://github.com/onion-4-dinner/yellowjacket/issues/45)) ([2d6e221](https://github.com/onion-4-dinner/yellowjacket/commit/2d6e22105d2daed1dc5b586c0442e2941949a165))
|
|
||||||
* **deps:** update dependency vite to v7 ([#53](https://github.com/onion-4-dinner/yellowjacket/issues/53)) ([f0006c4](https://github.com/onion-4-dinner/yellowjacket/commit/f0006c4c4335b60b58cccdd29de4792965e39694))
|
|
||||||
* **deps:** update module github.com/evilmartians/lefthook to v2 ([#61](https://github.com/onion-4-dinner/yellowjacket/issues/61)) ([e32b217](https://github.com/onion-4-dinner/yellowjacket/commit/e32b2179129ae7f26037697a125710ff7587566d))
|
|
||||||
|
|
||||||
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
|
|
||||||
|
|
||||||
### ⚠ BREAKING CHANGES
|
|
||||||
|
|
||||||
* **deps:** update actions/setup-node action to v6 (#48)
|
|
||||||
* **deps:** update dependency stylelint-config-standard to v40 (#52)
|
|
||||||
* **deps:** update dependency node to v24 (#51)
|
|
||||||
* **deps:** update dependency vite-plugin-static-copy to v3 (#54)
|
|
||||||
* **deps:** update golangci/golangci-lint-action action to v9 (#55)
|
|
||||||
* **deps:** update amannn/action-semantic-pull-request action to v6 (#50)
|
|
||||||
* **deps:** update actions/upload-artifact action to v6 (#49)
|
|
||||||
* **deps:** update actions/setup-go action to v6 (#47)
|
|
||||||
* **deps:** update actions/download-artifact action to v7 (#46)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** use allowedPostUpgradeCommands for Renovate post-upgrade tasks ([#60](https://github.com/onion-4-dinner/yellowjacket/issues/60)) ([0aef483](https://github.com/onion-4-dinner/yellowjacket/commit/0aef483b3cccd0616fd5be2d06d0856b46851d09))
|
|
||||||
|
|
||||||
### Miscellaneous
|
|
||||||
|
|
||||||
* **deps:** update actions/download-artifact action to v7 ([#46](https://github.com/onion-4-dinner/yellowjacket/issues/46)) ([1910f99](https://github.com/onion-4-dinner/yellowjacket/commit/1910f99cf64e9bdc5ce91e89cab254ecca15d030))
|
|
||||||
* **deps:** update actions/setup-go action to v6 ([#47](https://github.com/onion-4-dinner/yellowjacket/issues/47)) ([8911fb2](https://github.com/onion-4-dinner/yellowjacket/commit/8911fb2400047cf2f3dfa719edc1d1bf474cdaa5))
|
|
||||||
* **deps:** update actions/setup-node action to v6 ([#48](https://github.com/onion-4-dinner/yellowjacket/issues/48)) ([d7382fd](https://github.com/onion-4-dinner/yellowjacket/commit/d7382fd8444b6618dbfe991f5f97231528a07f13))
|
|
||||||
* **deps:** update actions/upload-artifact action to v6 ([#49](https://github.com/onion-4-dinner/yellowjacket/issues/49)) ([a2c644b](https://github.com/onion-4-dinner/yellowjacket/commit/a2c644b00eed83acc0ed38a2eb8c73868b7b79af))
|
|
||||||
* **deps:** update amannn/action-semantic-pull-request action to v6 ([#50](https://github.com/onion-4-dinner/yellowjacket/issues/50)) ([643ba27](https://github.com/onion-4-dinner/yellowjacket/commit/643ba27f066164aeb47e8d9aaf20fe98b9b69d30))
|
|
||||||
* **deps:** update dependency node to v24 ([#51](https://github.com/onion-4-dinner/yellowjacket/issues/51)) ([e7d3971](https://github.com/onion-4-dinner/yellowjacket/commit/e7d39711078ce86b0c029f0d03ff81162c5dc28a))
|
|
||||||
* **deps:** update dependency stylelint-config-standard to v40 ([#52](https://github.com/onion-4-dinner/yellowjacket/issues/52)) ([422aabc](https://github.com/onion-4-dinner/yellowjacket/commit/422aabcc07e9700ff189302b363e13d87c69163a))
|
|
||||||
* **deps:** update dependency vite-plugin-static-copy to v3 ([#54](https://github.com/onion-4-dinner/yellowjacket/issues/54)) ([77fa643](https://github.com/onion-4-dinner/yellowjacket/commit/77fa6435a5298f58ef83607d99c59b876132c66c))
|
|
||||||
* **deps:** update golangci/golangci-lint-action action to v9 ([#55](https://github.com/onion-4-dinner/yellowjacket/issues/55)) ([aedb7d1](https://github.com/onion-4-dinner/yellowjacket/commit/aedb7d1e6d204c56c468dd26b340752fd6bfeaeb))
|
|
||||||
|
|
||||||
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* resolve Renovate repo detection and pre-push hook hang ([#36](https://github.com/onion-4-dinner/yellowjacket/issues/36)) ([b205889](https://github.com/onion-4-dinner/yellowjacket/commit/b205889128f01e9eb75b607cf7c4034887cda3f4))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
|
|
||||||
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
|
|
||||||
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
|
|
||||||
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
* rename downloaded artifacts to platform-specific names for release ([e3bda0e](https://github.com/onion-4-dinner/yellowjacket/commit/e3bda0e2fc7700fad382cabe00aeb46f91fbb0a0))
|
|
||||||
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
|
|
||||||
* trigger build workflow from release event instead of tag push ([47772f7](https://github.com/onion-4-dinner/yellowjacket/commit/47772f73cc04093c55414bf20ebe2ef442418d19))
|
|
||||||
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
|
|
||||||
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
|
|
||||||
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
|
|
||||||
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
|
|
||||||
* trigger build workflow from release event instead of tag push ([47772f7](https://github.com/onion-4-dinner/yellowjacket/commit/47772f73cc04093c55414bf20ebe2ef442418d19))
|
|
||||||
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
|
|
||||||
|
|
||||||
## [1.0.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.2...v1.0.3) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
|
|
||||||
|
|
||||||
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
|
|
||||||
|
|
||||||
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
|
|
||||||
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
|
|
||||||
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
|
|
||||||
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
|
|
||||||
|
|
||||||
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|
||||||
### Bug Fixes
|
|
||||||
|
|
||||||
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
|
|
||||||
|
|
||||||
## 1.0.0 (2026-02-14)
|
|
||||||
|
|
||||||
### Features
|
|
||||||
|
|
||||||
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
|
|
||||||
|
|||||||
@@ -6,16 +6,111 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||||||
|
|
||||||
YellowJacket is a cross-platform desktop music player built with Go (backend) and TypeScript/Lit (frontend), using the Wails framework to bridge them. It supports MP3, FLAC, OGG Vorbis, and WAV playback.
|
YellowJacket is a cross-platform desktop music player built with Go (backend) and TypeScript/Lit (frontend), using the Wails framework to bridge them. It supports MP3, FLAC, OGG Vorbis, and WAV playback.
|
||||||
|
|
||||||
|
## Issues
|
||||||
|
|
||||||
|
**The tracker is the source of truth for what is wanted and what is
|
||||||
|
already being worked on**, and it is shared with a collaborator who
|
||||||
|
cannot see this session. `scripts/issue.sh` is the whole interface to
|
||||||
|
it (`list`, `mine`, `search`, `show`, `new`, `claim`, `unclaim`,
|
||||||
|
`comment`, `close`, `label`, `depends`, `labels`); it needs a
|
||||||
|
`GITEA_TOKEN` with `write:issue`.
|
||||||
|
|
||||||
|
**Search the tracker before starting any work, and claim what you
|
||||||
|
find.** Fifty-odd issues make that a real lookup rather than a
|
||||||
|
formality. `./scripts/issue.sh search <terms>` covers open and closed —
|
||||||
|
closed matters, because "that was fixed three weeks ago" is the
|
||||||
|
cheapest possible answer.
|
||||||
|
|
||||||
|
**Claiming happens before the first edit, not before the commit.** The
|
||||||
|
whole point is that the collaborator can see the work is taken *while
|
||||||
|
it is being done*, so `claim` sets the assignee, applies
|
||||||
|
`Status/In Progress` and posts a comment naming the branch and the
|
||||||
|
approach — all three, or none. It refuses outright if somebody else
|
||||||
|
already holds it, and that refusal is the feature: talk to them rather
|
||||||
|
than working around it.
|
||||||
|
|
||||||
|
**If no issue covers the work, open one first.** The issue exists
|
||||||
|
before the branch does. That is what makes the tracker a description
|
||||||
|
of the project rather than a description of the past.
|
||||||
|
|
||||||
|
**Findings get filed.** A bug tripped over while doing something else
|
||||||
|
is an issue with a reproduction, not a sentence in a chat message
|
||||||
|
nobody can search. So is a piece of work deliberately not done — the
|
||||||
|
issue is where "we decided not to, and here is why" survives.
|
||||||
|
|
||||||
|
Four conventions are already established and are not up for
|
||||||
|
reinvention:
|
||||||
|
|
||||||
|
- **The labels are a taxonomy**, not tags: `Kind/*`, `Area/*`,
|
||||||
|
`Priority/*`, `Platform/*`, plus `Reviewed/Confirmed` (the code was
|
||||||
|
read and the defect confirmed) and the `Status/*` family. `Status/*`
|
||||||
|
and `Reviewed/*` are **exclusive scopes** — one of each at most, so
|
||||||
|
applying a second replaces the first.
|
||||||
|
- **#73 is the roadmap.** It states the order the backlog should be
|
||||||
|
worked in and the soft relations that are not expressible as
|
||||||
|
blockers. Picking work off the open list by eye when a meta issue
|
||||||
|
states the sequence is how the sequence stops meaning anything.
|
||||||
|
- **Hard blockers are real Gitea dependencies**, which render on the
|
||||||
|
issue itself, and the blocked issue carries `Status/Blocked`.
|
||||||
|
- **A PR body carries a commit-to-issue table, the verification
|
||||||
|
actually run, and a `Closes` list** — PR #83 is the shape. That list
|
||||||
|
is for whoever reads the PR; what actually closes an issue is the
|
||||||
|
footer below.
|
||||||
|
|
||||||
|
**The closing keyword goes in the commit body, one issue per line.**
|
||||||
|
|
||||||
|
```
|
||||||
|
docs: delete four documents that contradict the code
|
||||||
|
|
||||||
|
<body>
|
||||||
|
|
||||||
|
Closes #98
|
||||||
|
```
|
||||||
|
|
||||||
|
**Gitea parses commit messages that reach `main`; it does not parse the
|
||||||
|
PR body**, which only closes anything if the merge happens to copy it
|
||||||
|
into the merge commit. Both halves of that were measured. #83's merge
|
||||||
|
commit carried `Closes #9, #13, #14, …` and closed **five of ten** — a
|
||||||
|
comma list is partially matched. #93's merge commit body was one
|
||||||
|
`Reviewed-on:` trailer, so #92 stayed open behind a perfectly correct
|
||||||
|
`Closes` line in the PR description.
|
||||||
|
|
||||||
|
A footer costs nothing elsewhere: Conventional Commits allows one,
|
||||||
|
`scripts/commit-check.sh` only regexes the subject, and
|
||||||
|
semantic-release reads the type from the subject — so this changes no
|
||||||
|
release decision. The rule that the issue number stays out of the
|
||||||
|
**subject** is unaffected, and was never about the body.
|
||||||
|
|
||||||
|
**Check it anyway.** A squash, or a merge message edited by hand,
|
||||||
|
still drops the footer. `./scripts/issue.sh list --state open` after a
|
||||||
|
merge, looking for what you just shipped; `./scripts/issue.sh close
|
||||||
|
<n>` for whatever did not take, with a comment naming the commit.
|
||||||
|
`close` also drops `Status/In Progress`, because a claim outlives the
|
||||||
|
work if nothing takes the label off.
|
||||||
|
|
||||||
## Planning
|
## Planning
|
||||||
|
|
||||||
Active and historical plans live in `.planning/`:
|
`.planning/` is **design documents and measured history**, not a queue
|
||||||
|
— the queue is the tracker, and a plan file that describes work nobody
|
||||||
|
has started is a second, staler answer to "what are we doing next".
|
||||||
|
|
||||||
- `.planning/NOTES.md` — gotchas, deferred items, open architecture questions, the "we already considered and rejected" list.
|
- `.planning/NOTES.md` — gotchas, measured facts, open architecture
|
||||||
- `.planning/plans/active/` — work currently in progress (read first).
|
questions, and the "we already considered and rejected" list. Dated,
|
||||||
- `.planning/plans/pending/` — sequenced future work.
|
because several are properties of someone else's server. **This is
|
||||||
- `.planning/plans/completed/` — one concise recap per shipped milestone.
|
where a decision reached on an issue gets written down** when it
|
||||||
|
outlives the issue.
|
||||||
|
- `.planning/plans/completed/` — one recap per shipped milestone, kept
|
||||||
|
for the arguments in it. Where a plan shipped incompletely, its
|
||||||
|
header says which issue carries the remainder.
|
||||||
|
- `.planning/audits/` — the read-only audits that produced the
|
||||||
|
reconciliation plans. Historical evidence; not a backlog.
|
||||||
|
- `.planning/plans/active/` — a multi-phase design document for work
|
||||||
|
**in flight**, linked from the issue that tracks it. Empty is the
|
||||||
|
normal state. There is no `pending/`: a plan nobody is executing is
|
||||||
|
an issue.
|
||||||
|
|
||||||
Numbering is sequential and stable across status moves (a plan keeps its `NNN-` prefix as it migrates between `pending → active → completed`). Abandoned plans are deleted; paused work stays in `pending/`.
|
Numbering is sequential and stable across status moves (a plan keeps
|
||||||
|
its `NNN-` prefix). Abandoned plans are deleted.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
@@ -233,6 +328,74 @@ rather than renaming them.
|
|||||||
the drift it caused before — `sql/schemas/` and the migrations
|
the drift it caused before — `sql/schemas/` and the migrations
|
||||||
disagreed, and sqlc generated against the stale one.
|
disagreed, and sqlc generated against the stale one.
|
||||||
|
|
||||||
|
**What that costs an existing database is repaired once, at open.**
|
||||||
|
`CREATE ... IF NOT EXISTS` reaches an existing table only if its shape
|
||||||
|
already matches and otherwise silently no-ops, so a *changed* table
|
||||||
|
never migrates. Plan 014 added `total_tracks` to `explore_index` and
|
||||||
|
to `indexRowFields` — the projection every explore read uses — and no
|
||||||
|
database that already existed grew the column: **every** Explore
|
||||||
|
search, browse, artist and album page on such an install failed with
|
||||||
|
`no such column: total_tracks`, while a fresh install was perfectly
|
||||||
|
healthy, which is exactly why no test saw it. Plan 013 was worse on
|
||||||
|
the same install: `applySchema` could not be applied at all over a
|
||||||
|
pre-013 `audio_files`, so the app did not open.
|
||||||
|
|
||||||
|
`backend/database/staleshape.go` runs before `applySchema` and
|
||||||
|
retires what is stale, so the create is a create. Five things about
|
||||||
|
it are load-bearing:
|
||||||
|
- **It parses `sql/schemas/` for the expectation** rather than
|
||||||
|
writing the column list down a second time, because a second list
|
||||||
|
is a second thing to forget — the fault it exists to repair.
|
||||||
|
- **It notices a changed *type*, not just a missing column.** 013
|
||||||
|
moved `mbid` from TEXT to BLOB, and SQLite does not coerce between
|
||||||
|
them: a comparison against 16 raw bytes returns no rows rather than
|
||||||
|
an error. `ALTER TABLE ADD COLUMN` would have handled
|
||||||
|
`total_tracks` alone and cannot express this at all, which is why
|
||||||
|
the repair drops rather than migrates.
|
||||||
|
- **`Authored` is never retired**, and that boundary is a test
|
||||||
|
(`TestAuthoredTablesAreNeverRetired`), not a comment. Everything
|
||||||
|
else is rebuildable: `Cache` by definition, `Owned` by a rescan —
|
||||||
|
plan 013's stated "delete and rescan" — and `Derived` from Owned.
|
||||||
|
A table the schema no longer describes at all goes too; 013 left
|
||||||
|
seven behind plus `schema_migrations`.
|
||||||
|
- **Whether a stale `Cache` table may be rebuilt is a build tag**, and
|
||||||
|
it is the most expensive thing in this file to get wrong. In the app
|
||||||
|
the catalog is *downloaded*, so a wrong shape costs a minute of
|
||||||
|
re-fetching the artifact and keeping it costs every Explore read. In
|
||||||
|
`cmd/indexbuild` the catalog is *derived*, and the only way back is
|
||||||
|
the ~205 GB dump stream the `/cache` volume exists to avoid — so
|
||||||
|
`retireStaleCache` is false there (`staleshape_policy_indexbuild.go`)
|
||||||
|
and `TestTheCatalogSurvivesAStaleShape` fails the moment it is not.
|
||||||
|
`TestNoCacheTableIsRetiredHere` is the same assertion made of *every*
|
||||||
|
`datamap` Cache table rather than one, because the risk is not that
|
||||||
|
shape recurring — it is the next destructive repair added to
|
||||||
|
`database.NewDB`, the chokepoint every binary here shares, without
|
||||||
|
asking which binary it is in.
|
||||||
|
This is written down because it already happened: the repair shipped
|
||||||
|
without the distinction and dropped the real CI catalog on its first
|
||||||
|
run, with `reason="column entity_type is TEXT, schema declares
|
||||||
|
INTEGER"`. The mismatch was genuine — that database is deliberately
|
||||||
|
kept in the older encoding, which `fix(indexexport): read an index
|
||||||
|
older than the binary` exists to tolerate — so it would have been
|
||||||
|
dropped on *every* run. The consequence is that a future
|
||||||
|
`explore_index` column fails the index job loudly on `applySchema`
|
||||||
|
rather than silently costing it a rebuild, which is the trade a
|
||||||
|
human should get to make.
|
||||||
|
- **The drops are one transaction with `defer_foreign_keys`.** Those
|
||||||
|
legacy tables reference each other, so dropping them in any order
|
||||||
|
fails on whichever goes first, and turning foreign keys *off*
|
||||||
|
instead would silently take `playlist_tracks.audio_file_id`'s
|
||||||
|
ON DELETE SET NULL with it — leaving playlist entries pointing at
|
||||||
|
ids a rescan reissues to *different songs*. Nulled entries are
|
||||||
|
empty; stale ones are wrong, and wrong quietly.
|
||||||
|
- **The order is sorted, so a failure reproduces.** Map order is
|
||||||
|
random, and the foreign-key bug above passed its own regression
|
||||||
|
test on two runs in three until the order was fixed.
|
||||||
|
|
||||||
|
Retiring `explore_index` takes its FTS and its meta with it, because
|
||||||
|
the `dump_import_done` marker is what would otherwise stop the
|
||||||
|
artifact ever being fetched again.
|
||||||
|
|
||||||
**What that costs an existing database is that it does not open**, and
|
**What that costs an existing database is that it does not open**, and
|
||||||
"delete and rescan" is the answer (plan 013, open question 1) — free
|
"delete and rescan" is the answer (plan 013, open question 1) — free
|
||||||
for everyone except one machine. The index job's `/cache` volume is a
|
for everyone except one machine. The index job's `/cache` volume is a
|
||||||
@@ -343,7 +506,26 @@ rather than renaming them.
|
|||||||
came about.
|
came about.
|
||||||
- `config` — TOML-based settings. Settings page uses HTMX + templ for server-rendered HTML fragments.
|
- `config` — TOML-based settings. Settings page uses HTMX + templ for server-rendered HTML fragments.
|
||||||
- `playlist` / `smartplaylist` — Playlist CRUD and rule-based smart playlists.
|
- `playlist` / `smartplaylist` — Playlist CRUD and rule-based smart playlists.
|
||||||
- `mediacontrols` — MPRIS integration on Linux via D-Bus.
|
- `mediacontrols` — OS media controls behind one `Handler`: MPRIS over
|
||||||
|
D-Bus on desktop Linux, a MediaSession on Android, a no-op stub
|
||||||
|
elsewhere. The split is by build tag and `android` implies `linux`,
|
||||||
|
so the three files read `linux && !android`, `android` and `!linux`.
|
||||||
|
Its Android half needs no JNI beyond what Wails exports — a JSON
|
||||||
|
payload out through `application.Android.StartForegroundService`, a
|
||||||
|
command event back through `WailsBridge.emitEvent` — and the Java it
|
||||||
|
talks to is `build/android/.../WailsForegroundService.java`. That
|
||||||
|
contract (payload keys, state words, command names) is in
|
||||||
|
`androidpayload.go` **without** the build tag, because a tagged file
|
||||||
|
is compiled by nothing `make lint` or `make test` runs and is
|
||||||
|
untestable off a phone.
|
||||||
|
|
||||||
|
`OnDuck` is the one callback MPRIS does not use: Android asks for
|
||||||
|
attenuation rather than a pause when something short needs the
|
||||||
|
output. `Player.SetDuck` keeps it as an offset on top of the user's
|
||||||
|
level rather than writing through to the volume, so it cannot
|
||||||
|
accumulate and nothing persists or emits a level the user did not
|
||||||
|
choose — and it only ever fires below API 26, where the framework
|
||||||
|
does not already duck the app itself.
|
||||||
- `system` — OS-specific paths (XDG on Linux, `%LOCALAPPDATA%` on Windows).
|
- `system` — OS-specific paths (XDG on Linux, `%LOCALAPPDATA%` on Windows).
|
||||||
- `explore` — Catalog search and browse over `explore_index`. See below.
|
- `explore` — Catalog search and browse over `explore_index`. See below.
|
||||||
Its **shelves** (`shelves.go`) are the page Explore shows before
|
Its **shelves** (`shelves.go`) are the page Explore shows before
|
||||||
@@ -489,6 +671,81 @@ selected as a literal `0`. Adding the column to the importer's SELECT
|
|||||||
list without that is how a published artifact — which nobody can re-cut
|
list without that is how a published artifact — which nobody can re-cut
|
||||||
retroactively — starts failing with `no such column`.
|
retroactively — starts failing with `no such column`.
|
||||||
|
|
||||||
|
**A credit is ordered parts, and the string is derived from them.** A
|
||||||
|
track credited to several artists had exactly one navigable artist and
|
||||||
|
the rest were punctuation: `primaryArtist()` string-parses the credit,
|
||||||
|
strips a " feat. " clause and discards the guest, and deliberately does
|
||||||
|
not split on `&`, `with` or `,` because those live inside real artist
|
||||||
|
names ("Simon & Garfunkel"). Measured on a real 26,069-file library,
|
||||||
|
**13%** of recordings are multi-artist upstream while only **0.86%** of
|
||||||
|
files carry a structured multi-artist tag — mp3 carries *zero* files
|
||||||
|
with multiple `MUSICBRAINZ_ARTISTID` across 19,840 — so this cannot be
|
||||||
|
a tag-parsing feature. (The "3 credits of 2,823" figure that justified
|
||||||
|
plan 013's removal of the credit tables measured our own *writer*:
|
||||||
|
`cachedLinkArtist` ran once per credit, so a collaboration could never
|
||||||
|
have been recorded. Dropping the join table was still right on cost.)
|
||||||
|
|
||||||
|
`artist_credit_part` / `artist_credit_ref` carry the decomposition for
|
||||||
|
multi-artist credits only — a single-artist credit is already
|
||||||
|
`explore_index`'s own `artist_name`, and storing those would triple the
|
||||||
|
table to say nothing. Five things about it are load-bearing:
|
||||||
|
|
||||||
|
- **Join phrases are assembly instructions, not disassembly ones.**
|
||||||
|
`creditLink` concatenates parts, so link boundaries are known by
|
||||||
|
construction. Locating a `credited_name` *inside* the stored credit
|
||||||
|
string would reintroduce the fault this exists to fix: that string may
|
||||||
|
come from the file's tags while the parts come from the catalog, and
|
||||||
|
the two disagree for ~1 in 3 multi-artist credits (`'Skrillex feat.
|
||||||
|
Swae Lee'` tagged against `'Skrillex & Swae Lee'` upstream).
|
||||||
|
- **`credited_name` is stored per row**, never joined from `artists`:
|
||||||
|
MusicBrainz credits "Snoop Dogg" on a track by the artist called
|
||||||
|
"Snoop Doggy Dogg". Display follows the credit, navigation the MBID.
|
||||||
|
- **The lookup is keyed on the recording MBID**, which the catalog and a
|
||||||
|
local file both carry (`library.Track.RecordingMBID`), so one binding
|
||||||
|
serves Explore and the library's own lists — which is why this needed
|
||||||
|
no local table. `file_artists` remains the offline-resilience step and
|
||||||
|
is deliberately *not* declared until something writes it.
|
||||||
|
- **Absence is cached as an answer.** `credit-store.ts` stores `[]` for
|
||||||
|
a single-artist credit — *asked*, not *answered* — or the ~87% that
|
||||||
|
have nothing to decompose are re-requested on every render forever.
|
||||||
|
`request()` is per-row and coalesces into one call per frame, because
|
||||||
|
a virtualized list cannot hand over "the whole list": 50,000 rows is
|
||||||
|
100 queries for the ~30 on screen.
|
||||||
|
- **The dump is a third source, and it had to be.** The canonical dump
|
||||||
|
CI already streams has no join phrases and no as-credited names, and
|
||||||
|
the JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
|
||||||
|
against a real library. So `mbdump.tar.bz2` — 7.1 GB, ~13.7 min in
|
||||||
|
pure-Go bzip2, whose members are alphabetical, which is what lets one
|
||||||
|
pass resolve an entity's credit without buffering 35M recordings. The
|
||||||
|
pass runs on **every** mode, because a complete import means
|
||||||
|
`refresh`, which never enters the importer at all, and it reports
|
||||||
|
whether it populated anything so `changed` republishes the artifact.
|
||||||
|
|
||||||
|
**A 0.6 GB download asks about the connection first.** `explore`'s
|
||||||
|
catalog artifact had no network awareness at all, which on a phone is a
|
||||||
|
month's data allowance spent without being asked (plan 016 B4).
|
||||||
|
`netpolicy.go` is the gate, and its shape is dictated by one constraint:
|
||||||
|
`explore` is imported by `cmd/indexbuild`, which is built with
|
||||||
|
`CGO_ENABLED=0` and must not link Wails — so the *policy* and the
|
||||||
|
*parsing* live here and are tested on every platform, while the platform
|
||||||
|
call is a closure injected from `app.go`. It is
|
||||||
|
`application.Mobile.NetworkJSON()`, not `application.Android`'s: the
|
||||||
|
latter exists only under the `android` build tag, and `Mobile`'s desktop
|
||||||
|
implementation is a stub returning `""`.
|
||||||
|
|
||||||
|
Three rules in it are load-bearing. **An unknown answer is not a metered
|
||||||
|
one** — only mobile answers at all, so treating silence as metered would
|
||||||
|
refuse the download on every desktop. **Cellular is the only signal
|
||||||
|
available**: the runtime reports `wifi|cellular|ethernet|none` and no
|
||||||
|
metered flag, so a metered *Wi-Fi* (a hotspot, a hotel) cannot be
|
||||||
|
detected and is not refused, which is a documented gap rather than an
|
||||||
|
oversight. And **the gate runs before anything is staged**, so declining
|
||||||
|
is a no-op rather than a job in the indicator and a status the user has
|
||||||
|
to dismiss. The permission (`AllowMeteredCatalogDownload`, default
|
||||||
|
false, so an existing config is careful without a migration) is read at
|
||||||
|
the moment a download would start, so turning it on takes effect on the
|
||||||
|
next attempt rather than the next launch.
|
||||||
|
|
||||||
**Background work yields, and says so in the context.** The post-scan
|
**Background work yields, and says so in the context.** The post-scan
|
||||||
backfills share MusicBrainz's rate limiters with every page the user
|
backfills share MusicBrainz's rate limiters with every page the user
|
||||||
can open, and both were FIFO — so a thousand-artist enrichment put an
|
can open, and both were FIFO — so a thousand-artist enrichment put an
|
||||||
@@ -664,6 +921,27 @@ moment it is most needed is the likeliest moment loading one fails.
|
|||||||
`first-run-wizard` and the startup chrome are eager for the ordinary
|
`first-run-wizard` and the startup chrome are eager for the ordinary
|
||||||
reason — they are the first paint.
|
reason — they are the first paint.
|
||||||
|
|
||||||
|
**A navigation is a history entry, and that is the whole back stack.**
|
||||||
|
`index.ts` records each navigation with `pushState` (same URL — the app
|
||||||
|
has no routes, and a path a reload cannot resolve is worse than none)
|
||||||
|
and replays `popstate` with `_isBack`. It exists for Android, whose back
|
||||||
|
button is not a key the page can bind: the scaffold's
|
||||||
|
`MainActivity.onBackPressed` asks `webView.canGoBack()` and finishes the
|
||||||
|
activity otherwise, so an app that never touched `history` quit from any
|
||||||
|
depth — which is what a device reported. Hooking the platform's own
|
||||||
|
mechanism rather than adding a JNI callback is also what makes it
|
||||||
|
testable in a browser (`page.goBack()`), and the Java half needed no
|
||||||
|
change at all.
|
||||||
|
|
||||||
|
Two rules hold it up. The **first** navigation *replaces* the launch
|
||||||
|
entry rather than pushing one, or every launch costs a back press before
|
||||||
|
the app will close. And the in-app back buttons (`navigate-back`, fired
|
||||||
|
by the detail views and `now-playing-view`) go through `history.back()`
|
||||||
|
rather than a stack of their own: the old `navStack` is **deleted**, not
|
||||||
|
kept beside it, because two stacks is precisely how a view's own back
|
||||||
|
button and the phone's gesture come to disagree about what one press
|
||||||
|
means.
|
||||||
|
|
||||||
**A primary view is cached, not unmounted.** `index.ts` keeps every
|
**A primary view is cached, not unmounted.** `index.ts` keeps every
|
||||||
primary view in the DOM and toggles a `.view-hidden` class, because that
|
primary view in the DOM and toggles a `.view-hidden` class, because that
|
||||||
is what preserves `scrollTop` across navigation — so
|
is what preserves `scrollTop` across navigation — so
|
||||||
@@ -819,6 +1097,20 @@ against the real components:
|
|||||||
moving focus without setting it leaves the highlight on whichever
|
moving focus without setting it leaves the highlight on whichever
|
||||||
item the mouse last touched.
|
item the mouse last touched.
|
||||||
|
|
||||||
|
**And a menu opens from a finger, through the event it already has.**
|
||||||
|
`utils/long-press.ts` is one document-capture listener installed once
|
||||||
|
from `index.ts`: a touch that holds still for 500 ms dispatches a
|
||||||
|
synthetic `contextmenu` at the touch point, so all six components that
|
||||||
|
bind one — delegated on a virtualizer, per row, per card — gained the
|
||||||
|
gesture without changing. The target is `composedPath()[0]` rather than
|
||||||
|
`elementFromPoint`, which stops at the outermost shadow host and so
|
||||||
|
reaches a delegated listener and no per-row one; a browser that fires
|
||||||
|
its own long-press `contextmenu` (Chromium does, WebKit and the WebView
|
||||||
|
vary) wins, ours being told from theirs by **identity** rather than
|
||||||
|
`isTrusted`, since no test can dispatch a trusted event; and the click
|
||||||
|
that ends the gesture is swallowed, keyed on the gesture rather than on
|
||||||
|
a time window so the first tap on the menu it opened is not eaten too.
|
||||||
|
|
||||||
Three lists had no focused row to open a menu *from* — the queue panel
|
Three lists had no focused row to open a menu *from* — the queue panel
|
||||||
and both playlist detail views — and gained a roving tab stop through
|
and both playlist detail views — and gained a roving tab stop through
|
||||||
`utils/roving-rows.ts`. **`track-list` deliberately does not use it**:
|
`utils/roving-rows.ts`. **`track-list` deliberately does not use it**:
|
||||||
@@ -989,6 +1281,65 @@ this app promises, no scrollbar appears. Note that `overflow: hidden`
|
|||||||
still permits *programmatic* scrolling, so a probe that sets
|
still permits *programmatic* scrolling, so a probe that sets
|
||||||
`scrollLeft` passes on the broken build; the spec uses a wheel gesture.
|
`scrollLeft` passes on the broken build; the spec uses a wheel gesture.
|
||||||
|
|
||||||
|
**Below 600px it reflows instead, and that is the phone.** The sideways
|
||||||
|
scroll above was the concession available while the shell had one
|
||||||
|
layout; plan 016 B2 gives it a second. Under 600px the grid drops its
|
||||||
|
sidebar column, `<bottom-nav>` takes over as the primary navigation,
|
||||||
|
the header's controls shrink or stand down, and the shell measures
|
||||||
|
exactly 320px in a 320px viewport — so `layout-overflow.spec.ts` now
|
||||||
|
asserts *nothing needs scrolling to*, which is what WCAG 1.4.10 wanted
|
||||||
|
all along. 600 rather than the sidebar's 900 because 900 is a laptop:
|
||||||
|
the answer there is a narrower sidebar, which is still a sidebar.
|
||||||
|
|
||||||
|
Three rules in it are load-bearing, and the second cost 30 specs.
|
||||||
|
|
||||||
|
**A grid item's implicit minimum is its content**, so one child that
|
||||||
|
insists on 580px makes the *body* 580px wide inside a 360px viewport
|
||||||
|
and `overflow-x: hidden` then hides a third of the app rather than
|
||||||
|
fitting it. Every box between the viewport and the content that must
|
||||||
|
shrink carries `min-width: 0`, and the things that cannot shrink say so
|
||||||
|
in their own stylesheet — `search-bar`'s 200px floor, `job-indicator`'s
|
||||||
|
label, `audio-player`'s seek bar and volume. A media query inside a
|
||||||
|
shadow root is answered by the viewport, so a component states what it
|
||||||
|
drops at phone width itself rather than the shell reaching in.
|
||||||
|
|
||||||
|
**A duplicated component duplicates its handles.** `bottom-nav`'s
|
||||||
|
"More" opens the *same* `<app-sidebar>` in a `wa-drawer` rather than
|
||||||
|
listing the destinations again — but rendering it unconditionally put a
|
||||||
|
second copy of every `data-testid="nav-*"` in the DOM, and 30 existing
|
||||||
|
specs failed with "strict mode violation: resolved to 2 elements" on a
|
||||||
|
desktop viewport where the element is not even visible. It renders only
|
||||||
|
while the drawer is open, and `bottom-nav.test.ts` asserts its absence
|
||||||
|
before that.
|
||||||
|
|
||||||
|
**The tab bar is four destinations and a way to the rest.** Three to
|
||||||
|
five is where touch targets stop being thumb-sized; eleven over 360px
|
||||||
|
is 32px each. Which four is plan 016's committed subset, and everything
|
||||||
|
else — Settings included, because a phone still needs it — is behind
|
||||||
|
"More".
|
||||||
|
|
||||||
|
**The phone section of `index.css` is last on purpose.** A media query
|
||||||
|
adds no specificity, so a `@media (max-width: 599px)` block placed
|
||||||
|
above the plain rules it overrides loses to them — which is how phase 1
|
||||||
|
shipped a header that kept its 2em gutters and 24px title on a 390px
|
||||||
|
phone with every declaration dead and nothing failing. The shell fitted
|
||||||
|
anyway, because the fitting is done by `min-width: 0` and by each
|
||||||
|
component's own media query, which live in their own stylesheets and
|
||||||
|
have no later rule to lose to. Cosmetic declarations are exactly what
|
||||||
|
no assertion sees; a screenshot found it.
|
||||||
|
|
||||||
|
**`<now-playing-view>` is where the seek bar and volume went.** It is a
|
||||||
|
*detail* view (`DETAIL_LOADERS`, so the nav stack carries the way out —
|
||||||
|
a tab you cannot leave by pressing again is not a tab), reached from a
|
||||||
|
phone-only button over the mini player's art, and it **composes the
|
||||||
|
real `<seek-bar>`, `<player-controls>` and `<volume-control>`** rather
|
||||||
|
than reimplementing them. While it is up, `index.css` hides the bottom
|
||||||
|
bar through `body:has(#main-content[data-active-view="now-playing"])` —
|
||||||
|
the active view is already published as an attribute, and a class
|
||||||
|
toggled from `index.ts` would be a second expression of the same fact.
|
||||||
|
The view therefore carries its own queue button, because that button
|
||||||
|
lives in the bar it hides.
|
||||||
|
|
||||||
**The playing row is a shape, not a hue.** `track-list` and
|
**The playing row is a shape, not a hue.** `track-list` and
|
||||||
`queue-panel` draw a `::before` triangle in each row's own left
|
`queue-panel` draw a `::before` triangle in each row's own left
|
||||||
padding, plus `aria-current` — before, both rows were a background tint
|
padding, plus `aria-current` — before, both rows were a background tint
|
||||||
@@ -1423,6 +1774,28 @@ by the three places that need them (the default widths, the
|
|||||||
normaliser, and the resize handles' positions), because they were
|
normaliser, and the resize handles' positions), because they were
|
||||||
written out separately and that is how they came to disagree.
|
written out separately and that is how they came to disagree.
|
||||||
|
|
||||||
|
**A phone draws one column of two lines, and that is a column set
|
||||||
|
rather than a second row template.** Measured on the device: at 424 px
|
||||||
|
the four configured columns fit the row *exactly* (`--grid-cols` came
|
||||||
|
out `24px 102px 101px 101px 80px`) and not one of them fit its content
|
||||||
|
— "Duration" did not fit its own header. The columns were never too
|
||||||
|
wide; there were too many of them. `PHONE_COLUMN_IDS` is `titleArtist`
|
||||||
|
(title over artist, sharing the row's whole width) plus the duration, so
|
||||||
|
the row, the delegated events, the selection semantics, the playing
|
||||||
|
marker and the virtualizer are all untouched: from their side only the
|
||||||
|
number of columns changed. Three rules come with it. **The row height
|
||||||
|
lives in two places and they must agree** — `PHONE_ROW_HEIGHT` and the
|
||||||
|
CSS rule — because the virtualizer positions rows from that number, so a
|
||||||
|
taller row overlaps its neighbour. **What is drawn and what can be
|
||||||
|
sorted are different questions**: the page header's sort list is built
|
||||||
|
from `configuredColumns`, or a phone (which has no column headers
|
||||||
|
either) could sort by nothing but title and duration. And **a phone's
|
||||||
|
widths are neither loaded nor saved**: `loadColumnWidths` is keyed by
|
||||||
|
column *id* and fills a gap with the minimum, so the stacked column —
|
||||||
|
which nothing can ever have saved a width for — came out at 148 px
|
||||||
|
beside a duration column of 236, and saving would have replaced the
|
||||||
|
width the user dragged on a desktop for the same id.
|
||||||
|
|
||||||
**The default columns are declared twice and must agree.**
|
**The default columns are declared twice and must agree.**
|
||||||
`tracklist.DefaultColumns` is what a fresh install persists;
|
`tracklist.DefaultColumns` is what a fresh install persists;
|
||||||
`DEFAULT_COLUMN_IDS` in `track-list/columns.ts` is what the list draws
|
`DEFAULT_COLUMN_IDS` in `track-list/columns.ts` is what the list draws
|
||||||
@@ -1749,13 +2122,31 @@ Pre-commit hooks verify generated code is fresh — always run `make generate` a
|
|||||||
two in step or semantic-release will decline to release something the
|
two in step or semantic-release will decline to release something the
|
||||||
check accepted.
|
check accepted.
|
||||||
|
|
||||||
`.releaserc.yml` is a complete semantic-release config that **nothing
|
`.releaserc.yml` **is** what runs now, from `release.yml`, and it is why
|
||||||
currently runs** — no workflow invokes it, and `CHANGELOG.md` is not
|
the commit grammar is load-bearing rather than decorative: a merge to
|
||||||
being written by it. That is deliberate for now (wiring it means pushing
|
`main` whose commits are all `chore`/`ci`/`docs` releases nothing, and a
|
||||||
tags, committing a changelog back, and interacting with the three
|
mistyped `feat` ships a minor version. `make release-dry` answers "what
|
||||||
publish workflows); it is recorded here rather than implied, because
|
would this merge release" without pushing.
|
||||||
this file claimed for five phases that commitlint gated CI and that
|
|
||||||
semantic release ran, and neither was true.
|
**`@semantic-release/github` is not in that config and must not be.**
|
||||||
|
Gitea's API is `/api/v1` and is not GitHub's surface, so
|
||||||
|
`@semantic-release/exec` calls `scripts/gitea-release.sh` instead — one
|
||||||
|
`POST`, which is the whole of the Gitea-shaped work. The community
|
||||||
|
plugin (`@saithodev/semantic-release-gitea`) was considered and
|
||||||
|
rejected: last published 2022, on `got@10`, declaring no peer
|
||||||
|
dependency on semantic-release at all.
|
||||||
|
|
||||||
|
Two things in it fail *silently* and are therefore pinned with their
|
||||||
|
reasons. **The notes come from `CHANGELOG.md`, not from an argument**:
|
||||||
|
release notes are rendered commit messages — arbitrary text carrying
|
||||||
|
backticks, quotes and `$` — so templating `${nextRelease.notes}` into
|
||||||
|
`publishCmd` would be a shell injection whose input is the commit log.
|
||||||
|
And **`conventional-changelog-conventionalcommits` is held at 9**,
|
||||||
|
because at 10 it is quietly incompatible with the writer
|
||||||
|
`release-notes-generator@14` pulls in: every release note renders as a
|
||||||
|
bare `## 0.0.1 (date)` heading with no sections and no commits beneath
|
||||||
|
it, no step fails, and the release ships with an empty body. Check the
|
||||||
|
rendered notes, never the exit code.
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
@@ -1764,18 +2155,98 @@ Tests use `database.NewTestDB(t)` for in-memory SQLite, built by the same
|
|||||||
|
|
||||||
## Git Workflow
|
## Git Workflow
|
||||||
|
|
||||||
Feature branches and PRs are the norm, but direct pushes to `main` are allowed. Pre-commit runs vet, lint, codegen check, and frontend typecheck in parallel. Pre-push runs the full test suite.
|
Feature branches and PRs are the only way in: **`main` is a protected
|
||||||
|
branch** (`enable_push: false`, an empty push whitelist, and `CI / check*`
|
||||||
|
+ `CI / e2e*` as required status checks), so a direct push is rejected by
|
||||||
|
the pre-receive hook. This file said otherwise for a long time. Tags are
|
||||||
|
*not* protected, which is what lets `release.yml` push one.
|
||||||
|
|
||||||
|
**A branch answers a claimed issue** — see "Issues" above. The commit
|
||||||
|
grammar is unchanged and is load-bearing for a different reason
|
||||||
|
(semantic-release reads it), so the issue number lives in the branch
|
||||||
|
name and the PR body rather than in the commit subject.
|
||||||
|
|
||||||
|
**A batch of small fixes can be one PR**, which is what #83 did: eight
|
||||||
|
branches preserved as merges under one integration branch, so
|
||||||
|
authorship survives and the batch lands as one release rather than
|
||||||
|
eight. The cost is that its `Closes` list has to be checked afterwards
|
||||||
|
— it half-worked.
|
||||||
|
|
||||||
|
Pre-commit runs vet, lint, codegen check, and frontend typecheck in parallel. Pre-push runs the full test suite.
|
||||||
|
|
||||||
## CI
|
## CI
|
||||||
|
|
||||||
Five workflows in `.gitea/workflows/`. Four of them package and
|
Seven workflows in `.gitea/workflows/`. Five of them package and
|
||||||
publish (`arch-package`, `homebrew-formula`, `index-artifact`,
|
publish (`arch-package`, `homebrew-formula`, `index-artifact`,
|
||||||
`android-apk`); only `ci.yml` gates, and it is the one to look at when
|
`android-apk`, `desktop-assets`); `release.yml` decides *whether* four of
|
||||||
|
those run at all; only `ci.yml` gates, and it is the one to look at when
|
||||||
deciding whether a push was healthy.
|
deciding whether a push was healthy.
|
||||||
|
|
||||||
**`android-apk.yml` is the only one keyed on a tag and the only one
|
**`release.yml` is the entry point for all of it.** On every push to
|
||||||
that can lose something irrecoverable.** It builds the signed fat APK
|
`main` it reads the Conventional Commits since the last tag and, if any
|
||||||
on every `v*` tag and publishes it to the *generic* registry, which is
|
is releasable, writes the changelog, pushes the tag and creates the Gitea
|
||||||
|
release whose body is that changelog section. `arch-package`,
|
||||||
|
`homebrew-formula`, `android-apk` and `desktop-assets` are all keyed on
|
||||||
|
`v*`, so **the tag push is what starts them** — nothing is released by
|
||||||
|
hand any more.
|
||||||
|
|
||||||
|
Four things about it are load-bearing:
|
||||||
|
|
||||||
|
- **The tag is pushed with a user PAT, not the Actions token.** Gitea,
|
||||||
|
like GitHub, does not start a workflow from a ref pushed by a
|
||||||
|
workflow's own token (go-gitea#33123). The token is what decides this,
|
||||||
|
so `PACKAGE_TOKEN` is handed to semantic-release as the
|
||||||
|
`repositoryUrl` credential and the push is attributed to a person.
|
||||||
|
- **That same limitation is used deliberately, once.** semantic-release
|
||||||
|
calls the first release of a tagless repo `1.0.0` and offers no way to
|
||||||
|
say otherwise, so a `v0.0.0` floor tag is what makes the first release
|
||||||
|
`0.0.1` — and it is pushed with the *Actions* token precisely so it
|
||||||
|
triggers nothing. All four publishers additionally skip `v0.0.0`
|
||||||
|
explicitly, cleanly rather than by failing, because a floor is not a
|
||||||
|
shipment.
|
||||||
|
- **The release page is the changelog, and that follows from the branch
|
||||||
|
protection.** `@semantic-release/git` would push a `chore(release):`
|
||||||
|
commit back to `main`, which the pre-receive hook rejects — *after* the
|
||||||
|
tag had been pushed, leaving a tagged release the run then reports as
|
||||||
|
failed. Whitelisting the CI user was the alternative and was declined:
|
||||||
|
it weakens a protection someone set on purpose and lets a bot push to
|
||||||
|
`main` without the checks every human PR passes. So the plugin is
|
||||||
|
absent, `@semantic-release/changelog` writes to a gitignored
|
||||||
|
`.release-notes.md` purely to carry the notes into
|
||||||
|
`scripts/gitea-release.sh`, and `CHANGELOG.md` is a signpost to the
|
||||||
|
releases page rather than a file that would silently stop updating.
|
||||||
|
The workflow keeps its `chore(release):` guard anyway, for the day
|
||||||
|
someone adds the plugin back.
|
||||||
|
- **An asset upload waits for the release to exist.** semantic-release
|
||||||
|
pushes the tag in `prepare` and creates the release in `publish`, so
|
||||||
|
the tag push that starts these workflows happens *before* there is a
|
||||||
|
release id to attach to. `scripts/release-asset.sh` polls for it. The
|
||||||
|
capacity-1 runner serialises things enough that this would usually work
|
||||||
|
by accident, which is the worst kind of bug.
|
||||||
|
|
||||||
|
**Releases restarted at `0.0.1`, which is a downgrade on every channel.**
|
||||||
|
pacman and Homebrew both silently offer no upgrade from the old `1.x`,
|
||||||
|
and Android refuses the install outright — its remedy is an uninstall
|
||||||
|
that takes the user's library. This was chosen over pacman's `epoch` and
|
||||||
|
over offsetting `versionCode`, on the grounds that both are permanent and
|
||||||
|
a reinstall is once. `packaging/homebrew/README.md` and
|
||||||
|
`docs/android-release.md` say so where a user would look.
|
||||||
|
|
||||||
|
**`desktop-assets.yml` publishes Linux and nothing else, and macOS is not
|
||||||
|
an oversight.** `GOOS=darwin CGO_ENABLED=0` fails at
|
||||||
|
`wails/v3/pkg/mac: build constraints exclude all Go files` — the darwin
|
||||||
|
backend is Objective-C behind cgo, so a `.app` needs a macOS host and the
|
||||||
|
runner is a Linux container. That is exactly why the Homebrew formula
|
||||||
|
builds from source on the user's own Mac. Windows *does* cross-compile
|
||||||
|
cleanly (`GOOS=windows CGO_ENABLED=0`, a couple of seconds — oto uses
|
||||||
|
WinMM through `x/sys`, sqlite is modernc's pure-Go driver, WebView2 is
|
||||||
|
COM syscalls, MPRIS is `linux && !android`-tagged) and is deliberately
|
||||||
|
not published: no Windows build of this app has ever been *run*, and no
|
||||||
|
tier here can exercise one.
|
||||||
|
|
||||||
|
**`android-apk.yml` is the one that can lose something irrecoverable.** It builds the signed
|
||||||
|
`arm64-v8a` APK (the only ABI Android can run this app on — see
|
||||||
|
`app/build.gradle`) on every `v*` tag and publishes it to the *generic* registry, which is
|
||||||
readable without credentials — the reason Obtainium can poll a plain
|
readable without credentials — the reason Obtainium can poll a plain
|
||||||
URL. Android refuses to update an app whose signing certificate
|
URL. Android refuses to update an app whose signing certificate
|
||||||
changed, and the only remedy is an uninstall that takes the user's
|
changed, and the only remedy is an uninstall that takes the user's
|
||||||
@@ -1881,6 +2352,33 @@ like source** — it was generated once into a scratch directory and
|
|||||||
copied across (plan 015), it carries one deliberate edit to its
|
copied across (plan 015), it carries one deliberate edit to its
|
||||||
`Taskfile.yml`, and only its output is gitignored. `build/ios/` is
|
`Taskfile.yml`, and only its output is gitignored. `build/ios/` is
|
||||||
still not carried and its `includes:` entry is still dropped.
|
still not carried and its `includes:` entry is still dropped.
|
||||||
|
**Its `MainActivity` owns the safe area, because `targetSdk 35` does
|
||||||
|
not leave that to the theme.** Android 15 lays every app out
|
||||||
|
edge-to-edge and ignores the `statusBarColor`/`navigationBarColor` the
|
||||||
|
scaffold's theme sets, and the WebView is `match_parent`, so the page's
|
||||||
|
bottom band — the transport and, on a phone, the tab bar — would be
|
||||||
|
drawn under the gesture bar. `applyWindowInsets()` pads the container by
|
||||||
|
`systemBars | displayCutout | ime` and returns the insets rather than
|
||||||
|
consuming them; the window background is black to match the app's own
|
||||||
|
ramp, since that padding is what shows through. It is **pre-emptive**:
|
||||||
|
the phone this was checked against is Android 14, where the system still
|
||||||
|
insets the window, and the enforcement applies to an app *running on*
|
||||||
|
15. No browser tier can see this class of fault either way — a viewport
|
||||||
|
has no system bars.
|
||||||
|
|
||||||
|
**And a device is an engine, not just a screen.** The phone this app was
|
||||||
|
first run on renders in **Chrome 113** — two years behind every browser
|
||||||
|
any other tier uses — at a 424x439 CSS px viewport. It has `:has()`,
|
||||||
|
`color-mix()` and `dialog.showModal()`; it does **not** have relaxed CSS
|
||||||
|
nesting (Chrome 120, so a nested rule beginning with a bare element
|
||||||
|
selector is silently dropped), the Popover API (114, which Web Awesome's
|
||||||
|
popups set `popover="manual"` for), `light-dark()` or relative colour
|
||||||
|
syntax. So "it renders at that size in Chromium" is not evidence about
|
||||||
|
the phone, and resizing a spec cannot recover the missing signal. `make
|
||||||
|
android-inspect` forwards the WebView's devtools socket and `make
|
||||||
|
android-eval` asks the real page — raw CDP, because `connectOverCDP`
|
||||||
|
calls `Browser.setDownloadBehavior` and a WebView refuses it.
|
||||||
|
|
||||||
`build/config.yml`'s `version` is the
|
`build/config.yml`'s `version` is the
|
||||||
*metadata* version and is not what the app reports — `main.version` is
|
*metadata* version and is not what the app reports — `main.version` is
|
||||||
stamped at link time from the packaging recipe's git-derived version.
|
stamped at link time from the packaging recipe's git-derived version.
|
||||||
|
|||||||
@@ -52,8 +52,13 @@ ANDROID_SDK ?= $(HOME)/Android/Sdk
|
|||||||
ANDROID_NDK ?= /opt/android-ndk
|
ANDROID_NDK ?= /opt/android-ndk
|
||||||
ANDROID_ENV := ANDROID_HOME=$(ANDROID_SDK) ANDROID_SDK_ROOT=$(ANDROID_SDK) ANDROID_NDK_HOME=$(ANDROID_NDK)
|
ANDROID_ENV := ANDROID_HOME=$(ANDROID_SDK) ANDROID_SDK_ROOT=$(ANDROID_SDK) ANDROID_NDK_HOME=$(ANDROID_NDK)
|
||||||
|
|
||||||
android: build-frontend ## Build the fat APK (arm64 + x86_64) into bin/
|
# `package`, not `package:fat`: x86_64 Android cannot run this app at
|
||||||
@$(ANDROID_ENV) PATH="$(TOOLBIN):$$PATH" go tool wails3 task android:package:fat
|
# all (modernc's raw lstat vs Android's seccomp -- see
|
||||||
|
# android-tier.md), so the second ABI was ~31 MB that could not run
|
||||||
|
# anywhere. app/build.gradle's abiFilters says the same thing to
|
||||||
|
# Gradle; both have to agree or the .so is built and then dropped.
|
||||||
|
android: build-frontend ## Build the arm64 APK into bin/
|
||||||
|
@$(ANDROID_ENV) PATH="$(TOOLBIN):$$PATH" go tool wails3 task android:package
|
||||||
|
|
||||||
android-setup: ## Install the SDK pieces and create the AVD (once, ~3.5GB)
|
android-setup: ## Install the SDK pieces and create the AVD (once, ~3.5GB)
|
||||||
@$(ANDROID_ENV) ./scripts/android-emulator.sh setup
|
@$(ANDROID_ENV) ./scripts/android-emulator.sh setup
|
||||||
@@ -73,6 +78,19 @@ android-launch: ## Force-stop, clear logcat, and start the app
|
|||||||
android-logs: ## Tail logcat, filtered to the app's own tags
|
android-logs: ## Tail logcat, filtered to the app's own tags
|
||||||
@$(ANDROID_ENV) ./scripts/android-emulator.sh logs
|
@$(ANDROID_ENV) ./scripts/android-emulator.sh logs
|
||||||
|
|
||||||
|
# The only tier that can see the platform is the one you can look at.
|
||||||
|
android-screenshot: ## Grab the device screen (OUT=<path>)
|
||||||
|
@$(ANDROID_ENV) ./scripts/android-emulator.sh screenshot $(OUT)
|
||||||
|
|
||||||
|
# The page's own answer, from the engine that is really rendering it.
|
||||||
|
# Needs the debug build installed (it is a sibling id, so it does not
|
||||||
|
# disturb the release app): see scripts/android-eval.mjs.
|
||||||
|
android-inspect: ## Forward the device WebView's devtools socket
|
||||||
|
@$(ANDROID_ENV) ./scripts/android-emulator.sh inspect
|
||||||
|
|
||||||
|
android-eval: ## Evaluate JS in the device WebView (EXPR='...')
|
||||||
|
@node ./scripts/android-eval.mjs $(if $(EXPR),'$(EXPR)',)
|
||||||
|
|
||||||
# "Did it start" is the wrong question — a crash-looping app starts
|
# "Did it start" is the wrong question — a crash-looping app starts
|
||||||
# several times a second. This asserts the *same pid* is still there.
|
# several times a second. This asserts the *same pid* is still there.
|
||||||
android-smoke: ## Launch and assert the app is still alive (SECONDS=<n>)
|
android-smoke: ## Launch and assert the app is still alive (SECONDS=<n>)
|
||||||
@@ -161,10 +179,12 @@ bindings-check: ## Fail if the generated bindings are stale
|
|||||||
css-check: ## Fail if a css`` literal was ended early by a backtick in a comment
|
css-check: ## Fail if a css`` literal was ended early by a backtick in a comment
|
||||||
@cd frontend && node scripts/check-css-literals.mjs
|
@cd frontend && node scripts/check-css-literals.mjs
|
||||||
|
|
||||||
# .pi/ documents commands, and a skill that documents a command wrongly
|
# .pi/ and CLAUDE.md document commands, and a doc that documents a
|
||||||
# is worse than no skill: an agent runs it confidently. Every command
|
# command wrongly is worse than no doc: an agent runs it confidently.
|
||||||
# in there is a make target on purpose, so this is checkable.
|
# Every command in them is a make target on purpose, so this is
|
||||||
skill-check: ## Fail if .pi/ documents a make target that does not exist
|
# checkable. It also asserts AGENTS.md is a symlink to CLAUDE.md, so the
|
||||||
|
# two harnesses cannot drift onto two descriptions of one project.
|
||||||
|
skill-check: ## Fail if the agent docs name a missing make target, or AGENTS.md is not a symlink
|
||||||
@./scripts/skill-check.sh
|
@./scripts/skill-check.sh
|
||||||
|
|
||||||
# Conventional Commits, which CLAUDE.md claimed CI enforced for a long
|
# Conventional Commits, which CLAUDE.md claimed CI enforced for a long
|
||||||
@@ -172,6 +192,24 @@ skill-check: ## Fail if .pi/ documents a make target that does not exist
|
|||||||
commit-check: ## Fail if a commit subject is not a Conventional Commit
|
commit-check: ## Fail if a commit subject is not a Conventional Commit
|
||||||
@./scripts/commit-check.sh $(if $(RANGE),--range $(RANGE))
|
@./scripts/commit-check.sh $(if $(RANGE),--range $(RANGE))
|
||||||
|
|
||||||
|
# What a merge to main would release, without releasing it. Reads the
|
||||||
|
# same .releaserc.yml CI does, so "why did that not cut a version" is
|
||||||
|
# answerable locally instead of by pushing and watching. Needs no
|
||||||
|
# credentials: --dry-run neither tags nor publishes.
|
||||||
|
#
|
||||||
|
# The pins must stay identical to release.yml's, which is where the note
|
||||||
|
# on holding the conventionalcommits preset at 9 lives -- at 10 the
|
||||||
|
# release notes come out empty with everything green.
|
||||||
|
release-dry: ## Print the version a merge to main would release
|
||||||
|
@npx --yes \
|
||||||
|
-p semantic-release@25 \
|
||||||
|
-p @semantic-release/commit-analyzer@13 \
|
||||||
|
-p @semantic-release/release-notes-generator@14 \
|
||||||
|
-p @semantic-release/changelog@7 \
|
||||||
|
-p @semantic-release/exec@7 \
|
||||||
|
-p conventional-changelog-conventionalcommits@9 \
|
||||||
|
semantic-release --dry-run --no-ci
|
||||||
|
|
||||||
# v3 generates TypeScript into frontend/bindings/, nested by Go import
|
# v3 generates TypeScript into frontend/bindings/, nested by Go import
|
||||||
# path, rather than v2's frontend/wailsjs/. The `@go` alias absorbs the
|
# path, rather than v2's frontend/wailsjs/. The `@go` alias absorbs the
|
||||||
# constant prefix, so a call site imports '@go/library/library.js'.
|
# constant prefix, so a call site imports '@go/library/library.js'.
|
||||||
@@ -187,7 +225,7 @@ bindings: ## Regenerate frontend/bindings from the bound Go services
|
|||||||
sandbox-seed sandbox-seed-bulk sandbox-seeds e2e e2e-setup e2e-report \
|
sandbox-seed sandbox-seed-bulk sandbox-seeds e2e e2e-setup e2e-report \
|
||||||
perf perf-compare \
|
perf perf-compare \
|
||||||
ui-test ui-watch ui-visual ui-visual-update ui-setup \
|
ui-test ui-watch ui-visual ui-visual-update ui-setup \
|
||||||
bindings bindings-check skill-check commit-check
|
bindings bindings-check skill-check commit-check release-dry
|
||||||
|
|
||||||
# Base directory for fresh-install sandboxes. Deliberately NOT $TMPDIR:
|
# Base directory for fresh-install sandboxes. Deliberately NOT $TMPDIR:
|
||||||
# on most Linux distros /tmp is tmpfs (RAM-backed) and only a few GB, so
|
# on most Linux distros /tmp is tmpfs (RAM-backed) and only a few GB, so
|
||||||
|
|||||||
@@ -106,5 +106,8 @@ make dev # run with hot-reload
|
|||||||
make build-prod # produce a release binary
|
make build-prod # produce a release binary
|
||||||
```
|
```
|
||||||
|
|
||||||
More detail for contributors lives in
|
More detail for contributors lives in [`CLAUDE.md`](./CLAUDE.md) — the
|
||||||
[`docs/dev/overview.md`](./docs/dev/overview.md) and [`CLAUDE.md`](./CLAUDE.md).
|
architecture, the conventions and the reasons behind them. What is
|
||||||
|
being worked on is [the issue
|
||||||
|
tracker](https://git.ljones.me/yonlu/yellowjacket/issues); #73 is the
|
||||||
|
roadmap.
|
||||||
|
|||||||
+28
-5
@@ -191,6 +191,24 @@ func NewYellowJacketApp(
|
|||||||
yjApp.library.SetJobRegistry(yjApp.jobs)
|
yjApp.library.SetJobRegistry(yjApp.jobs)
|
||||||
yjApp.explore.SetJobRegistry(yjApp.jobs)
|
yjApp.explore.SetJobRegistry(yjApp.jobs)
|
||||||
|
|
||||||
|
// Whether this connection is one to spend ~0.6 GB of catalog on
|
||||||
|
// (plan 016 B4). The probe is injected from here because `explore` is
|
||||||
|
// imported by `cmd/indexbuild`, which must not link Wails: naming
|
||||||
|
// `application` there is what `TestIndexToolsDoNotImportWails`
|
||||||
|
// forbids.
|
||||||
|
//
|
||||||
|
// `application.Mobile`, not `application.Android`: the latter exists
|
||||||
|
// only under the `android` build tag, while `Mobile` is the portable
|
||||||
|
// name whose desktop implementation is a stub returning "" — which
|
||||||
|
// parses to "unknown" and refuses nothing. Plan 016 named the tagged
|
||||||
|
// one; this is the same call by the name every build has.
|
||||||
|
yjApp.explore.SetNetworkPolicy(
|
||||||
|
func() explore.Network {
|
||||||
|
return explore.ParseNetworkJSON(application.Mobile.NetworkJSON())
|
||||||
|
},
|
||||||
|
yjApp.appConfig.GetAllowMeteredCatalogDownload,
|
||||||
|
)
|
||||||
|
|
||||||
// Let the release prefetch skip albums the user already owns in
|
// Let the release prefetch skip albums the user already owns in
|
||||||
// full — those open with no catalog call at all, so warming their
|
// full — those open with no catalog call at all, so warming their
|
||||||
// tracklists spends the most expensive request in the app on
|
// tracklists spends the most expensive request in the app on
|
||||||
@@ -484,20 +502,24 @@ func (yj *YellowJacketApp) OnStartup(ctx context.Context) {
|
|||||||
// Register playback finished handler to drive queue auto-advance.
|
// Register playback finished handler to drive queue auto-advance.
|
||||||
yj.player.SetPlaybackFinishedHandler(yj.queue.OnPlaybackFinished)
|
yj.player.SetPlaybackFinishedHandler(yj.queue.OnPlaybackFinished)
|
||||||
|
|
||||||
// Initialize OS media controls (MPRIS on Linux, no-op elsewhere).
|
// Initialize OS media controls (MPRIS on desktop Linux, a
|
||||||
|
// MediaSession on Android, no-op elsewhere). The callbacks are the
|
||||||
|
// same on every platform; only what delivers them differs.
|
||||||
yj.mediaControls = mediacontrols.NewHandler(yj.logger)
|
yj.mediaControls = mediacontrols.NewHandler(yj.logger)
|
||||||
|
|
||||||
if err := yj.mediaControls.Init(mediacontrols.Callbacks{
|
if err := yj.mediaControls.Init(mediacontrols.Callbacks{
|
||||||
OnPlay: yj.queue.Play,
|
OnPlay: yj.queue.Play,
|
||||||
OnPause: func() {
|
OnPause: func() {
|
||||||
if err := yj.player.Pause(); err != nil {
|
if err := yj.player.Pause(); err != nil {
|
||||||
yj.logger.Warn("MPRIS Pause failed", "err", err)
|
yj.logger.Warn("Media controls Pause failed", "err", err)
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
OnPlayPause: func() {
|
OnPlayPause: func() {
|
||||||
if yj.player.IsPlaying() {
|
if yj.player.IsPlaying() {
|
||||||
if err := yj.player.Pause(); err != nil {
|
if err := yj.player.Pause(); err != nil {
|
||||||
yj.logger.Warn("MPRIS PlayPause(pause) failed", "err", err)
|
yj.logger.Warn(
|
||||||
|
"Media controls PlayPause(pause) failed", "err", err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
yj.queue.Play()
|
yj.queue.Play()
|
||||||
@@ -505,14 +527,14 @@ func (yj *YellowJacketApp) OnStartup(ctx context.Context) {
|
|||||||
},
|
},
|
||||||
OnStop: func() {
|
OnStop: func() {
|
||||||
if err := yj.player.Pause(); err != nil {
|
if err := yj.player.Pause(); err != nil {
|
||||||
yj.logger.Warn("MPRIS Stop failed", "err", err)
|
yj.logger.Warn("Media controls Stop failed", "err", err)
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
OnNext: yj.queue.Next,
|
OnNext: yj.queue.Next,
|
||||||
OnPrevious: yj.queue.Previous,
|
OnPrevious: yj.queue.Previous,
|
||||||
OnSeek: func(positionSec int) {
|
OnSeek: func(positionSec int) {
|
||||||
if err := yj.player.Seek(positionSec); err != nil {
|
if err := yj.player.Seek(positionSec); err != nil {
|
||||||
yj.logger.Warn("MPRIS Seek failed", "err", err)
|
yj.logger.Warn("Media controls Seek failed", "err", err)
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
OnVolume: func(vol float64) {
|
OnVolume: func(vol float64) {
|
||||||
@@ -522,6 +544,7 @@ func (yj *YellowJacketApp) OnStartup(ctx context.Context) {
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
},
|
},
|
||||||
|
OnDuck: yj.player.SetDuck,
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
yj.logger.Error(
|
yj.logger.Error(
|
||||||
"Failed to initialize media controls",
|
"Failed to initialize media controls",
|
||||||
|
|||||||
@@ -411,9 +411,10 @@ func (c *Config) SetDownloadPreferences(prefs download.AutoDownloadPrefs) error
|
|||||||
formats = append(formats, string(f))
|
formats = append(formats, string(f))
|
||||||
}
|
}
|
||||||
|
|
||||||
c.Downloads.MinFileSizeMB = prefs.MinSizeMB
|
c.Downloads.MinKbps = prefs.MinKbps
|
||||||
|
c.Downloads.MaxKbps = prefs.MaxKbps
|
||||||
|
c.Downloads.PreferredKbps = prefs.PreferredKbps
|
||||||
c.Downloads.MaxFileSizeMB = prefs.MaxSizeMB
|
c.Downloads.MaxFileSizeMB = prefs.MaxSizeMB
|
||||||
c.Downloads.PreferredFileSizeMB = prefs.PreferredSizeMB
|
|
||||||
c.Downloads.AllowedFormats = formats
|
c.Downloads.AllowedFormats = formats
|
||||||
|
|
||||||
if err := c.Save(); err != nil {
|
if err := c.Save(); err != nil {
|
||||||
@@ -620,6 +621,51 @@ func (c *Config) SetQueueFallback(mode string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetAllowMeteredCatalogDownload reports whether the ~0.6 GB Explore
|
||||||
|
// catalog may be fetched on a metered connection.
|
||||||
|
func (c *Config) GetAllowMeteredCatalogDownload() bool {
|
||||||
|
if c.General == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
return c.General.AllowMeteredCatalogDownload
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetAllowMeteredCatalogDownload saves the metered-download permission.
|
||||||
|
//
|
||||||
|
// There is nothing to validate and nothing to restart: the policy is
|
||||||
|
// read at the moment a download would start, so turning it on takes
|
||||||
|
// effect on the next attempt rather than needing this launch to be over.
|
||||||
|
func (c *Config) SetAllowMeteredCatalogDownload(allow bool) error {
|
||||||
|
if c.General == nil {
|
||||||
|
c.General = &GeneralConfig{}
|
||||||
|
c.General.ApplyDefaults()
|
||||||
|
}
|
||||||
|
|
||||||
|
c.General.AllowMeteredCatalogDownload = allow
|
||||||
|
|
||||||
|
if err := c.Save(); err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"could not save config: %w", err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
events.Emit(
|
||||||
|
c.ctx,
|
||||||
|
events.GeneralConfigChanged,
|
||||||
|
map[string]any{
|
||||||
|
"AllowMeteredCatalogDownload": allow,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
c.logger.Info(
|
||||||
|
"metered catalog download permission updated",
|
||||||
|
"allow", allow,
|
||||||
|
)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// GetTrackListColumns returns the configured track-list columns.
|
// GetTrackListColumns returns the configured track-list columns.
|
||||||
func (c *Config) GetTrackListColumns() []tracklist.Column {
|
func (c *Config) GetTrackListColumns() []tracklist.Column {
|
||||||
if c.TrackList == nil {
|
if c.TrackList == nil {
|
||||||
|
|||||||
@@ -48,6 +48,12 @@ var errUnknownQueueFallback = errors.New("unknown queue fallback")
|
|||||||
type GeneralConfig struct {
|
type GeneralConfig struct {
|
||||||
DefaultPage DefaultPage `toml:"DefaultPage"`
|
DefaultPage DefaultPage `toml:"DefaultPage"`
|
||||||
QueueFallback QueueFallback `toml:"QueueFallback"`
|
QueueFallback QueueFallback `toml:"QueueFallback"`
|
||||||
|
// AllowMeteredCatalogDownload permits the ~0.6 GB Explore catalog to
|
||||||
|
// be fetched on a connection the platform calls cellular. It defaults
|
||||||
|
// to false, which is the whole point: the zero value is the safe one,
|
||||||
|
// so an existing config with no such key refuses by default rather
|
||||||
|
// than needing a migration to become careful.
|
||||||
|
AllowMeteredCatalogDownload bool `toml:"AllowMeteredCatalogDownload"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ApplyDefaults fills zero-value fields with sensible defaults.
|
// ApplyDefaults fills zero-value fields with sensible defaults.
|
||||||
|
|||||||
@@ -89,6 +89,14 @@ func NewDB(logger *slog.Logger) (*DB, error) {
|
|||||||
return nil, fmt.Errorf("could not apply PRAGMAs: %w", err)
|
return nil, fmt.Errorf("could not apply PRAGMAs: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Before the schema is applied, not after: applySchema is
|
||||||
|
// CREATE ... IF NOT EXISTS, which no-ops against a table that
|
||||||
|
// already exists in an older shape. Retiring the stale one first is
|
||||||
|
// what turns that no-op into a create.
|
||||||
|
if err := retireStaleTables(dbCtx, db, logger); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
if err := applySchema(dbCtx, db); err != nil {
|
if err := applySchema(dbCtx, db); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
-- The decomposition of a multi-artist credit, from the MusicBrainz
|
||||||
|
-- dump. One row per credited artist, in credit order.
|
||||||
|
--
|
||||||
|
-- A credit is ordered parts, and the credit *string* is derived from
|
||||||
|
-- them -- MusicBrainz's own `artist_credit.name` is a cached render and
|
||||||
|
-- nothing more. Rendering is a concatenation:
|
||||||
|
--
|
||||||
|
-- for each part in position order:
|
||||||
|
-- emit link(credited_name -> artist_mbid)
|
||||||
|
-- emit text(join_phrase)
|
||||||
|
--
|
||||||
|
-- so the link boundaries are known by construction. That is the whole
|
||||||
|
-- reason this table exists, and it is why nothing may reconstruct a
|
||||||
|
-- credit by *searching* for a name inside a credit string: the stored
|
||||||
|
-- string may have come from a file's tags while the parts come from the
|
||||||
|
-- catalog, and measured on a real library those disagree for about one
|
||||||
|
-- in three multi-artist credits ("Skrillex feat. Swae Lee" tagged
|
||||||
|
-- against "Skrillex & Swae Lee" upstream). A search would miss, or
|
||||||
|
-- match the wrong span.
|
||||||
|
--
|
||||||
|
-- `credited_name` is the name *as credited*, which is not the artist's
|
||||||
|
-- canonical name: MusicBrainz credits "Snoop Dogg" on a track by the
|
||||||
|
-- artist whose name is "Snoop Doggy Dogg". It is stored per row rather
|
||||||
|
-- than joined from an artist table for exactly that reason.
|
||||||
|
--
|
||||||
|
-- Only *multi-artist* credits are stored. A single-artist credit is
|
||||||
|
-- (name, "") and is already fully described by explore_index's
|
||||||
|
-- artist_name and artist_mbid; storing those would roughly triple the
|
||||||
|
-- table to say nothing new.
|
||||||
|
--
|
||||||
|
-- Credits are shared: an album's twelve tracks by one artist reference
|
||||||
|
-- one credit_id. That is the opposite of the local library's verdict
|
||||||
|
-- in plan 013, and correctly so -- credit sharing is 1:1 in one
|
||||||
|
-- person's files and genuinely many-to-one across a 2M-row catalog.
|
||||||
|
--
|
||||||
|
-- MBIDs are the same 16 raw bytes explore_index stores, for the same
|
||||||
|
-- size reason and with the same CHECK, so a stringly write fails at the
|
||||||
|
-- insert that made it rather than reading back as no rows at all. See
|
||||||
|
-- backend/explore/mbid.go.
|
||||||
|
CREATE TABLE IF NOT EXISTS artist_credit_part (
|
||||||
|
credit_id INTEGER NOT NULL,
|
||||||
|
position INTEGER NOT NULL,
|
||||||
|
artist_mbid BLOB NOT NULL CHECK(length(artist_mbid) = 16),
|
||||||
|
|
||||||
|
-- The name as credited on this release, which may differ from the
|
||||||
|
-- artist's canonical name. Display uses this; navigation uses the
|
||||||
|
-- MBID above.
|
||||||
|
credited_name TEXT NOT NULL,
|
||||||
|
|
||||||
|
-- The literal connector that follows this part -- " feat. ", " & ",
|
||||||
|
-- ", ", or "" on the last part. Rendered as plain text between two
|
||||||
|
-- links.
|
||||||
|
join_phrase TEXT NOT NULL DEFAULT '',
|
||||||
|
|
||||||
|
PRIMARY KEY (credit_id, position)
|
||||||
|
) WITHOUT ROWID;
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
-- Which credit a catalog entity is credited to. One row per recording
|
||||||
|
-- or release group whose credit names more than one artist.
|
||||||
|
--
|
||||||
|
-- This is a table rather than an `explore_index.artist_credit_id`
|
||||||
|
-- column, and that is a deliberate consequence of how this app applies
|
||||||
|
-- its schema. `applySchema` is CREATE ... IF NOT EXISTS and there is
|
||||||
|
-- no migration chain (plan 013), so a *column* added to an existing
|
||||||
|
-- table never reaches a database that already has it -- while a new
|
||||||
|
-- *table* is created on every install, old or new, for free.
|
||||||
|
-- explore_index is the one table nobody can afford to drop and rebuild
|
||||||
|
-- on a schema change: it is the artifact users download rather than
|
||||||
|
-- derive.
|
||||||
|
--
|
||||||
|
-- Only multi-artist credits are referenced here, matching
|
||||||
|
-- artist_credit_part. An entity with no row is credited to exactly one
|
||||||
|
-- artist, which explore_index's own artist_name and artist_mbid already
|
||||||
|
-- describe -- so absence is the common case and means "nothing to
|
||||||
|
-- decompose", not "unknown".
|
||||||
|
--
|
||||||
|
-- `credit_id` is opaque and is only meaningful against the
|
||||||
|
-- artist_credit_part rows built or imported alongside it. The two are
|
||||||
|
-- always written together; nothing persists a credit_id anywhere else.
|
||||||
|
-- The local library stores resolved parts, never this id.
|
||||||
|
CREATE TABLE IF NOT EXISTS artist_credit_ref (
|
||||||
|
mbid BLOB NOT NULL PRIMARY KEY CHECK(length(mbid) = 16),
|
||||||
|
credit_id INTEGER NOT NULL
|
||||||
|
) WITHOUT ROWID;
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_artist_credit_ref_credit
|
||||||
|
ON artist_credit_ref(credit_id);
|
||||||
@@ -27,6 +27,19 @@ type Artist struct {
|
|||||||
Mbid sql.NullString
|
Mbid sql.NullString
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type ArtistCreditPart struct {
|
||||||
|
CreditID int64
|
||||||
|
Position int64
|
||||||
|
ArtistMbid []byte
|
||||||
|
CreditedName string
|
||||||
|
JoinPhrase string
|
||||||
|
}
|
||||||
|
|
||||||
|
type ArtistCreditRef struct {
|
||||||
|
Mbid []byte
|
||||||
|
CreditID int64
|
||||||
|
}
|
||||||
|
|
||||||
type ArtistEnrichment struct {
|
type ArtistEnrichment struct {
|
||||||
ArtistMbid string
|
ArtistMbid string
|
||||||
BrowsedAt sql.NullTime
|
BrowsedAt sql.NullTime
|
||||||
|
|||||||
@@ -0,0 +1,560 @@
|
|||||||
|
package database
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"io/fs"
|
||||||
|
"log/slog"
|
||||||
|
"maps"
|
||||||
|
"path"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"yellowjacket/backend/datamap"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This file repairs the one thing `CREATE TABLE IF NOT EXISTS` cannot.
|
||||||
|
//
|
||||||
|
// `sql/schemas/` is the single description of the schema and there is no
|
||||||
|
// migration chain (plan 013): a schema change is one edit to one file.
|
||||||
|
// That works perfectly for a *new* table, which every install then
|
||||||
|
// creates, and not at all for a changed one -- `IF NOT EXISTS` reaches
|
||||||
|
// an existing table only if its shape already matches, and otherwise
|
||||||
|
// silently no-ops. The user's answer to that is "delete and rescan"
|
||||||
|
// (plan 013, open question 1), which is free for everything a rescan
|
||||||
|
// rebuilds.
|
||||||
|
//
|
||||||
|
// It is not free for the catalog. explore_index is a *downloaded
|
||||||
|
// artifact*, not something derived from the user's files, and it is the
|
||||||
|
// largest thing this app stores. So it went stale instead: plan 014
|
||||||
|
// added `total_tracks` to the schema and to `indexRowFields` -- the one
|
||||||
|
// projection every explore read uses -- and no database that already
|
||||||
|
// existed ever grew the column. Every Explore search, browse, artist
|
||||||
|
// page and album page on such an install fails with
|
||||||
|
// "no such column: total_tracks", while a fresh install is perfectly
|
||||||
|
// healthy, which is why the tests did not see it. The same databases
|
||||||
|
// are stale a second way, from the same plan: their `mbid` columns are
|
||||||
|
// still TEXT where the schema now declares BLOB, and SQLite does not
|
||||||
|
// coerce between the two -- a comparison against 16 raw bytes simply
|
||||||
|
// returns no rows.
|
||||||
|
//
|
||||||
|
// The repair is to notice and drop, not to migrate. A dropped catalog
|
||||||
|
// costs one artifact download (about a minute); the alternative --
|
||||||
|
// ALTER TABLE ADD COLUMN, which would handle `total_tracks` alone
|
||||||
|
// cheaply -- cannot express the TEXT-to-BLOB half at all, and would
|
||||||
|
// leave those installs quietly broken while reporting success.
|
||||||
|
//
|
||||||
|
// Everything except `Authored` is eligible. `Cache` is rebuildable by
|
||||||
|
// definition; `Owned` is a projection of the user's files and a rescan
|
||||||
|
// rebuilds it, which is plan 013's stated answer to exactly this
|
||||||
|
// situation ("delete and rescan", open question 1); `Derived` is
|
||||||
|
// computed from Owned. No `Authored` table is ever dropped here --
|
||||||
|
// that is the whole point of the datamap, and it is asserted by
|
||||||
|
// TestAuthoredTablesAreNeverRetired rather than only stated.
|
||||||
|
//
|
||||||
|
// What that does *not* buy is immunity for authored rows that reference
|
||||||
|
// a retired table. `audio_files` is MIXED KIND: `play_count`,
|
||||||
|
// `last_played` and `tag_status` are authored columns on an Owned
|
||||||
|
// table, and they go with it. Playlists survive as playlists, and
|
||||||
|
// their entries survive pointing at nothing. That cost was weighed and
|
||||||
|
// accepted rather than overlooked -- the alternative is to carry the
|
||||||
|
// authored columns across the rebuild keyed on file_path, which stays a
|
||||||
|
// real option if this ever bites harder than it is worth.
|
||||||
|
//
|
||||||
|
// **This relies on foreign_keys being ON**, which applyPRAGMAs has
|
||||||
|
// already done by the time NewDB calls it, and the dependency is not
|
||||||
|
// cosmetic. SQLite performs an implicit DELETE before dropping a table
|
||||||
|
// when foreign keys are enabled, so `playlist_tracks.audio_file_id` --
|
||||||
|
// declared ON DELETE SET NULL -- is nulled. With foreign keys off, no
|
||||||
|
// action fires and those rows keep the ids they had, which a rescan
|
||||||
|
// then reissues starting from 1: every playlist would silently fill
|
||||||
|
// with *different songs*. Nulled entries are merely empty; stale ones
|
||||||
|
// are wrong, and wrong quietly. TestRetiringOwnedTablesDoesNotDangle
|
||||||
|
// is what stops a future reordering turning one into the other.
|
||||||
|
|
||||||
|
// retireGroups are tables that must be retired together. A catalog
|
||||||
|
// whose rows are gone must not keep the full-text index built over
|
||||||
|
// them, nor the metadata claiming the import that produced them
|
||||||
|
// finished -- that marker is exactly what stops the artifact being
|
||||||
|
// fetched again. applySchema recreates all three empty immediately
|
||||||
|
// afterwards, and the ordinary "no index yet" path takes over.
|
||||||
|
var retireGroups = [][]string{
|
||||||
|
{
|
||||||
|
"explore_index",
|
||||||
|
"explore_index_fts",
|
||||||
|
"explore_index_meta",
|
||||||
|
"explore_champion_fts",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// schemaColumn is one column as the schema file declares it.
|
||||||
|
type schemaColumn struct {
|
||||||
|
name string
|
||||||
|
typ string
|
||||||
|
}
|
||||||
|
|
||||||
|
// retireStaleTables drops every non-authored table whose live shape no
|
||||||
|
// longer matches what sql/schemas/ declares, plus any table the schema
|
||||||
|
// no longer describes at all, so applySchema can create the current
|
||||||
|
// shape afresh. It runs before applySchema and is a no-op on a new
|
||||||
|
// database, where the tables do not exist yet.
|
||||||
|
func retireStaleTables(
|
||||||
|
ctx context.Context, db *sql.DB, logger *slog.Logger,
|
||||||
|
) error {
|
||||||
|
declared, err := declaredTables()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
stale := make(map[string]string)
|
||||||
|
|
||||||
|
for table, columns := range declared {
|
||||||
|
entry, ok := datamap.Lookup(table)
|
||||||
|
if !ok || entry.Kind == datamap.Authored || entry.FTS {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Whether a stale Cache table may be rebuilt is decided per
|
||||||
|
// binary, at compile time: the app re-downloads its catalog in
|
||||||
|
// about a minute, cmd/indexbuild would re-derive it from ~205 GB
|
||||||
|
// of dumps. See staleshape_policy.go.
|
||||||
|
if entry.Kind == datamap.Cache && !retireStaleCache {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
reason, err := staleReason(ctx, db, table, columns)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if reason != "" {
|
||||||
|
stale[table] = reason
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
obsolete, err := obsoleteTables(ctx, db)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
maps.Copy(stale, obsolete)
|
||||||
|
|
||||||
|
if len(stale) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return retireGroupsFor(ctx, db, logger, stale)
|
||||||
|
}
|
||||||
|
|
||||||
|
// obsoleteTables are live tables the schema no longer describes at all.
|
||||||
|
// TestCatalogCoversSchema makes the datamap a complete description of
|
||||||
|
// the current schema, so a table it does not know is one a past version
|
||||||
|
// created and this one does not -- plan 013 alone left seven behind
|
||||||
|
// (recordings, release_groups, artist_credit, artist_credit_artist,
|
||||||
|
// release_group_recordings, recording_genres) plus the
|
||||||
|
// schema_migrations table that squashing the chain retired. They are
|
||||||
|
// dead weight, and one of them holding a foreign key into a table being
|
||||||
|
// rebuilt is worse than dead weight.
|
||||||
|
//
|
||||||
|
// SQLite's own bookkeeping and FTS shadow tables are not obsolete:
|
||||||
|
// datamap.Lookup resolves a shadow table to its parent, and IsInternal
|
||||||
|
// covers the rest.
|
||||||
|
func obsoleteTables(ctx context.Context, db *sql.DB) (map[string]string, error) {
|
||||||
|
rows, err := db.QueryContext(
|
||||||
|
ctx, "SELECT name FROM sqlite_master WHERE type = 'table'",
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("could not list tables: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
out := make(map[string]string)
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var name string
|
||||||
|
|
||||||
|
if err := rows.Scan(&name); err != nil {
|
||||||
|
return nil, fmt.Errorf("could not scan table name: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if datamap.IsInternal(name) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, known := datamap.Lookup(name); !known {
|
||||||
|
out[name] = "the schema no longer describes this table"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return nil, fmt.Errorf("could not read table list: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// retireGroupsFor drops each stale table along with everything its
|
||||||
|
// retire group says must go with it.
|
||||||
|
func retireGroupsFor(
|
||||||
|
ctx context.Context, db *sql.DB, logger *slog.Logger,
|
||||||
|
stale map[string]string,
|
||||||
|
) error {
|
||||||
|
drop := make(map[string]string)
|
||||||
|
|
||||||
|
for table, reason := range stale {
|
||||||
|
drop[table] = reason
|
||||||
|
|
||||||
|
for _, group := range retireGroups {
|
||||||
|
if !slices.Contains(group, table) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, member := range group {
|
||||||
|
if _, already := drop[member]; !already {
|
||||||
|
drop[member] = "retired with " + table
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return dropDeferred(ctx, db, logger, drop)
|
||||||
|
}
|
||||||
|
|
||||||
|
// dropDeferred drops every named table in one transaction with foreign
|
||||||
|
// key enforcement deferred to the commit.
|
||||||
|
//
|
||||||
|
// The deferral is required and the two obvious alternatives are both
|
||||||
|
// wrong. These tables reference each other -- pre-013 `audio_files`
|
||||||
|
// has a foreign key into `recordings`, which is itself being retired --
|
||||||
|
// so dropping them one at a time in an arbitrary order fails with
|
||||||
|
// "FOREIGN KEY constraint failed" on whichever is unlucky enough to go
|
||||||
|
// first, and there is no order that is safe in general. Turning
|
||||||
|
// foreign keys *off* for the duration would fix that and silently take
|
||||||
|
// the ON DELETE SET NULL on `playlist_tracks.audio_file_id` with it,
|
||||||
|
// leaving playlist entries pointing at ids a rescan reissues to
|
||||||
|
// different songs -- the exact failure
|
||||||
|
// TestRetiringOwnedTablesDoesNotDangle exists to prevent.
|
||||||
|
//
|
||||||
|
// Deferring keeps the actions firing while tolerating the inconsistency
|
||||||
|
// in the middle, and the commit then checks that the end state is
|
||||||
|
// sound. It is set inside the transaction because SQLite resets it at
|
||||||
|
// every commit.
|
||||||
|
func dropDeferred(
|
||||||
|
ctx context.Context, db *sql.DB, logger *slog.Logger,
|
||||||
|
drop map[string]string,
|
||||||
|
) error {
|
||||||
|
tx, err := db.BeginTx(ctx, nil)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("could not begin the retire transaction: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = tx.Rollback() }()
|
||||||
|
|
||||||
|
if _, err := tx.ExecContext(ctx, "PRAGMA defer_foreign_keys = ON"); err != nil {
|
||||||
|
return fmt.Errorf("could not defer foreign keys: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sorted, so a failure is reproducible. Map order is random, and a
|
||||||
|
// bug that depends on which table happens to go first reproduces on
|
||||||
|
// one run in three and passes review on the other two -- which is
|
||||||
|
// exactly how the foreign-key ordering above reached a real
|
||||||
|
// database. Sorting does not make any order *safe*; the deferral
|
||||||
|
// does that.
|
||||||
|
for _, table := range slices.Sorted(maps.Keys(drop)) {
|
||||||
|
logger.Warn(
|
||||||
|
"retiring a table the schema no longer describes",
|
||||||
|
"table", table,
|
||||||
|
"reason", drop[table],
|
||||||
|
)
|
||||||
|
|
||||||
|
if _, err := tx.ExecContext(
|
||||||
|
ctx, "DROP TABLE IF EXISTS "+quoteIdent(table),
|
||||||
|
); err != nil {
|
||||||
|
return fmt.Errorf("could not retire stale table %s: %w", table, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
return fmt.Errorf("could not commit the retire: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// staleReason reports why a live table disagrees with its declaration,
|
||||||
|
// or "" when it agrees. A column the live table does not have is the
|
||||||
|
// additive case; a column whose declared type changed is the one an
|
||||||
|
// ALTER could not fix anyway. Columns the live table has and the
|
||||||
|
// schema no longer declares are ignored: they cost nothing and dropping
|
||||||
|
// the table over one would retire a healthy catalog.
|
||||||
|
func staleReason(
|
||||||
|
ctx context.Context, db *sql.DB, table string, columns []schemaColumn,
|
||||||
|
) (string, error) {
|
||||||
|
live, err := liveColumns(ctx, db, table)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(live) == 0 {
|
||||||
|
// Not present at all: applySchema is about to create it.
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, col := range columns {
|
||||||
|
liveType, present := live[col.name]
|
||||||
|
if !present {
|
||||||
|
return "missing column " + col.name, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if !sameDeclaredType(col.typ, liveType) {
|
||||||
|
return fmt.Sprintf(
|
||||||
|
"column %s is %s, schema declares %s",
|
||||||
|
col.name, liveType, col.typ,
|
||||||
|
), nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// liveColumns returns the live table's columns and their declared types,
|
||||||
|
// empty when the table does not exist.
|
||||||
|
func liveColumns(
|
||||||
|
ctx context.Context, db *sql.DB, table string,
|
||||||
|
) (map[string]string, error) {
|
||||||
|
rows, err := db.QueryContext(
|
||||||
|
ctx, "SELECT name, type FROM pragma_table_info(?)", table,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("could not inspect table %s: %w", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
out := make(map[string]string)
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var name, typ string
|
||||||
|
|
||||||
|
if err := rows.Scan(&name, &typ); err != nil {
|
||||||
|
return nil, fmt.Errorf("could not scan column of %s: %w", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
out[name] = typ
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return nil, fmt.Errorf("could not read columns of %s: %w", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// sameDeclaredType compares two SQLite type names. They are compared
|
||||||
|
// case-insensitively and only on the leading word, so INTEGER matches
|
||||||
|
// INTEGER and VARCHAR(20) matches VARCHAR -- SQLite's affinity rules
|
||||||
|
// make finer distinctions meaningless, and a difference that fine is
|
||||||
|
// not worth retiring a catalog over. An empty declared type matches
|
||||||
|
// anything, which is what a column declared with only constraints has.
|
||||||
|
func sameDeclaredType(declared, live string) bool {
|
||||||
|
d := strings.ToUpper(strings.Fields(declared + " ")[0])
|
||||||
|
l := strings.ToUpper(strings.Fields(live + " ")[0])
|
||||||
|
|
||||||
|
if d == "" || l == "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
if i := strings.IndexByte(d, '('); i >= 0 {
|
||||||
|
d = d[:i]
|
||||||
|
}
|
||||||
|
|
||||||
|
if i := strings.IndexByte(l, '('); i >= 0 {
|
||||||
|
l = l[:i]
|
||||||
|
}
|
||||||
|
|
||||||
|
return d == l
|
||||||
|
}
|
||||||
|
|
||||||
|
// declaredTables parses every CREATE TABLE in sql/schemas/ into its
|
||||||
|
// column list. Parsing the schema rather than writing the expectation
|
||||||
|
// down a second time is the point: a second list is a second thing to
|
||||||
|
// forget, which is the fault this whole file exists to repair.
|
||||||
|
func declaredTables() (map[string][]schemaColumn, error) {
|
||||||
|
dirEntries, err := schemas.ReadDir("sql/schemas")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("could not read schemas directory: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
out := make(map[string][]schemaColumn)
|
||||||
|
|
||||||
|
for _, dirEntry := range dirEntries {
|
||||||
|
if dirEntry.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
content, err := fs.ReadFile(schemas, path.Join("sql/schemas", dirEntry.Name()))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("could not read %s: %w", dirEntry.Name(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
maps.Copy(out, parseCreateTables(string(content)))
|
||||||
|
}
|
||||||
|
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// constraintKeywords begin a table constraint rather than a column.
|
||||||
|
var constraintKeywords = map[string]bool{
|
||||||
|
"PRIMARY": true, "FOREIGN": true, "UNIQUE": true,
|
||||||
|
"CHECK": true, "CONSTRAINT": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseCreateTables extracts the column names and declared types of
|
||||||
|
// every non-virtual CREATE TABLE in one schema file.
|
||||||
|
func parseCreateTables(content string) map[string][]schemaColumn {
|
||||||
|
out := make(map[string][]schemaColumn)
|
||||||
|
rest := stripLineComments(content)
|
||||||
|
|
||||||
|
for {
|
||||||
|
idx := indexFold(rest, "CREATE TABLE ")
|
||||||
|
if idx < 0 {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
rest = rest[idx+len("CREATE TABLE "):]
|
||||||
|
|
||||||
|
head, body, ok := splitTableBody(rest)
|
||||||
|
if !ok {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
if name := tableName(head); name != "" {
|
||||||
|
out[name] = parseColumns(body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// tableName pulls the table name out of the text between "CREATE TABLE"
|
||||||
|
// and its opening parenthesis, dropping an IF NOT EXISTS and any
|
||||||
|
// quoting.
|
||||||
|
func tableName(head string) string {
|
||||||
|
head = strings.TrimSpace(head)
|
||||||
|
head = strings.TrimPrefix(head, "IF NOT EXISTS ")
|
||||||
|
head = strings.TrimPrefix(head, "if not exists ")
|
||||||
|
|
||||||
|
fields := strings.Fields(head)
|
||||||
|
if len(fields) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return strings.Trim(fields[len(fields)-1], `"'`+"`")
|
||||||
|
}
|
||||||
|
|
||||||
|
// splitTableBody returns the text before the table's opening paren and
|
||||||
|
// the balanced text inside it.
|
||||||
|
func splitTableBody(s string) (head, body string, ok bool) {
|
||||||
|
open := strings.IndexByte(s, '(')
|
||||||
|
if open < 0 {
|
||||||
|
return "", "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
depth := 0
|
||||||
|
|
||||||
|
for i := open; i < len(s); i++ {
|
||||||
|
switch s[i] {
|
||||||
|
case '(':
|
||||||
|
depth++
|
||||||
|
case ')':
|
||||||
|
depth--
|
||||||
|
|
||||||
|
if depth == 0 {
|
||||||
|
return s[:open], s[open+1 : i], true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseColumns splits a table body on its top-level commas and keeps
|
||||||
|
// the parts that are columns rather than table constraints.
|
||||||
|
func parseColumns(body string) []schemaColumn {
|
||||||
|
var (
|
||||||
|
out []schemaColumn
|
||||||
|
depth int
|
||||||
|
start int
|
||||||
|
)
|
||||||
|
|
||||||
|
parts := make([]string, 0, 8)
|
||||||
|
|
||||||
|
for i := range len(body) {
|
||||||
|
switch body[i] {
|
||||||
|
case '(':
|
||||||
|
depth++
|
||||||
|
case ')':
|
||||||
|
depth--
|
||||||
|
case ',':
|
||||||
|
if depth == 0 {
|
||||||
|
parts = append(parts, body[start:i])
|
||||||
|
start = i + 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
parts = append(parts, body[start:])
|
||||||
|
|
||||||
|
for _, part := range parts {
|
||||||
|
fields := strings.Fields(part)
|
||||||
|
if len(fields) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// A table constraint need not be followed by a space --
|
||||||
|
// "UNIQUE(mbid)" is one field, and reading it as a column name
|
||||||
|
// makes an entirely healthy table look stale, which retires a
|
||||||
|
// catalog nobody asked to lose.
|
||||||
|
head := fields[0]
|
||||||
|
if i := strings.IndexByte(head, '('); i >= 0 {
|
||||||
|
head = head[:i]
|
||||||
|
}
|
||||||
|
|
||||||
|
if constraintKeywords[strings.ToUpper(head)] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
col := schemaColumn{name: strings.Trim(head, `"'`+"`")}
|
||||||
|
if len(fields) > 1 {
|
||||||
|
col.typ = fields[1]
|
||||||
|
}
|
||||||
|
|
||||||
|
out = append(out, col)
|
||||||
|
}
|
||||||
|
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// stripLineComments removes -- comments, which otherwise contribute
|
||||||
|
// stray parentheses and commas to the parse.
|
||||||
|
func stripLineComments(s string) string {
|
||||||
|
lines := strings.Split(s, "\n")
|
||||||
|
for i, line := range lines {
|
||||||
|
if idx := strings.Index(line, "--"); idx >= 0 {
|
||||||
|
lines[i] = line[:idx]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return strings.Join(lines, "\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
// indexFold is a case-insensitive strings.Index.
|
||||||
|
func indexFold(s, substr string) int {
|
||||||
|
return strings.Index(strings.ToUpper(s), strings.ToUpper(substr))
|
||||||
|
}
|
||||||
|
|
||||||
|
// quoteIdent quotes a table name for interpolation into DDL, which
|
||||||
|
// cannot take a bound parameter.
|
||||||
|
func quoteIdent(name string) string {
|
||||||
|
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
//go:build !indexbuild
|
||||||
|
|
||||||
|
package database
|
||||||
|
|
||||||
|
// retireStaleCache reports whether a Cache table whose shape no longer
|
||||||
|
// matches the schema may be dropped and rebuilt.
|
||||||
|
//
|
||||||
|
// In the app: yes. The only Cache table large enough to care about is
|
||||||
|
// the catalog, and the app does not derive it — it downloads it. A
|
||||||
|
// stale one costs about a minute of re-fetching the artifact, and
|
||||||
|
// keeping it costs every Explore read on the install, because a
|
||||||
|
// projection naming a column the table does not have fails outright.
|
||||||
|
//
|
||||||
|
// In cmd/indexbuild: no, and the file next to this one says why.
|
||||||
|
const retireStaleCache = true
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
//go:build indexbuild
|
||||||
|
|
||||||
|
package database
|
||||||
|
|
||||||
|
// retireStaleCache is false here, and this is the whole reason the
|
||||||
|
// policy is a build tag rather than a rule inside retireStaleTables.
|
||||||
|
//
|
||||||
|
// The index database is the one place in this project where the catalog
|
||||||
|
// is *derived* rather than downloaded. Rebuilding it is a ~205 GB dump
|
||||||
|
// stream over hours, resumed across runs from a checkpoint on a
|
||||||
|
// persistent volume; that volume exists for no other purpose. The app's
|
||||||
|
// answer to a stale catalog — drop it, fetch the artifact again — is
|
||||||
|
// not available here, because this database *is* what the artifact is
|
||||||
|
// cut from.
|
||||||
|
//
|
||||||
|
// This was not hypothetical. The repair shipped without it and dropped
|
||||||
|
// the CI catalog on its first run:
|
||||||
|
//
|
||||||
|
// retiring a table ... table=explore_index
|
||||||
|
// reason="column entity_type is TEXT, schema declares INTEGER"
|
||||||
|
// index maintenance mode=build reason="no completed import yet"
|
||||||
|
//
|
||||||
|
// The shape mismatch was real and the drop was correct by the app's
|
||||||
|
// rule. It was still wrong here: that database is deliberately kept in
|
||||||
|
// the older encoding, which is what `fix(indexexport): read an index
|
||||||
|
// older than the binary` exists to tolerate. A rule that is right for
|
||||||
|
// every install and catastrophic for one database has to be told which
|
||||||
|
// one it is in, and a build tag is how this project already tells the
|
||||||
|
// index tools apart (backend/events/runtime_indexbuild.go,
|
||||||
|
// backend/explore/servicestartup.go, dumpbuild_stub.go).
|
||||||
|
//
|
||||||
|
// cmd/indexbuild has its own repair for the half it *can* safely
|
||||||
|
// discard: retireLibraryTables drops every table the datamap does not
|
||||||
|
// classify as Cache, which is empty by construction in that database.
|
||||||
|
// Between the two, the library half is repaired and the catalog is
|
||||||
|
// never touched.
|
||||||
|
const retireStaleCache = false
|
||||||
@@ -0,0 +1,499 @@
|
|||||||
|
package database
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"log/slog"
|
||||||
|
"path"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
_ "modernc.org/sqlite"
|
||||||
|
)
|
||||||
|
|
||||||
|
// testLogger discards the repair's warnings; the tests assert on the
|
||||||
|
// database, not on the log.
|
||||||
|
func testLogger() *slog.Logger {
|
||||||
|
return slog.New(slog.DiscardHandler)
|
||||||
|
}
|
||||||
|
|
||||||
|
// openRaw opens a scratch database file with no schema applied, so a
|
||||||
|
// test can build an *old* shape and then let NewDB's repair meet it.
|
||||||
|
func openRaw(t *testing.T, dir string) *sql.DB {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
db, err := sql.Open("sqlite", path.Join(dir, "yj.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
|
return db
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiresIndexMissingAColumn is plan 014's bug, symptom first: an
|
||||||
|
// explore_index created before `total_tracks` existed, met by the
|
||||||
|
// projection every explore read uses. Before the repair this failed
|
||||||
|
// with "no such column: total_tracks" on every install that already had
|
||||||
|
// a catalog, while a fresh one was perfectly healthy.
|
||||||
|
func TestRetiresIndexMissingAColumn(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
dir := t.TempDir()
|
||||||
|
db := openRaw(t, dir)
|
||||||
|
|
||||||
|
// The pre-014 shape: the columns the projection needs, minus the
|
||||||
|
// one the plan added.
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE explore_index (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
entity_type INTEGER NOT NULL,
|
||||||
|
mbid BLOB NOT NULL,
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
artist_name TEXT NOT NULL,
|
||||||
|
artist_mbid BLOB NOT NULL
|
||||||
|
);
|
||||||
|
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
|
||||||
|
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'');
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The column the projection needs is there now.
|
||||||
|
var n int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
`SELECT COUNT(*) FROM pragma_table_info('explore_index')
|
||||||
|
WHERE name = 'total_tracks'`,
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("inspect: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatalf("explore_index still has no total_tracks column")
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the catalog really was retired rather than patched, so the
|
||||||
|
// artifact is fetched again instead of half a catalog being served.
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM explore_index",
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("count: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 0 {
|
||||||
|
t.Fatalf("stale rows survived the retire: %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiresIndexWithTextMBIDs is the half an ALTER could not have
|
||||||
|
// repaired: plan 013 changed mbid from TEXT to BLOB, and SQLite does not
|
||||||
|
// coerce between them, so a query against 16 raw bytes returns no rows
|
||||||
|
// rather than an error.
|
||||||
|
func TestRetiresIndexWithTextMBIDs(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
dir := t.TempDir()
|
||||||
|
db := openRaw(t, dir)
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE explore_index (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
entity_type TEXT NOT NULL,
|
||||||
|
mbid TEXT NOT NULL,
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
artist_name TEXT NOT NULL,
|
||||||
|
artist_mbid TEXT NOT NULL,
|
||||||
|
total_tracks INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var typ string
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
`SELECT type FROM pragma_table_info('explore_index') WHERE name = 'mbid'`,
|
||||||
|
).Scan(&typ); err != nil {
|
||||||
|
t.Fatalf("inspect: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if typ != "BLOB" {
|
||||||
|
t.Fatalf("mbid is still %s, want BLOB", typ)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiringTheIndexTakesItsMetaWithIt guards the thing that makes the
|
||||||
|
// repair actually repair: the marker saying the import finished is what
|
||||||
|
// stops the artifact being fetched again, so a catalog dropped without
|
||||||
|
// it would stay empty forever.
|
||||||
|
func TestRetiringTheIndexTakesItsMetaWithIt(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
dir := t.TempDir()
|
||||||
|
db := openRaw(t, dir)
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE explore_index (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
entity_type INTEGER NOT NULL,
|
||||||
|
mbid BLOB NOT NULL
|
||||||
|
);
|
||||||
|
CREATE TABLE explore_index_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
|
||||||
|
INSERT INTO explore_index_meta VALUES ('dump_import_done', '1');
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var n int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM explore_index_meta WHERE key = 'dump_import_done'",
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("meta: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 0 {
|
||||||
|
t.Fatalf("the import-done marker survived a retired catalog")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHealthyDatabaseIsUntouched is the other half, and the one that
|
||||||
|
// would make this dangerous if it failed: a current schema must survive
|
||||||
|
// a launch with its catalog intact. A repair that retires a healthy
|
||||||
|
// catalog costs every user an artifact download on every start.
|
||||||
|
func TestHealthyDatabaseIsUntouched(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
dir := t.TempDir()
|
||||||
|
db := openRaw(t, dir)
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
|
||||||
|
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'')
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var n int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM explore_index",
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("count: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatalf("a healthy catalog was retired: %d rows left", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestAuthoredTablesAreNeverRetired states the boundary in a test rather
|
||||||
|
// than only in a comment: this mechanism deletes data, and the only
|
||||||
|
// thing standing between it and a user's playlists is the Kind filter.
|
||||||
|
func TestAuthoredTablesAreNeverRetired(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
dir := t.TempDir()
|
||||||
|
db := openRaw(t, dir)
|
||||||
|
|
||||||
|
// A playlists table missing most of its current columns.
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE playlists (id INTEGER PRIMARY KEY, name TEXT NOT NULL);
|
||||||
|
INSERT INTO playlists (name) VALUES ('irreplaceable');
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var n int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM playlists",
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("count: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatalf("an authored table was retired; rows left: %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiresTablesTheSchemaNoLongerDescribes covers what plan 013 left
|
||||||
|
// behind on every database that predates it: seven tables the schema
|
||||||
|
// stopped describing, plus the schema_migrations table that squashing
|
||||||
|
// the chain retired. They are not stale in shape — they are simply not
|
||||||
|
// ours any more.
|
||||||
|
func TestRetiresTablesTheSchemaNoLongerDescribes(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
db := openRaw(t, t.TempDir())
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
|
||||||
|
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
|
||||||
|
CREATE TABLE schema_migrations (version INTEGER PRIMARY KEY);
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, table := range []string{"recordings", "artist_credit", "schema_migrations"} {
|
||||||
|
var n int
|
||||||
|
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
|
||||||
|
table,
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("inspect %s: %v", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 0 {
|
||||||
|
t.Errorf("%s survived; the schema no longer describes it", table)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestFTSShadowTablesAreNotObsolete is the sweep's sharp edge: an FTS5
|
||||||
|
// virtual table is backed by four shadow tables that appear in
|
||||||
|
// sqlite_master under their own names and are in no schema file.
|
||||||
|
// Dropping one destroys the index it belongs to.
|
||||||
|
func TestFTSShadowTablesAreNotObsolete(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
db := openRaw(t, t.TempDir())
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
obsolete, err := obsoleteTables(ctx, db)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("obsoleteTables: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(obsolete) != 0 {
|
||||||
|
t.Fatalf("a freshly created schema reported obsolete tables: %v", obsolete)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiringOwnedTablesDoesNotDangle pins the one behaviour that is
|
||||||
|
// silently wrong rather than loudly broken.
|
||||||
|
//
|
||||||
|
// Retiring audio_files leaves playlist entries behind. With
|
||||||
|
// foreign_keys ON — which applyPRAGMAs has done before NewDB gets here —
|
||||||
|
// SET NULL fires and they point at nothing. With it OFF they keep ids
|
||||||
|
// that the rescan reissues from 1, so every playlist quietly fills with
|
||||||
|
// different songs. Nothing about the schema makes that ordering
|
||||||
|
// obvious, so it is asserted rather than assumed.
|
||||||
|
func TestRetiringOwnedTablesDoesNotDangle(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
db := openRaw(t, t.TempDir())
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
|
||||||
|
t.Fatalf("pragma: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Break audio_files' shape so it is retired, keeping a playlist
|
||||||
|
// entry that references it.
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
INSERT INTO playlists (id, name) VALUES (1, 'keepme');
|
||||||
|
INSERT INTO libraries (id, name, path) VALUES (0, 'test', '/music');
|
||||||
|
INSERT INTO audio_files (id, file_path, file_type_id, length_milliseconds)
|
||||||
|
VALUES (7, '/music/a.flac', 1, 1000);
|
||||||
|
INSERT INTO playlist_tracks (playlist_id, audio_file_id, position)
|
||||||
|
VALUES (1, 7, 0);
|
||||||
|
DROP VIEW IF EXISTS track_metadata;
|
||||||
|
ALTER TABLE audio_files DROP COLUMN artist_credit;
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var dangling int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM playlist_tracks WHERE audio_file_id IS NOT NULL",
|
||||||
|
).Scan(&dangling); err != nil {
|
||||||
|
t.Fatalf("count: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if dangling != 0 {
|
||||||
|
t.Fatalf(
|
||||||
|
"%d playlist entries still point at retired audio_files ids; "+
|
||||||
|
"a rescan will reissue those ids to different tracks",
|
||||||
|
dangling,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The playlist itself is authored and must be untouched.
|
||||||
|
var playlists int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM playlists",
|
||||||
|
).Scan(&playlists); err != nil {
|
||||||
|
t.Fatalf("playlists: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if playlists != 1 {
|
||||||
|
t.Fatalf("authored playlist lost: %d", playlists)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRetiringInterlinkedLegacyTables is the bug the unit tests missed
|
||||||
|
// and a real database found.
|
||||||
|
//
|
||||||
|
// The tables plan 013 retired reference each other -- pre-013
|
||||||
|
// audio_files has a foreign key into recordings -- so with foreign keys
|
||||||
|
// ON, dropping them one at a time fails with "FOREIGN KEY constraint
|
||||||
|
// failed" on whichever goes first, and map iteration order decides
|
||||||
|
// which that is. Every other test in this file ran with foreign keys
|
||||||
|
// off and passed happily; the app enables them in applyPRAGMAs before
|
||||||
|
// the repair runs, so only the real launch path showed it.
|
||||||
|
func TestRetiringInterlinkedLegacyTables(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
db := openRaw(t, t.TempDir())
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
|
||||||
|
t.Fatalf("pragma: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The pre-013 shape, with the reference that makes ordering matter.
|
||||||
|
// release_group_recordings sorts *after* recordings and references
|
||||||
|
// it, so the deterministic order retires the parent while the child
|
||||||
|
// still holds rows pointing at it -- which is the case that fails
|
||||||
|
// without the deferral, rather than one that fails on some runs.
|
||||||
|
if _, err := db.ExecContext(ctx, `
|
||||||
|
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
|
||||||
|
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
|
||||||
|
CREATE TABLE release_group_recordings (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
recording_id INTEGER NOT NULL,
|
||||||
|
FOREIGN KEY(recording_id) REFERENCES recordings(id)
|
||||||
|
);
|
||||||
|
CREATE TABLE audio_files (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
file_path TEXT NOT NULL UNIQUE,
|
||||||
|
recording_id INTEGER,
|
||||||
|
FOREIGN KEY(recording_id) REFERENCES recordings(id)
|
||||||
|
);
|
||||||
|
INSERT INTO recordings (id, name) VALUES (1, 'x');
|
||||||
|
INSERT INTO release_group_recordings (id, recording_id) VALUES (1, 1);
|
||||||
|
INSERT INTO audio_files (id, file_path, recording_id)
|
||||||
|
VALUES (1, '/music/a.flac', 1);
|
||||||
|
`); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
|
||||||
|
t.Fatalf("retire: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := applySchema(ctx, db); err != nil {
|
||||||
|
t.Fatalf("applySchema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, table := range []string{"recordings", "artist_credit"} {
|
||||||
|
var n int
|
||||||
|
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
|
||||||
|
table,
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("inspect %s: %v", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 0 {
|
||||||
|
t.Errorf("%s survived the retire", table)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the rebuilt audio_files is the current shape, which is the
|
||||||
|
// whole reason the old one had to go.
|
||||||
|
var n int
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
`SELECT COUNT(*) FROM pragma_table_info('audio_files')
|
||||||
|
WHERE name = 'artist_credit'`,
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
t.Fatalf("inspect audio_files: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n != 1 {
|
||||||
|
t.Fatal("audio_files was not rebuilt in the current shape")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseCreateTablesReadsTheRealSchema keeps the parser honest
|
||||||
|
// against the files it actually runs on: a parser that silently found
|
||||||
|
// no columns would report every table healthy and repair nothing.
|
||||||
|
func TestParseCreateTablesReadsTheRealSchema(t *testing.T) {
|
||||||
|
declared, err := declaredTables()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("declaredTables: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cols, ok := declared["explore_index"]
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("explore_index was not parsed out of the schema files")
|
||||||
|
}
|
||||||
|
|
||||||
|
want := map[string]string{
|
||||||
|
"mbid": "BLOB",
|
||||||
|
"total_tracks": "INTEGER",
|
||||||
|
"artist_name": "TEXT",
|
||||||
|
}
|
||||||
|
|
||||||
|
got := make(map[string]string, len(cols))
|
||||||
|
for _, c := range cols {
|
||||||
|
got[c.name] = c.typ
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, typ := range want {
|
||||||
|
if got[name] != typ {
|
||||||
|
t.Errorf("explore_index.%s parsed as %q, want %q", name, got[name], typ)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A table constraint must not be mistaken for a column.
|
||||||
|
for _, c := range cols {
|
||||||
|
switch c.name {
|
||||||
|
case "PRIMARY", "FOREIGN", "UNIQUE", "CHECK", "CONSTRAINT":
|
||||||
|
t.Errorf("parsed table constraint %q as a column", c.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -185,6 +185,21 @@ var tables = []Table{
|
|||||||
Note: "Full-text index over the champion entities of the " +
|
Note: "Full-text index over the champion entities of the " +
|
||||||
"MusicBrainz dump. Rebuilt only by a full index build.",
|
"MusicBrainz dump. Rebuilt only by a full index build.",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
Name: "artist_credit_part", Kind: Cache, Lifetime: Retained,
|
||||||
|
Note: "The decomposition of a multi-artist credit, from the " +
|
||||||
|
"MusicBrainz dump: one row per credited artist, with the " +
|
||||||
|
"name as credited and the join phrase that follows it. " +
|
||||||
|
"Arrives with the downloaded artifact, so rebuilding it " +
|
||||||
|
"costs a dump stream and it is never swept.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Name: "artist_credit_ref", Kind: Cache, Lifetime: Retained,
|
||||||
|
Note: "Which credit a catalog recording or release group is " +
|
||||||
|
"credited to. Present only for multi-artist credits; " +
|
||||||
|
"absence means one artist, which explore_index already " +
|
||||||
|
"describes. Ships and dies with artist_credit_part.",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
Name: "explore_index", Kind: Cache, Lifetime: Retained,
|
Name: "explore_index", Kind: Cache, Lifetime: Retained,
|
||||||
Note: "The offline MusicBrainz search index. Rebuilding costs a " +
|
Note: "The offline MusicBrainz search index. Rebuilding costs a " +
|
||||||
|
|||||||
+28
-11
@@ -34,13 +34,29 @@ type UserConfig struct {
|
|||||||
// in one burst that every provider sees as a flood.
|
// in one burst that every provider sees as a flood.
|
||||||
WantedBatch int `toml:"WantedBatch"`
|
WantedBatch int `toml:"WantedBatch"`
|
||||||
|
|
||||||
// MinFileSizeMB, MaxFileSizeMB and PreferredFileSizeMB bound and
|
// MinKbps, MaxKbps and PreferredKbps bound and nudge what auto-pick
|
||||||
// nudge what auto-pick (interactive or via the request list) may
|
// (interactive or via the request list) may grab without asking.
|
||||||
// grab without asking. Zero on any of them is permissive: see
|
// Zero on any of them is permissive: see AutoDownloadPrefs.
|
||||||
// AutoDownloadPrefs.
|
//
|
||||||
MinFileSizeMB int `toml:"MinFileSizeMB"`
|
// They replaced MinFileSizeMB / MaxFileSizeMB /
|
||||||
MaxFileSizeMB int `toml:"MaxFileSizeMB"`
|
// PreferredFileSizeMB, which were megabytes and so said nothing
|
||||||
PreferredFileSizeMB int `toml:"PreferredFileSizeMB"`
|
// without knowing how long the release was. The old keys are
|
||||||
|
// deliberately *not* read back: a number that meant "300 MB" cannot
|
||||||
|
// be reinterpreted as a bitrate without knowing the album it was
|
||||||
|
// aimed at, so migrating it would be inventing an intent the user
|
||||||
|
// never expressed. An existing config falls back to no window,
|
||||||
|
// which is the permissive default and matches a fresh install —
|
||||||
|
// and MaxFileSizeMB is the one that does carry over, because a
|
||||||
|
// ceiling on total bytes still means exactly what it did.
|
||||||
|
MinKbps int `toml:"MinKbps"`
|
||||||
|
MaxKbps int `toml:"MaxKbps"`
|
||||||
|
PreferredKbps int `toml:"PreferredKbps"`
|
||||||
|
|
||||||
|
// MaxFileSizeMB is a hard ceiling on a candidate's total size, kept
|
||||||
|
// in megabytes on purpose — it is a question about disk space, not
|
||||||
|
// about quality, and it has to apply to a candidate whose bitrate
|
||||||
|
// cannot be worked out at all.
|
||||||
|
MaxFileSizeMB int `toml:"MaxFileSizeMB"`
|
||||||
|
|
||||||
// AllowedFormats restricts auto-pick to these formats. Empty means
|
// AllowedFormats restricts auto-pick to these formats. Empty means
|
||||||
// no restriction. Values are Format strings ("flac", "mp3", ...).
|
// no restriction. Values are Format strings ("flac", "mp3", ...).
|
||||||
@@ -56,10 +72,11 @@ func (c *UserConfig) AutoDownloadPrefs() AutoDownloadPrefs {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return AutoDownloadPrefs{
|
return AutoDownloadPrefs{
|
||||||
MinSizeMB: c.MinFileSizeMB,
|
MinKbps: c.MinKbps,
|
||||||
MaxSizeMB: c.MaxFileSizeMB,
|
MaxKbps: c.MaxKbps,
|
||||||
PreferredSizeMB: c.PreferredFileSizeMB,
|
PreferredKbps: c.PreferredKbps,
|
||||||
AllowedFormats: formats,
|
MaxSizeMB: c.MaxFileSizeMB,
|
||||||
|
AllowedFormats: formats,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -236,6 +236,12 @@ func (m *Manager) AutoPickable(dl Download, ranked []Candidate) bool {
|
|||||||
return AutoPickable(dl, ranked, m.preferences())
|
return AutoPickable(dl, ranked, m.preferences())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AutoPickVeto wraps the package function the same way, and is what the
|
||||||
|
// request list quotes back to the user.
|
||||||
|
func (m *Manager) AutoPickVeto(dl Download, ranked []Candidate) string {
|
||||||
|
return AutoPickVeto(dl, ranked, m.preferences())
|
||||||
|
}
|
||||||
|
|
||||||
// Reload rebuilds every provider from stored config. Called at startup
|
// Reload rebuilds every provider from stored config. Called at startup
|
||||||
// and after any provider settings change.
|
// and after any provider settings change.
|
||||||
//
|
//
|
||||||
@@ -612,16 +618,8 @@ func (m *Manager) Attempt(
|
|||||||
return false, "", err
|
return false, "", err
|
||||||
}
|
}
|
||||||
|
|
||||||
if !m.AutoPickable(dl, ranked) {
|
if veto := m.AutoPickVeto(dl, ranked); veto != "" {
|
||||||
best := ranked[0]
|
return false, veto, nil
|
||||||
|
|
||||||
return false, fmt.Sprintf(
|
|
||||||
"best of %d found is not a confident enough match "+
|
|
||||||
"(match %.0f%%, quality %.0f%%)",
|
|
||||||
len(ranked),
|
|
||||||
best.Match.Overall*100, //nolint:mnd // percent
|
|
||||||
best.Quality.Overall*100,
|
|
||||||
), nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := m.store.CreateDownload(ctx, dl); err != nil {
|
if err := m.store.CreateDownload(ctx, dl); err != nil {
|
||||||
|
|||||||
@@ -199,28 +199,36 @@ func TestManagerEndToEndAutoPick(t *testing.T) {
|
|||||||
t.Errorf("expected imported file at %s: %v", want, err)
|
t.Errorf("expected imported file at %s: %v", want, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Staging was released only after a successful import.
|
// Staging release and the rescan happen *after* the state is
|
||||||
entries, err := os.ReadDir(f.staging.Root())
|
// recorded (manager.go sets StateComplete, then releases, then
|
||||||
if err != nil {
|
// scans), so waiting on the state is not waiting on these. Under
|
||||||
t.Fatalf("read staging root: %v", err)
|
// load the worker is descheduled in between and asserting straight
|
||||||
}
|
// away reads the world one step too early -- which is exactly how
|
||||||
|
// this test failed on a busy machine while passing alone.
|
||||||
|
waitFor(t, func() bool {
|
||||||
|
entries, err := os.ReadDir(f.staging.Root())
|
||||||
|
if err != nil || len(entries) != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
if len(entries) != 0 {
|
f.lib.mu.Lock()
|
||||||
t.Errorf("staging not released: %d dirs remain", len(entries))
|
defer f.lib.mu.Unlock()
|
||||||
}
|
|
||||||
|
|
||||||
// The library was told to rescan.
|
return len(f.lib.scanned) == 1
|
||||||
f.lib.mu.Lock()
|
}, "staging was never released, or the library was never rescanned")
|
||||||
scanned := len(f.lib.scanned)
|
|
||||||
f.lib.mu.Unlock()
|
|
||||||
|
|
||||||
if scanned != 1 {
|
|
||||||
t.Errorf("library scans = %d, want 1", scanned)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// An ambiguous result set must park for the user rather than guess.
|
// Two equally good copies are not an ambiguity — they are a spare.
|
||||||
func TestManagerWaitsWhenAmbiguous(t *testing.T) {
|
//
|
||||||
|
// This asserted the opposite for as long as auto-pick required 0.08 of
|
||||||
|
// daylight over the runner-up, and that rule was wrong in exactly the
|
||||||
|
// case it fired hardest: a popular album turns up several *correct*
|
||||||
|
// copies, all matching the tracklist, differing only in format and
|
||||||
|
// seeders. There is no question there about what to fetch, only about
|
||||||
|
// which copy, and the ranking already answers that — closest to the
|
||||||
|
// preferred bitrate first. A candidate does not have to be better than
|
||||||
|
// the field, only good enough on its own terms.
|
||||||
|
func TestManagerAutoPicksAmongEquallyGoodCopies(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
f := newManagerFixture(t)
|
f := newManagerFixture(t)
|
||||||
@@ -238,11 +246,41 @@ func TestManagerWaitsWhenAmbiguous(t *testing.T) {
|
|||||||
t.Fatalf("Start: %v", err)
|
t.Fatalf("Start: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if f.manager.AutoPickable(dl, ranked) {
|
if veto := f.manager.AutoPickVeto(dl, ranked); veto != "" {
|
||||||
t.Fatal("two equivalent candidates must not auto-pick")
|
t.Fatalf("two equally good copies must auto-pick, got veto: %s", veto)
|
||||||
|
}
|
||||||
|
|
||||||
|
waitForDownloadState(t, f.store, dl.ID, StateComplete)
|
||||||
|
|
||||||
|
// Exactly one of them was fetched, not both.
|
||||||
|
if grabs := a.GrabCalls + b.GrabCalls; grabs != 1 {
|
||||||
|
t.Errorf("grabs = %d, want exactly 1", grabs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The user can still pick explicitly when auto-pick is not what
|
||||||
|
// happened — a candidate the ranking did not choose is still grabbable.
|
||||||
|
func TestManagerPickIsExplicit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
f := newManagerFixture(t)
|
||||||
|
|
||||||
|
a := fakeWithAlbum(1, "source-a", ".flac")
|
||||||
|
b := fakeWithAlbum(2, "source-b", ".flac")
|
||||||
|
|
||||||
|
f.manager.installProvider(Config{ID: 1, Priority: 50}, a)
|
||||||
|
f.manager.installProvider(Config{ID: 2, Priority: 50}, b)
|
||||||
|
|
||||||
|
// No tracklist: never auto-picks, so the result set parks for the
|
||||||
|
// user and Pick is the only way anything is fetched.
|
||||||
|
dl := fourTrackDownload()
|
||||||
|
dl.Expected = nil
|
||||||
|
|
||||||
|
ranked, err := f.manager.Start(context.Background(), dl)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Start: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Nothing was grabbed while waiting for the user.
|
|
||||||
if a.GrabCalls != 0 || b.GrabCalls != 0 {
|
if a.GrabCalls != 0 || b.GrabCalls != 0 {
|
||||||
t.Errorf(
|
t.Errorf(
|
||||||
"grabs happened without a pick: a=%d b=%d",
|
"grabs happened without a pick: a=%d b=%d",
|
||||||
@@ -259,7 +297,6 @@ func TestManagerWaitsWhenAmbiguous(t *testing.T) {
|
|||||||
t.Errorf("stored request id = %s, want %s", stored.ID, dl.ID)
|
t.Errorf("stored request id = %s, want %s", stored.ID, dl.ID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The user picks the second one explicitly.
|
|
||||||
if err := f.manager.Pick(
|
if err := f.manager.Pick(
|
||||||
context.Background(), dl.ID, ranked[1].ID,
|
context.Background(), dl.ID, ranked[1].ID,
|
||||||
); err != nil {
|
); err != nil {
|
||||||
|
|||||||
+313
-77
@@ -1,6 +1,7 @@
|
|||||||
package download
|
package download
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"fmt"
|
||||||
"math"
|
"math"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -34,38 +35,102 @@ const (
|
|||||||
weightArtistFit = 0.12
|
weightArtistFit = 0.12
|
||||||
)
|
)
|
||||||
|
|
||||||
// Quality sub-weights. They sum to 1.0 along with weightSizeFit below.
|
// Quality sub-weights. Each set sums to 1.0.
|
||||||
|
//
|
||||||
|
// There are two of them because a stated preference changes what the
|
||||||
|
// other numbers are *for*. `formatRank` and `bitrateScore` are the
|
||||||
|
// app guessing at how good a copy is — FLAC over MP3, 320 over 128 —
|
||||||
|
// and that guess exists precisely because the user has not said. Once
|
||||||
|
// they have, the guess should not outvote them: with the old single set
|
||||||
|
// a preference of 320 kbps moved a candidate's score by at most 0.05
|
||||||
|
// against the 0.42 riding on format, so asking for 320 and being handed
|
||||||
|
// a FLAC every time was the *designed* behaviour. That is the same
|
||||||
|
// fault the megabyte window had — a preference the user can express and
|
||||||
|
// the ranking can ignore.
|
||||||
const (
|
const (
|
||||||
weightFormat = 0.42
|
weightFormat = 0.42
|
||||||
weightBitrate = 0.23
|
weightBitrate = 0.23
|
||||||
weightHealth = 0.20
|
weightHealth = 0.20
|
||||||
weightPriority = 0.10
|
weightPriority = 0.10
|
||||||
weightSizeFit = 0.05
|
weightBitrateFit = 0.05
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// Quality sub-weights when the user has named a preferred bitrate.
|
||||||
|
// The weight comes off format and bitrate — the two proxies the
|
||||||
|
// preference replaces — and health and priority are untouched, since
|
||||||
|
// neither is a stand-in for anything the user just said.
|
||||||
|
const (
|
||||||
|
statedWeightFormat = 0.20
|
||||||
|
statedWeightBitrate = 0.10
|
||||||
|
statedWeightHealth = 0.20
|
||||||
|
statedWeightPriority = 0.10
|
||||||
|
statedWeightBitrateFit = 0.40
|
||||||
|
)
|
||||||
|
|
||||||
|
// qualityWeights picks the set, in the order scoreQuality applies them.
|
||||||
|
func qualityWeights(p AutoDownloadPrefs) (
|
||||||
|
format, bitrate, health, priority, fit float64,
|
||||||
|
) {
|
||||||
|
if p.PreferredKbps > 0 {
|
||||||
|
return statedWeightFormat,
|
||||||
|
statedWeightBitrate,
|
||||||
|
statedWeightHealth,
|
||||||
|
statedWeightPriority,
|
||||||
|
statedWeightBitrateFit
|
||||||
|
}
|
||||||
|
|
||||||
|
return weightFormat,
|
||||||
|
weightBitrate,
|
||||||
|
weightHealth,
|
||||||
|
weightPriority,
|
||||||
|
weightBitrateFit
|
||||||
|
}
|
||||||
|
|
||||||
// unanchoredCap bounds the match score of a free-text request. Without
|
// unanchoredCap bounds the match score of a free-text request. Without
|
||||||
// an MBID there is no tracklist to be right about, so a confident-
|
// an MBID there is no tracklist to be right about, so a confident-
|
||||||
// looking score would be a lie — and auto-pick keys off this.
|
// looking score would be a lie — and auto-pick keys off this.
|
||||||
const unanchoredCap = 0.65
|
const unanchoredCap = 0.65
|
||||||
|
|
||||||
// AutoDownloadPrefs gates and scores what AutoPickable may choose
|
// AutoDownloadPrefs gates and scores what AutoPickable may choose
|
||||||
// without asking. Zero values are permissive: no size window and no
|
// without asking. Zero values are permissive: no bitrate window, no
|
||||||
// format restriction.
|
// size ceiling and no format restriction.
|
||||||
|
//
|
||||||
|
// **The window is a rate, not a size.** It used to be three numbers in
|
||||||
|
// megabytes, which cannot mean anything on their own: 300 MB is a
|
||||||
|
// generous FLAC single and a suspiciously small boxset, and the user
|
||||||
|
// setting the number has no idea which release the pipeline will
|
||||||
|
// eventually apply it to. A bitrate is the same statement normalised
|
||||||
|
// by how long the music is, so one number holds across a 9-minute EP
|
||||||
|
// and a 3-hour opera — and it is the unit the thing being described is
|
||||||
|
// actually measured in. The runtime is known for every request
|
||||||
|
// auto-pick can act on (`Download.Expected` carries per-track lengths,
|
||||||
|
// and an anchored request is the only kind that reaches here), so this
|
||||||
|
// costs no extra lookup.
|
||||||
type AutoDownloadPrefs struct {
|
type AutoDownloadPrefs struct {
|
||||||
// MinSizeMB and MaxSizeMB bound what auto-pick will grab. Zero
|
// MinKbps and MaxKbps bound the average bitrate auto-pick will
|
||||||
// means no bound on that side. A candidate outside the window is
|
// grab. Zero means no bound on that side. A candidate outside the
|
||||||
// filtered out of auto-pick entirely, not merely scored down — a
|
// window is filtered out of auto-pick entirely, not merely scored
|
||||||
// tiny "sampler" torrent or a boxset ten times the expected size is
|
// down — a 96 kbps rip of the right album is not a worse copy the
|
||||||
// usually the wrong thing entirely, not a worse copy of the right
|
// user might accept, it is one they said not to take unattended.
|
||||||
// thing.
|
//
|
||||||
MinSizeMB int `json:"minSizeMb"`
|
// For reference: 320 is the top of MP3, ~500–1000 is FLAC depending
|
||||||
MaxSizeMB int `json:"maxSizeMb"`
|
// on the material, and anything under ~128 is a transcode.
|
||||||
|
MinKbps int `json:"minKbps"`
|
||||||
|
MaxKbps int `json:"maxKbps"`
|
||||||
|
|
||||||
// PreferredSizeMB nudges the score toward a target size within the
|
// PreferredKbps nudges the score toward a target rate within the
|
||||||
// min/max window (a lossless rip and a heavily-padded lossless rip
|
// window, and breaks the tie when several candidates are equally
|
||||||
// can both pass the window). Zero disables the nudge; sizeFit then
|
// good matches. Zero disables the nudge; bitrateFit then returns a
|
||||||
// returns a neutral value that does not affect ranking.
|
// neutral value that does not affect ranking.
|
||||||
PreferredSizeMB int `json:"preferredSizeMb"`
|
PreferredKbps int `json:"preferredKbps"`
|
||||||
|
|
||||||
|
// MaxSizeMB is a hard ceiling on the whole candidate, and it is
|
||||||
|
// deliberately still a size. It answers a different question from
|
||||||
|
// the window above — not "is this the quality I want" but "is this
|
||||||
|
// going to fill the disk" — and it has to hold even for a candidate
|
||||||
|
// whose bitrate cannot be worked out, which is exactly the shape a
|
||||||
|
// mislabelled boxset arrives in. Zero means no ceiling.
|
||||||
|
MaxSizeMB int `json:"maxSizeMb"`
|
||||||
|
|
||||||
// AllowedFormats restricts auto-pick to candidates whose audio
|
// AllowedFormats restricts auto-pick to candidates whose audio
|
||||||
// files are all in one of these formats. Empty means no
|
// files are all in one of these formats. Empty means no
|
||||||
@@ -74,19 +139,33 @@ type AutoDownloadPrefs struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// eligible reports whether a candidate may be auto-picked under these
|
// eligible reports whether a candidate may be auto-picked under these
|
||||||
// preferences: within the size window (when set) and, when a format
|
// preferences: inside the bitrate window and the size ceiling (when
|
||||||
// list is given, every audio file in an allowed format.
|
// set) and, when a format list is given, every audio file in an
|
||||||
func (p AutoDownloadPrefs) eligible(c Candidate) bool {
|
// allowed format.
|
||||||
|
//
|
||||||
|
// `runtimeMillis` is how long the requested release is, and 0 means
|
||||||
|
// nobody knows. An unknown runtime **passes** the bitrate window
|
||||||
|
// rather than failing it: the window is a statement about quality, and
|
||||||
|
// refusing everything the moment a tracklist is missing a length would
|
||||||
|
// turn a gap in MusicBrainz into a silent embargo. The size ceiling
|
||||||
|
// still applies, which is why it exists separately.
|
||||||
|
func (p AutoDownloadPrefs) eligible(c Candidate, runtimeMillis int64) bool {
|
||||||
const bytesPerMB = 1 << 20
|
const bytesPerMB = 1 << 20
|
||||||
|
|
||||||
if p.MinSizeMB > 0 && c.TotalSize < int64(p.MinSizeMB)*bytesPerMB {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
if p.MaxSizeMB > 0 && c.TotalSize > int64(p.MaxSizeMB)*bytesPerMB {
|
if p.MaxSizeMB > 0 && c.TotalSize > int64(p.MaxSizeMB)*bytesPerMB {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if kbps := candidateKbps(c, runtimeMillis); kbps > 0 {
|
||||||
|
if p.MinKbps > 0 && kbps < float64(p.MinKbps) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
if p.MaxKbps > 0 && kbps > float64(p.MaxKbps) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if len(p.AllowedFormats) == 0 {
|
if len(p.AllowedFormats) == 0 {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
@@ -107,11 +186,14 @@ func (p AutoDownloadPrefs) eligible(c Candidate) bool {
|
|||||||
|
|
||||||
// filter returns only the candidates these preferences allow to be
|
// filter returns only the candidates these preferences allow to be
|
||||||
// auto-picked, in the same (already ranked) order.
|
// auto-picked, in the same (already ranked) order.
|
||||||
func (p AutoDownloadPrefs) filter(ranked []Candidate) []Candidate {
|
func (p AutoDownloadPrefs) filter(
|
||||||
|
ranked []Candidate,
|
||||||
|
runtimeMillis int64,
|
||||||
|
) []Candidate {
|
||||||
out := make([]Candidate, 0, len(ranked))
|
out := make([]Candidate, 0, len(ranked))
|
||||||
|
|
||||||
for _, c := range ranked {
|
for _, c := range ranked {
|
||||||
if p.eligible(c) {
|
if p.eligible(c, runtimeMillis) {
|
||||||
out = append(out, c)
|
out = append(out, c)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -119,32 +201,116 @@ func (p AutoDownloadPrefs) filter(ranked []Candidate) []Candidate {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
// sizeFit scores how close totalSize is to PreferredSizeMB, 0..1,
|
// bitrateFit scores how close a candidate's average bitrate is to
|
||||||
// falling off linearly as the size doubles or halves away from it.
|
// PreferredKbps, falling off linearly as it doubles or halves away
|
||||||
// Returns a neutral 0.5 when no preference is set, so the absence of a
|
// from it.
|
||||||
// preference does not bias ranking.
|
//
|
||||||
func (p AutoDownloadPrefs) sizeFit(totalSize int64) float64 {
|
// The range is **0.5 to 1.0, not 0 to 1**, and the floor is the point.
|
||||||
|
// This carries 0.40 of the quality score once a preference is set, so a
|
||||||
|
// span down to zero would let a preference of 320 kbps push a perfectly
|
||||||
|
// good FLAC under `minQuality` and out of auto-pick altogether —
|
||||||
|
// turning "I like 320" into "never take anything else", silently. A
|
||||||
|
// preference may promote the copy that matches it; it may not
|
||||||
|
// disqualify the others. That is what `MinKbps`/`MaxKbps` are for, and
|
||||||
|
// they say so out loud.
|
||||||
|
//
|
||||||
|
// Returns the neutral floor when no preference is set or the rate
|
||||||
|
// cannot be worked out, so neither an absent preference nor an absent
|
||||||
|
// runtime biases ranking.
|
||||||
|
func (p AutoDownloadPrefs) bitrateFit(
|
||||||
|
c Candidate,
|
||||||
|
runtimeMillis int64,
|
||||||
|
) float64 {
|
||||||
const (
|
const (
|
||||||
bytesPerMB = 1 << 20
|
neutral = 0.5
|
||||||
neutral = 0.5
|
span = 0.5
|
||||||
)
|
)
|
||||||
|
|
||||||
if p.PreferredSizeMB <= 0 || totalSize <= 0 {
|
if p.PreferredKbps <= 0 {
|
||||||
return neutral
|
return neutral
|
||||||
}
|
}
|
||||||
|
|
||||||
preferred := float64(p.PreferredSizeMB) * bytesPerMB
|
kbps := candidateKbps(c, runtimeMillis)
|
||||||
ratio := float64(totalSize) / preferred
|
if kbps <= 0 {
|
||||||
|
return neutral
|
||||||
|
}
|
||||||
|
|
||||||
|
ratio := kbps / float64(p.PreferredKbps)
|
||||||
if ratio < 1 {
|
if ratio < 1 {
|
||||||
ratio = 1 / ratio
|
ratio = 1 / ratio
|
||||||
}
|
}
|
||||||
|
|
||||||
// ratio is now >= 1: 1.0 is an exact match, 2.0 is double or half
|
// ratio is now >= 1: 1.0 is an exact match, 2.0 is double or half
|
||||||
// the preferred size. Falls to 0 at 2x away and beyond.
|
// the preferred rate, where the closeness term reaches 0.
|
||||||
fit := 1 - (ratio - 1)
|
return neutral + span*clamp01(1-(ratio-1))
|
||||||
|
}
|
||||||
|
|
||||||
return clamp01(fit)
|
// candidateKbps is a candidate's average audio bitrate, or 0 when it
|
||||||
|
// cannot be worked out.
|
||||||
|
//
|
||||||
|
// Two sources, in this order, and the order matters:
|
||||||
|
//
|
||||||
|
// - **Derived from bytes over runtime**, which is the honest one. It
|
||||||
|
// covers lossless (where a stated bitrate rarely exists), it cannot
|
||||||
|
// be lied to by a filename, and it is what the user's window means.
|
||||||
|
// Only the *audio* files count: cover scans and a log file are not
|
||||||
|
// part of the bitrate, and a folder with 30 MB of artwork would
|
||||||
|
// otherwise read as a better rip than the same music without it.
|
||||||
|
// - **The mean stated bitrate**, when the runtime is unknown. Weaker
|
||||||
|
// — a provider that parses it from an MP3 header states it and one
|
||||||
|
// that guesses from the filename also "states" it — but a number
|
||||||
|
// from the file itself beats no number at all.
|
||||||
|
func candidateKbps(c Candidate, runtimeMillis int64) float64 {
|
||||||
|
const bitsPerByte = 8
|
||||||
|
|
||||||
|
audio := c.AudioFiles()
|
||||||
|
if len(audio) == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
if runtimeMillis > 0 {
|
||||||
|
var bytes int64
|
||||||
|
for _, f := range audio {
|
||||||
|
bytes += f.Size
|
||||||
|
}
|
||||||
|
|
||||||
|
if bytes > 0 {
|
||||||
|
// bytes×8 bits over seconds, expressed in kbps: the two
|
||||||
|
// factors of 1000 (millis→seconds, bits→kilobits) cancel.
|
||||||
|
return float64(bytes) * bitsPerByte /
|
||||||
|
float64(runtimeMillis)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
sum int
|
||||||
|
count int
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, f := range audio {
|
||||||
|
if f.Bitrate > 0 {
|
||||||
|
sum += f.Bitrate
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if count == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
return float64(sum) / float64(count)
|
||||||
|
}
|
||||||
|
|
||||||
|
// runtimeMillis is how long the requested release is, summed over its
|
||||||
|
// expected tracklist. Zero when the tracklist is absent or carries no
|
||||||
|
// lengths, which is what every caller here treats as "unknown".
|
||||||
|
func (d Download) runtimeMillis() int64 {
|
||||||
|
var total int64
|
||||||
|
for _, t := range d.Expected {
|
||||||
|
total += t.LengthMillis
|
||||||
|
}
|
||||||
|
|
||||||
|
return total
|
||||||
}
|
}
|
||||||
|
|
||||||
// Score fills a candidate's Match, Quality and Score fields.
|
// Score fills a candidate's Match, Quality and Score fields.
|
||||||
@@ -160,7 +326,9 @@ func Score(dl Download, c Candidate, priority int, prefs AutoDownloadPrefs) Cand
|
|||||||
c.Files = mergeMatched(c.Files, matched)
|
c.Files = mergeMatched(c.Files, matched)
|
||||||
|
|
||||||
c.Match = scoreMatch(dl, c, audio, titleFit)
|
c.Match = scoreMatch(dl, c, audio, titleFit)
|
||||||
c.Quality = scoreQuality(c, audio, priority, prefs)
|
c.Quality = scoreQuality(
|
||||||
|
c, audio, priority, prefs, dl.runtimeMillis(),
|
||||||
|
)
|
||||||
|
|
||||||
c.Score = weightMatch*c.Match.Overall + weightQuality*c.Quality.Overall
|
c.Score = weightMatch*c.Match.Overall + weightQuality*c.Quality.Overall
|
||||||
|
|
||||||
@@ -279,11 +447,12 @@ func scoreQuality(
|
|||||||
audio []CandidateFile,
|
audio []CandidateFile,
|
||||||
priority int,
|
priority int,
|
||||||
prefs AutoDownloadPrefs,
|
prefs AutoDownloadPrefs,
|
||||||
|
runtimeMillis int64,
|
||||||
) QualityScore {
|
) QualityScore {
|
||||||
q := QualityScore{
|
q := QualityScore{
|
||||||
Health: clamp01(c.Health),
|
Health: clamp01(c.Health),
|
||||||
Priority: clamp01(float64(priority) / 100.0),
|
Priority: clamp01(float64(priority) / 100.0),
|
||||||
SizeFit: prefs.sizeFit(c.TotalSize),
|
BitrateFit: prefs.bitrateFit(c, runtimeMillis),
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(audio) == 0 {
|
if len(audio) == 0 {
|
||||||
@@ -310,11 +479,13 @@ func scoreQuality(
|
|||||||
q.FormatRank = worst
|
q.FormatRank = worst
|
||||||
q.Bitrate = bitrateScore(audio)
|
q.Bitrate = bitrateScore(audio)
|
||||||
|
|
||||||
q.Overall = weightFormat*q.FormatRank +
|
wFormat, wBitrate, wHealth, wPriority, wFit := qualityWeights(prefs)
|
||||||
weightBitrate*q.Bitrate +
|
|
||||||
weightHealth*q.Health +
|
q.Overall = wFormat*q.FormatRank +
|
||||||
weightPriority*q.Priority +
|
wBitrate*q.Bitrate +
|
||||||
weightSizeFit*q.SizeFit
|
wHealth*q.Health +
|
||||||
|
wPriority*q.Priority +
|
||||||
|
wFit*q.BitrateFit
|
||||||
|
|
||||||
if q.Mixed {
|
if q.Mixed {
|
||||||
q.Overall *= 0.9
|
q.Overall *= 0.9
|
||||||
@@ -444,6 +615,19 @@ func Rank(
|
|||||||
return out[i].Match.Overall > out[j].Match.Overall
|
return out[i].Match.Overall > out[j].Match.Overall
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Closest to the preferred bitrate wins the tie.
|
||||||
|
//
|
||||||
|
// This is what decides which copy is taken now that auto-pick
|
||||||
|
// no longer requires the winner to be clear of the field: when
|
||||||
|
// several candidates are equally good matches of equal overall
|
||||||
|
// quality, the one the user said they wanted the shape of is
|
||||||
|
// the answer, ahead of provider priority. With no preference
|
||||||
|
// set every BitrateFit is the same neutral value and this
|
||||||
|
// falls through, exactly as before.
|
||||||
|
if out[i].Quality.BitrateFit != out[j].Quality.BitrateFit {
|
||||||
|
return out[i].Quality.BitrateFit > out[j].Quality.BitrateFit
|
||||||
|
}
|
||||||
|
|
||||||
if out[i].Quality.Priority != out[j].Quality.Priority {
|
if out[i].Quality.Priority != out[j].Quality.Priority {
|
||||||
return out[i].Quality.Priority > out[j].Quality.Priority
|
return out[i].Quality.Priority > out[j].Quality.Priority
|
||||||
}
|
}
|
||||||
@@ -454,19 +638,58 @@ func Rank(
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
// AutoPickable reports whether a ranked list has a clear enough winner
|
// Auto-pick gates. Named rather than inlined because AutoPickVeto
|
||||||
// to grab without asking. It demands an anchored request, a high match,
|
// reports which of them refused, and a number in a sentence the user
|
||||||
// decent quality, and daylight between first and second place — if two
|
// reads should be the same number the decision used.
|
||||||
// candidates are close, the choice is the user's.
|
const (
|
||||||
func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool {
|
minMatch = 0.85
|
||||||
const (
|
minQuality = 0.5
|
||||||
minMatch = 0.85
|
)
|
||||||
minQuality = 0.5
|
|
||||||
minLead = 0.08
|
|
||||||
)
|
|
||||||
|
|
||||||
if !dl.Anchored() || len(ranked) == 0 {
|
// AutoPickable reports whether a ranked list has a candidate worth
|
||||||
return false
|
// grabbing without asking: an anchored request with a tracklist behind
|
||||||
|
// it, and a candidate that clears the match and quality bars inside the
|
||||||
|
// user's guardrails.
|
||||||
|
//
|
||||||
|
// **It does not require the winner to be better than the runner-up.**
|
||||||
|
// It used to demand 0.08 of daylight on the combined score, which meant
|
||||||
|
// the check fired hardest in the case it was never written for: a
|
||||||
|
// popular album turns up five *correct* copies, all matching the
|
||||||
|
// tracklist at 95%+ and differing only in format and seeders, their
|
||||||
|
// scores land within a point of each other, and auto-pick refused
|
||||||
|
// forever on the grounds that the choice was the user's. It was not.
|
||||||
|
// There was no question about *what* to fetch, only about which copy —
|
||||||
|
// and abundance is the one condition under which that question matters
|
||||||
|
// least. A candidate does not need to be the best one, only one that
|
||||||
|
// meets the criteria; where several do, `Rank` puts the one closest to
|
||||||
|
// the preferred bitrate first.
|
||||||
|
func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool {
|
||||||
|
return AutoPickVeto(dl, ranked, prefs) == ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// AutoPickVeto returns the reason auto-pick declined, or "" when it
|
||||||
|
// would go ahead.
|
||||||
|
//
|
||||||
|
// It exists because "it rejected all of them" was indistinguishable
|
||||||
|
// from "it found nothing good". The request list's message was built
|
||||||
|
// from `ranked[0]` — the best candidate *before* the size and format
|
||||||
|
// guardrails, and before the lead check — so a request refused because
|
||||||
|
// the user's maximum size excluded every copy, or because three equally
|
||||||
|
// good copies were found, reported "best of 12 found is not a confident
|
||||||
|
// enough match (match 96%, quality 88%)". Numbers that clear both
|
||||||
|
// thresholds, beside a refusal, is a message that teaches the user the
|
||||||
|
// matcher is broken. Each gate names itself now.
|
||||||
|
func AutoPickVeto(
|
||||||
|
dl Download,
|
||||||
|
ranked []Candidate,
|
||||||
|
prefs AutoDownloadPrefs,
|
||||||
|
) string {
|
||||||
|
if len(ranked) == 0 {
|
||||||
|
return "nothing found"
|
||||||
|
}
|
||||||
|
|
||||||
|
if !dl.Anchored() {
|
||||||
|
return "the request is free text, so there is no release to be right about"
|
||||||
}
|
}
|
||||||
|
|
||||||
// An anchor with no tracklist behind it is an anchor in name only:
|
// An anchor with no tracklist behind it is an anchor in name only:
|
||||||
@@ -474,29 +697,42 @@ func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool
|
|||||||
// is exactly the evidence a wrong-album candidate also has. This
|
// is exactly the evidence a wrong-album candidate also has. This
|
||||||
// matters most for the request list, where nobody is watching.
|
// matters most for the request list, where nobody is watching.
|
||||||
if len(dl.Expected) == 0 {
|
if len(dl.Expected) == 0 {
|
||||||
return false
|
return "no tracklist for this release is known yet, so a candidate cannot be checked against it"
|
||||||
}
|
}
|
||||||
|
|
||||||
// The guardrails apply before the match/quality/lead checks: a
|
// The guardrails apply before the match and quality checks: a
|
||||||
// candidate outside the allowed size or format is not a worse
|
// candidate outside the allowed bitrate, size or format is not a
|
||||||
// choice, it is not a choice auto-pick may make at all, so it must
|
// worse choice, it is not a choice auto-pick may make at all, so it
|
||||||
// not count as "the winner" nor as "second place" for the lead
|
// must not count as "the winner" either.
|
||||||
// check below.
|
eligible := prefs.filter(ranked, dl.runtimeMillis())
|
||||||
eligible := prefs.filter(ranked)
|
|
||||||
if len(eligible) == 0 {
|
if len(eligible) == 0 {
|
||||||
return false
|
return fmt.Sprintf(
|
||||||
|
"all %d found are outside the auto-download bitrate, size or format limits",
|
||||||
|
len(ranked),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
best := eligible[0]
|
best := eligible[0]
|
||||||
if best.Match.Overall < minMatch || best.Quality.Overall < minQuality {
|
|
||||||
return false
|
if best.Match.Overall < minMatch {
|
||||||
|
return fmt.Sprintf(
|
||||||
|
"best of %d found matches this release only %.0f%% (needs %.0f%%)",
|
||||||
|
len(ranked),
|
||||||
|
best.Match.Overall*100, //nolint:mnd // percent
|
||||||
|
minMatch*100, //nolint:mnd // percent
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(eligible) > 1 && best.Score-eligible[1].Score < minLead {
|
if best.Quality.Overall < minQuality {
|
||||||
return false
|
return fmt.Sprintf(
|
||||||
|
"best of %d found is the right release but scores %.0f%% on quality (needs %.0f%%)",
|
||||||
|
len(ranked),
|
||||||
|
best.Quality.Overall*100, //nolint:mnd // percent
|
||||||
|
minQuality*100, //nolint:mnd // percent
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return true
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// mergeMatched copies MatchedTo assignments from the audio-only slice
|
// mergeMatched copies MatchedTo assignments from the audio-only slice
|
||||||
|
|||||||
+360
-57
@@ -1,6 +1,34 @@
|
|||||||
package download
|
package download
|
||||||
|
|
||||||
import "testing"
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// trackMillis is five minutes; okComputer's four of them make a
|
||||||
|
// twenty-minute release, which is what turns a candidate's byte count
|
||||||
|
// into a bitrate the assertions below can name.
|
||||||
|
const trackMillis = 5 * 60 * 1000
|
||||||
|
|
||||||
|
// okComputerRuntime is that release's runtime, for the helpers that
|
||||||
|
// need it directly.
|
||||||
|
const okComputerRuntime = 4 * trackMillis
|
||||||
|
|
||||||
|
// kbpsCandidate builds an annotated candidate whose audio adds up to
|
||||||
|
// the given average bitrate over okComputer's runtime.
|
||||||
|
func kbpsCandidate(id, ext string, kbps int) Candidate {
|
||||||
|
// bits = kbps × 1000 × (runtimeMillis / 1000), so the thousands
|
||||||
|
// cancel and the byte count is kbps × runtimeMillis / 8.
|
||||||
|
const bitsPerByte = 8
|
||||||
|
|
||||||
|
total := int64(kbps) * okComputerRuntime / bitsPerByte
|
||||||
|
|
||||||
|
c := candidateFor(id, allTitles(), ext, total/int64(len(allTitles())))
|
||||||
|
c.Files = AnnotateFiles(c.Files)
|
||||||
|
c.TotalSize = total
|
||||||
|
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
// okComputer is the reference request used across ranking tests.
|
// okComputer is the reference request used across ranking tests.
|
||||||
func okComputer() Download {
|
func okComputer() Download {
|
||||||
@@ -8,11 +36,15 @@ func okComputer() Download {
|
|||||||
ReleaseMBID: "mbid-ok-computer",
|
ReleaseMBID: "mbid-ok-computer",
|
||||||
Artist: "Radiohead",
|
Artist: "Radiohead",
|
||||||
Album: "OK Computer",
|
Album: "OK Computer",
|
||||||
|
// Four five-minute tracks: twenty minutes, so a candidate's
|
||||||
|
// bitrate is a number these tests can state exactly. Without
|
||||||
|
// lengths there is no runtime and the bitrate window has
|
||||||
|
// nothing to divide by.
|
||||||
Expected: []ExpectedTrack{
|
Expected: []ExpectedTrack{
|
||||||
{Position: 1, Title: "Airbag"},
|
{Position: 1, Title: "Airbag", LengthMillis: trackMillis},
|
||||||
{Position: 2, Title: "Paranoid Android"},
|
{Position: 2, Title: "Paranoid Android", LengthMillis: trackMillis},
|
||||||
{Position: 3, Title: "Subterranean Homesick Alien"},
|
{Position: 3, Title: "Subterranean Homesick Alien", LengthMillis: trackMillis},
|
||||||
{Position: 4, Title: "Exit Music (For a Film)"},
|
{Position: 4, Title: "Exit Music (For a Film)", LengthMillis: trackMillis},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -187,7 +219,7 @@ func TestUnanchoredMatchIsCapped(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAutoPickableRequiresAnchorAndLead(t *testing.T) {
|
func TestAutoPickableRequiresAnchorAndTracklist(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
dl := okComputer()
|
dl := okComputer()
|
||||||
@@ -211,14 +243,18 @@ func TestAutoPickableRequiresAnchorAndLead(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Run("two close candidates are not", func(t *testing.T) {
|
// Two identical copies are a spare, not an ambiguity. This
|
||||||
|
// asserted the opposite while auto-pick required daylight over the
|
||||||
|
// runner-up — a rule that made abundance the thing that stopped a
|
||||||
|
// request being satisfied, which is backwards.
|
||||||
|
t.Run("two equally good candidates still are", func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
twin := best
|
twin := best
|
||||||
twin.ID = "twin"
|
twin.ID = "twin"
|
||||||
|
|
||||||
if AutoPickable(dl, []Candidate{best, twin}, AutoDownloadPrefs{}) {
|
if !AutoPickable(dl, []Candidate{best, twin}, AutoDownloadPrefs{}) {
|
||||||
t.Error("identical candidates must not auto-pick")
|
t.Error("identical good candidates must auto-pick")
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -300,18 +336,11 @@ func TestProviderPriorityBreaksTies(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const mb = 1 << 20
|
|
||||||
|
|
||||||
func TestAutoDownloadPrefsEligible(t *testing.T) {
|
func TestAutoDownloadPrefsEligible(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
flacCandidate := candidateFor("c", allTitles(), ".flac", 30_000_000)
|
flacCandidate := kbpsCandidate("c", ".flac", 900)
|
||||||
flacCandidate.Files = AnnotateFiles(flacCandidate.Files)
|
mp3Candidate := kbpsCandidate("c", ".mp3", 128)
|
||||||
flacCandidate.TotalSize = 300 * mb
|
|
||||||
|
|
||||||
mp3Candidate := candidateFor("c", allTitles(), ".mp3", 3_000_000)
|
|
||||||
mp3Candidate.Files = AnnotateFiles(mp3Candidate.Files)
|
|
||||||
mp3Candidate.TotalSize = 30 * mb
|
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
@@ -321,18 +350,25 @@ func TestAutoDownloadPrefsEligible(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{"zero value is permissive", AutoDownloadPrefs{}, flacCandidate, true},
|
{"zero value is permissive", AutoDownloadPrefs{}, flacCandidate, true},
|
||||||
{
|
{
|
||||||
"within min/max window",
|
"within the bitrate window",
|
||||||
AutoDownloadPrefs{MinSizeMB: 100, MaxSizeMB: 500},
|
AutoDownloadPrefs{MinKbps: 320, MaxKbps: 1200},
|
||||||
flacCandidate, true,
|
flacCandidate, true,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"below minimum",
|
"below the minimum bitrate",
|
||||||
AutoDownloadPrefs{MinSizeMB: 400},
|
AutoDownloadPrefs{MinKbps: 500},
|
||||||
|
mp3Candidate, false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"above the maximum bitrate",
|
||||||
|
AutoDownloadPrefs{MaxKbps: 500},
|
||||||
flacCandidate, false,
|
flacCandidate, false,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"above maximum",
|
// The ceiling is bytes, not a rate, and it is the guard
|
||||||
AutoDownloadPrefs{MaxSizeMB: 200},
|
// that still works when the bitrate cannot be worked out.
|
||||||
|
"above the hard size ceiling",
|
||||||
|
AutoDownloadPrefs{MaxSizeMB: 50},
|
||||||
flacCandidate, false,
|
flacCandidate, false,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -351,57 +387,131 @@ func TestAutoDownloadPrefsEligible(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
if got := tt.prefs.eligible(tt.c); got != tt.want {
|
got := tt.prefs.eligible(tt.c, okComputerRuntime)
|
||||||
|
if got != tt.want {
|
||||||
t.Errorf("eligible() = %v, want %v", got, tt.want)
|
t.Errorf("eligible() = %v, want %v", got, tt.want)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A release nobody knows the length of cannot be judged on bitrate, and
|
||||||
|
// the window must not become a silent embargo because MusicBrainz is
|
||||||
|
// missing a track length. The size ceiling still applies — that is why
|
||||||
|
// it is a separate field.
|
||||||
|
func TestBitrateWindowPassesAnUnknownRuntime(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
c := kbpsCandidate("c", ".mp3", 128)
|
||||||
|
prefs := AutoDownloadPrefs{MinKbps: 900}
|
||||||
|
|
||||||
|
if !prefs.eligible(c, 0) {
|
||||||
|
t.Error("an unknown runtime must pass the bitrate window")
|
||||||
|
}
|
||||||
|
|
||||||
|
if prefs.eligible(c, okComputerRuntime) {
|
||||||
|
t.Error("a known runtime must still be judged")
|
||||||
|
}
|
||||||
|
|
||||||
|
ceiling := AutoDownloadPrefs{MaxSizeMB: 1}
|
||||||
|
if ceiling.eligible(c, 0) {
|
||||||
|
t.Error("the size ceiling must apply even with no runtime")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Artwork is not part of the bitrate. A folder carrying 30 MB of
|
||||||
|
// scans would otherwise read as a better rip than the same music
|
||||||
|
// without them, which is backwards.
|
||||||
|
func TestBitrateIgnoresNonAudioFiles(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
c := kbpsCandidate("c", ".mp3", 320)
|
||||||
|
bare := candidateKbps(c, okComputerRuntime)
|
||||||
|
|
||||||
|
c.Files = append(c.Files, CandidateFile{
|
||||||
|
Path: "Radiohead - OK Computer/cover.jpg",
|
||||||
|
Size: 30 << 20,
|
||||||
|
})
|
||||||
|
c.Files = AnnotateFiles(c.Files)
|
||||||
|
|
||||||
|
if got := candidateKbps(c, okComputerRuntime); got != bare {
|
||||||
|
t.Errorf("bitrate with artwork = %f, want %f", got, bare)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Where no runtime is known, a stated per-file bitrate is better than
|
||||||
|
// no answer at all.
|
||||||
|
func TestBitrateFallsBackToTheStatedRate(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
c := candidateFor("c", allTitles(), ".mp3", 3_000_000)
|
||||||
|
for i := range c.Files {
|
||||||
|
c.Files[i].Bitrate = 192
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Files = AnnotateFiles(c.Files)
|
||||||
|
|
||||||
|
if got := candidateKbps(c, 0); got != 192 {
|
||||||
|
t.Errorf("stated bitrate = %f, want 192", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestAutoDownloadPrefsFilter(t *testing.T) {
|
func TestAutoDownloadPrefsFilter(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
small := candidateFor("small", allTitles(), ".flac", 10_000_000)
|
lossy := kbpsCandidate("lossy", ".mp3", 128)
|
||||||
small.TotalSize = 50 * mb
|
lossless := kbpsCandidate("lossless", ".flac", 900)
|
||||||
|
|
||||||
big := candidateFor("big", allTitles(), ".flac", 30_000_000)
|
prefs := AutoDownloadPrefs{MinKbps: 500}
|
||||||
big.TotalSize = 500 * mb
|
|
||||||
|
|
||||||
prefs := AutoDownloadPrefs{MinSizeMB: 100, MaxSizeMB: 600}
|
filtered := prefs.filter(
|
||||||
|
[]Candidate{lossy, lossless}, okComputerRuntime,
|
||||||
|
)
|
||||||
|
|
||||||
filtered := prefs.filter([]Candidate{small, big})
|
if len(filtered) != 1 || filtered[0].ID != "lossless" {
|
||||||
|
|
||||||
if len(filtered) != 1 || filtered[0].ID != "big" {
|
|
||||||
t.Errorf("filter() = %v, want only the in-window candidate", filtered)
|
t.Errorf("filter() = %v, want only the in-window candidate", filtered)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAutoDownloadPrefsSizeFit(t *testing.T) {
|
func TestAutoDownloadPrefsBitrateFit(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const neutral = 0.5
|
const neutral = 0.5
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
prefs AutoDownloadPrefs
|
prefs AutoDownloadPrefs
|
||||||
totalSize int64
|
c Candidate
|
||||||
want float64
|
want float64
|
||||||
}{
|
}{
|
||||||
{"no preference is neutral", AutoDownloadPrefs{}, 300 * mb, neutral},
|
{
|
||||||
|
"no preference is neutral",
|
||||||
|
AutoDownloadPrefs{},
|
||||||
|
kbpsCandidate("c", ".flac", 900), neutral,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"exact match scores 1",
|
"exact match scores 1",
|
||||||
AutoDownloadPrefs{PreferredSizeMB: 300},
|
AutoDownloadPrefs{PreferredKbps: 320},
|
||||||
300 * mb, 1.0,
|
kbpsCandidate("c", ".mp3", 320), 1.0,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"double the preferred size scores 0",
|
// The floor is neutral, not zero: this term carries 0.40
|
||||||
AutoDownloadPrefs{PreferredSizeMB: 300},
|
// of the quality score once a preference is set, and a
|
||||||
600 * mb, 0.0,
|
// span to zero would let "I like 320" quietly disqualify
|
||||||
|
// every FLAC from auto-pick.
|
||||||
|
"double the preferred rate falls to the neutral floor",
|
||||||
|
AutoDownloadPrefs{PreferredKbps: 320},
|
||||||
|
kbpsCandidate("c", ".flac", 640), neutral,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"half the preferred size scores 0",
|
"half the preferred rate falls to the neutral floor",
|
||||||
AutoDownloadPrefs{PreferredSizeMB: 300},
|
AutoDownloadPrefs{PreferredKbps: 320},
|
||||||
150 * mb, 0.0,
|
kbpsCandidate("c", ".mp3", 160), neutral,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"an unknowable rate is neutral",
|
||||||
|
AutoDownloadPrefs{PreferredKbps: 320},
|
||||||
|
kbpsCandidate("c", ".mp3", 320), neutral,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -409,30 +519,223 @@ func TestAutoDownloadPrefsSizeFit(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
if got := tt.prefs.sizeFit(tt.totalSize); got != tt.want {
|
// The last case deliberately withholds the runtime.
|
||||||
t.Errorf("sizeFit(%d) = %f, want %f", tt.totalSize, got, tt.want)
|
runtime := int64(okComputerRuntime)
|
||||||
|
if tt.name == "an unknowable rate is neutral" {
|
||||||
|
runtime = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := tt.prefs.bitrateFit(tt.c, runtime); got != tt.want {
|
||||||
|
t.Errorf("bitrateFit() = %f, want %f", got, tt.want)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// An otherwise-perfect candidate must not auto-pick when it falls
|
// An otherwise-perfect candidate must not auto-pick when it falls
|
||||||
// outside the configured size guard: the guardrail applies before the
|
// outside the configured guardrails: they apply before the match and
|
||||||
// match/quality/lead checks, not as one more input averaged into them.
|
// quality checks, not as one more input averaged into them.
|
||||||
func TestAutoPickableRejectsCandidateOutsideSizeGuard(t *testing.T) {
|
func TestAutoPickableRejectsCandidateOutsideTheGuardrails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
dl := okComputer()
|
dl := okComputer()
|
||||||
best := Score(dl, candidateFor("a", allTitles(), ".flac", 30_000_000), 50, AutoDownloadPrefs{})
|
best := Score(dl, kbpsCandidate("a", ".flac", 900), 50, AutoDownloadPrefs{})
|
||||||
best.TotalSize = 500 * mb
|
|
||||||
|
|
||||||
if !AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{}) {
|
if !AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{}) {
|
||||||
t.Fatal("expected this candidate to be auto-pickable with no guardrails")
|
t.Fatal("expected this candidate to be auto-pickable with no guardrails")
|
||||||
}
|
}
|
||||||
|
|
||||||
tight := AutoDownloadPrefs{MinSizeMB: 10, MaxSizeMB: 100}
|
if AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{MaxKbps: 320}) {
|
||||||
|
t.Error("candidate above the bitrate window must not auto-pick")
|
||||||
|
}
|
||||||
|
|
||||||
if AutoPickable(dl, []Candidate{best}, tight) {
|
if AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{MaxSizeMB: 1}) {
|
||||||
t.Error("candidate outside the size guard must not auto-pick")
|
t.Error("candidate above the size ceiling must not auto-pick")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The refusal has to name the gate that refused.
|
||||||
|
//
|
||||||
|
// Before AutoPickVeto, every one of these came back as the same
|
||||||
|
// sentence built from `ranked[0]` — the best candidate before the size
|
||||||
|
// and format guardrails — so a request refused because the user's size
|
||||||
|
// window excluded every copy reported a match and a quality that both
|
||||||
|
// cleared their thresholds. A refusal quoting numbers that pass is
|
||||||
|
// what made the matcher look broken from outside.
|
||||||
|
func TestAutoPickVetoNamesTheGate(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dl := okComputer()
|
||||||
|
best := Score(
|
||||||
|
dl,
|
||||||
|
candidateFor("a", allTitles(), ".flac", 30_000_000),
|
||||||
|
50,
|
||||||
|
AutoDownloadPrefs{},
|
||||||
|
)
|
||||||
|
|
||||||
|
// candidateFor sizes the files and leaves TotalSize at 0, which is
|
||||||
|
// what the guardrails read.
|
||||||
|
sized := func(c Candidate, total int64) Candidate {
|
||||||
|
c.TotalSize = total
|
||||||
|
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
dl Download
|
||||||
|
ranked []Candidate
|
||||||
|
prefs AutoDownloadPrefs
|
||||||
|
wantSub string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "nothing found",
|
||||||
|
dl: dl,
|
||||||
|
ranked: nil,
|
||||||
|
wantSub: "nothing found",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "free text",
|
||||||
|
dl: Download{Artist: "Radiohead", Album: "OK Computer"},
|
||||||
|
ranked: []Candidate{best},
|
||||||
|
wantSub: "free text",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "no tracklist behind the anchor",
|
||||||
|
dl: Download{
|
||||||
|
ReleaseMBID: "mbid-ok-computer",
|
||||||
|
Artist: "Radiohead",
|
||||||
|
Album: "OK Computer",
|
||||||
|
},
|
||||||
|
ranked: []Candidate{best},
|
||||||
|
wantSub: "no tracklist",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// The candidate is 120 MB and the window tops out at 1 MB:
|
||||||
|
// the old message reported its match and quality instead.
|
||||||
|
name: "outside the size window",
|
||||||
|
dl: dl,
|
||||||
|
ranked: []Candidate{sized(best, 120<<20)},
|
||||||
|
prefs: AutoDownloadPrefs{MaxSizeMB: 1},
|
||||||
|
wantSub: "bitrate, size or format limits",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
got := AutoPickVeto(tt.dl, tt.ranked, tt.prefs)
|
||||||
|
if !strings.Contains(got, tt.wantSub) {
|
||||||
|
t.Errorf("veto = %q, want it to mention %q", got, tt.wantSub)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A clear winner has no veto at all — the sentence is empty, which is
|
||||||
|
// what AutoPickable reads.
|
||||||
|
func TestAutoPickVetoIsEmptyForAClearWinner(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dl := okComputer()
|
||||||
|
best := Score(
|
||||||
|
dl,
|
||||||
|
candidateFor("a", allTitles(), ".flac", 30_000_000),
|
||||||
|
50,
|
||||||
|
AutoDownloadPrefs{},
|
||||||
|
)
|
||||||
|
weak := Score(
|
||||||
|
dl,
|
||||||
|
candidateFor("b", allTitles()[:2], ".mp3", 1_000_000),
|
||||||
|
50,
|
||||||
|
AutoDownloadPrefs{},
|
||||||
|
)
|
||||||
|
|
||||||
|
if got := AutoPickVeto(dl, []Candidate{best, weak}, AutoDownloadPrefs{}); got != "" {
|
||||||
|
t.Errorf("veto = %q, want none", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// With several candidates that all clear the bar, the preferred
|
||||||
|
// bitrate decides which one is taken.
|
||||||
|
//
|
||||||
|
// This is what replaced the daylight requirement. Auto-pick no longer
|
||||||
|
// refuses when the field is close; it takes the copy nearest the shape
|
||||||
|
// the user asked for, which is the question they actually answered in
|
||||||
|
// Settings.
|
||||||
|
func TestPreferredBitrateBreaksTheTie(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dl := okComputer()
|
||||||
|
prefs := AutoDownloadPrefs{PreferredKbps: 320}
|
||||||
|
|
||||||
|
// Same album, same completeness, same health, same provider — the
|
||||||
|
// only difference between them is the rate.
|
||||||
|
lossless := kbpsCandidate("lossless", ".flac", 900)
|
||||||
|
perfect := kbpsCandidate("perfect", ".mp3", 320)
|
||||||
|
|
||||||
|
ranked := Rank(
|
||||||
|
dl, []Candidate{lossless, perfect}, nil, prefs,
|
||||||
|
)
|
||||||
|
|
||||||
|
if ranked[0].ID != "perfect" {
|
||||||
|
t.Errorf(
|
||||||
|
"winner = %q (fit %f) over %q (fit %f), want the 320 kbps copy",
|
||||||
|
ranked[0].ID, ranked[0].Quality.BitrateFit,
|
||||||
|
ranked[1].ID, ranked[1].Quality.BitrateFit,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if AutoPickVeto(dl, ranked, prefs) != "" {
|
||||||
|
t.Error("a close field must still auto-pick")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// With no preference set, nothing changes: BitrateFit is the same
|
||||||
|
// neutral value for every candidate and the older tie-breaks decide.
|
||||||
|
func TestNoPreferredBitrateLeavesRankingAlone(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dl := okComputer()
|
||||||
|
|
||||||
|
lossless := kbpsCandidate("lossless", ".flac", 900)
|
||||||
|
lossy := kbpsCandidate("lossy", ".mp3", 320)
|
||||||
|
|
||||||
|
ranked := Rank(
|
||||||
|
dl, []Candidate{lossy, lossless}, nil, AutoDownloadPrefs{},
|
||||||
|
)
|
||||||
|
|
||||||
|
if ranked[0].ID != "lossless" {
|
||||||
|
t.Errorf(
|
||||||
|
"winner = %q, want the lossless copy on format alone",
|
||||||
|
ranked[0].ID,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A preferred bitrate promotes the copy that matches it and must never
|
||||||
|
// disqualify the ones that do not. It carries 0.40 of the quality
|
||||||
|
// score, so a fit spanning down to zero would put a perfectly good FLAC
|
||||||
|
// under minQuality and out of auto-pick — turning a preference into a
|
||||||
|
// prohibition without saying so. MinKbps and MaxKbps are how a user
|
||||||
|
// says that on purpose.
|
||||||
|
func TestAPreferredBitrateNeverDisqualifies(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dl := okComputer()
|
||||||
|
far := AutoDownloadPrefs{PreferredKbps: 128}
|
||||||
|
|
||||||
|
lossless := Score(dl, kbpsCandidate("flac", ".flac", 900), 50, far)
|
||||||
|
|
||||||
|
if lossless.Quality.Overall < minQuality {
|
||||||
|
t.Errorf(
|
||||||
|
"quality = %f under a far-off preference, want >= %f",
|
||||||
|
lossless.Quality.Overall, minQuality,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if veto := AutoPickVeto(dl, []Candidate{lossless}, far); veto != "" {
|
||||||
|
t.Errorf("a far-off preference vetoed the candidate: %s", veto)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -310,15 +310,35 @@ func (r *Reconciler) run(ctx context.Context, force bool) (Summary, error) {
|
|||||||
|
|
||||||
summary.Synced = r.syncExternalLists(ctx)
|
summary.Synced = r.syncExternalLists(ctx)
|
||||||
|
|
||||||
attempted, started, err := r.attemptDue(ctx, force)
|
// Nothing is searched for when there is nothing to search with, and
|
||||||
if err != nil {
|
// the point is what that *does not* do to the list.
|
||||||
return summary, err
|
//
|
||||||
|
// Attempting anyway is not merely wasted work: every request comes
|
||||||
|
// back "no download clients are enabled", which RecordAttempt writes
|
||||||
|
// down as an attempt and schedules a retry for -- so a user who has
|
||||||
|
// deliberately built a wanted list with no client watched their
|
||||||
|
// requests accrue failures and announce "next check in 6 hours"
|
||||||
|
// about a check that cannot happen. Wanting something without a way
|
||||||
|
// to fetch it is a supported thing to do; being told it is being
|
||||||
|
// looked for is a lie.
|
||||||
|
//
|
||||||
|
// Everything above this line still runs: an artist subscription
|
||||||
|
// still expands, and a request the user satisfied by some other
|
||||||
|
// route -- ripped, bought, copied in -- is still retired, because
|
||||||
|
// neither needs a provider.
|
||||||
|
summary.NoProviders = len(r.manager.enabledProviders()) == 0
|
||||||
|
|
||||||
|
if !summary.NoProviders {
|
||||||
|
attempted, started, err := r.attemptDue(ctx, force)
|
||||||
|
if err != nil {
|
||||||
|
return summary, err
|
||||||
|
}
|
||||||
|
|
||||||
|
summary.Attempted = attempted
|
||||||
|
summary.Started = started
|
||||||
}
|
}
|
||||||
|
|
||||||
summary.Attempted = attempted
|
|
||||||
summary.Started = started
|
|
||||||
summary.Waiting = r.countWaiting(ctx)
|
summary.Waiting = r.countWaiting(ctx)
|
||||||
summary.NoProviders = len(r.manager.enabledProviders()) == 0
|
|
||||||
|
|
||||||
r.logger.Info(
|
r.logger.Info(
|
||||||
"reconciled request list",
|
"reconciled request list",
|
||||||
|
|||||||
@@ -453,6 +453,15 @@ func TestReconcileRespectsBatchSize(t *testing.T) {
|
|||||||
f := newReconcileFixture(t)
|
f := newReconcileFixture(t)
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
|
|
||||||
|
// A client that searches and finds nothing. The batch size is about
|
||||||
|
// how many requests one pass *searches for*, which only means
|
||||||
|
// anything when there is something to search with -- a pass with no
|
||||||
|
// provider now attempts nothing at all, deliberately.
|
||||||
|
f.manager.installProvider(
|
||||||
|
Config{ID: 1, Priority: 50},
|
||||||
|
NewFakeProvider(1, "finds-nothing", Caps{CanSearch: true}),
|
||||||
|
)
|
||||||
|
|
||||||
f.reconciler.SetBatch(2)
|
f.reconciler.SetBatch(2)
|
||||||
|
|
||||||
for _, mbid := range []string{"rg-1", "rg-2", "rg-3", "rg-4"} {
|
for _, mbid := range []string{"rg-1", "rg-2", "rg-3", "rg-4"} {
|
||||||
@@ -593,3 +602,72 @@ func TestSummaryReportsNoProviders(t *testing.T) {
|
|||||||
t.Error("summary did not report that no download client is enabled")
|
t.Error("summary did not report that no download client is enabled")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ...and it does not search, which is the part the user sees.
|
||||||
|
//
|
||||||
|
// Attempting with no provider fails every request with "no download
|
||||||
|
// clients are enabled", and RecordAttempt writes that down as an
|
||||||
|
// attempt and schedules a retry -- so a wanted list built deliberately
|
||||||
|
// without a client accrued failures and announced "next check in 6
|
||||||
|
// hours" about a check that cannot happen. Wanting something with no
|
||||||
|
// way to fetch it is supported; being told it is being looked for is
|
||||||
|
// a lie.
|
||||||
|
func TestNoProvidersMeansNoAttempt(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
f := newReconcileFixture(t)
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
id, err := f.store.AddRequest(ctx, Request{
|
||||||
|
MBID: "rg-1",
|
||||||
|
Entity: EntityReleaseGroup,
|
||||||
|
LibraryID: 1,
|
||||||
|
Title: "OK Computer",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("AddRequest: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
f.catalog.tracklists["rg-1"] = fourTrackDownload().Expected
|
||||||
|
|
||||||
|
summary, err := f.reconciler.RunNow(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunNow: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if summary.Attempted != 0 {
|
||||||
|
t.Errorf("attempted %d requests with no client to search with, want 0",
|
||||||
|
summary.Attempted)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The list still knows what is on it: "nothing happened" has to be
|
||||||
|
// reportable as "nothing was searched for, of the one thing you
|
||||||
|
// want" rather than as silence.
|
||||||
|
if summary.Waiting != 1 {
|
||||||
|
t.Errorf("summary reported %d waiting, want 1", summary.Waiting)
|
||||||
|
}
|
||||||
|
|
||||||
|
req, err := f.store.GetRequest(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetRequest: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if req.Attempts != 0 {
|
||||||
|
t.Errorf("attempts = %d, want 0: a pass that could not search did not",
|
||||||
|
req.Attempts)
|
||||||
|
}
|
||||||
|
|
||||||
|
if req.LastError != "" {
|
||||||
|
t.Errorf("lastError = %q, want empty: the request did not fail, it "+
|
||||||
|
"was never tried", req.LastError)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A new request is due immediately (next_try_at is set to now on
|
||||||
|
// insert), so the fault is not the presence of a time -- it is a
|
||||||
|
// time pushed into the future by a failed attempt, which is what the
|
||||||
|
// UI renders as "next check in 6 hours".
|
||||||
|
if req.NextTryAt.After(time.Now().Add(time.Minute)) {
|
||||||
|
t.Errorf("next try scheduled for %v: a check that cannot happen was "+
|
||||||
|
"put on the clock", req.NextTryAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -20,6 +20,41 @@ func newServiceFixture(t *testing.T) serviceFixture {
|
|||||||
mf := newManagerFixture(t)
|
mf := newManagerFixture(t)
|
||||||
svc := NewService(slogDiscard(), mf.manager, mf.store, NewMemSecretStore())
|
svc := NewService(slogDiscard(), mf.manager, mf.store, NewMemSecretStore())
|
||||||
|
|
||||||
|
// Every test here is about the durable Request that `StartDownload`
|
||||||
|
// leaves behind, and none of them is about the download itself -- but
|
||||||
|
// their fixture is an anchored four-track request with a healthy
|
||||||
|
// provider, which is exactly what `AutoPickable` says yes to. So
|
||||||
|
// `Manager.Start` was firing `go m.grab(...)`, detached and with
|
||||||
|
// `context.WithoutCancel`, and the test then raced it.
|
||||||
|
//
|
||||||
|
// It lost, twice, in CI (`check` on c03c0b8, and nowhere locally):
|
||||||
|
//
|
||||||
|
// service_test.go:66: state = "satisfied", want wanted
|
||||||
|
// testing.go:1369: TempDir RemoveAll cleanup: ... directory not empty
|
||||||
|
//
|
||||||
|
// The first is the request reaching its *next* state before the
|
||||||
|
// assertion read it; the second is that same goroutine still writing
|
||||||
|
// into `t.TempDir()` after the test returned. One cause, two shapes.
|
||||||
|
//
|
||||||
|
// Putting the candidate outside the auto-pick guardrails stops the
|
||||||
|
// grab from ever starting, which is better than waiting for it: there
|
||||||
|
// is no goroutine to be slow, so the tests state what they mean
|
||||||
|
// ("the request exists, in this state") without a timing assumption
|
||||||
|
// underneath. A test that does want the download has `managerFixture`
|
||||||
|
// and sets its own preferences.
|
||||||
|
//
|
||||||
|
// The guard is a *format* the fake never produces, and it used to be
|
||||||
|
// `MaxSizeMB: 1`, which never fired: the size gates read
|
||||||
|
// `Candidate.TotalSize`, which real providers fill and the fake
|
||||||
|
// leaves at zero, and zero is under every ceiling. So the grab went
|
||||||
|
// ahead anyway and the second failure shape above — the TempDir
|
||||||
|
// cleanup race — kept happening, reproducibly, roughly one run in
|
||||||
|
// fifteen. A guard has to be keyed on something the fixture
|
||||||
|
// actually sets.
|
||||||
|
mf.manager.SetPreferences(AutoDownloadPrefs{
|
||||||
|
AllowedFormats: []Format{FormatWMA},
|
||||||
|
})
|
||||||
|
|
||||||
return serviceFixture{managerFixture: mf, svc: svc}
|
return serviceFixture{managerFixture: mf, svc: svc}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -158,6 +193,11 @@ func TestManualDownloadSatisfiesRequestOnSuccess(t *testing.T) {
|
|||||||
f := newServiceFixture(t)
|
f := newServiceFixture(t)
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
|
|
||||||
|
// This is the one test here that is *about* the download, so it
|
||||||
|
// undoes the fixture's guard rather than relying on it — which is
|
||||||
|
// what it was doing implicitly while the guard did not work.
|
||||||
|
f.manager.SetPreferences(AutoDownloadPrefs{})
|
||||||
|
|
||||||
provider := fakeWithAlbum(1, "source", ".flac")
|
provider := fakeWithAlbum(1, "source", ".flac")
|
||||||
f.manager.installProvider(Config{ID: 1, Priority: 50}, provider)
|
f.manager.installProvider(Config{ID: 1, Priority: 50}, provider)
|
||||||
|
|
||||||
|
|||||||
@@ -302,7 +302,12 @@ type QualityScore struct {
|
|||||||
Bitrate float64 `json:"bitrate"`
|
Bitrate float64 `json:"bitrate"`
|
||||||
Health float64 `json:"health"` // seeders, free slots
|
Health float64 `json:"health"` // seeders, free slots
|
||||||
Priority float64 `json:"priority"` // user's per-provider preference
|
Priority float64 `json:"priority"` // user's per-provider preference
|
||||||
SizeFit float64 `json:"sizeFit"` // closeness to the preferred download size
|
// BitrateFit is closeness to the preferred *rate*, which is what
|
||||||
|
// the auto-download window is expressed in. It replaced a
|
||||||
|
// `SizeFit` measured in megabytes: a size means nothing without
|
||||||
|
// knowing how long the music is, so the same number described a
|
||||||
|
// generous single and a suspiciously small boxset.
|
||||||
|
BitrateFit float64 `json:"bitrateFit"`
|
||||||
|
|
||||||
// Mixed marks a candidate whose files are not all the same format,
|
// Mixed marks a candidate whose files are not all the same format,
|
||||||
// which usually means a hand-assembled folder rather than a rip.
|
// which usually means a hand-assembled folder rather than a rip.
|
||||||
|
|||||||
@@ -27,6 +27,20 @@ var artifactStageNames = [...]string{
|
|||||||
// failure path is non-fatal by design: the caller falls back, and a
|
// failure path is non-fatal by design: the caller falls back, and a
|
||||||
// fresh install with no network still gets its own library in Explore.
|
// fresh install with no network still gets its own library in Explore.
|
||||||
func (si *SearchIndex) tryCoreArtifact(ctx context.Context) error {
|
func (si *SearchIndex) tryCoreArtifact(ctx context.Context) error {
|
||||||
|
// Before anything is staged: ~0.6 GB is not a download to start on
|
||||||
|
// someone's cellular allowance without being asked (plan 016 B4).
|
||||||
|
// This is checked first so no job appears and no status changes --
|
||||||
|
// declining is a no-op, not a failure the user has to dismiss.
|
||||||
|
if si.netPolicy.refuses() {
|
||||||
|
si.logIndexJob(
|
||||||
|
jobs.LevelInfo,
|
||||||
|
"Skipping the catalog download on a metered connection. "+
|
||||||
|
"Enable it in Settings to download anyway.",
|
||||||
|
)
|
||||||
|
|
||||||
|
return ErrMeteredNetwork
|
||||||
|
}
|
||||||
|
|
||||||
si.mu.Lock()
|
si.mu.Lock()
|
||||||
si.buildStatus = IndexStatus{
|
si.buildStatus = IndexStatus{
|
||||||
Building: true,
|
Building: true,
|
||||||
|
|||||||
@@ -283,6 +283,9 @@ func (si *SearchIndex) importCoreArtifact(ctx context.Context, path string) erro
|
|||||||
}
|
}
|
||||||
|
|
||||||
merged, mergeErr := si.mergeArtifactRows(ctx, info.rows)
|
merged, mergeErr := si.mergeArtifactRows(ctx, info.rows)
|
||||||
|
if mergeErr == nil {
|
||||||
|
si.mergeArtifactCredits(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
if ftsSuspended {
|
if ftsSuspended {
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
@@ -473,3 +476,75 @@ func (si *SearchIndex) removeArtifactFile(path string) {
|
|||||||
si.logger.Warn("core artifact: cleanup failed", "path", path, "error", err)
|
si.logger.Warn("core artifact: cleanup failed", "path", path, "error", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// artifactHasCredits reports whether the attached artifact carries the
|
||||||
|
// multi-artist credit tables.
|
||||||
|
//
|
||||||
|
// The same shape, and the same handle, as artifactHasTotals above: an
|
||||||
|
// artifact published before credits existed is still a perfectly good
|
||||||
|
// catalog, and there is one already out there. Selecting from a table
|
||||||
|
// that is not in it would fail an import that should have succeeded, so
|
||||||
|
// it is asked rather than assumed -- on the *writer*, because `core` is
|
||||||
|
// attached to that one connection and the read pool cannot see it.
|
||||||
|
func (si *SearchIndex) artifactHasCredits() bool {
|
||||||
|
var n int
|
||||||
|
|
||||||
|
err := si.db.QueryRowWriter(
|
||||||
|
`SELECT COUNT(*) FROM core.sqlite_master
|
||||||
|
WHERE type = 'table' AND name IN ('artist_credit_part', 'artist_credit_ref')`,
|
||||||
|
).Scan(&n)
|
||||||
|
|
||||||
|
return err == nil && n == 2
|
||||||
|
}
|
||||||
|
|
||||||
|
// mergeArtifactCredits copies the credit decomposition out of the
|
||||||
|
// attached artifact.
|
||||||
|
//
|
||||||
|
// Credits are replaced wholesale rather than merged: they are derived
|
||||||
|
// entirely from one dump build, they are keyed by ids that are only
|
||||||
|
// meaningful within the artifact that carried them, and a half-updated
|
||||||
|
// credit renders as the wrong artists rather than as missing ones.
|
||||||
|
//
|
||||||
|
// A failure here is logged and not returned. The catalog has already
|
||||||
|
// merged at this point, and a catalog without credits is the catalog
|
||||||
|
// this app had before them -- every credit falls back to its single
|
||||||
|
// artist, which is the same fallback an untagged file already gets.
|
||||||
|
func (si *SearchIndex) mergeArtifactCredits(ctx context.Context) {
|
||||||
|
if !si.artifactHasCredits() {
|
||||||
|
si.logger.Info("core artifact: no credit tables, keeping single-artist credits")
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
start := time.Now()
|
||||||
|
|
||||||
|
for _, stmt := range []string{
|
||||||
|
"DELETE FROM artist_credit_part",
|
||||||
|
"DELETE FROM artist_credit_ref",
|
||||||
|
`INSERT OR REPLACE INTO artist_credit_part
|
||||||
|
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
SELECT credit_id, position, artist_mbid, credited_name, join_phrase
|
||||||
|
FROM core.artist_credit_part`,
|
||||||
|
`INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id)
|
||||||
|
SELECT mbid, credit_id FROM core.artist_credit_ref`,
|
||||||
|
} {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := si.db.ExecContext(stmt); err != nil {
|
||||||
|
si.logger.Warn("core artifact: credit merge failed", "error", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var refs int
|
||||||
|
|
||||||
|
_ = si.db.QueryRowWriter("SELECT COUNT(*) FROM artist_credit_ref").Scan(&refs)
|
||||||
|
|
||||||
|
si.logger.Info("core artifact: credits merged",
|
||||||
|
"entities", refs,
|
||||||
|
"elapsed", time.Since(start).Round(time.Millisecond),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package explore
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
|
"encoding/hex"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -597,3 +598,153 @@ func TestImportCoreArtifactReadsTotalsWhenPresent(t *testing.T) {
|
|||||||
t.Errorf("TotalTracks = %d, want 0 (the catalog does not say)", old.TotalTracks)
|
t.Errorf("TotalTracks = %d, want 0 (the catalog does not say)", old.TotalTracks)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// addArtifactCredits gives an artifact file the credit tables the
|
||||||
|
// exporter now writes, so the import path can be exercised against one
|
||||||
|
// that has them.
|
||||||
|
func addArtifactCredits(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
db, err := sql.Open("sqlite", "file:"+path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open artifact: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
for _, stmt := range []string{
|
||||||
|
`CREATE TABLE artist_credit_part (
|
||||||
|
credit_id INTEGER NOT NULL,
|
||||||
|
position INTEGER NOT NULL,
|
||||||
|
artist_mbid BLOB NOT NULL,
|
||||||
|
credited_name TEXT NOT NULL,
|
||||||
|
join_phrase TEXT NOT NULL DEFAULT '',
|
||||||
|
PRIMARY KEY (credit_id, position)
|
||||||
|
) WITHOUT ROWID`,
|
||||||
|
`CREATE TABLE artist_credit_ref (
|
||||||
|
mbid BLOB NOT NULL PRIMARY KEY,
|
||||||
|
credit_id INTEGER NOT NULL
|
||||||
|
) WITHOUT ROWID`,
|
||||||
|
} {
|
||||||
|
if _, err := db.Exec(stmt); err != nil {
|
||||||
|
t.Fatalf("create credit tables: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The packed form the catalog stores. uuid16/parseUUID live behind
|
||||||
|
// the indexbuild tag, so this file decodes for itself.
|
||||||
|
pack := func(mbid string) []byte {
|
||||||
|
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
|
||||||
|
if err != nil || len(raw) != 16 {
|
||||||
|
t.Fatalf("fixture MBID %q is not a UUID: %v", mbid, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
|
a, b, rec := pack(artA), pack(artB), pack(recA)
|
||||||
|
|
||||||
|
for _, part := range [][]any{
|
||||||
|
{7, 0, a, "Artist A", " feat. "},
|
||||||
|
{7, 1, b, "Artist B", ""},
|
||||||
|
} {
|
||||||
|
if _, err := db.Exec(`INSERT INTO artist_credit_part
|
||||||
|
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
VALUES (?, ?, ?, ?, ?)`, part...); err != nil {
|
||||||
|
t.Fatalf("insert part: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.Exec(
|
||||||
|
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)", rec, 7,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("insert ref: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImportCoreArtifactMergesCredits is the positive half of the
|
||||||
|
// compatibility pair: an artifact that carries credits delivers them,
|
||||||
|
// rendering back to the credit string they decompose.
|
||||||
|
func TestImportCoreArtifactMergesCredits(t *testing.T) {
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
si := NewSearchIndex(db, nil, nil, testLogger())
|
||||||
|
|
||||||
|
path := writeTestArtifact(t, validMeta(), []artifactRow{
|
||||||
|
{"recording", recA, "Song A", "Artist A feat. Artist B", artA, 2000},
|
||||||
|
})
|
||||||
|
|
||||||
|
addArtifactCredits(t, path)
|
||||||
|
|
||||||
|
if err := si.importCoreArtifact(context.Background(), path); err != nil {
|
||||||
|
t.Fatalf("importCoreArtifact: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := db.QueryContext(
|
||||||
|
`SELECT p.credited_name, p.join_phrase
|
||||||
|
FROM artist_credit_ref r
|
||||||
|
JOIN artist_credit_part p ON p.credit_id = r.credit_id
|
||||||
|
ORDER BY p.position`,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("query credits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
var rendered strings.Builder
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var name, join string
|
||||||
|
|
||||||
|
if err := rows.Scan(&name, &join); err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rendered.WriteString(name)
|
||||||
|
rendered.WriteString(join)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := rendered.String(); got != "Artist A feat. Artist B" {
|
||||||
|
t.Errorf("rendered credit = %q, want %q", got, "Artist A feat. Artist B")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImportCoreArtifactWithoutCredits is the regression that matters
|
||||||
|
// most here: an artifact published before credits existed cannot be
|
||||||
|
// re-cut retroactively, so it must import as a catalog that declines to
|
||||||
|
// answer rather than failing outright. writeTestArtifact deliberately
|
||||||
|
// builds one without the tables.
|
||||||
|
func TestImportCoreArtifactWithoutCredits(t *testing.T) {
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
si := NewSearchIndex(db, nil, nil, testLogger())
|
||||||
|
|
||||||
|
path := writeTestArtifact(t, validMeta(), []artifactRow{
|
||||||
|
{"recording", recA, "Song A", "Artist A", artA, 2000},
|
||||||
|
})
|
||||||
|
|
||||||
|
if err := si.importCoreArtifact(context.Background(), path); err != nil {
|
||||||
|
t.Fatalf("an artifact without credit tables must still import: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var rows int
|
||||||
|
if err := db.QueryRowWriter(
|
||||||
|
"SELECT COUNT(*) FROM explore_index",
|
||||||
|
).Scan(&rows); err != nil {
|
||||||
|
t.Fatalf("count: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rows != 1 {
|
||||||
|
t.Errorf("catalog rows = %d, want 1", rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
var refs int
|
||||||
|
if err := db.QueryRowWriter(
|
||||||
|
"SELECT COUNT(*) FROM artist_credit_ref",
|
||||||
|
).Scan(&refs); err != nil {
|
||||||
|
t.Fatalf("count refs: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if refs != 0 {
|
||||||
|
t.Errorf("credit refs = %d, want 0", refs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -46,23 +46,42 @@ func TestCacheMiss(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestCacheTTLExpiry checks both halves of the TTL contract, and uses two
|
||||||
|
// entries to do it.
|
||||||
|
//
|
||||||
|
// **No assertion here may depend on an upper bound of elapsed wall-clock
|
||||||
|
// time**, which is what the single-entry version of this test did: it set
|
||||||
|
// a 1s TTL and immediately asserted a *hit*, so on a loaded runner — one
|
||||||
|
// goroutine descheduled for over a second while the rest of the suite
|
||||||
|
// runs — the entry was correctly gone and the test failed with "expected
|
||||||
|
// cache hit immediately after set". It did exactly that in CI while
|
||||||
|
// passing five times out of five locally.
|
||||||
|
//
|
||||||
|
// Sleeping *past* a TTL is always safe, so the expiry half keeps a short
|
||||||
|
// one; the presence half gets a TTL nothing can outrun.
|
||||||
func TestCacheTTLExpiry(t *testing.T) {
|
func TestCacheTTLExpiry(t *testing.T) {
|
||||||
c := newTestCache(t)
|
c := newTestCache(t)
|
||||||
|
|
||||||
data := []byte(`{"ephemeral":true}`)
|
data := []byte(`{"ephemeral":true}`)
|
||||||
c.Set("ttl-test-key", data, 1*time.Second, "", "")
|
c.Set("ttl-live-key", data, time.Hour, "", "")
|
||||||
|
c.Set("ttl-expiring-key", data, 1*time.Second, "", "")
|
||||||
|
|
||||||
// Verify it's there immediately.
|
if _, ok := c.Get("ttl-live-key"); !ok {
|
||||||
if _, ok := c.Get("ttl-test-key"); !ok {
|
t.Fatal("expected a cache hit on an entry with an hour to live")
|
||||||
t.Fatal("expected cache hit immediately after set")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Wait for expiry.
|
// Wait for the short one to expire.
|
||||||
time.Sleep(2 * time.Second)
|
time.Sleep(2 * time.Second)
|
||||||
|
|
||||||
if _, ok := c.Get("ttl-test-key"); ok {
|
if _, ok := c.Get("ttl-expiring-key"); ok {
|
||||||
t.Error("expected cache miss after TTL expiry, got hit")
|
t.Error("expected cache miss after TTL expiry, got hit")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// And the long-lived entry is still there, which is what says the
|
||||||
|
// sweep above expired an entry rather than the cache.
|
||||||
|
if _, ok := c.Get("ttl-live-key"); !ok {
|
||||||
|
t.Error("the hour-long entry expired too")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCacheMBID(t *testing.T) {
|
func TestCacheMBID(t *testing.T) {
|
||||||
|
|||||||
@@ -25,8 +25,26 @@ const (
|
|||||||
// where cached cover art thumbnails are stored.
|
// where cached cover art thumbnails are stored.
|
||||||
thumbnailDir = CoverArtCacheDirName
|
thumbnailDir = CoverArtCacheDirName
|
||||||
|
|
||||||
// thumbnailTimeout is the HTTP timeout for fetching a thumbnail.
|
// thumbnailTimeout is the HTTP timeout for fetching a thumbnail,
|
||||||
thumbnailTimeout = 10 * time.Second
|
// and it has to cover a redirect the Cover Art Archive does not
|
||||||
|
// serve itself.
|
||||||
|
//
|
||||||
|
// `coverartarchive.org` answers `front-250` with a 307 to an
|
||||||
|
// Internet Archive storage node (`dn######.us.archive.org`), and
|
||||||
|
// those nodes are routinely slow: measured against the twelve
|
||||||
|
// albums on Explore's own shelves, a successful fetch took 14–16 s
|
||||||
|
// and a failing one 13–17 s. At 10 s *every* cover on the page
|
||||||
|
// timed out — 24 cards, 5 of which had art, all of those from the
|
||||||
|
// disk cache — which reads as "Explore has no album art" rather
|
||||||
|
// than as a slow upstream, because a timeout writes nothing and
|
||||||
|
// says nothing.
|
||||||
|
//
|
||||||
|
// 30 s is chosen to clear that measured range with room, not to be
|
||||||
|
// generous: the fetch is off the critical path (each one is its own
|
||||||
|
// goroutine behind an 8/s limiter, and the frontend renders a
|
||||||
|
// placeholder until it lands), so the cost of waiting is nothing
|
||||||
|
// and the cost of giving up early is a blank page.
|
||||||
|
thumbnailTimeout = 30 * time.Second
|
||||||
|
|
||||||
// thumbnailMaxSize is the maximum image size to cache (2 MB).
|
// thumbnailMaxSize is the maximum image size to cache (2 MB).
|
||||||
thumbnailMaxSize = 2 * 1024 * 1024
|
thumbnailMaxSize = 2 * 1024 * 1024
|
||||||
@@ -97,6 +115,20 @@ func (p *CoverArtProxy) GetThumbnail(
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A 404 is an answer, and it is already on disk.
|
||||||
|
//
|
||||||
|
// `writeCache(mbid, nil)` has recorded "the archive has no art for
|
||||||
|
// this" as an empty file since this was written, and nothing has
|
||||||
|
// ever read it back: `readCache` returns "" for an empty file,
|
||||||
|
// which is indistinguishable from a miss, so every art-less release
|
||||||
|
// group was re-fetched from the network on every render that asked
|
||||||
|
// about it. On Explore's shelves a third of the cards are art-less,
|
||||||
|
// so that was a third of the page spending a live CAA request to be
|
||||||
|
// told again what the last one said.
|
||||||
|
if p.knownMissing(releaseGroupMBID) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
// Source 3: fetch from Cover Art Archive (slow, cached to disk).
|
// Source 3: fetch from Cover Art Archive (slow, cached to disk).
|
||||||
url := CoverArtGroupURL(releaseGroupMBID)
|
url := CoverArtGroupURL(releaseGroupMBID)
|
||||||
data, cacheable, err := p.fetch(url)
|
data, cacheable, err := p.fetch(url)
|
||||||
@@ -177,8 +209,9 @@ func (p *CoverArtProxy) GetCandidateThumbnail(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Network fetch on release group.
|
// Network fetch on release group — unless a previous one was told
|
||||||
if releaseGroupMBID != "" {
|
// there is none. See `knownMissing`.
|
||||||
|
if releaseGroupMBID != "" && !p.knownMissing(releaseGroupMBID) {
|
||||||
url := CoverArtGroupURL(releaseGroupMBID)
|
url := CoverArtGroupURL(releaseGroupMBID)
|
||||||
data, cacheable, err := p.fetch(url)
|
data, cacheable, err := p.fetch(url)
|
||||||
|
|
||||||
@@ -194,7 +227,7 @@ func (p *CoverArtProxy) GetCandidateThumbnail(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Network fetch on release (fallback).
|
// Network fetch on release (fallback).
|
||||||
if releaseMBID != "" {
|
if releaseMBID != "" && !p.knownMissing(releaseMBID) {
|
||||||
url := CoverArtURL(releaseMBID)
|
url := CoverArtURL(releaseMBID)
|
||||||
data, cacheable, err := p.fetch(url)
|
data, cacheable, err := p.fetch(url)
|
||||||
|
|
||||||
@@ -285,6 +318,17 @@ func (p *CoverArtProxy) cachePath(mbid string) string {
|
|||||||
return filepath.Join(p.cacheDir, mbid+".jpg")
|
return filepath.Join(p.cacheDir, mbid+".jpg")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// knownMissing reports whether a previous fetch was told the archive
|
||||||
|
// has no art for this MBID — the empty file `writeCache(mbid, nil)`
|
||||||
|
// leaves behind. It is deliberately separate from `readCache`, which
|
||||||
|
// answers "what are the bytes" and cannot express the difference
|
||||||
|
// between no answer and an answer of none.
|
||||||
|
func (p *CoverArtProxy) knownMissing(mbid string) bool {
|
||||||
|
info, err := os.Stat(p.cachePath(mbid))
|
||||||
|
|
||||||
|
return err == nil && info.Size() == 0
|
||||||
|
}
|
||||||
|
|
||||||
func (p *CoverArtProxy) readCache(mbid string) string {
|
func (p *CoverArtProxy) readCache(mbid string) string {
|
||||||
path := p.cachePath(mbid)
|
path := p.cachePath(mbid)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Reading multi-artist credits back out of the catalog.
|
||||||
|
//
|
||||||
|
// The tables are filled centrally (backend/explore/dumpcredits.go, and
|
||||||
|
// the artifact import) and hold only credits naming more than one
|
||||||
|
// artist: an entity with no rows here is credited to one artist, which
|
||||||
|
// explore_index's own artist_name and artist_mbid already describe.
|
||||||
|
// Absence is the common case and means "nothing to decompose", never
|
||||||
|
// "unknown".
|
||||||
|
//
|
||||||
|
// The lookup is keyed on the *recording* MBID, which both sides of the
|
||||||
|
// app already have -- a catalog row carries it and so does a local
|
||||||
|
// file (library.Track.RecordingMBID) -- so one query serves the Explore
|
||||||
|
// pages and the library's own lists without either needing to know
|
||||||
|
// where the other gets its rows.
|
||||||
|
|
||||||
|
// CreditPart is one credited artist within a credit, in credit order.
|
||||||
|
//
|
||||||
|
// CreditedName is the name *as credited*, which is not the artist's own
|
||||||
|
// name: MusicBrainz credits "Snoop Dogg" on a track by the artist
|
||||||
|
// called "Snoop Doggy Dogg". Display uses it; navigation uses
|
||||||
|
// ArtistMBID. JoinPhrase is the literal connector that follows this
|
||||||
|
// part, so a credit renders by concatenation and never by searching a
|
||||||
|
// name inside a credit string.
|
||||||
|
type CreditPart struct {
|
||||||
|
Position int `json:"position"`
|
||||||
|
ArtistMBID string `json:"artistMbid"`
|
||||||
|
CreditedName string `json:"creditedName"`
|
||||||
|
JoinPhrase string `json:"joinPhrase"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// creditLookupBatch bounds how many MBIDs go into one IN clause. A
|
||||||
|
// tracklist is the caller here, so the realistic ceiling is a few
|
||||||
|
// hundred; the bound exists so a 50,000-row selection cannot build a
|
||||||
|
// statement SQLite refuses to parse.
|
||||||
|
const creditLookupBatch = 500
|
||||||
|
|
||||||
|
// GetCredits returns the decomposition of every multi-artist credit
|
||||||
|
// among the given entity MBIDs, keyed by MBID.
|
||||||
|
//
|
||||||
|
// MBIDs with a single-artist credit are simply absent from the result,
|
||||||
|
// which is what the caller wants: it renders its existing single link
|
||||||
|
// for those, and that is the same answer it would have rendered anyway.
|
||||||
|
func (si *SearchIndex) GetCredits(mbids []string) (map[string][]CreditPart, error) {
|
||||||
|
out := make(map[string][]CreditPart)
|
||||||
|
|
||||||
|
for start := 0; start < len(mbids); start += creditLookupBatch {
|
||||||
|
end := min(start+creditLookupBatch, len(mbids))
|
||||||
|
|
||||||
|
if err := si.appendCredits(mbids[start:end], out); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// appendCredits runs one batch into the accumulating result.
|
||||||
|
func (si *SearchIndex) appendCredits(
|
||||||
|
mbids []string, out map[string][]CreditPart,
|
||||||
|
) error {
|
||||||
|
args := make([]any, 0, len(mbids))
|
||||||
|
holders := make([]string, 0, len(mbids))
|
||||||
|
|
||||||
|
for _, mbid := range mbids {
|
||||||
|
if mbid == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
args = append(args, dbMBID(mbid))
|
||||||
|
holders = append(holders, "?")
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(args) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ordered by position because that ordering *is* the credit's
|
||||||
|
// meaning; the caller concatenates in the order it receives.
|
||||||
|
rows, err := si.db.QueryContext(
|
||||||
|
`SELECT r.mbid, p.position, p.artist_mbid, p.credited_name, p.join_phrase
|
||||||
|
FROM artist_credit_ref r
|
||||||
|
JOIN artist_credit_part p ON p.credit_id = r.credit_id
|
||||||
|
WHERE r.mbid IN (`+strings.Join(holders, ",")+`)
|
||||||
|
ORDER BY r.mbid, p.position`,
|
||||||
|
args...,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("read artist credits: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var (
|
||||||
|
entity dbMBID
|
||||||
|
artist dbMBID
|
||||||
|
part CreditPart
|
||||||
|
)
|
||||||
|
|
||||||
|
if err := rows.Scan(
|
||||||
|
&entity, &part.Position, &artist, &part.CreditedName, &part.JoinPhrase,
|
||||||
|
); err != nil {
|
||||||
|
return fmt.Errorf("scan artist credit: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
part.ArtistMBID = string(artist)
|
||||||
|
out[string(entity)] = append(out[string(entity)], part)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return fmt.Errorf("read artist credits: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetCredits is the bound form: the frontend asks for a tracklist's
|
||||||
|
// worth of MBIDs at once rather than one per row.
|
||||||
|
//
|
||||||
|
// Batched for the reason every other per-row backend question here is:
|
||||||
|
// asking on hover or on render turns a list into N IPC round trips, and
|
||||||
|
// this one is asked about every row of every list in the app.
|
||||||
|
func (e *Service) GetCredits(mbids []string) (map[string][]CreditPart, error) {
|
||||||
|
return e.index.GetCredits(mbids)
|
||||||
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"yellowjacket/backend/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// seedCredit writes one multi-artist credit and points an entity at it,
|
||||||
|
// the way the dump import and the artifact import both do.
|
||||||
|
func seedCredit(t *testing.T, db *database.DB, entity string, id int, parts []CreditPart) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
pack := func(mbid string) []byte {
|
||||||
|
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
|
||||||
|
if err != nil || len(raw) != 16 {
|
||||||
|
t.Fatalf("bad fixture mbid %q: %v", mbid, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.ExecContext(
|
||||||
|
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
|
||||||
|
pack(entity), id,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("seed ref: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, p := range parts {
|
||||||
|
if _, err := db.ExecContext(
|
||||||
|
`INSERT INTO artist_credit_part
|
||||||
|
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
VALUES (?, ?, ?, ?, ?)`,
|
||||||
|
id, p.Position, pack(p.ArtistMBID), p.CreditedName, p.JoinPhrase,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("seed part: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetCreditsDecomposes: the parts come back in position order and
|
||||||
|
// concatenate to the credit they describe.
|
||||||
|
func TestGetCreditsDecomposes(t *testing.T) {
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
si := NewSearchIndex(db, nil, nil, testLogger())
|
||||||
|
|
||||||
|
rec := testMBID("rec-1")
|
||||||
|
a, b := testMBID("artist-a"), testMBID("artist-b")
|
||||||
|
|
||||||
|
seedCredit(t, db, rec, 7, []CreditPart{
|
||||||
|
{Position: 0, ArtistMBID: a, CreditedName: "2Pac", JoinPhrase: " feat. "},
|
||||||
|
{Position: 1, ArtistMBID: b, CreditedName: "Snoop Dogg"},
|
||||||
|
})
|
||||||
|
|
||||||
|
got, err := si.GetCredits([]string{rec})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetCredits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
parts := got[rec]
|
||||||
|
if len(parts) != 2 {
|
||||||
|
t.Fatalf("parts = %d, want 2", len(parts))
|
||||||
|
}
|
||||||
|
|
||||||
|
var rendered strings.Builder
|
||||||
|
for _, p := range parts {
|
||||||
|
rendered.WriteString(p.CreditedName)
|
||||||
|
rendered.WriteString(p.JoinPhrase)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rendered.String() != "2Pac feat. Snoop Dogg" {
|
||||||
|
t.Errorf("rendered = %q, want %q", rendered.String(), "2Pac feat. Snoop Dogg")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dashed on the way out: a blob reaching the frontend is sixteen
|
||||||
|
// bytes of mojibake, and nothing above mbid.go speaks that.
|
||||||
|
if parts[0].ArtistMBID != a {
|
||||||
|
t.Errorf("artist mbid = %q, want %q", parts[0].ArtistMBID, a)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetCreditsOmitsSingleArtist: absence is the common case and means
|
||||||
|
// "nothing to decompose", so the caller renders its existing one link.
|
||||||
|
func TestGetCreditsOmitsSingleArtist(t *testing.T) {
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
si := NewSearchIndex(db, nil, nil, testLogger())
|
||||||
|
|
||||||
|
got, err := si.GetCredits([]string{testMBID("untagged"), ""})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetCredits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(got) != 0 {
|
||||||
|
t.Errorf("got %d credits, want none", len(got))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetCreditsBatches: the lookup is asked about whole tracklists, so
|
||||||
|
// it must not build one statement per row or one SQLite refuses to
|
||||||
|
// parse.
|
||||||
|
func TestGetCreditsBatches(t *testing.T) {
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
si := NewSearchIndex(db, nil, nil, testLogger())
|
||||||
|
|
||||||
|
mbids := make([]string, 0, creditLookupBatch*2+7)
|
||||||
|
for i := range creditLookupBatch*2 + 7 {
|
||||||
|
mbids = append(mbids, testMBID(fmt.Sprintf("batch-%d", i)))
|
||||||
|
}
|
||||||
|
|
||||||
|
// One real credit somewhere past the first batch boundary.
|
||||||
|
seedCredit(t, db, mbids[creditLookupBatch+3], 9, []CreditPart{
|
||||||
|
{Position: 0, ArtistMBID: testMBID("a"), CreditedName: "A", JoinPhrase: " & "},
|
||||||
|
{Position: 1, ArtistMBID: testMBID("b"), CreditedName: "B"},
|
||||||
|
})
|
||||||
|
|
||||||
|
got, err := si.GetCredits(mbids)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetCredits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(got[mbids[creditLookupBatch+3]]) != 2 {
|
||||||
|
t.Errorf("a credit past the first batch boundary was not returned")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,637 @@
|
|||||||
|
//go:build indexbuild
|
||||||
|
|
||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"bufio"
|
||||||
|
"compress/bzip2"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"path"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Multi-artist credits, from the core MusicBrainz dump.
|
||||||
|
//
|
||||||
|
// A credit is ordered parts and the credit *string* is derived from
|
||||||
|
// them; MusicBrainz's own artist_credit.name is a cached render. What
|
||||||
|
// this pass extracts is the decomposition: for each catalog recording
|
||||||
|
// and release group whose credit names more than one artist, the
|
||||||
|
// credited artists in order, each with the name *as credited* and the
|
||||||
|
// join phrase that follows it. See artist_credit_part.sql for why that
|
||||||
|
// is stored rather than derived, and why nothing may reconstruct a
|
||||||
|
// credit by searching a name inside a credit string.
|
||||||
|
//
|
||||||
|
// It is a separate dump from everything else here, and it has to be.
|
||||||
|
// The canonical dump this importer already streams gives artist_mbids
|
||||||
|
// (an ordered list) and artist_credit_name (the *rendered* string) --
|
||||||
|
// no join phrases, and no per-artist as-credited names. Splitting the
|
||||||
|
// rendered string using canonical artist names fails on exactly the
|
||||||
|
// credits that matter: measured on a real library, 21% of multi-artist
|
||||||
|
// credits name an artist differently from the artist's own name
|
||||||
|
// ("Snoop Dogg" credited on a track by "Snoop Doggy Dogg"), so the
|
||||||
|
// substring is simply not there. The JSON dumps were checked too and
|
||||||
|
// cover 153,691 recordings of ~35M, with zero overlap against a real
|
||||||
|
// library. This dump is the only source.
|
||||||
|
//
|
||||||
|
// Cost, measured on the 20260815 export: 7.1 GB compressed, decompressed
|
||||||
|
// by pure-Go compress/bzip2 at ~26 MB/s uncompressed (~13.7 min for the
|
||||||
|
// whole file, single-threaded). cmd/indexbuild is built CGO_ENABLED=0,
|
||||||
|
// so the stdlib decompressor is what there is -- and it is fine, because
|
||||||
|
// the 2 MB/s origin throttle dominates, as it does for every other dump
|
||||||
|
// here.
|
||||||
|
|
||||||
|
const (
|
||||||
|
// defaultMBDumpBaseURL is the core MusicBrainz export. Only
|
||||||
|
// mbdump.tar.bz2 is fetched; the other tarballs there hold data this
|
||||||
|
// app has no use for.
|
||||||
|
defaultMBDumpBaseURL = "https://data.metabrainz.org/pub/musicbrainz/data/fullexport/"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
mbdumpDirRe = regexp.MustCompile(`^\d{8}-\d+$`)
|
||||||
|
mbdumpFileRe = regexp.MustCompile(`^mbdump\.tar\.bz2$`)
|
||||||
|
|
||||||
|
// ErrDumpShape is returned when a dump member does not have the
|
||||||
|
// columns this code was written against. It is deliberately fatal:
|
||||||
|
// reading the wrong column silently produces a catalog whose credits
|
||||||
|
// are subtly wrong, which is far worse than a failed build.
|
||||||
|
ErrDumpShape = errors.New("musicbrainz dump member has an unexpected shape")
|
||||||
|
)
|
||||||
|
|
||||||
|
// Column positions in the Postgres COPY output, verified against the
|
||||||
|
// 20260815 export. There is no header row to read them from, so they
|
||||||
|
// are asserted instead -- see checkShape.
|
||||||
|
const (
|
||||||
|
artistColID = 0
|
||||||
|
artistColGID = 1
|
||||||
|
artistColMin = 2
|
||||||
|
|
||||||
|
creditColID = 0
|
||||||
|
creditColArtistCount = 2
|
||||||
|
creditColMin = 3
|
||||||
|
|
||||||
|
partColCredit = 0
|
||||||
|
partColPosition = 1
|
||||||
|
partColArtist = 2
|
||||||
|
partColName = 3
|
||||||
|
partColJoin = 4
|
||||||
|
partColMin = 5
|
||||||
|
|
||||||
|
// recording and release_group share a layout in the columns this
|
||||||
|
// pass reads: id, gid, name, artist_credit, ...
|
||||||
|
entityColGID = 1
|
||||||
|
entityColCredit = 3
|
||||||
|
entityColMin = 4
|
||||||
|
)
|
||||||
|
|
||||||
|
// creditPart is one credited artist within a credit.
|
||||||
|
type creditPart struct {
|
||||||
|
position int
|
||||||
|
artistID int32
|
||||||
|
name string
|
||||||
|
join string
|
||||||
|
}
|
||||||
|
|
||||||
|
// creditScan is what one pass over the dump collects.
|
||||||
|
type creditScan struct {
|
||||||
|
// artistGIDs maps an artist row id to its MBID. artist_credit_name
|
||||||
|
// references artists by row id, and the tar orders `artist` before
|
||||||
|
// it, so this is complete by the time it is read.
|
||||||
|
artistGIDs map[int32]uuid16
|
||||||
|
|
||||||
|
// multiCredits are the credit ids naming more than one artist, from
|
||||||
|
// artist_credit.artist_count. Taking the count from the dump rather
|
||||||
|
// than counting parts means a credit can be rejected before its
|
||||||
|
// parts are stored.
|
||||||
|
multiCredits map[int32]struct{}
|
||||||
|
|
||||||
|
// parts are the decompositions of multiCredits, keyed by credit id.
|
||||||
|
parts map[int32][]creditPart
|
||||||
|
|
||||||
|
// refs maps a kept catalog entity to its credit. Only entities in
|
||||||
|
// explore_index and only multi-artist credits: everything else is
|
||||||
|
// already described by explore_index's own artist_name/artist_mbid.
|
||||||
|
refs map[uuid16]int32
|
||||||
|
|
||||||
|
// used are the credits some ref actually points at, which is a small
|
||||||
|
// fraction of multiCredits -- the catalog keeps ~1.8M entities of
|
||||||
|
// MusicBrainz's tens of millions.
|
||||||
|
used map[int32]struct{}
|
||||||
|
|
||||||
|
skippedUnknownArtist int
|
||||||
|
}
|
||||||
|
|
||||||
|
// creditsImportDoneKey marks in explore_index_meta that the credit pass
|
||||||
|
// has run against the current catalog.
|
||||||
|
//
|
||||||
|
// It is its own marker rather than part of the import's stage state for
|
||||||
|
// a resume reason: the credit pass runs *after* the catalog is
|
||||||
|
// assembled, and a failure in it must not send the next run back
|
||||||
|
// through the ~205 GB it just finished. Marking separately means a
|
||||||
|
// retry retries only this.
|
||||||
|
const creditsImportDoneKey = "credits_import_done"
|
||||||
|
|
||||||
|
// ensureArtistCredits runs the credit pass unless it has already run
|
||||||
|
// against this catalog, reporting whether it newly populated them.
|
||||||
|
//
|
||||||
|
// Called from both of run's paths -- the full import and the resume
|
||||||
|
// that finds the rows already assembled -- and from the maintenance
|
||||||
|
// entry point below, since a catalog built before credits existed is
|
||||||
|
// otherwise never offered a chance to gain them: the index job picks
|
||||||
|
// its mode from the index's own state, and a complete import means
|
||||||
|
// "refresh", which never enters run() at all.
|
||||||
|
//
|
||||||
|
// The return value is what tells the job there is something new worth
|
||||||
|
// publishing. A refresh otherwise reports "changed" only when the
|
||||||
|
// listens series advanced, so credits would sit in the CI database and
|
||||||
|
// never reach an artifact.
|
||||||
|
func (imp *dumpImporter) ensureArtistCredits(ctx context.Context) bool {
|
||||||
|
if imp.si.hasMeta(creditsImportDoneKey) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
url, err := discoverDumpFile(
|
||||||
|
ctx, imp.httpClient, imp.mbdumpBaseURL, mbdumpDirRe, mbdumpFileRe,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
imp.logger.Warn("credit import: could not find the dump", "error", err)
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := imp.importArtistCredits(ctx, url); err != nil {
|
||||||
|
// A catalog without credits is the catalog this app shipped
|
||||||
|
// before them: every credit falls back to its single artist.
|
||||||
|
// That is worth far less than failing an import that otherwise
|
||||||
|
// succeeded.
|
||||||
|
imp.logger.Warn("credit import: failed", "error", err)
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
imp.si.setMeta(creditsImportDoneKey, "1")
|
||||||
|
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureArtistCredits tops up the credit tables outside a full import.
|
||||||
|
//
|
||||||
|
// It exists because the index job's modes are decided from the index's
|
||||||
|
// own state: a cache holding a completed import chooses `refresh`,
|
||||||
|
// which folds in incremental listens and never enters the dump
|
||||||
|
// importer. Without this, a catalog built before the credit pass
|
||||||
|
// existed could only gain credits from a `rebuild` -- and a rebuild
|
||||||
|
// re-downloads ~205 GB to reproduce rows it already has, to add
|
||||||
|
// something that costs 7 GB on its own.
|
||||||
|
//
|
||||||
|
// Reports whether credits were newly populated, so the caller knows
|
||||||
|
// there is a new artifact worth publishing.
|
||||||
|
func (e *Service) EnsureArtistCredits(ctx context.Context) bool {
|
||||||
|
imp, err := newDumpImporter(e.index, e.lb)
|
||||||
|
if err != nil {
|
||||||
|
e.index.logger.Warn("credit import: could not start", "error", err)
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
return imp.ensureArtistCredits(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// importArtistCredits streams the core MusicBrainz dump and fills
|
||||||
|
// artist_credit_part and artist_credit_ref for the entities the catalog
|
||||||
|
// kept.
|
||||||
|
//
|
||||||
|
// It runs after assembleIndex because it asks explore_index which
|
||||||
|
// entities those are: the popularity filter decides what is worth
|
||||||
|
// carrying credits for, and asking the table rather than the kept sets
|
||||||
|
// means this stays correct if that filter changes.
|
||||||
|
func (imp *dumpImporter) importArtistCredits(ctx context.Context, url string) error {
|
||||||
|
kept, err := imp.keptEntityMBIDs(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(kept) == 0 {
|
||||||
|
imp.logger.Warn("credit import: no catalog entities, skipping")
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
imp.logger.Info("credit import: starting", "url", url, "entities", len(kept))
|
||||||
|
imp.logJob("Streaming MusicBrainz dump for artist credits")
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditDump(ctx, url, kept)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
imp.logger.Info("credit import: scanned",
|
||||||
|
"multiArtistCredits", len(scan.multiCredits),
|
||||||
|
"entitiesWithMultiArtistCredit", len(scan.refs),
|
||||||
|
"creditsUsed", len(scan.used),
|
||||||
|
)
|
||||||
|
|
||||||
|
return imp.writeCredits(ctx, scan)
|
||||||
|
}
|
||||||
|
|
||||||
|
// keptEntityMBIDs is every recording and release group in the catalog.
|
||||||
|
// Artists are excluded: an artist is not credited to a credit.
|
||||||
|
func (imp *dumpImporter) keptEntityMBIDs(ctx context.Context) (map[uuid16]struct{}, error) {
|
||||||
|
rows, err := imp.si.db.QueryContextWith(ctx,
|
||||||
|
`SELECT mbid FROM explore_index
|
||||||
|
WHERE entity_type IN (2 /* release_group */, 3 /* recording */)`,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
out := make(map[uuid16]struct{})
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var raw []byte
|
||||||
|
|
||||||
|
if err := rows.Scan(&raw); err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: scan mbid: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(raw) != len(uuid16{}) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var id uuid16
|
||||||
|
|
||||||
|
copy(id[:], raw)
|
||||||
|
|
||||||
|
out[id] = struct{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCreditDump makes one sequential pass over mbdump.tar.bz2.
|
||||||
|
//
|
||||||
|
// The tar's members are alphabetical, which is what makes a single pass
|
||||||
|
// possible without buffering the big ones: `artist` and
|
||||||
|
// `artist_credit_name` both arrive before `recording` and
|
||||||
|
// `release_group`, so by the time an entity names a credit, that
|
||||||
|
// credit's parts and their artists' MBIDs are already known and the
|
||||||
|
// entity can be resolved and dropped. 35M recording rows are never
|
||||||
|
// held.
|
||||||
|
//
|
||||||
|
// The order is not depended on blindly: an entity naming a credit that
|
||||||
|
// has not been seen is counted and reported rather than silently
|
||||||
|
// producing an empty catalog, which is what a reordered export would
|
||||||
|
// otherwise look like.
|
||||||
|
func (imp *dumpImporter) scanCreditDump(
|
||||||
|
ctx context.Context, url string, kept map[uuid16]struct{},
|
||||||
|
) (*creditScan, error) {
|
||||||
|
stream := imp.openDumpStream(ctx, url, 0)
|
||||||
|
|
||||||
|
defer func() { _ = stream.Close() }()
|
||||||
|
|
||||||
|
return imp.scanCreditTar(
|
||||||
|
ctx,
|
||||||
|
tar.NewReader(bzip2.NewReader(bufio.NewReaderSize(stream, 1<<20))),
|
||||||
|
kept,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCreditTar is the parse, separated from the fetch so it can be
|
||||||
|
// driven by a tar built in a test. compress/bzip2 is decompress-only,
|
||||||
|
// so a test cannot produce the real container.
|
||||||
|
func (imp *dumpImporter) scanCreditTar(
|
||||||
|
ctx context.Context, tr *tar.Reader, kept map[uuid16]struct{},
|
||||||
|
) (*creditScan, error) {
|
||||||
|
scan := &creditScan{
|
||||||
|
artistGIDs: make(map[int32]uuid16),
|
||||||
|
multiCredits: make(map[int32]struct{}),
|
||||||
|
parts: make(map[int32][]creditPart),
|
||||||
|
refs: make(map[uuid16]int32),
|
||||||
|
used: make(map[int32]struct{}),
|
||||||
|
}
|
||||||
|
|
||||||
|
for {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
hdr, err := tr.Next()
|
||||||
|
if errors.Is(err, io.EOF) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: tar: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if hdr.Typeflag != tar.TypeReg {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
done, err := imp.scanCreditMember(ctx, hdr.Name, tr, kept, scan)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if done {
|
||||||
|
// Everything this pass needs has been read; the rest of the
|
||||||
|
// tarball is other entities' data and decompressing it would
|
||||||
|
// cost minutes for nothing.
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if scan.skippedUnknownArtist > 0 {
|
||||||
|
imp.logger.Warn("credit import: credits dropped for unknown artists",
|
||||||
|
"count", scan.skippedUnknownArtist,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return scan, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCreditMember dispatches one tar member, reporting whether the
|
||||||
|
// pass has everything it needs.
|
||||||
|
func (imp *dumpImporter) scanCreditMember(
|
||||||
|
ctx context.Context, name string, r io.Reader,
|
||||||
|
kept map[uuid16]struct{}, scan *creditScan,
|
||||||
|
) (bool, error) {
|
||||||
|
switch path.Base(name) {
|
||||||
|
case "artist":
|
||||||
|
return false, imp.scanArtists(ctx, r, scan)
|
||||||
|
case "artist_credit":
|
||||||
|
return false, imp.scanCredits(ctx, r, scan)
|
||||||
|
case "artist_credit_name":
|
||||||
|
return false, imp.scanCreditParts(ctx, r, scan)
|
||||||
|
case "recording", "release_group":
|
||||||
|
if err := imp.scanCreditedEntities(ctx, r, kept, scan); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// release_group sorts after recording, so the pass is complete
|
||||||
|
// once it has been read.
|
||||||
|
return path.Base(name) == "release_group", nil
|
||||||
|
default:
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanArtists records every artist's MBID by row id.
|
||||||
|
func (imp *dumpImporter) scanArtists(
|
||||||
|
ctx context.Context, r io.Reader, scan *creditScan,
|
||||||
|
) error {
|
||||||
|
return scanTSV(ctx, r, artistColMin, "artist", func(fields []string) error {
|
||||||
|
id, ok := parseInt32(fields[artistColID])
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var gid uuid16
|
||||||
|
|
||||||
|
if !parseUUID(fields[artistColGID], gid[:]) {
|
||||||
|
return fmt.Errorf("%w: artist.gid is not a UUID: %q",
|
||||||
|
ErrDumpShape, truncate(fields[artistColGID]))
|
||||||
|
}
|
||||||
|
|
||||||
|
scan.artistGIDs[id] = gid
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCredits records which credits name more than one artist.
|
||||||
|
func (imp *dumpImporter) scanCredits(
|
||||||
|
ctx context.Context, r io.Reader, scan *creditScan,
|
||||||
|
) error {
|
||||||
|
return scanTSV(ctx, r, creditColMin, "artist_credit", func(fields []string) error {
|
||||||
|
id, ok := parseInt32(fields[creditColID])
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
count, ok := parseInt32(fields[creditColArtistCount])
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("%w: artist_credit.artist_count is not a number: %q",
|
||||||
|
ErrDumpShape, truncate(fields[creditColArtistCount]))
|
||||||
|
}
|
||||||
|
|
||||||
|
if count > 1 {
|
||||||
|
scan.multiCredits[id] = struct{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCreditParts records the decomposition of every multi-artist
|
||||||
|
// credit.
|
||||||
|
func (imp *dumpImporter) scanCreditParts(
|
||||||
|
ctx context.Context, r io.Reader, scan *creditScan,
|
||||||
|
) error {
|
||||||
|
return scanTSV(ctx, r, partColMin, "artist_credit_name", func(fields []string) error {
|
||||||
|
credit, ok := parseInt32(fields[partColCredit])
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, multi := scan.multiCredits[credit]; !multi {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
position, ok := parseInt32(fields[partColPosition])
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
artist, ok := parseInt32(fields[partColArtist])
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
scan.parts[credit] = append(scan.parts[credit], creditPart{
|
||||||
|
position: int(position),
|
||||||
|
artistID: artist,
|
||||||
|
name: fields[partColName],
|
||||||
|
join: fields[partColJoin],
|
||||||
|
})
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanCreditedEntities resolves recordings and release groups against
|
||||||
|
// the catalog, keeping only those the catalog holds and whose credit
|
||||||
|
// names more than one artist.
|
||||||
|
func (imp *dumpImporter) scanCreditedEntities(
|
||||||
|
ctx context.Context, r io.Reader, kept map[uuid16]struct{}, scan *creditScan,
|
||||||
|
) error {
|
||||||
|
return scanTSV(ctx, r, entityColMin, "recording/release_group",
|
||||||
|
func(fields []string) error {
|
||||||
|
var gid uuid16
|
||||||
|
|
||||||
|
if !parseUUID(fields[entityColGID], gid[:]) {
|
||||||
|
return fmt.Errorf("%w: entity gid is not a UUID: %q",
|
||||||
|
ErrDumpShape, truncate(fields[entityColGID]))
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, want := kept[gid]; !want {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
credit, ok := parseInt32(fields[entityColCredit])
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("%w: entity artist_credit is not a number: %q",
|
||||||
|
ErrDumpShape, truncate(fields[entityColCredit]))
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, multi := scan.multiCredits[credit]; !multi {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
scan.refs[gid] = credit
|
||||||
|
scan.used[credit] = struct{}{}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// scanTSV reads Postgres COPY output a line at a time, unescaping each
|
||||||
|
// field and handing the row to fn.
|
||||||
|
//
|
||||||
|
// The shape is asserted on the first row rather than trusted: this dump
|
||||||
|
// has no header, so a column that moved would otherwise be read as a
|
||||||
|
// neighbouring one and produce a catalog that is quietly wrong.
|
||||||
|
func scanTSV(
|
||||||
|
ctx context.Context, r io.Reader, minCols int, member string,
|
||||||
|
fn func(fields []string) error,
|
||||||
|
) error {
|
||||||
|
sc := bufio.NewScanner(r)
|
||||||
|
sc.Buffer(make([]byte, 0, 1<<20), 1<<24)
|
||||||
|
|
||||||
|
checked := false
|
||||||
|
rows := 0
|
||||||
|
|
||||||
|
for sc.Scan() {
|
||||||
|
rows++
|
||||||
|
|
||||||
|
if rows%(1<<20) == 0 {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
line := sc.Text()
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
fields := strings.Split(line, "\t")
|
||||||
|
if len(fields) < minCols {
|
||||||
|
if !checked {
|
||||||
|
return fmt.Errorf("%w: %s has %d columns, need at least %d",
|
||||||
|
ErrDumpShape, member, len(fields), minCols)
|
||||||
|
}
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
checked = true
|
||||||
|
|
||||||
|
for i := range fields {
|
||||||
|
fields[i] = unescapeCopy(fields[i])
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := fn(fields); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := sc.Err(); err != nil {
|
||||||
|
return fmt.Errorf("credit import: read %s: %w", member, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// unescapeCopy undoes Postgres COPY's text escaping. A NULL (\N) is
|
||||||
|
// returned as an empty string: every field this pass reads is either a
|
||||||
|
// number it will reject or a name whose absence means the same as
|
||||||
|
// empty.
|
||||||
|
func unescapeCopy(s string) string {
|
||||||
|
if s == `\N` {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.ContainsRune(s, '\\') {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
var b strings.Builder
|
||||||
|
|
||||||
|
b.Grow(len(s))
|
||||||
|
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
if s[i] != '\\' || i+1 >= len(s) {
|
||||||
|
b.WriteByte(s[i])
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
i++
|
||||||
|
|
||||||
|
switch s[i] {
|
||||||
|
case 'n':
|
||||||
|
b.WriteByte('\n')
|
||||||
|
case 't':
|
||||||
|
b.WriteByte('\t')
|
||||||
|
case 'r':
|
||||||
|
b.WriteByte('\r')
|
||||||
|
case 'b':
|
||||||
|
b.WriteByte('\b')
|
||||||
|
case 'f':
|
||||||
|
b.WriteByte('\f')
|
||||||
|
case 'v':
|
||||||
|
b.WriteByte('\v')
|
||||||
|
case '\\':
|
||||||
|
b.WriteByte('\\')
|
||||||
|
default:
|
||||||
|
b.WriteByte('\\')
|
||||||
|
b.WriteByte(s[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseInt32(s string) (int32, bool) {
|
||||||
|
n, err := strconv.ParseInt(s, 10, 32)
|
||||||
|
if err != nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
|
||||||
|
return int32(n), true
|
||||||
|
}
|
||||||
|
|
||||||
|
// truncate bounds an error message built from dump data, which is
|
||||||
|
// attacker-free but can be long.
|
||||||
|
func truncate(s string) string {
|
||||||
|
const limit = 64
|
||||||
|
|
||||||
|
if len(s) <= limit {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
return s[:limit] + "..."
|
||||||
|
}
|
||||||
@@ -0,0 +1,406 @@
|
|||||||
|
//go:build indexbuild
|
||||||
|
|
||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"yellowjacket/backend/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// tarOf builds an uncompressed tar of the named members, in the order
|
||||||
|
// given. Order is the point of several of these tests: the real dump's
|
||||||
|
// members are alphabetical, which is what lets one pass resolve an
|
||||||
|
// entity's credit without buffering 35M recordings.
|
||||||
|
func tarOf(t *testing.T, members ...[2]string) *tar.Reader {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
|
||||||
|
tw := tar.NewWriter(&buf)
|
||||||
|
|
||||||
|
for _, m := range members {
|
||||||
|
body := []byte(m[1])
|
||||||
|
|
||||||
|
if err := tw.WriteHeader(&tar.Header{
|
||||||
|
Name: "mbdump/" + m[0],
|
||||||
|
Mode: 0o644,
|
||||||
|
Size: int64(len(body)),
|
||||||
|
Typeflag: tar.TypeReg,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("tar header: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := tw.Write(body); err != nil {
|
||||||
|
t.Fatalf("tar write: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tw.Close(); err != nil {
|
||||||
|
t.Fatalf("tar close: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return tar.NewReader(&buf)
|
||||||
|
}
|
||||||
|
|
||||||
|
func tsv(rows ...[]string) string {
|
||||||
|
var b strings.Builder
|
||||||
|
|
||||||
|
for _, r := range rows {
|
||||||
|
b.WriteString(strings.Join(r, "\t"))
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// mustMBID is testMBID in the packed form the catalog stores.
|
||||||
|
func mustMBID(label string) uuid16 {
|
||||||
|
var u uuid16
|
||||||
|
|
||||||
|
if !parseUUID(testMBID(label), u[:]) {
|
||||||
|
panic("testMBID did not produce a UUID for " + label)
|
||||||
|
}
|
||||||
|
|
||||||
|
return u
|
||||||
|
}
|
||||||
|
|
||||||
|
// The two artists of the worked example, and the entities they credit.
|
||||||
|
var (
|
||||||
|
creditRecMBID = mustMBID("recording-1")
|
||||||
|
creditRGMBID = mustMBID("release-group-1")
|
||||||
|
)
|
||||||
|
|
||||||
|
// sampleDump is the shape verified against the 20260815 export:
|
||||||
|
// artist(id, gid, ...), artist_credit(id, name, artist_count, ...),
|
||||||
|
// artist_credit_name(credit, position, artist, name, join_phrase),
|
||||||
|
// recording/release_group(id, gid, name, artist_credit, ...).
|
||||||
|
func sampleDump(t *testing.T) *tar.Reader {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
return tarOf(t,
|
||||||
|
[2]string{"artist", tsv(
|
||||||
|
[]string{"11", testMBID("artist-a"), "Snoop Doggy Dogg", "Snoop Doggy Dogg"},
|
||||||
|
[]string{"22", testMBID("artist-b"), "2Pac", "2Pac"},
|
||||||
|
)},
|
||||||
|
[2]string{"artist_credit", tsv(
|
||||||
|
[]string{"900", "2Pac feat. Snoop Dogg", "2", "1", "", "0", ""},
|
||||||
|
[]string{"901", "Solo Artist", "1", "1", "", "0", ""},
|
||||||
|
)},
|
||||||
|
[2]string{"artist_credit_name", tsv(
|
||||||
|
// Deliberately out of position order: the dump is not
|
||||||
|
// obliged to emit them sorted and the credit's meaning is
|
||||||
|
// the order, not the file's.
|
||||||
|
[]string{"900", "1", "11", "Snoop Dogg", ""},
|
||||||
|
[]string{"900", "0", "22", "2Pac", " feat. "},
|
||||||
|
[]string{"901", "0", "11", "Solo Artist", ""},
|
||||||
|
)},
|
||||||
|
[2]string{"recording", tsv(
|
||||||
|
[]string{"1", testMBID("recording-1"), "Some Song", "900", "180000"},
|
||||||
|
[]string{"2", testMBID("not-kept"), "Other", "900", "1"},
|
||||||
|
[]string{"3", testMBID("solo"), "Solo", "901", "1"},
|
||||||
|
)},
|
||||||
|
[2]string{"release_group", tsv(
|
||||||
|
[]string{"5", testMBID("release-group-1"), "Some Album", "900", "1"},
|
||||||
|
)},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func creditTestImporter(t *testing.T) *dumpImporter {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
|
||||||
|
return &dumpImporter{
|
||||||
|
si: NewSearchIndex(db, nil, nil, testLogger()),
|
||||||
|
logger: testLogger(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestScanCreditDumpDecomposes is the worked example end to end: the
|
||||||
|
// credit's parts come back in position order, with the *credited*
|
||||||
|
// names and the join phrase between them.
|
||||||
|
func TestScanCreditDumpDecomposes(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
kept := map[uuid16]struct{}{
|
||||||
|
creditRecMBID: {},
|
||||||
|
creditRGMBID: {},
|
||||||
|
}
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := len(scan.refs); got != 2 {
|
||||||
|
t.Fatalf("refs = %d, want 2 (the recording and the release group)", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
if scan.refs[creditRecMBID] != 900 {
|
||||||
|
t.Errorf("recording credit = %d, want 900", scan.refs[creditRecMBID])
|
||||||
|
}
|
||||||
|
|
||||||
|
parts := scan.parts[900]
|
||||||
|
if len(parts) != 2 {
|
||||||
|
t.Fatalf("parts = %d, want 2", len(parts))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sorting happens on write, so assert the pieces are all present
|
||||||
|
// and let the render test below check the order.
|
||||||
|
byPos := map[int]creditPart{}
|
||||||
|
for _, p := range parts {
|
||||||
|
byPos[p.position] = p
|
||||||
|
}
|
||||||
|
|
||||||
|
if byPos[0].name != "2Pac" || byPos[0].join != " feat. " {
|
||||||
|
t.Errorf("position 0 = %q/%q, want \"2Pac\"/\" feat. \"",
|
||||||
|
byPos[0].name, byPos[0].join)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The credited name, not the artist's own name: this is the whole
|
||||||
|
// reason credited_name is stored per row.
|
||||||
|
if byPos[1].name != "Snoop Dogg" {
|
||||||
|
t.Errorf("position 1 credited name = %q, want \"Snoop Dogg\"", byPos[1].name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSingleArtistCreditsAreNotStored: a one-artist credit is already
|
||||||
|
// described by explore_index's artist_name/artist_mbid, and storing it
|
||||||
|
// would roughly triple the table to say nothing new.
|
||||||
|
func TestSingleArtistCreditsAreNotStored(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
solo := mustMBID("solo")
|
||||||
|
kept := map[uuid16]struct{}{solo: {}}
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(scan.refs) != 0 {
|
||||||
|
t.Fatalf("a single-artist credit was referenced: %v", scan.refs)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, ok := scan.multiCredits[901]; ok {
|
||||||
|
t.Error("credit 901 has artist_count 1 and should not be multi")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOnlyKeptEntitiesAreReferenced: the catalog's popularity filter
|
||||||
|
// decides what is worth carrying credits for, and an entity outside it
|
||||||
|
// must not produce a row pointing at nothing.
|
||||||
|
func TestOnlyKeptEntitiesAreReferenced(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
kept := map[uuid16]struct{}{creditRecMBID: {}}
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, ok := scan.refs[mustMBID("not-kept")]; ok {
|
||||||
|
t.Error("an entity outside the catalog was referenced")
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(scan.used) != 1 {
|
||||||
|
t.Errorf("used credits = %d, want 1", len(scan.used))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestWriteCreditsRoundTrips checks what the frontend will actually
|
||||||
|
// read: parts in position order, dashed MBIDs out of the 16 raw bytes,
|
||||||
|
// and a rendered credit that reassembles to the tagged string.
|
||||||
|
func TestWriteCreditsRoundTrips(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
kept := map[uuid16]struct{}{creditRecMBID: {}, creditRGMBID: {}}
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := imp.writeCredits(context.Background(), scan); err != nil {
|
||||||
|
t.Fatalf("writeCredits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := imp.si.db.QueryContext(
|
||||||
|
`SELECT p.position, p.artist_mbid, p.credited_name, p.join_phrase
|
||||||
|
FROM artist_credit_ref r
|
||||||
|
JOIN artist_credit_part p ON p.credit_id = r.credit_id
|
||||||
|
WHERE r.mbid = ?
|
||||||
|
ORDER BY p.position`,
|
||||||
|
creditRecMBID[:],
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("query: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
|
||||||
|
var rendered strings.Builder
|
||||||
|
|
||||||
|
names := []string{}
|
||||||
|
|
||||||
|
for rows.Next() {
|
||||||
|
var (
|
||||||
|
pos int
|
||||||
|
mbid []byte
|
||||||
|
name string
|
||||||
|
join string
|
||||||
|
)
|
||||||
|
|
||||||
|
if err := rows.Scan(&pos, &mbid, &name, &join); err != nil {
|
||||||
|
t.Fatalf("scan row: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(mbid) != 16 {
|
||||||
|
t.Fatalf("artist_mbid is %d bytes, want 16", len(mbid))
|
||||||
|
}
|
||||||
|
|
||||||
|
names = append(names, name)
|
||||||
|
|
||||||
|
rendered.WriteString(name)
|
||||||
|
rendered.WriteString(join)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
t.Fatalf("rows: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Concatenation is the contract: names in order, join phrases
|
||||||
|
// between them, and no searching a name inside a credit string.
|
||||||
|
if got := rendered.String(); got != "2Pac feat. Snoop Dogg" {
|
||||||
|
t.Errorf("rendered credit = %q, want %q", got, "2Pac feat. Snoop Dogg")
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(names) != 2 || names[0] != "2Pac" {
|
||||||
|
t.Errorf("parts came back out of position order: %v", names)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCreditRefsNeverDangle: a ref whose parts were not stored renders
|
||||||
|
// as a credit with no artists at all, which is worse than the
|
||||||
|
// single-artist fallback it replaced.
|
||||||
|
func TestCreditRefsNeverDangle(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
kept := map[uuid16]struct{}{creditRecMBID: {}}
|
||||||
|
|
||||||
|
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// An artist the dump never named: the credit cannot be navigated to
|
||||||
|
// and must be dropped whole, taking its ref with it.
|
||||||
|
scan.artistGIDs = map[int32]uuid16{}
|
||||||
|
|
||||||
|
if err := imp.writeCredits(context.Background(), scan); err != nil {
|
||||||
|
t.Fatalf("writeCredits: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var refs, parts int
|
||||||
|
|
||||||
|
if err := imp.si.db.QueryRowWriter(
|
||||||
|
"SELECT COUNT(*) FROM artist_credit_ref",
|
||||||
|
).Scan(&refs); err != nil {
|
||||||
|
t.Fatalf("count refs: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := imp.si.db.QueryRowWriter(
|
||||||
|
"SELECT COUNT(*) FROM artist_credit_part",
|
||||||
|
).Scan(&parts); err != nil {
|
||||||
|
t.Fatalf("count parts: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if refs != 0 || parts != 0 {
|
||||||
|
t.Fatalf("refs=%d parts=%d, want 0/0 when the artists are unknown", refs, parts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCreditDumpShapeIsAsserted: the dump has no header row, so a
|
||||||
|
// column that moved would be read as its neighbour and produce a
|
||||||
|
// catalog that is quietly wrong. Loud is the requirement.
|
||||||
|
func TestCreditDumpShapeIsAsserted(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
short := tarOf(t, [2]string{"artist", tsv([]string{"11", "only-two-columns"})})
|
||||||
|
|
||||||
|
_, err := imp.scanCreditTar(context.Background(), short, map[uuid16]struct{}{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("a member with a non-UUID gid was accepted")
|
||||||
|
}
|
||||||
|
|
||||||
|
if !errors.Is(err, ErrDumpShape) {
|
||||||
|
t.Errorf("error = %v, want ErrDumpShape", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUnescapeCopy covers Postgres COPY's text escaping, which reaches
|
||||||
|
// artist names routinely -- a tab or backslash in a name would
|
||||||
|
// otherwise shift every field after it.
|
||||||
|
func TestUnescapeCopy(t *testing.T) {
|
||||||
|
tests := []struct{ in, want string }{
|
||||||
|
{`plain`, `plain`},
|
||||||
|
{`\N`, ``},
|
||||||
|
{`a\tb`, "a\tb"},
|
||||||
|
{`a\nb`, "a\nb"},
|
||||||
|
{`back\\slash`, `back\slash`},
|
||||||
|
{`AC\/DC`, `AC\/DC`},
|
||||||
|
{`trailing\`, `trailing\`},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
if got := unescapeCopy(tt.in); got != tt.want {
|
||||||
|
t.Errorf("unescapeCopy(%q) = %q, want %q", tt.in, got, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEnsureArtistCreditsIsIdempotent pins what the index job depends
|
||||||
|
// on to decide whether to publish.
|
||||||
|
//
|
||||||
|
// The pass runs on every mode, including the `refresh` that a complete
|
||||||
|
// catalog always chooses — so it must be free when there is nothing to
|
||||||
|
// do, and it must say so. A `true` here republishes the artifact; a
|
||||||
|
// `true` on every run would republish an identical one weekly, and a
|
||||||
|
// permanent `false` would mean a catalog that never gains credits at
|
||||||
|
// all.
|
||||||
|
func TestEnsureArtistCreditsIsIdempotent(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
|
||||||
|
// The marker is what "already done" means; with it set, the pass
|
||||||
|
// must not reach the network or report a change.
|
||||||
|
imp.si.setMeta(creditsImportDoneKey, "1")
|
||||||
|
|
||||||
|
if imp.ensureArtistCredits(context.Background()) {
|
||||||
|
t.Fatal("a second run reported new credits; the artifact would republish forever")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEnsureArtistCreditsReportsFailureAsNoChange: a dump that cannot be
|
||||||
|
// reached leaves the catalog exactly as it was, and must not claim
|
||||||
|
// otherwise — publishing on it would ship an artifact with no credits
|
||||||
|
// and mark the work done.
|
||||||
|
func TestEnsureArtistCreditsReportsFailureAsNoChange(t *testing.T) {
|
||||||
|
imp := creditTestImporter(t)
|
||||||
|
imp.httpClient = newDumpHTTPClient()
|
||||||
|
imp.mbdumpBaseURL = "http://127.0.0.1:1/nonexistent/"
|
||||||
|
|
||||||
|
if imp.ensureArtistCredits(context.Background()) {
|
||||||
|
t.Fatal("an unreachable dump reported new credits")
|
||||||
|
}
|
||||||
|
|
||||||
|
if imp.si.hasMeta(creditsImportDoneKey) {
|
||||||
|
t.Error("a failed pass marked itself done; it would never retry")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,164 @@
|
|||||||
|
//go:build indexbuild
|
||||||
|
|
||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
)
|
||||||
|
|
||||||
|
// writeCredits persists the scanned decompositions.
|
||||||
|
//
|
||||||
|
// Only credits some catalog entity actually points at are written: the
|
||||||
|
// dump has millions of multi-artist credits and the catalog keeps ~1.8M
|
||||||
|
// entities, so storing every credit would be most of a table nothing
|
||||||
|
// can reach.
|
||||||
|
//
|
||||||
|
// The two tables are written in one transaction, because a ref pointing
|
||||||
|
// at parts that are not there renders as a credit with no artists --
|
||||||
|
// worse than the single-artist fallback it replaced.
|
||||||
|
func (imp *dumpImporter) writeCredits(ctx context.Context, scan *creditScan) error {
|
||||||
|
tx, err := imp.si.db.BeginTx()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("credit import: begin: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = tx.Rollback() }()
|
||||||
|
|
||||||
|
// A rebuild replaces the previous pass wholesale. These are Cache
|
||||||
|
// tables derived entirely from the dump, so there is nothing to
|
||||||
|
// merge and a stale row is a wrong credit.
|
||||||
|
for _, table := range []string{"artist_credit_part", "artist_credit_ref"} {
|
||||||
|
if _, err := tx.ExecContext(ctx, "DELETE FROM "+table); err != nil {
|
||||||
|
return fmt.Errorf("credit import: clear %s: %w", table, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
written, err := imp.writeCreditParts(ctx, tx, scan)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
refs, err := imp.writeCreditRefs(ctx, tx, scan, written)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
return fmt.Errorf("credit import: commit: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
imp.logger.Info("credit import: complete",
|
||||||
|
"credits", len(written),
|
||||||
|
"refs", refs,
|
||||||
|
)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeCreditParts inserts the parts of every used credit and returns
|
||||||
|
// the set of credits that were actually stored.
|
||||||
|
//
|
||||||
|
// A credit is stored whole or not at all. If any of its artists has no
|
||||||
|
// MBID -- which should not happen, the dump being self-consistent, but
|
||||||
|
// would leave a part that cannot be navigated to -- the credit is
|
||||||
|
// dropped and the entity falls back to explore_index's single artist,
|
||||||
|
// which is a worse answer rather than a broken one.
|
||||||
|
func (imp *dumpImporter) writeCreditParts(
|
||||||
|
ctx context.Context, tx *sql.Tx, scan *creditScan,
|
||||||
|
) (map[int32]struct{}, error) {
|
||||||
|
stmt, err := tx.PrepareContext(ctx,
|
||||||
|
`INSERT INTO artist_credit_part
|
||||||
|
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
VALUES (?, ?, ?, ?, ?)`,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: prepare part insert: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = stmt.Close() }()
|
||||||
|
|
||||||
|
written := make(map[int32]struct{}, len(scan.used))
|
||||||
|
|
||||||
|
for credit := range scan.used {
|
||||||
|
parts := scan.parts[credit]
|
||||||
|
if len(parts) < 2 {
|
||||||
|
// artist_credit said more than one artist and
|
||||||
|
// artist_credit_name did not deliver them. Nothing to
|
||||||
|
// decompose, so leave the entity to its single artist.
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Position order is the credit's meaning, and the dump is not
|
||||||
|
// obliged to emit it sorted.
|
||||||
|
sort.Slice(parts, func(i, j int) bool {
|
||||||
|
return parts[i].position < parts[j].position
|
||||||
|
})
|
||||||
|
|
||||||
|
resolved := make([][]any, 0, len(parts))
|
||||||
|
ok := true
|
||||||
|
|
||||||
|
for _, part := range parts {
|
||||||
|
gid, found := scan.artistGIDs[part.artistID]
|
||||||
|
if !found {
|
||||||
|
scan.skippedUnknownArtist++
|
||||||
|
ok = false
|
||||||
|
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved = append(resolved, []any{
|
||||||
|
credit, part.position, gid[:], part.name, part.join,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, args := range resolved {
|
||||||
|
if _, err := stmt.ExecContext(ctx, args...); err != nil {
|
||||||
|
return nil, fmt.Errorf("credit import: insert part: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
written[credit] = struct{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return written, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeCreditRefs points each kept entity at its credit, skipping any
|
||||||
|
// whose credit was not stored so a ref never dangles.
|
||||||
|
func (imp *dumpImporter) writeCreditRefs(
|
||||||
|
ctx context.Context, tx *sql.Tx, scan *creditScan, written map[int32]struct{},
|
||||||
|
) (int, error) {
|
||||||
|
stmt, err := tx.PrepareContext(ctx,
|
||||||
|
"INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("credit import: prepare ref insert: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = stmt.Close() }()
|
||||||
|
|
||||||
|
count := 0
|
||||||
|
|
||||||
|
for mbid, credit := range scan.refs {
|
||||||
|
if _, stored := written[credit]; !stored {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
id := mbid
|
||||||
|
|
||||||
|
if _, err := stmt.ExecContext(ctx, id[:], credit); err != nil {
|
||||||
|
return 0, fmt.Errorf("credit import: insert ref: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
|
||||||
|
return count, nil
|
||||||
|
}
|
||||||
@@ -104,6 +104,7 @@ type dumpImporter struct {
|
|||||||
|
|
||||||
canonicalBaseURL string
|
canonicalBaseURL string
|
||||||
listensBaseURL string
|
listensBaseURL string
|
||||||
|
mbdumpBaseURL string
|
||||||
|
|
||||||
// Disk safety floors (fields so tests can relax them).
|
// Disk safety floors (fields so tests can relax them).
|
||||||
minStartFreeBytes uint64
|
minStartFreeBytes uint64
|
||||||
@@ -144,6 +145,7 @@ func newDumpImporter(si *SearchIndex, lb *ListenBrainzClient) (*dumpImporter, er
|
|||||||
stagingDir: stagingDir,
|
stagingDir: stagingDir,
|
||||||
canonicalBaseURL: defaultCanonicalBaseURL,
|
canonicalBaseURL: defaultCanonicalBaseURL,
|
||||||
listensBaseURL: defaultListensBaseURL,
|
listensBaseURL: defaultListensBaseURL,
|
||||||
|
mbdumpBaseURL: defaultMBDumpBaseURL,
|
||||||
minStartFreeBytes: dumpMinStartFreeBytes,
|
minStartFreeBytes: dumpMinStartFreeBytes,
|
||||||
abortFreeBytes: dumpAbortFreeBytes,
|
abortFreeBytes: dumpAbortFreeBytes,
|
||||||
}, nil
|
}, nil
|
||||||
@@ -171,6 +173,7 @@ func (imp *dumpImporter) run(ctx context.Context) error {
|
|||||||
// Fast path: rows already assembled, only patch passes remain.
|
// Fast path: rows already assembled, only patch passes remain.
|
||||||
if state.Stage == dumpStageAssembled {
|
if state.Stage == dumpStageAssembled {
|
||||||
imp.si.MarkReadyIfPopulated()
|
imp.si.MarkReadyIfPopulated()
|
||||||
|
imp.ensureArtistCredits(ctx)
|
||||||
imp.runPatchPasses(ctx)
|
imp.runPatchPasses(ctx)
|
||||||
|
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
@@ -305,6 +308,11 @@ func (imp *dumpImporter) run(ctx context.Context) error {
|
|||||||
imp.si.MarkReadyIfPopulated()
|
imp.si.MarkReadyIfPopulated()
|
||||||
imp.si.refreshStatusCounts()
|
imp.si.refreshStatusCounts()
|
||||||
|
|
||||||
|
// Multi-artist credits, from a different dump. After the catalog,
|
||||||
|
// because it asks explore_index which entities are worth carrying
|
||||||
|
// credits for.
|
||||||
|
imp.ensureArtistCredits(ctx)
|
||||||
|
|
||||||
// Stage 4: API patch passes (idempotent).
|
// Stage 4: API patch passes (idempotent).
|
||||||
imp.runPatchPasses(ctx)
|
imp.runPatchPasses(ctx)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Whether the catalog artifact may be downloaded on this connection
|
||||||
|
// (plan 016 B4).
|
||||||
|
//
|
||||||
|
// The artifact is ~0.6 GB. On a desktop that is a minute of someone
|
||||||
|
// else's bandwidth; on a phone it can be a month's allowance, and the
|
||||||
|
// app had no awareness of the difference at all.
|
||||||
|
//
|
||||||
|
// Three decisions shape this file.
|
||||||
|
//
|
||||||
|
// **The policy lives here and the platform call does not.** `explore` is
|
||||||
|
// imported by `cmd/indexbuild`, which is built with `CGO_ENABLED=0` in a
|
||||||
|
// plain Go container, so naming `application` here would break the one
|
||||||
|
// job that must not fail (see `TestIndexToolsDoNotImportWails`). What is
|
||||||
|
// injected is a closure; what is *tested* is the parsing and the
|
||||||
|
// decision, on every platform.
|
||||||
|
//
|
||||||
|
// **An unknown answer is not a metered one.** Only mobile answers this
|
||||||
|
// question — the desktop stub returns an empty string — so a policy that
|
||||||
|
// treated silence as "metered" would refuse the download on every
|
||||||
|
// desktop in the world. Silence means "no reason to refuse".
|
||||||
|
//
|
||||||
|
// **Cellular is the signal, and it is the only one available.** Wails
|
||||||
|
// reports `{"connected":bool,"type":"wifi|cellular|ethernet|none"}` and
|
||||||
|
// no metered flag, so a metered *wifi* — a phone hotspot, a hotel — is
|
||||||
|
// invisible to us and will not be refused. That is a known gap rather
|
||||||
|
// than an oversight: Android knows (`NET_CAPABILITY_NOT_METERED`) and
|
||||||
|
// the runtime does not pass it on.
|
||||||
|
|
||||||
|
// ErrMeteredNetwork is returned instead of downloading the catalog when
|
||||||
|
// the connection looks metered and the user has not opted in. Every
|
||||||
|
// failure path in `tryCoreArtifact` is already non-fatal, so this
|
||||||
|
// behaves like any other reason the artifact is not available yet.
|
||||||
|
var ErrMeteredNetwork = errors.New(
|
||||||
|
"explore: catalog download declined on a metered connection",
|
||||||
|
)
|
||||||
|
|
||||||
|
// Network is what the platform can say about the connection.
|
||||||
|
type Network struct {
|
||||||
|
// Known is false when nothing answered — every desktop, and any
|
||||||
|
// mobile build whose bridge is not up yet.
|
||||||
|
Known bool
|
||||||
|
// Connected reports a usable connection of any kind.
|
||||||
|
Connected bool
|
||||||
|
// Metered reports a connection the user is plausibly paying for by
|
||||||
|
// the byte. See the note above on what this cannot see.
|
||||||
|
Metered bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// NetworkProbe answers "what kind of connection is this", or an unknown
|
||||||
|
// Network when the platform does not say.
|
||||||
|
type NetworkProbe func() Network
|
||||||
|
|
||||||
|
// ParseNetworkJSON reads the runtime's network payload.
|
||||||
|
//
|
||||||
|
// Anything unparseable is `Known: false` rather than an error: this
|
||||||
|
// decides whether to *skip* an optional download, and a malformed
|
||||||
|
// payload is not a reason to refuse one.
|
||||||
|
func ParseNetworkJSON(payload string) Network {
|
||||||
|
var raw struct {
|
||||||
|
Connected bool `json:"connected"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.TrimSpace(payload) == "" {
|
||||||
|
return Network{}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := json.Unmarshal([]byte(payload), &raw); err != nil {
|
||||||
|
return Network{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return Network{
|
||||||
|
Known: true,
|
||||||
|
Connected: raw.Connected,
|
||||||
|
Metered: strings.EqualFold(raw.Type, "cellular"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// networkPolicy is the injected half: how to ask, and whether the user
|
||||||
|
// has said yes anyway.
|
||||||
|
type networkPolicy struct {
|
||||||
|
mu sync.RWMutex
|
||||||
|
probe NetworkProbe
|
||||||
|
allowMetered func() bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *networkPolicy) set(probe NetworkProbe, allowMetered func() bool) {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
|
||||||
|
p.probe = probe
|
||||||
|
p.allowMetered = allowMetered
|
||||||
|
}
|
||||||
|
|
||||||
|
// refuses reports whether a large optional download should be skipped.
|
||||||
|
func (p *networkPolicy) refuses() bool {
|
||||||
|
p.mu.RLock()
|
||||||
|
probe, allow := p.probe, p.allowMetered
|
||||||
|
p.mu.RUnlock()
|
||||||
|
|
||||||
|
if probe == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
if allow != nil && allow() {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
state := probe()
|
||||||
|
|
||||||
|
return state.Known && state.Metered
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetNetworkPolicy wires how the catalog download decides whether this
|
||||||
|
// connection is one to spend 0.6 GB on. Both arguments may be nil, which
|
||||||
|
// is the desktop's answer: never refuse.
|
||||||
|
//
|
||||||
|
//wails:ignore // internal wiring, not part of the app's IPC surface.
|
||||||
|
func (si *SearchIndex) SetNetworkPolicy(probe NetworkProbe, allowMetered func() bool) {
|
||||||
|
si.netPolicy.set(probe, allowMetered)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetNetworkPolicy wires the metered-connection policy into the index.
|
||||||
|
//
|
||||||
|
//wails:ignore // internal wiring, not part of the app's IPC surface.
|
||||||
|
func (e *Service) SetNetworkPolicy(probe NetworkProbe, allowMetered func() bool) {
|
||||||
|
e.index.SetNetworkPolicy(probe, allowMetered)
|
||||||
|
}
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
package explore
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The catalog is ~0.6 GB and the decision not to fetch it is the only
|
||||||
|
// part of plan 016 B4 that can be tested anywhere but on a phone: the
|
||||||
|
// platform call is a one-line closure injected from app.go, and
|
||||||
|
// everything that decides anything is here.
|
||||||
|
|
||||||
|
func TestParseNetworkJSON(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
payload string
|
||||||
|
want Network
|
||||||
|
}{{
|
||||||
|
name: "cellular is metered",
|
||||||
|
payload: `{"connected":true,"type":"cellular"}`,
|
||||||
|
want: Network{Known: true, Connected: true, Metered: true},
|
||||||
|
}, {
|
||||||
|
name: "wifi is not",
|
||||||
|
payload: `{"connected":true,"type":"wifi"}`,
|
||||||
|
want: Network{Known: true, Connected: true},
|
||||||
|
}, {
|
||||||
|
name: "ethernet is not",
|
||||||
|
payload: `{"connected":true,"type":"ethernet"}`,
|
||||||
|
want: Network{Known: true, Connected: true},
|
||||||
|
}, {
|
||||||
|
name: "the case is the platform's business, not ours",
|
||||||
|
payload: `{"connected":true,"type":"Cellular"}`,
|
||||||
|
want: Network{Known: true, Connected: true, Metered: true},
|
||||||
|
}, {
|
||||||
|
name: "offline is known and unmetered",
|
||||||
|
payload: `{"connected":false,"type":"none"}`,
|
||||||
|
want: Network{Known: true},
|
||||||
|
}, {
|
||||||
|
// The desktop stub. This is the case that must not read as
|
||||||
|
// "metered": every desktop in the world answers this way.
|
||||||
|
name: "an empty payload is unknown",
|
||||||
|
payload: "",
|
||||||
|
want: Network{},
|
||||||
|
}, {
|
||||||
|
name: "so is a malformed one",
|
||||||
|
payload: `{"connected":`,
|
||||||
|
want: Network{},
|
||||||
|
}}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
if got := ParseNetworkJSON(tt.payload); got != tt.want {
|
||||||
|
t.Errorf("ParseNetworkJSON(%q) = %+v, want %+v", tt.payload, got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNetworkPolicyRefuses(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cellular := func() Network {
|
||||||
|
return Network{Known: true, Connected: true, Metered: true}
|
||||||
|
}
|
||||||
|
wifi := func() Network { return Network{Known: true, Connected: true} }
|
||||||
|
unknown := func() Network { return Network{} }
|
||||||
|
yes := func() bool { return true }
|
||||||
|
no := func() bool { return false }
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
probe NetworkProbe
|
||||||
|
allowMetered func() bool
|
||||||
|
want bool
|
||||||
|
}{{
|
||||||
|
name: "no probe wired refuses nothing",
|
||||||
|
probe: nil,
|
||||||
|
want: false,
|
||||||
|
}, {
|
||||||
|
name: "an unknown connection refuses nothing",
|
||||||
|
probe: unknown,
|
||||||
|
want: false,
|
||||||
|
}, {
|
||||||
|
name: "wifi refuses nothing",
|
||||||
|
probe: wifi,
|
||||||
|
want: false,
|
||||||
|
}, {
|
||||||
|
name: "cellular refuses by default",
|
||||||
|
probe: cellular,
|
||||||
|
want: true,
|
||||||
|
}, {
|
||||||
|
name: "cellular with no permission refuses",
|
||||||
|
probe: cellular,
|
||||||
|
allowMetered: no,
|
||||||
|
want: true,
|
||||||
|
}, {
|
||||||
|
name: "cellular the user opted into does not",
|
||||||
|
probe: cellular,
|
||||||
|
allowMetered: yes,
|
||||||
|
want: false,
|
||||||
|
}, {
|
||||||
|
// The permission is read at decision time rather than captured,
|
||||||
|
// so turning it on takes effect on the next attempt instead of
|
||||||
|
// the next launch.
|
||||||
|
name: "permission is asked, not remembered",
|
||||||
|
probe: cellular,
|
||||||
|
allowMetered: yes,
|
||||||
|
want: false,
|
||||||
|
}}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var p networkPolicy
|
||||||
|
|
||||||
|
p.set(tt.probe, tt.allowMetered)
|
||||||
|
|
||||||
|
if got := p.refuses(); got != tt.want {
|
||||||
|
t.Errorf("refuses() = %v, want %v", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The gate has to come before anything is staged: a declined download is
|
||||||
|
// a no-op, not a job in the indicator or a status the user must dismiss.
|
||||||
|
func TestTryCoreArtifactDeclinesMeteredWithoutStaging(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
si := &SearchIndex{}
|
||||||
|
|
||||||
|
si.SetNetworkPolicy(
|
||||||
|
func() Network { return Network{Known: true, Connected: true, Metered: true} },
|
||||||
|
nil,
|
||||||
|
)
|
||||||
|
|
||||||
|
err := si.tryCoreArtifact(t.Context())
|
||||||
|
|
||||||
|
if !errors.Is(err, ErrMeteredNetwork) {
|
||||||
|
t.Fatalf("tryCoreArtifact() error = %v, want ErrMeteredNetwork", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nothing announced itself: no build status, no tiers, no job. A
|
||||||
|
// SearchIndex with no database would panic on any of the work below
|
||||||
|
// the gate, which is itself part of the assertion.
|
||||||
|
if si.buildStatus.Building {
|
||||||
|
t.Error("declining a metered download still reported a build in progress")
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(si.buildStatus.Tiers) != 0 {
|
||||||
|
t.Errorf("declining staged %d tiers, want none", len(si.buildStatus.Tiers))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -212,6 +212,11 @@ type SearchIndex struct {
|
|||||||
cancel context.CancelFunc
|
cancel context.CancelFunc
|
||||||
done chan struct{}
|
done chan struct{}
|
||||||
|
|
||||||
|
// netPolicy decides whether this connection is one to spend ~0.6 GB
|
||||||
|
// of catalog on. Its own lock: it is written once at startup and read
|
||||||
|
// from the build goroutine (netpolicy.go).
|
||||||
|
netPolicy networkPolicy
|
||||||
|
|
||||||
mu sync.RWMutex
|
mu sync.RWMutex
|
||||||
ready bool
|
ready bool
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
package frontendutil
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"sort"
|
||||||
|
)
|
||||||
|
|
||||||
|
// errNotADirectory is returned when a caller asks to list something
|
||||||
|
// that exists but is not a directory.
|
||||||
|
var errNotADirectory = errors.New("not a directory")
|
||||||
|
|
||||||
|
// DirEntry is one selectable directory in a listing.
|
||||||
|
type DirEntry struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Path string `json:"path"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// DirListing is one level of the filesystem, as a folder picker needs
|
||||||
|
// it: where we are, what is above, and the directories below.
|
||||||
|
//
|
||||||
|
// Parent is empty at a root, which is what tells the UI not to draw an
|
||||||
|
// "up" affordance rather than having it compute that from the path
|
||||||
|
// separator.
|
||||||
|
type DirListing struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
Parent string `json:"parent"`
|
||||||
|
Entries []DirEntry `json:"entries"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListDirectories returns the directories directly inside path, so the
|
||||||
|
// frontend can draw a folder picker.
|
||||||
|
//
|
||||||
|
// **It exists because Android has no directory picker.** Wails' file
|
||||||
|
// dialog can choose directories on every desktop platform, and on
|
||||||
|
// Android it returns an error: the Storage Access Framework yields tree
|
||||||
|
// URIs rather than filesystem paths, and a path is what this app's
|
||||||
|
// entire library model is keyed on. Rather than teach the backend about
|
||||||
|
// tree URIs, the app browses the filesystem itself — which it can do
|
||||||
|
// because it holds all-files access (see the manifest).
|
||||||
|
//
|
||||||
|
// Three rules, each of which a picker gets wrong if it is not stated:
|
||||||
|
// only directories are returned, because the caller is choosing a
|
||||||
|
// library root and files are noise; unreadable children are skipped
|
||||||
|
// rather than failing the whole listing, since Android's storage root
|
||||||
|
// contains directories no app may enter; and hidden directories are
|
||||||
|
// omitted, because a music library is not in one and `.thumbnails`
|
||||||
|
// alone would swamp the list.
|
||||||
|
func (fe *FrontendUtil) ListDirectories(path string) (DirListing, error) {
|
||||||
|
if path == "" {
|
||||||
|
path = fe.DefaultBrowseRoot()
|
||||||
|
}
|
||||||
|
|
||||||
|
path = filepath.Clean(path)
|
||||||
|
|
||||||
|
info, err := os.Stat(path)
|
||||||
|
if err != nil {
|
||||||
|
return DirListing{}, fmt.Errorf("could not open %s: %w", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !info.IsDir() {
|
||||||
|
return DirListing{}, fmt.Errorf("%w: %s", errNotADirectory, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
entries, err := os.ReadDir(path)
|
||||||
|
if err != nil {
|
||||||
|
return DirListing{}, fmt.Errorf("could not read %s: %w", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dirs := make([]DirEntry, 0, len(entries))
|
||||||
|
|
||||||
|
for _, e := range entries {
|
||||||
|
name := e.Name()
|
||||||
|
if name == "" || name[0] == '.' {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// A symlink reports itself, not its target, so ask the
|
||||||
|
// filesystem: a symlinked music directory is ordinary and
|
||||||
|
// skipping it would be a bug the user cannot explain.
|
||||||
|
child := filepath.Join(path, name)
|
||||||
|
|
||||||
|
info, err := os.Stat(child)
|
||||||
|
if err != nil || !info.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
dirs = append(dirs, DirEntry{Name: name, Path: child})
|
||||||
|
}
|
||||||
|
|
||||||
|
sort.Slice(dirs, func(i, j int) bool { return dirs[i].Name < dirs[j].Name })
|
||||||
|
|
||||||
|
parent := filepath.Dir(path)
|
||||||
|
if parent == path {
|
||||||
|
parent = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return DirListing{Path: path, Parent: parent, Entries: dirs}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// androidSharedStorage is where a user's music lives on Android. It is
|
||||||
|
// not derivable from the environment the way a desktop home directory
|
||||||
|
// is: HOME inside an Android app process is "/", so os.UserHomeDir()
|
||||||
|
// would start the picker at the filesystem root with nothing readable
|
||||||
|
// under it.
|
||||||
|
const androidSharedStorage = "/storage/emulated/0"
|
||||||
|
|
||||||
|
// DefaultBrowseRoot is where a folder picker should open.
|
||||||
|
func (fe *FrontendUtil) DefaultBrowseRoot() string {
|
||||||
|
if runtime.GOOS == "android" {
|
||||||
|
for _, candidate := range []string{androidSharedStorage, "/storage"} {
|
||||||
|
if info, err := os.Stat(candidate); err == nil && info.IsDir() {
|
||||||
|
return candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "/"
|
||||||
|
}
|
||||||
|
|
||||||
|
if home, err := os.UserHomeDir(); err == nil && home != "" {
|
||||||
|
return home
|
||||||
|
}
|
||||||
|
|
||||||
|
return string(filepath.Separator)
|
||||||
|
}
|
||||||
|
|
||||||
|
// StorageAccess reports whether the app can actually read the place the
|
||||||
|
// user's music lives.
|
||||||
|
type StorageAccess struct {
|
||||||
|
Root string `json:"root"`
|
||||||
|
Readable bool `json:"readable"`
|
||||||
|
Reason string `json:"reason"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckStorageAccess asks the filesystem rather than the permission
|
||||||
|
// system.
|
||||||
|
//
|
||||||
|
// On Android this app holds MANAGE_EXTERNAL_STORAGE, which the user
|
||||||
|
// grants on a Settings screen rather than in a dialog — so it can be
|
||||||
|
// refused, revoked later, or simply never answered, and the permission
|
||||||
|
// API is one more thing that can disagree with reality. Reading the
|
||||||
|
// directory is the question the library scanner will actually ask, so
|
||||||
|
// it is the one worth answering.
|
||||||
|
//
|
||||||
|
// It is deliberately not an error return: "we cannot read your music
|
||||||
|
// yet" is a state the UI renders, not a failure of the call.
|
||||||
|
func (fe *FrontendUtil) CheckStorageAccess() StorageAccess {
|
||||||
|
root := fe.DefaultBrowseRoot()
|
||||||
|
|
||||||
|
if _, err := os.ReadDir(root); err != nil {
|
||||||
|
return StorageAccess{
|
||||||
|
Root: root,
|
||||||
|
Readable: false,
|
||||||
|
Reason: err.Error(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return StorageAccess{Root: root, Readable: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
// HasNativeDirectoryPicker reports whether this platform can open a
|
||||||
|
// directory dialog at all.
|
||||||
|
//
|
||||||
|
// It is asked of the backend rather than tested in the frontend with
|
||||||
|
// `System.IsAndroid()`, for three reasons. The dialog *is* backend code
|
||||||
|
// — `DirectoryPicker` above — so this is the same package saying what
|
||||||
|
// it can do. It answers for iOS too without the frontend enumerating
|
||||||
|
// platforms. And it makes the frontend's fallback testable through the
|
||||||
|
// ordinary transport fake instead of a module mock of the Wails
|
||||||
|
// runtime, whose platform helpers read build constants.
|
||||||
|
func (fe *FrontendUtil) HasNativeDirectoryPicker() bool {
|
||||||
|
switch runtime.GOOS {
|
||||||
|
case "android", "ios":
|
||||||
|
return false
|
||||||
|
default:
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,207 @@
|
|||||||
|
package frontendutil
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// listing helper: a tree with the three shapes the picker has to get
|
||||||
|
// right — an ordinary directory, a file (never listed), and a hidden
|
||||||
|
// directory (never listed).
|
||||||
|
func browseFixture(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
|
||||||
|
for _, dir := range []string{"Music", "Podcasts", "aaa", ".thumbnails"} {
|
||||||
|
if err := os.Mkdir(filepath.Join(root, dir), 0o755); err != nil {
|
||||||
|
t.Fatalf("mkdir %s: %v", dir, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "track.mp3"), []byte("x"), 0o600); err != nil {
|
||||||
|
t.Fatalf("write file: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return root
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListDirectories(t *testing.T) {
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
root := browseFixture(t)
|
||||||
|
|
||||||
|
got, err := fe.ListDirectories(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ListDirectories: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sorted, directories only, no file and no dotted entry.
|
||||||
|
want := []string{"Music", "Podcasts", "aaa"}
|
||||||
|
if len(got.Entries) != len(want) {
|
||||||
|
t.Fatalf("got %d entries %v, want %v", len(got.Entries), got.Entries, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, w := range want {
|
||||||
|
if got.Entries[i].Name != w {
|
||||||
|
t.Errorf("entry %d = %q, want %q", i, got.Entries[i].Name, w)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.Entries[i].Path != filepath.Join(root, w) {
|
||||||
|
t.Errorf("entry %d path = %q, want %q", i, got.Entries[i].Path, filepath.Join(root, w))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.Path != root {
|
||||||
|
t.Errorf("Path = %q, want %q", got.Path, root)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.Parent != filepath.Dir(root) {
|
||||||
|
t.Errorf("Parent = %q, want %q", got.Parent, filepath.Dir(root))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A symlink reports itself rather than its target, so a listing that
|
||||||
|
// trusts DirEntry.IsDir() silently drops a symlinked music folder --
|
||||||
|
// which is an ordinary thing to have and an unexplainable thing to
|
||||||
|
// lose.
|
||||||
|
func TestListDirectoriesFollowsSymlinks(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("symlinks need elevation on Windows")
|
||||||
|
}
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
target := filepath.Join(root, "real")
|
||||||
|
|
||||||
|
if err := os.Mkdir(target, 0o755); err != nil {
|
||||||
|
t.Fatalf("mkdir: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
link := filepath.Join(root, "linked")
|
||||||
|
if err := os.Symlink(target, link); err != nil {
|
||||||
|
t.Fatalf("symlink: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
got, err := fe.ListDirectories(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ListDirectories: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(got.Entries) != 2 {
|
||||||
|
t.Fatalf("got %v, want both 'linked' and 'real'", got.Entries)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A dangling symlink, and anything else os.Stat refuses, must be
|
||||||
|
// skipped rather than failing the whole listing: Android's storage
|
||||||
|
// root holds directories no app may enter, and one of them must not
|
||||||
|
// cost the user the picker.
|
||||||
|
func TestListDirectoriesSkipsUnreadable(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("symlinks need elevation on Windows")
|
||||||
|
}
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(root, "good"), 0o755); err != nil {
|
||||||
|
t.Fatalf("mkdir: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dangling := filepath.Join(root, "dangling")
|
||||||
|
if err := os.Symlink(filepath.Join(root, "nowhere"), dangling); err != nil {
|
||||||
|
t.Fatalf("symlink: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
got, err := fe.ListDirectories(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ListDirectories: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(got.Entries) != 1 || got.Entries[0].Name != "good" {
|
||||||
|
t.Errorf("got %v, want only 'good'", got.Entries)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListDirectoriesRejectsFiles(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
|
||||||
|
file := filepath.Join(root, "track.mp3")
|
||||||
|
if err := os.WriteFile(file, []byte("x"), 0o600); err != nil {
|
||||||
|
t.Fatalf("write: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
if _, err := fe.ListDirectories(file); err == nil {
|
||||||
|
t.Error("listing a file should be an error")
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := fe.ListDirectories(filepath.Join(root, "missing")); err == nil {
|
||||||
|
t.Error("listing a missing path should be an error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An empty path means "start where the picker should open", so the
|
||||||
|
// frontend never has to know the platform.
|
||||||
|
func TestListDirectoriesDefaultsToBrowseRoot(t *testing.T) {
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
got, err := fe.ListDirectories("")
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("default root not listable in this environment: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.Path != fe.DefaultBrowseRoot() {
|
||||||
|
t.Errorf("Path = %q, want the default root %q", got.Path, fe.DefaultBrowseRoot())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parent is empty at a root, which is what tells the picker not to draw
|
||||||
|
// an "up" control rather than making it reason about separators.
|
||||||
|
func TestListDirectoriesRootHasNoParent(t *testing.T) {
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
got, err := fe.ListDirectories(string(filepath.Separator))
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("filesystem root not listable: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.Parent != "" {
|
||||||
|
t.Errorf("Parent = %q at the root, want empty", got.Parent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckStorageAccess(t *testing.T) {
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
got := fe.CheckStorageAccess()
|
||||||
|
if got.Root == "" {
|
||||||
|
t.Error("Root should never be empty")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The developer machine running this test can read its own home
|
||||||
|
// directory; the assertion is that the two fields agree, not that
|
||||||
|
// access is granted.
|
||||||
|
if got.Readable && got.Reason != "" {
|
||||||
|
t.Errorf("readable but Reason = %q", got.Reason)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !got.Readable && got.Reason == "" {
|
||||||
|
t.Error("not readable but no Reason given")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The picker's fallback is chosen from this, so a platform that gains
|
||||||
|
// a working dialog must flip it here rather than in the frontend.
|
||||||
|
func TestHasNativeDirectoryPicker(t *testing.T) {
|
||||||
|
fe := &FrontendUtil{}
|
||||||
|
|
||||||
|
want := runtime.GOOS != "android" && runtime.GOOS != "ios"
|
||||||
|
if got := fe.HasNativeDirectoryPicker(); got != want {
|
||||||
|
t.Errorf("HasNativeDirectoryPicker() on %s = %v, want %v", runtime.GOOS, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
+141
-15
@@ -289,8 +289,13 @@ func (l *Library) scanInternal(
|
|||||||
l.mu.Unlock()
|
l.mu.Unlock()
|
||||||
}()
|
}()
|
||||||
|
|
||||||
|
// The configured mode, not a hardcoded "auto". `ScanConcurrency`
|
||||||
|
// has been a validated config field with three values and one
|
||||||
|
// caller passing a constant, so choosing `ssd` or `hdd` by hand
|
||||||
|
// did nothing at all.
|
||||||
|
diskProfile := system.ProfileForPath(libraryPath)
|
||||||
workerCount := resolveScanWorkerCount(
|
workerCount := resolveScanWorkerCount(
|
||||||
ScanConcurrencyAuto,
|
l.conf.ScanConcurrency,
|
||||||
libraryPath,
|
libraryPath,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -300,6 +305,10 @@ func (l *Library) scanInternal(
|
|||||||
"libraryName", libraryName,
|
"libraryName", libraryName,
|
||||||
"libraryPath", libraryPath,
|
"libraryPath", libraryPath,
|
||||||
"workers", workerCount,
|
"workers", workerCount,
|
||||||
|
"mode", l.conf.ScanConcurrency,
|
||||||
|
"device", diskProfile.Device,
|
||||||
|
"rotational", diskProfile.Rotational,
|
||||||
|
"queueDepth", diskProfile.QueueDepth,
|
||||||
)
|
)
|
||||||
|
|
||||||
// Helper to build a ScanProgress with library identification.
|
// Helper to build a ScanProgress with library identification.
|
||||||
@@ -818,7 +827,7 @@ func (l *Library) scanInternal(
|
|||||||
g := new(errgroup.Group)
|
g := new(errgroup.Group)
|
||||||
g.SetLimit(workerCount)
|
g.SetLimit(workerCount)
|
||||||
|
|
||||||
for work := range workChan {
|
for work := range readaheadWork(scanCtx, workChan, diskProfile) {
|
||||||
g.Go(func() error {
|
g.Go(func() error {
|
||||||
if err := l.waitIfPaused(scanCtx); err != nil {
|
if err := l.waitIfPaused(scanCtx); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -1285,9 +1294,101 @@ func surveyAudioFiles(
|
|||||||
return count, maxModTime
|
return count, maxModTime
|
||||||
}
|
}
|
||||||
|
|
||||||
// hddWorkerCount is the maximum number of concurrent extraction
|
// How many extraction workers a spinning disk gets, and why it is two
|
||||||
// workers when the library resides on a spinning disk.
|
// numbers rather than one.
|
||||||
const hddWorkerCount = 2
|
//
|
||||||
|
// Extraction is not CPU work — every parser here reads headers and
|
||||||
|
// returns — so on a spinning disk the whole cost is seek latency, and
|
||||||
|
// the only question worth asking is how many reads should be in flight
|
||||||
|
// at once. That has two different right answers and the drive says
|
||||||
|
// which:
|
||||||
|
//
|
||||||
|
// - A drive with command queueing (NCQ: /sys/block/<dev>/device/
|
||||||
|
// queue_depth reports 31 or 32 on any SATA disk with it enabled)
|
||||||
|
// reorders outstanding reads into the order its head passes over
|
||||||
|
// them. Handing it several at once is most of why a parallel scan
|
||||||
|
// beats a serial one at all, and four is where the returns flatten:
|
||||||
|
// the drive needs a few requests to have anything to reorder, and
|
||||||
|
// past that it is queueing requests it was already going to
|
||||||
|
// service in that order.
|
||||||
|
// - A drive without it — queue_depth 1, which is what a USB bridge
|
||||||
|
// or a pre-2004 disk reports — services one command at a time in
|
||||||
|
// the order given. Every extra worker there is one more seek
|
||||||
|
// competing for one head, and the scan gets *slower* the harder it
|
||||||
|
// is pushed. Two is kept rather than one because the readahead
|
||||||
|
// hints (see readaheadWork) do the overlapping that concurrency
|
||||||
|
// was standing in for, and one worker cannot hide a stall.
|
||||||
|
//
|
||||||
|
// This used to be a flat 2 for anything rotational, which is a
|
||||||
|
// pre-NCQ assumption: it left a modern spinning disk with a quarter of
|
||||||
|
// the queue depth it can use.
|
||||||
|
const (
|
||||||
|
hddWorkerCountQueued = 4
|
||||||
|
hddWorkerCountSerial = 2
|
||||||
|
)
|
||||||
|
|
||||||
|
// Readahead tuning.
|
||||||
|
const (
|
||||||
|
// readaheadDepth is how many files ahead of the workers the
|
||||||
|
// prefetcher runs. It is the channel's buffer, so it is also the
|
||||||
|
// number of `WILLNEED` hints outstanding at once — comfortably more
|
||||||
|
// than a queueing drive's 32-command window is worth filling with
|
||||||
|
// one library, and small enough that a cancelled scan is not
|
||||||
|
// holding a long tail of queued reads.
|
||||||
|
readaheadDepth = 16
|
||||||
|
|
||||||
|
// readaheadBytes is how much of each file to pull in. Everything
|
||||||
|
// the scanner reads lives at the head: ID3v2 and FLAC's
|
||||||
|
// STREAMINFO/VORBIS_COMMENT/PICTURE blocks, and the first MPEG
|
||||||
|
// frame with its Xing header. 512 KB covers a tag carrying
|
||||||
|
// embedded cover art, which is the large case — and reading a
|
||||||
|
// little too much sequentially costs a spinning disk almost
|
||||||
|
// nothing next to the seek that got there.
|
||||||
|
readaheadBytes = 512 << 10
|
||||||
|
)
|
||||||
|
|
||||||
|
// readaheadWork forwards scan work while asking the kernel to fetch
|
||||||
|
// each file's header before a worker reaches it.
|
||||||
|
//
|
||||||
|
// The buffered channel *is* the lookahead: this goroutine runs ahead
|
||||||
|
// of the workers until the buffer fills, hinting every file as it goes,
|
||||||
|
// so by the time a worker takes an item the read it needs has been in
|
||||||
|
// flight for `readaheadDepth` files' worth of parsing. That is the
|
||||||
|
// only thing that helps a spinning disk here, because the per-file work
|
||||||
|
// is already header-only — every parser in `backend/metadata` reads a
|
||||||
|
// few hundred bytes and returns, so the scan is not waiting on CPU or
|
||||||
|
// on bytes, it is waiting on the head to arrive.
|
||||||
|
//
|
||||||
|
// It runs on rotational disks only. An SSD has no seek to hide and
|
||||||
|
// already has one worker per core; issuing hints there is pure syscall
|
||||||
|
// overhead against an OS readahead that is already ahead of us.
|
||||||
|
func readaheadWork(
|
||||||
|
ctx context.Context,
|
||||||
|
in <-chan scanWork,
|
||||||
|
profile system.DiskProfile,
|
||||||
|
) <-chan scanWork {
|
||||||
|
if !profile.Rotational {
|
||||||
|
return in
|
||||||
|
}
|
||||||
|
|
||||||
|
out := make(chan scanWork, readaheadDepth)
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
defer close(out)
|
||||||
|
|
||||||
|
for work := range in {
|
||||||
|
hintReadahead(work.absolutePath, readaheadBytes)
|
||||||
|
|
||||||
|
select {
|
||||||
|
case out <- work:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// resolveScanWorkerCount returns the number of concurrent
|
// resolveScanWorkerCount returns the number of concurrent
|
||||||
// extraction workers based on the configured concurrency mode
|
// extraction workers based on the configured concurrency mode
|
||||||
@@ -1296,20 +1397,45 @@ func resolveScanWorkerCount(
|
|||||||
mode ScanConcurrency,
|
mode ScanConcurrency,
|
||||||
libraryPath string,
|
libraryPath string,
|
||||||
) int {
|
) int {
|
||||||
|
return workersForProfile(
|
||||||
|
mode,
|
||||||
|
system.ProfileForPath(libraryPath),
|
||||||
|
goruntime.NumCPU(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// workersForProfile is the policy on its own, so it can be tested
|
||||||
|
// against drives this machine does not have.
|
||||||
|
//
|
||||||
|
// `hdd` and `ssd` override what the device says rather than being a
|
||||||
|
// separate branch: the mode is the user overruling detection, and
|
||||||
|
// detection is right about the queue depth either way — a user who
|
||||||
|
// picks `hdd` on a queueing drive still wants that drive's queue used.
|
||||||
|
func workersForProfile(
|
||||||
|
mode ScanConcurrency,
|
||||||
|
profile system.DiskProfile,
|
||||||
|
cpus int,
|
||||||
|
) int {
|
||||||
|
spinning := profile.Rotational
|
||||||
|
|
||||||
switch mode {
|
switch mode {
|
||||||
case ScanConcurrencySSD:
|
case ScanConcurrencySSD:
|
||||||
return goruntime.NumCPU()
|
spinning = false
|
||||||
case ScanConcurrencyHDD:
|
case ScanConcurrencyHDD:
|
||||||
return min(hddWorkerCount, goruntime.NumCPU())
|
spinning = true
|
||||||
default: // auto
|
case ScanConcurrencyAuto:
|
||||||
if system.IsRotationalDisk(libraryPath) {
|
|
||||||
return min(
|
|
||||||
hddWorkerCount, goruntime.NumCPU(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
return goruntime.NumCPU()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !spinning {
|
||||||
|
return cpus
|
||||||
|
}
|
||||||
|
|
||||||
|
workers := hddWorkerCountSerial
|
||||||
|
if profile.Queues() {
|
||||||
|
workers = hddWorkerCountQueued
|
||||||
|
}
|
||||||
|
|
||||||
|
return min(workers, cpus)
|
||||||
}
|
}
|
||||||
|
|
||||||
// scanWork represents a file to be processed by a worker.
|
// scanWork represents a file to be processed by a worker.
|
||||||
|
|||||||
@@ -0,0 +1,38 @@
|
|||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package library
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"golang.org/x/sys/unix"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hintReadahead asks the kernel to start fetching the head of a file
|
||||||
|
// that is about to be read.
|
||||||
|
//
|
||||||
|
// `POSIX_FADV_WILLNEED` returns immediately and queues the read, which
|
||||||
|
// is the whole point: on a spinning disk the first access to a file
|
||||||
|
// costs a seek of several milliseconds, and that latency can only be
|
||||||
|
// hidden by having the next seek already in flight while the current
|
||||||
|
// file is being parsed. A drive with command queueing can then service
|
||||||
|
// the queued reads in head order rather than in the order they were
|
||||||
|
// asked for.
|
||||||
|
//
|
||||||
|
// Errors are dropped on purpose. This is a hint: a file that has since
|
||||||
|
// been deleted, a filesystem that does not implement fadvise, or a
|
||||||
|
// permission the walk saw and this open does not, all mean "no
|
||||||
|
// prefetch", never "fail the scan". The read that follows is what
|
||||||
|
// reports a genuine problem.
|
||||||
|
func hintReadahead(path string, bytes int64) {
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = f.Close() }()
|
||||||
|
|
||||||
|
_ = unix.Fadvise(
|
||||||
|
int(f.Fd()), 0, bytes, unix.FADV_WILLNEED,
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
//go:build !linux
|
||||||
|
|
||||||
|
package library
|
||||||
|
|
||||||
|
// hintReadahead is a no-op off Linux.
|
||||||
|
//
|
||||||
|
// macOS has `F_RDADVISE` and Windows has `FILE_FLAG_SEQUENTIAL_SCAN`,
|
||||||
|
// and neither is wired up here for the reason the scan concurrency
|
||||||
|
// heuristic is not either: this package cannot tell a spinning disk
|
||||||
|
// from an SSD on those platforms (see system.ProfileForPath), so it
|
||||||
|
// would be prefetching without knowing whether prefetching is what the
|
||||||
|
// device wants.
|
||||||
|
func hintReadahead(_ string, _ int64) {}
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
package library
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"yellowjacket/backend/system"
|
||||||
|
)
|
||||||
|
|
||||||
|
// How many workers a scan gets is decided by two facts about the
|
||||||
|
// device, and the second one is new: a spinning disk that can queue
|
||||||
|
// commands wants several reads in flight, and one that cannot wants
|
||||||
|
// almost none. Before this it was a flat 2 for anything rotational,
|
||||||
|
// which is a pre-NCQ assumption — a modern SATA disk reports a queue
|
||||||
|
// depth of 32 and was being given a quarter of what it can use.
|
||||||
|
func TestWorkersForProfile(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const cpus = 16
|
||||||
|
|
||||||
|
ssd := system.DiskProfile{Device: "sda", QueueDepth: 32}
|
||||||
|
hddQueued := system.DiskProfile{
|
||||||
|
Device: "sdb", Rotational: true, QueueDepth: 32,
|
||||||
|
}
|
||||||
|
hddSerial := system.DiskProfile{
|
||||||
|
Device: "sdc", Rotational: true, QueueDepth: 1,
|
||||||
|
}
|
||||||
|
// Neither NVMe nor a device-mapper volume publishes queue_depth.
|
||||||
|
// An unknown depth must not be read as "cannot queue", or every
|
||||||
|
// such device would be scanned as if it were a 2003 drive.
|
||||||
|
unknown := system.DiskProfile{Device: "dm-0", Rotational: true}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
mode ScanConcurrency
|
||||||
|
profile system.DiskProfile
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{"ssd auto", ScanConcurrencyAuto, ssd, cpus},
|
||||||
|
{"queueing hdd auto", ScanConcurrencyAuto, hddQueued, hddWorkerCountQueued},
|
||||||
|
{"serial hdd auto", ScanConcurrencyAuto, hddSerial, hddWorkerCountSerial},
|
||||||
|
{"unknown depth queues", ScanConcurrencyAuto, unknown, hddWorkerCountQueued},
|
||||||
|
|
||||||
|
// The mode overrules detection about the *disk*, never about
|
||||||
|
// its queue: forcing hdd on a queueing drive still uses it.
|
||||||
|
{"forced hdd on an ssd", ScanConcurrencyHDD, ssd, hddWorkerCountQueued},
|
||||||
|
{"forced ssd on an hdd", ScanConcurrencySSD, hddQueued, cpus},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
if got := workersForProfile(tt.mode, tt.profile, cpus); got != tt.want {
|
||||||
|
t.Errorf(
|
||||||
|
"workersForProfile(%q, %+v) = %d, want %d",
|
||||||
|
tt.mode, tt.profile, got, tt.want,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A machine with fewer cores than the policy asks for gets its cores.
|
||||||
|
func TestWorkersNeverExceedTheCPUCount(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
hdd := system.DiskProfile{Rotational: true, QueueDepth: 32}
|
||||||
|
|
||||||
|
if got := workersForProfile(ScanConcurrencyAuto, hdd, 1); got != 1 {
|
||||||
|
t.Errorf("single-core hdd = %d workers, want 1", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The prefetch stage must forward every item and nothing else: it is a
|
||||||
|
// pass-through with a side effect, and a scan that drops a file because
|
||||||
|
// of a *hint* would be a spectacular way to lose part of a library.
|
||||||
|
func TestReadaheadForwardsEveryFile(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
in := make(chan scanWork, 4)
|
||||||
|
for _, p := range []string{"/a", "/b", "/c", "/d"} {
|
||||||
|
in <- scanWork{absolutePath: p}
|
||||||
|
}
|
||||||
|
|
||||||
|
close(in)
|
||||||
|
|
||||||
|
var got []string
|
||||||
|
for w := range readaheadWork(
|
||||||
|
context.Background(),
|
||||||
|
in,
|
||||||
|
system.DiskProfile{Rotational: true, QueueDepth: 32},
|
||||||
|
) {
|
||||||
|
got = append(got, w.absolutePath)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := []string{"/a", "/b", "/c", "/d"}
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Fatalf("forwarded %v, want %v", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := range want {
|
||||||
|
if got[i] != want[i] {
|
||||||
|
t.Errorf("item %d = %q, want %q", i, got[i], want[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// On an SSD the stage is not inserted at all — the channel comes back
|
||||||
|
// unchanged, so a scan there pays nothing for a feature it cannot use.
|
||||||
|
func TestReadaheadIsSkippedOnSolidState(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
in := make(chan scanWork)
|
||||||
|
out := readaheadWork(
|
||||||
|
context.Background(), in, system.DiskProfile{QueueDepth: 32},
|
||||||
|
)
|
||||||
|
|
||||||
|
if out != (<-chan scanWork)(in) {
|
||||||
|
t.Error("an ssd must get the original channel, unwrapped")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,215 @@
|
|||||||
|
//go:build android
|
||||||
|
|
||||||
|
// Android's answer to MPRIS is a MediaSession, and reaching it needs no
|
||||||
|
// new JNI: Wails exports application.Android.StartForegroundService(json)
|
||||||
|
// going out, and Java's WailsBridge.emitEvent lands on the application
|
||||||
|
// event bus coming back. So this handler is one JSON payload pushed to
|
||||||
|
// the foreground service and one command event read from it. The Java
|
||||||
|
// half is
|
||||||
|
// build/android/app/src/main/java/com/wails/app/WailsForegroundService.java
|
||||||
|
// and the payload keys below are its contract.
|
||||||
|
|
||||||
|
package mediacontrols
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"github.com/wailsapp/wails/v3/pkg/application"
|
||||||
|
)
|
||||||
|
|
||||||
|
// commandEvent is the event name the Java side emits transport
|
||||||
|
// commands on. It is a plain string on both sides; changing it means
|
||||||
|
// changing WailsForegroundService too.
|
||||||
|
const commandEvent = "yj:media:command"
|
||||||
|
|
||||||
|
var errNoApplication = errors.New(
|
||||||
|
"no running application to attach media controls to",
|
||||||
|
)
|
||||||
|
|
||||||
|
// androidHandler drives the media notification, the lock-screen
|
||||||
|
// transport and audio focus through the foreground service.
|
||||||
|
type androidHandler struct {
|
||||||
|
logger *slog.Logger
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
callbacks Callbacks
|
||||||
|
meta Metadata
|
||||||
|
state PlaybackState
|
||||||
|
positionSec int
|
||||||
|
|
||||||
|
// running tracks whether the foreground service has been started.
|
||||||
|
// Android 12+ forbids starting one from the background, so it is
|
||||||
|
// started when playback starts -- a user action, in a visible app
|
||||||
|
// -- and stopped only when playback stops, which is what keeps
|
||||||
|
// queue auto-advance working with the screen off.
|
||||||
|
running bool
|
||||||
|
|
||||||
|
// lastPayload is the last JSON sent. An unchanged payload is not
|
||||||
|
// an event here either: every push crosses JNI and re-delivers an
|
||||||
|
// Intent, and the player pushes state on several paths that can
|
||||||
|
// agree.
|
||||||
|
lastPayload string
|
||||||
|
|
||||||
|
unsubscribe func()
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHandler returns the Android media-session handler.
|
||||||
|
func NewHandler(logger *slog.Logger) Handler {
|
||||||
|
return &androidHandler{logger: logger, state: StateStopped}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Init subscribes to the transport commands the Java side emits.
|
||||||
|
func (a *androidHandler) Init(callbacks Callbacks) error {
|
||||||
|
app := application.Get()
|
||||||
|
if app == nil {
|
||||||
|
return errNoApplication
|
||||||
|
}
|
||||||
|
|
||||||
|
a.mu.Lock()
|
||||||
|
a.callbacks = callbacks
|
||||||
|
a.mu.Unlock()
|
||||||
|
|
||||||
|
a.unsubscribe = app.Event.On(commandEvent, a.onCommand)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// onCommand dispatches one transport command from the notification,
|
||||||
|
// the lock screen, a headset button or an audio-focus change.
|
||||||
|
//
|
||||||
|
// Every callback runs on its own goroutine, for the reason the MPRIS
|
||||||
|
// handler does the same: they take the player and queue mutexes, and
|
||||||
|
// this runs on the event processor's dispatch goroutine.
|
||||||
|
func (a *androidHandler) onCommand(event *application.CustomEvent) {
|
||||||
|
data, ok := event.Data.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
command := parseMediaCommand(data)
|
||||||
|
|
||||||
|
a.mu.Lock()
|
||||||
|
cb := a.callbacks
|
||||||
|
a.mu.Unlock()
|
||||||
|
|
||||||
|
switch command.name {
|
||||||
|
case cmdPlay:
|
||||||
|
run(cb.OnPlay)
|
||||||
|
case cmdPause:
|
||||||
|
run(cb.OnPause)
|
||||||
|
case cmdPlayPause:
|
||||||
|
run(cb.OnPlayPause)
|
||||||
|
case cmdStop:
|
||||||
|
run(cb.OnStop)
|
||||||
|
case cmdNext:
|
||||||
|
run(cb.OnNext)
|
||||||
|
case cmdPrevious:
|
||||||
|
run(cb.OnPrevious)
|
||||||
|
case cmdSeek:
|
||||||
|
if cb.OnSeek != nil {
|
||||||
|
go cb.OnSeek(command.positionSec)
|
||||||
|
}
|
||||||
|
case cmdDuck:
|
||||||
|
if cb.OnDuck != nil {
|
||||||
|
go cb.OnDuck(command.duck)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
a.logger.Warn("Unknown media command", "command", command.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// run invokes a callback on its own goroutine, tolerating a nil one.
|
||||||
|
func run(fn func()) {
|
||||||
|
if fn != nil {
|
||||||
|
go fn()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateMetadata pushes new track details to the notification.
|
||||||
|
func (a *androidHandler) UpdateMetadata(meta Metadata) {
|
||||||
|
a.mu.Lock()
|
||||||
|
defer a.mu.Unlock()
|
||||||
|
|
||||||
|
a.meta = meta
|
||||||
|
a.push()
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdatePlaybackState pushes the state and a fresh position anchor;
|
||||||
|
// the MediaSession interpolates from there while playing.
|
||||||
|
func (a *androidHandler) UpdatePlaybackState(
|
||||||
|
state PlaybackState,
|
||||||
|
positionSec int,
|
||||||
|
) {
|
||||||
|
a.mu.Lock()
|
||||||
|
defer a.mu.Unlock()
|
||||||
|
|
||||||
|
a.state = state
|
||||||
|
a.positionSec = positionSec
|
||||||
|
a.push()
|
||||||
|
}
|
||||||
|
|
||||||
|
// NotifySeek re-anchors the position. Unlike MPRIS, a MediaSession has
|
||||||
|
// no separate seeked signal -- a new state with a new position is the
|
||||||
|
// whole mechanism.
|
||||||
|
func (a *androidHandler) NotifySeek(positionSec int) {
|
||||||
|
a.mu.Lock()
|
||||||
|
defer a.mu.Unlock()
|
||||||
|
|
||||||
|
a.positionSec = positionSec
|
||||||
|
a.push()
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateVolume is deliberately a no-op. Android's volume keys act on
|
||||||
|
// the media stream, which the OS owns; an app that also moved its own
|
||||||
|
// volume in response would move it twice.
|
||||||
|
func (a *androidHandler) UpdateVolume(_ float64) {}
|
||||||
|
|
||||||
|
// Close stops the service and drops the command subscription.
|
||||||
|
func (a *androidHandler) Close() {
|
||||||
|
a.mu.Lock()
|
||||||
|
defer a.mu.Unlock()
|
||||||
|
|
||||||
|
if a.unsubscribe != nil {
|
||||||
|
a.unsubscribe()
|
||||||
|
a.unsubscribe = nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if a.running {
|
||||||
|
application.Android.StopForegroundService()
|
||||||
|
a.running = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// push sends the current state to the Java side, if it has changed.
|
||||||
|
// The caller holds a.mu.
|
||||||
|
func (a *androidHandler) push() {
|
||||||
|
if a.state == StateStopped {
|
||||||
|
// Nothing is playing, so nothing justifies an ongoing
|
||||||
|
// notification or the process staying alive.
|
||||||
|
if a.running {
|
||||||
|
application.Android.StopForegroundService()
|
||||||
|
a.running = false
|
||||||
|
a.lastPayload = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
payload, err := mediaPayload(a.meta, a.state, a.positionSec)
|
||||||
|
if err != nil {
|
||||||
|
a.logger.Error("Failed to encode media payload", "err", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if payload == a.lastPayload {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
a.lastPayload = payload
|
||||||
|
a.running = true
|
||||||
|
|
||||||
|
application.Android.StartForegroundService(payload)
|
||||||
|
}
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
// The contract between the Android handler and the Java
|
||||||
|
// WailsForegroundService is two JSON documents -- one pushed out with
|
||||||
|
// the track and the state, one read back with a transport command --
|
||||||
|
// and neither side can check the other.
|
||||||
|
//
|
||||||
|
// It lives here, *without* the android build tag, so that `go test` on
|
||||||
|
// any platform exercises it. android.go itself can only be compiled by
|
||||||
|
// a cross-compiler and only be run by a phone, so anything left in it
|
||||||
|
// is untested by construction; this is the half worth not leaving
|
||||||
|
// there.
|
||||||
|
|
||||||
|
package mediacontrols
|
||||||
|
|
||||||
|
import "encoding/json"
|
||||||
|
|
||||||
|
// Media command names, as the Java side spells them.
|
||||||
|
const (
|
||||||
|
cmdPlay = "play"
|
||||||
|
cmdPause = "pause"
|
||||||
|
cmdPlayPause = "playpause"
|
||||||
|
cmdStop = "stop"
|
||||||
|
cmdNext = "next"
|
||||||
|
cmdPrevious = "previous"
|
||||||
|
cmdSeek = "seek"
|
||||||
|
cmdDuck = "duck"
|
||||||
|
)
|
||||||
|
|
||||||
|
// stateNames are what the payload's "state" key carries. Words rather
|
||||||
|
// than the PlaybackState integers, because the Java side reads them as
|
||||||
|
// JSON and a renumbered constant would silently mean something else
|
||||||
|
// there.
|
||||||
|
var stateNames = map[PlaybackState]string{
|
||||||
|
StateStopped: "stopped",
|
||||||
|
StatePlaying: "playing",
|
||||||
|
StatePaused: "paused",
|
||||||
|
}
|
||||||
|
|
||||||
|
// mediaCommand is one transport command from the notification, the
|
||||||
|
// lock screen, a headset button or an audio-focus change.
|
||||||
|
type mediaCommand struct {
|
||||||
|
name string
|
||||||
|
positionSec int
|
||||||
|
duck bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// mediaPayload encodes the state the notification and MediaSession
|
||||||
|
// render.
|
||||||
|
func mediaPayload(
|
||||||
|
meta Metadata,
|
||||||
|
state PlaybackState,
|
||||||
|
positionSec int,
|
||||||
|
) (string, error) {
|
||||||
|
payload, err := json.Marshal(map[string]any{
|
||||||
|
"title": meta.Title,
|
||||||
|
"artist": meta.Artist,
|
||||||
|
"album": meta.Album,
|
||||||
|
"artPath": meta.ArtFilePath,
|
||||||
|
"durationSec": meta.DurationSec,
|
||||||
|
"positionSec": positionSec,
|
||||||
|
"state": stateNames[state],
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
|
||||||
|
return string(payload), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseMediaCommand reads one command out of the event payload.
|
||||||
|
//
|
||||||
|
// The numbers arrive as float64 because they came through
|
||||||
|
// encoding/json as an untyped document -- asserting int here is the
|
||||||
|
// way a seek silently becomes a seek to zero.
|
||||||
|
func parseMediaCommand(data map[string]any) mediaCommand {
|
||||||
|
cmd := mediaCommand{}
|
||||||
|
cmd.name, _ = data["command"].(string)
|
||||||
|
|
||||||
|
if position, ok := data["positionSec"].(float64); ok {
|
||||||
|
cmd.positionSec = int(position)
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd.duck, _ = data["on"].(bool)
|
||||||
|
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
package mediacontrols
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestMediaPayloadKeys pins the document the Java side parses. The
|
||||||
|
// keys are the contract: a rename here is silently a track with no
|
||||||
|
// title on the lock screen, because WailsForegroundService reads them
|
||||||
|
// with optString and a missing key is simply "".
|
||||||
|
func TestMediaPayloadKeys(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
payload, err := mediaPayload(Metadata{
|
||||||
|
Title: "Tideline",
|
||||||
|
Artist: "Sea Change",
|
||||||
|
Album: "Ebb",
|
||||||
|
ArtFilePath: "/covers/ebb_lg.jpg",
|
||||||
|
DurationSec: 245,
|
||||||
|
}, StatePlaying, 30)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("mediaPayload: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var got map[string]any
|
||||||
|
if err := json.Unmarshal([]byte(payload), &got); err != nil {
|
||||||
|
t.Fatalf("payload is not JSON: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := map[string]any{
|
||||||
|
"title": "Tideline",
|
||||||
|
"artist": "Sea Change",
|
||||||
|
"album": "Ebb",
|
||||||
|
"artPath": "/covers/ebb_lg.jpg",
|
||||||
|
"durationSec": float64(245),
|
||||||
|
"positionSec": float64(30),
|
||||||
|
"state": "playing",
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Errorf("payload has %d keys, want %d: %s", len(got), len(want), payload)
|
||||||
|
}
|
||||||
|
|
||||||
|
for key, expected := range want {
|
||||||
|
if got[key] != expected {
|
||||||
|
t.Errorf("payload[%q] = %v, want %v", key, got[key], expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMediaPayloadStateNames covers the one value the Java side
|
||||||
|
// compares against a literal.
|
||||||
|
func TestMediaPayloadStateNames(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
state PlaybackState
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{StatePlaying, "playing"},
|
||||||
|
{StatePaused, "paused"},
|
||||||
|
{StateStopped, "stopped"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
payload, err := mediaPayload(Metadata{}, tt.state, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("mediaPayload: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var got struct {
|
||||||
|
State string `json:"state"`
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := json.Unmarshal([]byte(payload), &got); err != nil {
|
||||||
|
t.Fatalf("payload is not JSON: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.State != tt.want {
|
||||||
|
t.Errorf("state %d encoded as %q, want %q", tt.state, got.State, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseMediaCommand covers the direction that arrives untyped.
|
||||||
|
// The seek case is the one with teeth: the position crosses as a JSON
|
||||||
|
// number, so it is a float64 in the map and an int assertion would
|
||||||
|
// make every seek a seek to zero.
|
||||||
|
func TestParseMediaCommand(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
data map[string]any
|
||||||
|
want mediaCommand
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "play",
|
||||||
|
data: map[string]any{"command": "play"},
|
||||||
|
want: mediaCommand{name: cmdPlay},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "seek carries a position",
|
||||||
|
data: map[string]any{"command": "seek", "positionSec": float64(93)},
|
||||||
|
want: mediaCommand{name: cmdSeek, positionSec: 93},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "duck carries a flag",
|
||||||
|
data: map[string]any{"command": "duck", "on": true},
|
||||||
|
want: mediaCommand{name: cmdDuck, duck: true},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unduck",
|
||||||
|
data: map[string]any{"command": "duck", "on": false},
|
||||||
|
want: mediaCommand{name: cmdDuck},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "a command with nothing in it is not a panic",
|
||||||
|
data: map[string]any{},
|
||||||
|
want: mediaCommand{},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrongly typed fields fall back to zero",
|
||||||
|
data: map[string]any{"command": "seek", "positionSec": "93"},
|
||||||
|
want: mediaCommand{name: cmdSeek},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
if got := parseMediaCommand(tt.data); got != tt.want {
|
||||||
|
t.Errorf("parseMediaCommand(%v) = %+v, want %+v", tt.data, got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMediaCommandNamesAreWhatJavaSends is a spelling check against
|
||||||
|
// the Java side, which builds these strings by hand. It is a list, not
|
||||||
|
// a mechanism: nothing can reach across into the .java file, so the
|
||||||
|
// point is that changing one of these constants fails a test that
|
||||||
|
// names the file to change with it.
|
||||||
|
//
|
||||||
|
// See build/android/app/src/main/java/com/wails/app/WailsForegroundService.java.
|
||||||
|
func TestMediaCommandNamesAreWhatJavaSends(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
want := []string{
|
||||||
|
"play", "pause", "playpause", "stop",
|
||||||
|
"next", "previous", "seek", "duck",
|
||||||
|
}
|
||||||
|
got := []string{
|
||||||
|
cmdPlay, cmdPause, cmdPlayPause, cmdStop,
|
||||||
|
cmdNext, cmdPrevious, cmdSeek, cmdDuck,
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, name := range want {
|
||||||
|
if got[i] != name {
|
||||||
|
t.Errorf("command %d = %q, want %q", i, got[i], name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -34,6 +34,13 @@ type Callbacks struct {
|
|||||||
OnPrevious func()
|
OnPrevious func()
|
||||||
OnSeek func(positionSec int)
|
OnSeek func(positionSec int)
|
||||||
OnVolume func(volume float64) // 0.0–1.0 linear scale.
|
OnVolume func(volume float64) // 0.0–1.0 linear scale.
|
||||||
|
|
||||||
|
// OnDuck asks for playback to be attenuated (true) or restored
|
||||||
|
// (false) without changing the user's volume. Android alone sends
|
||||||
|
// it, and only below API 26 -- from Oreo the audio framework ducks
|
||||||
|
// the app itself and reports no such focus change, so doing both
|
||||||
|
// would attenuate twice.
|
||||||
|
OnDuck func(ducked bool)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Handler manages the OS media control integration.
|
// Handler manages the OS media control integration.
|
||||||
|
|||||||
@@ -1,4 +1,11 @@
|
|||||||
//go:build linux
|
//go:build linux && !android
|
||||||
|
|
||||||
|
// MPRIS is a D-Bus desktop specification, and `android` implies the
|
||||||
|
// `linux` build tag -- so without the `!android` this file compiled
|
||||||
|
// into the Android app and went looking for a session bus that does
|
||||||
|
// not exist. Desktop-Linux-only files need both halves; see Wails'
|
||||||
|
// mobile guide, which names this as the Android analogue of
|
||||||
|
// ios/darwin.
|
||||||
|
|
||||||
package mediacontrols
|
package mediacontrols
|
||||||
|
|
||||||
@@ -272,17 +279,19 @@ func (h *MPRISHandler) enqueue(fn func()) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// UpdateMetadata pushes track metadata to D-Bus.
|
// metadataMap builds the org.mpris.MediaPlayer2.Player Metadata value
|
||||||
func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
|
// for one track.
|
||||||
h.mu.Lock()
|
//
|
||||||
h.trackID++
|
// It is separated from UpdateMetadata, which needs a live D-Bus
|
||||||
tid := h.trackID
|
// connection, so the map's contents can be asserted on: this file is
|
||||||
h.mu.Unlock()
|
// behind a build tag and everything in it that touches h is reachable
|
||||||
|
// only from a session bus, which is the same reason the Android
|
||||||
m := map[string]interface{}{
|
// contract lives in an untagged androidpayload.go.
|
||||||
|
func metadataMap(meta Metadata, trackID uint64) map[string]any {
|
||||||
|
m := map[string]any{
|
||||||
"mpris:trackid": dbus.ObjectPath(
|
"mpris:trackid": dbus.ObjectPath(
|
||||||
fmt.Sprintf(
|
fmt.Sprintf(
|
||||||
"/org/yellowjacket/Track/%d", tid,
|
"/org/yellowjacket/Track/%d", trackID,
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
}
|
}
|
||||||
@@ -299,16 +308,45 @@ func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
|
|||||||
m["xesam:album"] = meta.Album
|
m["xesam:album"] = meta.Album
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Always present, even with nothing to point at.
|
||||||
|
//
|
||||||
|
// Every other key here can be omitted safely because a client
|
||||||
|
// reading the map sees a track with no title or no album and
|
||||||
|
// renders it that way. Art is different: KDE's applet (and
|
||||||
|
// others) treat an *absent* mpris:artUrl as "no news about the
|
||||||
|
// art" and keep drawing whatever the last track had, so playing
|
||||||
|
// something with no cover left the previous album's sleeve on
|
||||||
|
// screen — which reads as the wrong track playing rather than as
|
||||||
|
// missing artwork.
|
||||||
|
//
|
||||||
|
// An empty string is the honest answer and is what the spec's
|
||||||
|
// "URI" type degrades to; a client that cannot load it falls back
|
||||||
|
// to its own placeholder, which is the behaviour wanted.
|
||||||
|
artURL := ""
|
||||||
if meta.ArtFilePath != "" {
|
if meta.ArtFilePath != "" {
|
||||||
m["mpris:artUrl"] = "file://" + meta.ArtFilePath
|
artURL = "file://" + meta.ArtFilePath
|
||||||
}
|
}
|
||||||
|
|
||||||
|
m["mpris:artUrl"] = artURL
|
||||||
|
|
||||||
if meta.DurationSec > 0 {
|
if meta.DurationSec > 0 {
|
||||||
m["mpris:length"] = int64(
|
m["mpris:length"] = int64(
|
||||||
meta.DurationSec,
|
meta.DurationSec,
|
||||||
) * usPerSec
|
) * usPerSec
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpdateMetadata pushes track metadata to D-Bus.
|
||||||
|
func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
|
||||||
|
h.mu.Lock()
|
||||||
|
h.trackID++
|
||||||
|
tid := h.trackID
|
||||||
|
h.mu.Unlock()
|
||||||
|
|
||||||
|
m := metadataMap(meta, tid)
|
||||||
|
|
||||||
h.enqueue(func() {
|
h.enqueue(func() {
|
||||||
h.props.SetMust(playerIf, "Metadata", m)
|
h.props.SetMust(playerIf, "Metadata", m)
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
//go:build linux && !android
|
||||||
|
|
||||||
|
package mediacontrols
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
// The one key that must be present even when it is empty.
|
||||||
|
//
|
||||||
|
// Everything else in the map may be omitted, because a client reading
|
||||||
|
// it renders a track with no title as a track with no title. Art is
|
||||||
|
// different: KDE's applet treats an *absent* mpris:artUrl as no news
|
||||||
|
// about the art and keeps drawing the last one it saw, so a track with
|
||||||
|
// no cover wore the previous album's sleeve — which reads as the wrong
|
||||||
|
// track playing rather than as missing artwork.
|
||||||
|
func TestMetadataMapAlwaysCarriesArtURL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
meta Metadata
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "no art at all",
|
||||||
|
meta: Metadata{Title: "Blue in Green"},
|
||||||
|
want: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "art on disk",
|
||||||
|
meta: Metadata{
|
||||||
|
Title: "Blue in Green",
|
||||||
|
ArtFilePath: "/covers/kind-of-blue_lg.jpg",
|
||||||
|
},
|
||||||
|
want: "file:///covers/kind-of-blue_lg.jpg",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
m := metadataMap(tt.meta, 1)
|
||||||
|
|
||||||
|
got, ok := m["mpris:artUrl"]
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("mpris:artUrl is absent; it must always be sent")
|
||||||
|
}
|
||||||
|
|
||||||
|
if got != tt.want {
|
||||||
|
t.Errorf("mpris:artUrl = %v, want %q", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The trackid has to change between tracks or a client is entitled to
|
||||||
|
// treat the metadata as describing the same track it already has.
|
||||||
|
func TestMetadataMapTrackIDVaries(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
first := metadataMap(Metadata{Title: "A"}, 1)["mpris:trackid"]
|
||||||
|
second := metadataMap(Metadata{Title: "B"}, 2)["mpris:trackid"]
|
||||||
|
|
||||||
|
if first == second {
|
||||||
|
t.Errorf("trackid did not change: %v", first)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The optional keys stay optional — this is what makes artUrl's
|
||||||
|
// always-present treatment a deliberate exception rather than drift.
|
||||||
|
func TestMetadataMapOmitsEmptyOptionalFields(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
m := metadataMap(Metadata{}, 1)
|
||||||
|
|
||||||
|
for _, key := range []string{
|
||||||
|
"xesam:title",
|
||||||
|
"xesam:artist",
|
||||||
|
"xesam:album",
|
||||||
|
"mpris:length",
|
||||||
|
} {
|
||||||
|
if _, ok := m[key]; ok {
|
||||||
|
t.Errorf("%s is present for an empty Metadata", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,5 +1,10 @@
|
|||||||
//go:build !linux
|
//go:build !linux
|
||||||
|
|
||||||
|
// Windows and macOS have no media-control integration yet. `!linux`
|
||||||
|
// covers Android too without naming it, since `android` implies the
|
||||||
|
// `linux` tag -- android.go claims it, mpris_linux.go excludes it, and
|
||||||
|
// this file is left with the platforms neither wants.
|
||||||
|
|
||||||
package mediacontrols
|
package mediacontrols
|
||||||
|
|
||||||
import "log/slog"
|
import "log/slog"
|
||||||
|
|||||||
@@ -56,6 +56,13 @@ type Player struct {
|
|||||||
trackChangeID uint64
|
trackChangeID uint64
|
||||||
mediaControls mediacontrols.Handler
|
mediaControls mediacontrols.Handler
|
||||||
|
|
||||||
|
// duckAmount is the attenuation currently applied on top of the
|
||||||
|
// user's volume, in the same base-2 exponent effects.Volume uses.
|
||||||
|
// It is deliberately not persisted and emits no VolumeChanged: a
|
||||||
|
// duck is something the OS did for the length of a notification,
|
||||||
|
// not something the user chose.
|
||||||
|
duckAmount float64
|
||||||
|
|
||||||
// trackLengthMs holds the authoritative track duration in
|
// trackLengthMs holds the authoritative track duration in
|
||||||
// milliseconds, sourced from the database (which uses the
|
// milliseconds, sourced from the database (which uses the
|
||||||
// custom header parser). The go-mp3 decoder's Len() can be
|
// custom header parser). The go-mp3 decoder's Len() can be
|
||||||
@@ -793,12 +800,40 @@ func (p *Player) setVolumeLocked(desiredVolume UserVolume) {
|
|||||||
speaker.Lock()
|
speaker.Lock()
|
||||||
|
|
||||||
volume := clampVolume(desiredVolume)
|
volume := clampVolume(desiredVolume)
|
||||||
p.volume.Volume = float64(volume.ToVolume())
|
p.volume.Volume = float64(volume.ToVolume()) - p.duckAmount
|
||||||
p.volume.Silent = volume == MinUserVol
|
p.volume.Silent = volume == MinUserVol
|
||||||
|
|
||||||
speaker.Unlock()
|
speaker.Unlock()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetDuck attenuates playback (or restores it) without changing the
|
||||||
|
// user's volume, for an OS that has asked us to get out of the way of
|
||||||
|
// something short -- a navigation prompt, a notification tone.
|
||||||
|
//
|
||||||
|
// It re-applies the *user's* level through setVolumeLocked rather than
|
||||||
|
// nudging the effect directly, so the offset cannot accumulate across
|
||||||
|
// repeated ducks, and it neither emits nor persists: the level the user
|
||||||
|
// set has not changed and the UI must not claim it has.
|
||||||
|
//
|
||||||
|
//wails:ignore // driven by OS audio focus, not by the frontend.
|
||||||
|
func (p *Player) SetDuck(ducked bool) {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
|
||||||
|
amount := 0.0
|
||||||
|
if ducked {
|
||||||
|
amount = duckAttenuation
|
||||||
|
}
|
||||||
|
|
||||||
|
if p.volume == nil || amount == p.duckAmount {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
current := p.getUserVolume()
|
||||||
|
p.duckAmount = amount
|
||||||
|
p.setVolumeLocked(current)
|
||||||
|
}
|
||||||
|
|
||||||
// ChangeVolume adjusts the volume by a relative amount.
|
// ChangeVolume adjusts the volume by a relative amount.
|
||||||
func (p *Player) ChangeVolume(deltaVolume int) error {
|
func (p *Player) ChangeVolume(deltaVolume int) error {
|
||||||
p.mu.Lock()
|
p.mu.Lock()
|
||||||
@@ -812,7 +847,9 @@ func (p *Player) ChangeVolume(deltaVolume int) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (p *Player) getUserVolume() UserVolume {
|
func (p *Player) getUserVolume() UserVolume {
|
||||||
return Volume(p.volume.Volume).ToUserVolume()
|
// Undo any duck, so every caller -- the event, the persisted
|
||||||
|
// state, a relative change -- sees the level the user chose.
|
||||||
|
return Volume(p.volume.Volume + p.duckAmount).ToUserVolume()
|
||||||
}
|
}
|
||||||
|
|
||||||
// Muted reports whether playback is currently silenced.
|
// Muted reports whether playback is currently silenced.
|
||||||
|
|||||||
@@ -19,6 +19,12 @@ const (
|
|||||||
MaxVol Volume = 0
|
MaxVol Volume = 0
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// duckAttenuation is how far playback drops when the OS asks us to
|
||||||
|
// duck, on the same base-2 exponent scale: two steps is a quarter of
|
||||||
|
// the amplitude (-12 dB), which is audible under a spoken notification
|
||||||
|
// without sounding like a pause.
|
||||||
|
const duckAttenuation = 2.0
|
||||||
|
|
||||||
// ToVolume converts user volume to internal player volume.
|
// ToVolume converts user volume to internal player volume.
|
||||||
func (oldVol UserVolume) ToVolume() Volume {
|
func (oldVol UserVolume) ToVolume() Volume {
|
||||||
var newVol Volume
|
var newVol Volume
|
||||||
|
|||||||
@@ -1,9 +1,12 @@
|
|||||||
package player
|
package player
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"log/slog"
|
||||||
"math"
|
"math"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"github.com/gopxl/beep/v2/effects"
|
||||||
|
|
||||||
"yellowjacket/backend/mediacontrols"
|
"yellowjacket/backend/mediacontrols"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -202,3 +205,60 @@ func TestStateToMediaControls(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestSetDuck covers the property the duck rests on: the attenuation
|
||||||
|
// is applied to the output and is invisible to everything that asks
|
||||||
|
// what the volume is -- the event, the persisted state, a relative
|
||||||
|
// change. Getting that wrong would let one notification tone
|
||||||
|
// permanently rewrite the user's volume.
|
||||||
|
func TestSetDuck(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
p := NewPlayer(slog.Default(), nil)
|
||||||
|
p.volume = &effects.Volume{Base: 2}
|
||||||
|
p.setVolumeLocked(80)
|
||||||
|
|
||||||
|
unducked := p.volume.Volume
|
||||||
|
|
||||||
|
p.SetDuck(true)
|
||||||
|
|
||||||
|
if p.volume.Volume >= unducked {
|
||||||
|
t.Errorf(
|
||||||
|
"ducked output volume = %v, want less than %v",
|
||||||
|
p.volume.Volume, unducked,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := p.getUserVolume(); got != 80 {
|
||||||
|
t.Errorf("user volume while ducked = %d, want 80", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A second duck must not stack: the offset is re-applied to the
|
||||||
|
// user's level, never subtracted again from the current output.
|
||||||
|
ducked := p.volume.Volume
|
||||||
|
|
||||||
|
p.SetDuck(true)
|
||||||
|
|
||||||
|
if p.volume.Volume != ducked {
|
||||||
|
t.Errorf(
|
||||||
|
"duck applied twice = %v, want %v", p.volume.Volume, ducked,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Changing the volume while ducked keeps the attenuation.
|
||||||
|
p.setVolumeLocked(60)
|
||||||
|
|
||||||
|
if got := p.getUserVolume(); got != 60 {
|
||||||
|
t.Errorf("user volume set while ducked = %d, want 60", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
if want := float64(UserVolume(60).ToVolume()) - duckAttenuation; p.volume.Volume != want {
|
||||||
|
t.Errorf("output while ducked = %v, want %v", p.volume.Volume, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
p.SetDuck(false)
|
||||||
|
|
||||||
|
if want := float64(UserVolume(60).ToVolume()); p.volume.Volume != want {
|
||||||
|
t.Errorf("output after unduck = %v, want %v", p.volume.Volume, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -81,6 +81,7 @@ func (q *Queue) emitTracksModified(
|
|||||||
Index: index,
|
Index: index,
|
||||||
Positions: positions,
|
Positions: positions,
|
||||||
CurrentIndex: q.currentIndex,
|
CurrentIndex: q.currentIndex,
|
||||||
|
Source: q.source,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -219,6 +219,56 @@ func TestEmit_AddTrackSendsDeltaNotSnapshot(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The append clears the source, and the delta is the only event those
|
||||||
|
// paths emit — so if it does not carry the source, the frontend keeps
|
||||||
|
// the label it was last given and goes on offering a link back to an
|
||||||
|
// album the queue no longer holds until something forces a full state.
|
||||||
|
func TestEmit_AppendDeltaCarriesClearedSource(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
q, db, rec := setupRecordedQueue(t)
|
||||||
|
paths := seedAudioFiles(t, db, 4)
|
||||||
|
|
||||||
|
q.SetQueue(
|
||||||
|
paths[:3], 0, false,
|
||||||
|
Source{Type: "album", ID: 1, Label: "Abbey Road"},
|
||||||
|
)
|
||||||
|
|
||||||
|
if _, ok := rec.Wait(events.QueueChanged, waitFor); !ok {
|
||||||
|
t.Fatalf("no QueueChanged after SetQueue; got %v", rec.Names())
|
||||||
|
}
|
||||||
|
|
||||||
|
rec.Reset()
|
||||||
|
q.AddTrack(paths[3])
|
||||||
|
|
||||||
|
if got := modifiedOf(t, rec).Source; got != (Source{}) {
|
||||||
|
t.Errorf("delta source = %+v, want zero value", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// And a delta that did not clear it still reports the source it has,
|
||||||
|
// or the frontend would drop a perfectly good label on every removal.
|
||||||
|
func TestEmit_NonAppendDeltaCarriesSource(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
q, db, rec := setupRecordedQueue(t)
|
||||||
|
paths := seedAudioFiles(t, db, 4)
|
||||||
|
|
||||||
|
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
|
||||||
|
q.SetQueue(paths, 0, false, album)
|
||||||
|
|
||||||
|
if _, ok := rec.Wait(events.QueueChanged, waitFor); !ok {
|
||||||
|
t.Fatalf("no QueueChanged after SetQueue; got %v", rec.Names())
|
||||||
|
}
|
||||||
|
|
||||||
|
rec.Reset()
|
||||||
|
q.RemoveTrack(3)
|
||||||
|
|
||||||
|
if got := modifiedOf(t, rec).Source; got != album {
|
||||||
|
t.Errorf("delta source = %+v, want %+v", got, album)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestEmit_RemoveTracksReportsPositions(t *testing.T) {
|
func TestEmit_RemoveTracksReportsPositions(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -156,12 +156,21 @@ type PlaybackFailure struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TracksModified is the payload for the QueueTracksModified event.
|
// TracksModified is the payload for the QueueTracksModified event.
|
||||||
|
//
|
||||||
|
// Source is carried because an append is exactly what can *invalidate*
|
||||||
|
// it: a queue built from one album stops being that album the moment a
|
||||||
|
// track from somewhere else is added to it. The delta is the only event
|
||||||
|
// those paths emit, so without this the frontend would keep the label
|
||||||
|
// it was last given and go on saying "Playing from" an album that is no
|
||||||
|
// longer what is queued — an event carrying what its consumer needs, so
|
||||||
|
// nothing has to invalidate anything.
|
||||||
type TracksModified struct {
|
type TracksModified struct {
|
||||||
Action string `json:"action"`
|
Action string `json:"action"`
|
||||||
Tracks []Track `json:"tracks,omitempty"`
|
Tracks []Track `json:"tracks,omitempty"`
|
||||||
Index int `json:"index"`
|
Index int `json:"index"`
|
||||||
Positions []int `json:"positions,omitempty"`
|
Positions []int `json:"positions,omitempty"`
|
||||||
CurrentIndex int `json:"currentIndex"`
|
CurrentIndex int `json:"currentIndex"`
|
||||||
|
Source Source `json:"source"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Queue manages an ordered list of tracks for playback.
|
// Queue manages an ordered list of tracks for playback.
|
||||||
@@ -455,6 +464,8 @@ func (q *Queue) AddTrack(filePath string) {
|
|||||||
q.generateShuffleOrder()
|
q.generateShuffleOrder()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
q.dropSource()
|
||||||
|
|
||||||
q.persistAddTrack(track)
|
q.persistAddTrack(track)
|
||||||
q.persistState()
|
q.persistState()
|
||||||
q.emitTracksModified(
|
q.emitTracksModified(
|
||||||
@@ -505,6 +516,8 @@ func (q *Queue) AddTracks(filePaths []string) {
|
|||||||
q.generateShuffleOrder()
|
q.generateShuffleOrder()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
q.dropSource()
|
||||||
|
|
||||||
q.persistAddTracks(newTracks)
|
q.persistAddTracks(newTracks)
|
||||||
q.persistState()
|
q.persistState()
|
||||||
q.emitTracksModified(
|
q.emitTracksModified(
|
||||||
@@ -563,6 +576,8 @@ func (q *Queue) InsertNextTracks(filePaths []string) {
|
|||||||
q.generateShuffleOrder()
|
q.generateShuffleOrder()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
q.dropSource()
|
||||||
|
|
||||||
q.persistInsertTracks(newTracks, insertPos)
|
q.persistInsertTracks(newTracks, insertPos)
|
||||||
q.persistState()
|
q.persistState()
|
||||||
q.emitTracksModified(
|
q.emitTracksModified(
|
||||||
@@ -613,6 +628,8 @@ func (q *Queue) InsertNext(filePath string) {
|
|||||||
q.generateShuffleOrder()
|
q.generateShuffleOrder()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
q.dropSource()
|
||||||
|
|
||||||
q.persistInsertTracks([]Track{track}, insertPos)
|
q.persistInsertTracks([]Track{track}, insertPos)
|
||||||
q.persistState()
|
q.persistState()
|
||||||
q.emitTracksModified(
|
q.emitTracksModified(
|
||||||
@@ -680,6 +697,8 @@ func (q *Queue) InsertTracksAt(filePaths []string, index int) {
|
|||||||
q.generateShuffleOrder()
|
q.generateShuffleOrder()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
q.dropSource()
|
||||||
|
|
||||||
q.persistInsertTracks(newTracks, index)
|
q.persistInsertTracks(newTracks, index)
|
||||||
q.persistState()
|
q.persistState()
|
||||||
q.emitTracksModified(
|
q.emitTracksModified(
|
||||||
@@ -1537,6 +1556,31 @@ func (q *Queue) reindexPositions() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// dropSource forgets which collection the queue was built from.
|
||||||
|
//
|
||||||
|
// A Source is a claim that everything queued came from one album,
|
||||||
|
// playlist, genre or artist, and the frontend renders it as a
|
||||||
|
// "Playing from X" link back to that page. Adding or inserting a track
|
||||||
|
// makes the claim false — the queue is now that album *plus* something
|
||||||
|
// else — so every path that does so calls this.
|
||||||
|
//
|
||||||
|
// It was set by SetQueue and cleared in exactly one place, Clear, so a
|
||||||
|
// label survived every append. It is persisted too (source_type /
|
||||||
|
// source_id / source_label on the queue state row), which is what made
|
||||||
|
// a wrong label outlive the session that earned it: an album queued on
|
||||||
|
// Monday, added to on Tuesday, still offered a link back to that album
|
||||||
|
// on Friday.
|
||||||
|
//
|
||||||
|
// Removing, reordering and shuffling deliberately do not call this. A
|
||||||
|
// queue with a track taken out of it, or played in another order, is
|
||||||
|
// still that album — the link still goes somewhere true. Only the
|
||||||
|
// arrival of a track from elsewhere makes it a lie.
|
||||||
|
//
|
||||||
|
// The caller must hold q.mu.
|
||||||
|
func (q *Queue) dropSource() {
|
||||||
|
q.source = Source{}
|
||||||
|
}
|
||||||
|
|
||||||
// commitMutation persists the current queue state after a mutation.
|
// commitMutation persists the current queue state after a mutation.
|
||||||
// When reindex is true, track positions are renumbered first.
|
// When reindex is true, track positions are renumbered first.
|
||||||
// The caller must hold q.mu.
|
// The caller must hold q.mu.
|
||||||
|
|||||||
@@ -126,6 +126,117 @@ func TestClear_ResetsSource(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A queue built from one album stops being that album the moment a
|
||||||
|
// track from somewhere else joins it, so every path that adds one
|
||||||
|
// drops the source. Before this, SetQueue was the only writer and
|
||||||
|
// Clear the only clearer, so "Playing from Abbey Road" outlived every
|
||||||
|
// append — and, being persisted, every restart too.
|
||||||
|
func TestAppendPathsDropSource(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
append func(q *Queue, paths []string)
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "AddTrack",
|
||||||
|
append: func(q *Queue, paths []string) {
|
||||||
|
q.AddTrack(paths[5])
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "AddTracks",
|
||||||
|
append: func(q *Queue, paths []string) {
|
||||||
|
q.AddTracks(paths[5:7])
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "InsertNext",
|
||||||
|
append: func(q *Queue, paths []string) {
|
||||||
|
q.InsertNext(paths[5])
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "InsertNextTracks",
|
||||||
|
append: func(q *Queue, paths []string) {
|
||||||
|
q.InsertNextTracks(paths[5:7])
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "InsertTracksAt",
|
||||||
|
append: func(q *Queue, paths []string) {
|
||||||
|
q.InsertTracksAt(paths[5:7], 1)
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
q, db := setupTestQueue(t)
|
||||||
|
paths := seedAudioFiles(t, db, 8)
|
||||||
|
|
||||||
|
q.SetQueue(paths[:5], 0, false, album)
|
||||||
|
|
||||||
|
if got := q.GetState().Source; got != album {
|
||||||
|
t.Fatalf("source before append: got %+v, want %+v", got, album)
|
||||||
|
}
|
||||||
|
|
||||||
|
tt.append(q, paths)
|
||||||
|
|
||||||
|
if got := q.GetState().Source; got != (Source{}) {
|
||||||
|
t.Errorf(
|
||||||
|
"source after %s: got %+v, want zero value",
|
||||||
|
tt.name, got,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Removing and reordering deliberately do not drop it: a queue with a
|
||||||
|
// track taken out of it is still that album, and the link still goes
|
||||||
|
// somewhere true.
|
||||||
|
func TestRemoveAndMoveKeepSource(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
|
||||||
|
|
||||||
|
t.Run("RemoveTrack", func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
q, db := setupTestQueue(t)
|
||||||
|
paths := seedAudioFiles(t, db, 5)
|
||||||
|
|
||||||
|
q.SetQueue(paths, 0, false, album)
|
||||||
|
q.RemoveTrack(3)
|
||||||
|
|
||||||
|
if got := q.GetState().Source; got != album {
|
||||||
|
t.Errorf("source after RemoveTrack: got %+v, want %+v", got, album)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("MoveQueueTracks", func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
q, db := setupTestQueue(t)
|
||||||
|
paths := seedAudioFiles(t, db, 5)
|
||||||
|
|
||||||
|
q.SetQueue(paths, 0, false, album)
|
||||||
|
q.MoveQueueTracks([]int{0}, 3)
|
||||||
|
|
||||||
|
if got := q.GetState().Source; got != album {
|
||||||
|
t.Errorf(
|
||||||
|
"source after MoveQueueTracks: got %+v, want %+v",
|
||||||
|
got, album,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func TestSetQueue_WithStartIndex(t *testing.T) {
|
func TestSetQueue_WithStartIndex(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ var (
|
|||||||
errUnsupportedOp = errors.New("unsupported operator")
|
errUnsupportedOp = errors.New("unsupported operator")
|
||||||
errInvalidSortField = errors.New("invalid sort field: not in allowed field list")
|
errInvalidSortField = errors.New("invalid sort field: not in allowed field list")
|
||||||
errNotNumeric = errors.New("value must be numeric")
|
errNotNumeric = errors.New("value must be numeric")
|
||||||
|
errInvalidMatch = errors.New("match must be \"all\" or \"any\"")
|
||||||
)
|
)
|
||||||
|
|
||||||
// Rule represents a single filter condition for a smart playlist.
|
// Rule represents a single filter condition for a smart playlist.
|
||||||
@@ -37,13 +38,45 @@ type Rule struct {
|
|||||||
Value string `json:"value"`
|
Value string `json:"value"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MatchType decides how a rule set's conditions combine.
|
||||||
|
//
|
||||||
|
// The rules used to be joined with " AND " and nothing else, so a
|
||||||
|
// playlist could only ever narrow: "jazz released after 1960" was
|
||||||
|
// expressible and "jazz or blues" was not, which is most of what
|
||||||
|
// anyone reaches for a second rule to say.
|
||||||
|
type MatchType string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// MatchAll requires every rule to hold — the historical behaviour,
|
||||||
|
// and what an empty match means so that every rule set written
|
||||||
|
// before this existed keeps the meaning it was saved with.
|
||||||
|
MatchAll MatchType = "all"
|
||||||
|
// MatchAny requires at least one rule to hold.
|
||||||
|
MatchAny MatchType = "any"
|
||||||
|
)
|
||||||
|
|
||||||
|
// joiner returns the SQL keyword that combines two conditions.
|
||||||
|
// An unrecognised value cannot reach here — ParseRuleSet rejects one
|
||||||
|
// — so the default is about the empty string, which is every rule set
|
||||||
|
// saved before this field existed.
|
||||||
|
func (m MatchType) joiner() string {
|
||||||
|
if m == MatchAny {
|
||||||
|
return " OR "
|
||||||
|
}
|
||||||
|
|
||||||
|
return " AND "
|
||||||
|
}
|
||||||
|
|
||||||
// RuleSet holds the complete filter configuration for a smart
|
// RuleSet holds the complete filter configuration for a smart
|
||||||
// playlist, including optional sort and limit.
|
// playlist, including optional sort and limit.
|
||||||
type RuleSet struct {
|
type RuleSet struct {
|
||||||
Rules []Rule `json:"rules"`
|
Rules []Rule `json:"rules"`
|
||||||
Limit int `json:"limit,omitempty"`
|
// Match is "all" or "any"; empty means "all". It is omitempty so
|
||||||
SortField string `json:"sort_field,omitempty"`
|
// an untouched playlist's stored JSON does not change shape.
|
||||||
SortDir string `json:"sort_dir,omitempty"`
|
Match MatchType `json:"match,omitempty"`
|
||||||
|
Limit int `json:"limit,omitempty"`
|
||||||
|
SortField string `json:"sort_field,omitempty"`
|
||||||
|
SortDir string `json:"sort_dir,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// fieldMap maps user-facing rule field names to track_metadata column
|
// fieldMap maps user-facing rule field names to track_metadata column
|
||||||
@@ -116,7 +149,12 @@ const genreDelimiter = "||"
|
|||||||
// slice of rules. It is a pure function — no database access needed.
|
// slice of rules. It is a pure function — no database access needed.
|
||||||
// Returns the clause (without the leading "WHERE"), the parameter
|
// Returns the clause (without the leading "WHERE"), the parameter
|
||||||
// args, and any validation error.
|
// args, and any validation error.
|
||||||
func BuildWhereClause(rules []Rule) (string, []any, error) {
|
//
|
||||||
|
// match decides how the conditions combine; an empty match is MatchAll,
|
||||||
|
// which is what every rule set saved before the field existed means.
|
||||||
|
func BuildWhereClause(
|
||||||
|
rules []Rule, match MatchType,
|
||||||
|
) (string, []any, error) {
|
||||||
if len(rules) == 0 {
|
if len(rules) == 0 {
|
||||||
return "", nil, nil
|
return "", nil, nil
|
||||||
}
|
}
|
||||||
@@ -179,7 +217,28 @@ func BuildWhereClause(rules []Rule) (string, []any, error) {
|
|||||||
args = append(args, condArgs...)
|
args = append(args, condArgs...)
|
||||||
}
|
}
|
||||||
|
|
||||||
return strings.Join(conditions, " AND "), args, nil
|
// Under OR, each condition is parenthesised; under AND it is not.
|
||||||
|
//
|
||||||
|
// The asymmetry is deliberate rather than an omission. AND is the
|
||||||
|
// tighter operator in SQL, so an OR-join has to protect any
|
||||||
|
// condition that contains a top-level AND of its own or the halves
|
||||||
|
// come apart: `days_since_played less_than` is
|
||||||
|
// `last_played IS NOT NULL AND <expr> < ?`, which read without
|
||||||
|
// brackets under an OR-join happens to still parse correctly and
|
||||||
|
// would stop doing so the moment a condition grows a top-level OR.
|
||||||
|
// Bracketing under AND would be a no-op semantically and would
|
||||||
|
// rewrite the clause every existing test pins, so the brackets go
|
||||||
|
// exactly where they change something.
|
||||||
|
if match == MatchAny {
|
||||||
|
bracketed := make([]string, len(conditions))
|
||||||
|
for i, cond := range conditions {
|
||||||
|
bracketed[i] = "(" + cond + ")"
|
||||||
|
}
|
||||||
|
|
||||||
|
conditions = bracketed
|
||||||
|
}
|
||||||
|
|
||||||
|
return strings.Join(conditions, match.joiner()), args, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// validateOperator checks that the operator is valid for the field
|
// validateOperator checks that the operator is valid for the field
|
||||||
@@ -599,7 +658,7 @@ func Evaluate(
|
|||||||
start := time.Now()
|
start := time.Now()
|
||||||
logger := db.Logger()
|
logger := db.Logger()
|
||||||
|
|
||||||
where, args, err := BuildWhereClause(ruleSet.Rules)
|
where, args, err := BuildWhereClause(ruleSet.Rules, ruleSet.Match)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf(
|
return nil, fmt.Errorf(
|
||||||
"smart playlist rule error: %w", err,
|
"smart playlist rule error: %w", err,
|
||||||
@@ -1036,6 +1095,16 @@ func ParseRuleSet(jsonStr string) (RuleSet, error) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A match nobody recognises would otherwise fall through to AND,
|
||||||
|
// which is a playlist quietly returning the wrong tracks rather
|
||||||
|
// than refusing to be saved. This is the only place a rule set
|
||||||
|
// enters the backend, so it is the only place that has to ask.
|
||||||
|
if rs.Match != "" && rs.Match != MatchAll && rs.Match != MatchAny {
|
||||||
|
return RuleSet{}, fmt.Errorf(
|
||||||
|
"%w: %q", errInvalidMatch, rs.Match,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
return rs, nil
|
return rs, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package smartplaylist
|
package smartplaylist
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"errors"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -170,7 +171,7 @@ func TestBuildWhereClause_TextIs(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "artist", Operator: "is", Value: "Queen"},
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -189,7 +190,7 @@ func TestBuildWhereClause_TextIsNot(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "artist", Operator: "is_not", Value: "Queen"},
|
{Field: "artist", Operator: "is_not", Value: "Queen"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -209,7 +210,7 @@ func TestBuildWhereClause_TextContains(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "title", Operator: "contains", Value: "Black"},
|
{Field: "title", Operator: "contains", Value: "Black"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -231,7 +232,7 @@ func TestBuildWhereClause_TextDoesNotContain(t *testing.T) {
|
|||||||
Field: "title", Operator: "does_not_contain",
|
Field: "title", Operator: "does_not_contain",
|
||||||
Value: "Black",
|
Value: "Black",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -251,7 +252,7 @@ func TestBuildWhereClause_TextStartsWith(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "title", Operator: "starts_with", Value: "Back"},
|
{Field: "title", Operator: "starts_with", Value: "Back"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -270,7 +271,7 @@ func TestBuildWhereClause_TextEndsWith(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "title", Operator: "ends_with", Value: "Black"},
|
{Field: "title", Operator: "ends_with", Value: "Black"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -292,7 +293,7 @@ func TestBuildWhereClause_TextIsAnyOf(t *testing.T) {
|
|||||||
Field: "artist", Operator: "is_any_of",
|
Field: "artist", Operator: "is_any_of",
|
||||||
Value: `["Queen","AC/DC"]`,
|
Value: `["Queen","AC/DC"]`,
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -312,7 +313,7 @@ func TestBuildWhereClause_NumericIs(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "year", Operator: "is", Value: "1980"},
|
{Field: "year", Operator: "is", Value: "1980"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -331,7 +332,7 @@ func TestBuildWhereClause_NumericIsNot(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "year", Operator: "is_not", Value: "1980"},
|
{Field: "year", Operator: "is_not", Value: "1980"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -350,7 +351,7 @@ func TestBuildWhereClause_NumericGreaterThan(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "year", Operator: "greater_than", Value: "2000"},
|
{Field: "year", Operator: "greater_than", Value: "2000"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -369,7 +370,7 @@ func TestBuildWhereClause_NumericLessThan(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "year", Operator: "less_than", Value: "1980"},
|
{Field: "year", Operator: "less_than", Value: "1980"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -391,7 +392,7 @@ func TestBuildWhereClause_NumericBetween(t *testing.T) {
|
|||||||
Field: "year", Operator: "between",
|
Field: "year", Operator: "between",
|
||||||
Value: "1975,1985",
|
Value: "1975,1985",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -414,7 +415,7 @@ func TestBuildWhereClause_NumericBetweenJSON(t *testing.T) {
|
|||||||
Field: "year", Operator: "between",
|
Field: "year", Operator: "between",
|
||||||
Value: `["1975","1985"]`,
|
Value: `["1975","1985"]`,
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -434,7 +435,7 @@ func TestBuildWhereClause_GenreIsProducesSubquery(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "genre", Operator: "is", Value: "Rock"},
|
{Field: "genre", Operator: "is", Value: "Rock"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -466,7 +467,7 @@ func TestBuildWhereClause_GenreIsNotProducesSubquery(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "genre", Operator: "is_not", Value: "Rock"},
|
{Field: "genre", Operator: "is_not", Value: "Rock"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -495,7 +496,7 @@ func TestBuildWhereClause_GenreIsAnyOfProducesSubquery(t *testing.T) {
|
|||||||
Field: "genre", Operator: "is_any_of",
|
Field: "genre", Operator: "is_any_of",
|
||||||
Value: `["Rock","Pop"]`,
|
Value: `["Rock","Pop"]`,
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -524,7 +525,7 @@ func TestBuildWhereClause_GenreContainsUsesSubquery(t *testing.T) {
|
|||||||
|
|
||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "genre", Operator: "contains", Value: "Rock"},
|
{Field: "genre", Operator: "contains", Value: "Rock"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -557,7 +558,7 @@ func TestBuildWhereClause_MultipleRulesAND(t *testing.T) {
|
|||||||
clause, args, err := BuildWhereClause([]Rule{
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
{Field: "artist", Operator: "is", Value: "Queen"},
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
{Field: "year", Operator: "greater_than", Value: "1975"},
|
{Field: "year", Operator: "greater_than", Value: "1975"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -572,6 +573,110 @@ func TestBuildWhereClause_MultipleRulesAND(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBuildWhereClause_MultipleRulesOR(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
clause, args, err := BuildWhereClause([]Rule{
|
||||||
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
|
{Field: "year", Operator: "greater_than", Value: "1975"},
|
||||||
|
}, MatchAny)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := "(artist_name = ? COLLATE NOCASE) OR (year > ?)"
|
||||||
|
if clause != want {
|
||||||
|
t.Errorf("clause = %q, want %q", clause, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(args) != 2 || args[0] != "Queen" || args[1] != int64(1975) {
|
||||||
|
t.Errorf("args = %v, want [Queen 1975]", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An empty match is what every rule set saved before the field existed
|
||||||
|
// carries, and it has to keep meaning AND — a playlist silently
|
||||||
|
// widening to OR on upgrade is the whole risk of adding this field.
|
||||||
|
func TestBuildWhereClause_EmptyMatchIsAll(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
rules := []Rule{
|
||||||
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
|
{Field: "year", Operator: "greater_than", Value: "1975"},
|
||||||
|
}
|
||||||
|
|
||||||
|
empty, _, err := BuildWhereClause(rules, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
all, _, err := BuildWhereClause(rules, MatchAll)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if empty != all {
|
||||||
|
t.Errorf("empty match = %q, want the same as MatchAll %q",
|
||||||
|
empty, all)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A condition carrying its own top-level AND is what makes the
|
||||||
|
// bracketing under OR load-bearing: `days_since_played less_than`
|
||||||
|
// is two predicates, and both belong to the same rule.
|
||||||
|
func TestBuildWhereClause_ORBracketsCompoundCondition(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
clause, _, err := BuildWhereClause([]Rule{
|
||||||
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
|
{
|
||||||
|
Field: "days_since_played",
|
||||||
|
Operator: "less_than",
|
||||||
|
Value: "30",
|
||||||
|
},
|
||||||
|
}, MatchAny)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.Contains(clause, "(last_played IS NOT NULL AND") {
|
||||||
|
t.Errorf(
|
||||||
|
"compound condition is not bracketed under OR: %q",
|
||||||
|
clause,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseRuleSet_RejectsUnknownMatch(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := ParseRuleSet(`{"rules":[],"match":"either"}`)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected an error for an unknown match type")
|
||||||
|
}
|
||||||
|
|
||||||
|
if !errors.Is(err, errInvalidMatch) {
|
||||||
|
t.Errorf("err = %v, want errInvalidMatch", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseRuleSet_AcceptsAnyAndAll(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, want := range []MatchType{MatchAll, MatchAny} {
|
||||||
|
rs, err := ParseRuleSet(
|
||||||
|
`{"rules":[],"match":"` + string(want) + `"}`,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("match %q: unexpected error: %v", want, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rs.Match != want {
|
||||||
|
t.Errorf("match = %q, want %q", rs.Match, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
|
func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -581,7 +686,7 @@ func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
|
|||||||
Field: "genre", Operator: "does_not_contain",
|
Field: "genre", Operator: "does_not_contain",
|
||||||
Value: "Punk",
|
Value: "Punk",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -609,7 +714,7 @@ func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
|
|||||||
func TestBuildWhereClause_EmptyRules(t *testing.T) {
|
func TestBuildWhereClause_EmptyRules(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
clause, args, err := BuildWhereClause(nil)
|
clause, args, err := BuildWhereClause(nil, MatchAll)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
}
|
}
|
||||||
@@ -631,7 +736,7 @@ func TestBuildWhereClause_InvalidField(t *testing.T) {
|
|||||||
Field: "nonexistent", Operator: "is",
|
Field: "nonexistent", Operator: "is",
|
||||||
Value: "anything",
|
Value: "anything",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error for invalid field, got nil")
|
t.Fatal("expected error for invalid field, got nil")
|
||||||
}
|
}
|
||||||
@@ -654,7 +759,7 @@ func TestBuildWhereClause_InvalidOperatorForNumeric(t *testing.T) {
|
|||||||
|
|
||||||
_, _, err := BuildWhereClause([]Rule{
|
_, _, err := BuildWhereClause([]Rule{
|
||||||
{Field: "year", Operator: "contains", Value: "1980"},
|
{Field: "year", Operator: "contains", Value: "1980"},
|
||||||
})
|
}, MatchAll)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal(
|
t.Fatal(
|
||||||
"expected error for text operator on numeric field",
|
"expected error for text operator on numeric field",
|
||||||
@@ -676,7 +781,7 @@ func TestBuildWhereClause_InvalidOperatorForText(t *testing.T) {
|
|||||||
Field: "artist", Operator: "greater_than",
|
Field: "artist", Operator: "greater_than",
|
||||||
Value: "Queen",
|
Value: "Queen",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal(
|
t.Fatal(
|
||||||
"expected error for numeric operator on text field",
|
"expected error for numeric operator on text field",
|
||||||
@@ -723,6 +828,80 @@ func TestEvaluate_TextIs(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Two rules that share no track at all: under AND this is empty, and
|
||||||
|
// under OR it is the union. Before Match existed only the first was
|
||||||
|
// expressible, so a playlist could only ever narrow — "jazz or blues"
|
||||||
|
// had no way to be said.
|
||||||
|
func TestEvaluate_MatchAnyUnionsWhereMatchAllIntersects(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
seedSmartPlaylistData(t, db)
|
||||||
|
|
||||||
|
// Queen has two tracks; Beyoncé has one; no track is by both.
|
||||||
|
rules := []Rule{
|
||||||
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
|
{Field: "artist", Operator: "is", Value: "Beyoncé"},
|
||||||
|
}
|
||||||
|
|
||||||
|
all, err := Evaluate(db, RuleSet{Rules: rules, Match: MatchAll})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Evaluate(all): %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(all) != 0 {
|
||||||
|
t.Errorf("match=all returned %d tracks, want 0", len(all))
|
||||||
|
}
|
||||||
|
|
||||||
|
either, err := Evaluate(db, RuleSet{Rules: rules, Match: MatchAny})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Evaluate(any): %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(either) != 3 {
|
||||||
|
t.Fatalf("match=any returned %d tracks, want 3", len(either))
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tr := range either {
|
||||||
|
if tr.ArtistName != "Queen" && tr.ArtistName != "Beyoncé" {
|
||||||
|
t.Errorf(
|
||||||
|
"track %q has artist %q, want Queen or Beyoncé",
|
||||||
|
tr.TrackName, tr.ArtistName,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An empty match is what every playlist saved before the field existed
|
||||||
|
// carries, and it has to keep meaning AND all the way through Evaluate
|
||||||
|
// — a stored playlist silently widening on upgrade is the only real
|
||||||
|
// risk in adding this.
|
||||||
|
func TestEvaluate_EmptyMatchStillIntersects(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := database.NewTestDB(t)
|
||||||
|
seedSmartPlaylistData(t, db)
|
||||||
|
|
||||||
|
tracks, err := Evaluate(db, RuleSet{
|
||||||
|
Rules: []Rule{
|
||||||
|
{Field: "artist", Operator: "is", Value: "Queen"},
|
||||||
|
{Field: "year", Operator: "greater_than", Value: "1979"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Evaluate: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only "Another One Bites the Dust" (Queen, 1980) satisfies both.
|
||||||
|
if len(tracks) != 1 {
|
||||||
|
t.Fatalf("got %d tracks, want 1", len(tracks))
|
||||||
|
}
|
||||||
|
|
||||||
|
if want := "Another One Bites the Dust"; tracks[0].TrackName != want {
|
||||||
|
t.Errorf("got %q, want %q", tracks[0].TrackName, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestEvaluate_ArtworkEnrichment verifies the presentation-only
|
// TestEvaluate_ArtworkEnrichment verifies the presentation-only
|
||||||
// cover-art and MusicBrainz-ID fields are attached to matched tracks
|
// cover-art and MusicBrainz-ID fields are attached to matched tracks
|
||||||
// by the batched fetchArtwork pass (they are no longer part of the
|
// by the batched fetchArtwork pass (they are no longer part of the
|
||||||
@@ -1340,7 +1519,7 @@ func TestSQLInjection_FieldName(t *testing.T) {
|
|||||||
Field: "title; DROP TABLE playlists",
|
Field: "title; DROP TABLE playlists",
|
||||||
Operator: "is", Value: "x",
|
Operator: "is", Value: "x",
|
||||||
},
|
},
|
||||||
})
|
}, MatchAll)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal(
|
t.Fatal(
|
||||||
"expected error for injected field name, got nil",
|
"expected error for injected field name, got nil",
|
||||||
|
|||||||
@@ -16,32 +16,107 @@ var errNoBlockDevice = errors.New(
|
|||||||
"no matching block device found",
|
"no matching block device found",
|
||||||
)
|
)
|
||||||
|
|
||||||
// IsRotationalDisk reports whether the block device backing the
|
// DiskProfile is what the scanner needs to know about the device a
|
||||||
// given path is a rotational (spinning) disk. Detection uses the
|
// library sits on. Both fields are about the same question — how many
|
||||||
// Linux sysfs interface at /sys/block/<dev>/queue/rotational.
|
// reads should be in flight at once — and they answer different halves
|
||||||
// Returns false on any error (assumes SSD).
|
// of it, so they travel together rather than as two probes.
|
||||||
func IsRotationalDisk(path string) bool {
|
type DiskProfile struct {
|
||||||
dev, err := deviceForPath(path)
|
// Device is the whole-disk kernel name ("sdb"), or "" when the
|
||||||
if err != nil {
|
// path could not be resolved to one.
|
||||||
return false
|
Device string
|
||||||
}
|
|
||||||
|
|
||||||
rotational, err := os.ReadFile(
|
// Rotational is /sys/block/<dev>/queue/rotational: true for a
|
||||||
filepath.Join(
|
// spinning disk, where a seek costs milliseconds.
|
||||||
"/sys/block", dev, "queue", "rotational",
|
Rotational bool
|
||||||
),
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
return strings.TrimSpace(string(rotational)) == "1"
|
// QueueDepth is /sys/block/<dev>/device/queue_depth — how many
|
||||||
|
// commands the drive will accept and reorder at once. This is
|
||||||
|
// NCQ: a SATA disk with it enabled reports 31 or 32, and one
|
||||||
|
// without reports 1. Zero means the file was not there to read,
|
||||||
|
// which is the case for anything that is not a SCSI/SATA device
|
||||||
|
// (NVMe, MMC, device-mapper, loop, a VM's virtio disk).
|
||||||
|
//
|
||||||
|
// It is the difference between concurrency helping and hurting.
|
||||||
|
// With queueing, several outstanding reads let the drive service
|
||||||
|
// them in the order its head passes over them, which is most of
|
||||||
|
// why a parallel scan is faster at all. Without it, every extra
|
||||||
|
// worker is one more seek competing for one head, and the scan
|
||||||
|
// gets slower the harder it is pushed.
|
||||||
|
QueueDepth int
|
||||||
}
|
}
|
||||||
|
|
||||||
// deviceForPath resolves a filesystem path to its underlying block
|
// Queues reports whether the drive can reorder outstanding commands.
|
||||||
// device name (e.g. "sda") by matching the device major:minor
|
//
|
||||||
// from stat(2) against /sys/block/ entries.
|
// An unknown depth (0) counts as queueing: everything that does not
|
||||||
func deviceForPath(path string) (string, error) {
|
// publish this file is a device where concurrency is fine — NVMe has
|
||||||
|
// its own queues, virtio and device-mapper are not the physical layer
|
||||||
|
// at all. The only case worth being careful about is the one that
|
||||||
|
// says so explicitly.
|
||||||
|
func (p DiskProfile) Queues() bool {
|
||||||
|
return p.QueueDepth != 1
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsRotationalDisk reports whether the block device backing the
|
||||||
|
// given path is a rotational (spinning) disk. Returns false on any
|
||||||
|
// error (assumes SSD).
|
||||||
|
func IsRotationalDisk(path string) bool {
|
||||||
|
return ProfileForPath(path).Rotational
|
||||||
|
}
|
||||||
|
|
||||||
|
// ProfileForPath describes the device backing a filesystem path. A
|
||||||
|
// path that cannot be resolved yields the zero profile, which reads as
|
||||||
|
// "not rotational, queueing" — the permissive answer, since assuming a
|
||||||
|
// spinning disk on an SSD would halve a scan for nothing.
|
||||||
|
func ProfileForPath(path string) DiskProfile {
|
||||||
|
dev, err := diskForPath(path)
|
||||||
|
if err != nil {
|
||||||
|
return DiskProfile{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return DiskProfile{
|
||||||
|
Device: dev,
|
||||||
|
Rotational: sysfsInt(dev, "queue", "rotational") == 1,
|
||||||
|
QueueDepth: sysfsInt(dev, "device", "queue_depth"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// sysfsInt reads one small integer out of /sys/block/<dev>/<parts...>,
|
||||||
|
// returning 0 when it is absent or unparseable. Every attribute here
|
||||||
|
// is optional: sysfs layout varies by driver, and a missing file is
|
||||||
|
// "this device does not say", never an error worth propagating.
|
||||||
|
func sysfsInt(dev string, parts ...string) int {
|
||||||
|
p := filepath.Join(
|
||||||
|
append([]string{"/sys/block", dev}, parts...)...,
|
||||||
|
)
|
||||||
|
|
||||||
|
data, err := os.ReadFile(p) //nolint:gosec // sysfs, name from the kernel
|
||||||
|
if err != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := strconv.Atoi(strings.TrimSpace(string(data)))
|
||||||
|
if err != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// diskForPath resolves a filesystem path to the *whole disk* backing
|
||||||
|
// it — "sdb" for a file on "sdb3".
|
||||||
|
//
|
||||||
|
// It goes through /sys/dev/block/<major>:<minor>, which the kernel
|
||||||
|
// maintains as a symlink to the device's own sysfs directory, and then
|
||||||
|
// walks up to the parent when that directory turns out to be a
|
||||||
|
// partition. The previous implementation scanned /sys/block comparing
|
||||||
|
// dev numbers and, failing an exact match, took the first entry whose
|
||||||
|
// *major* agreed — and every SATA disk shares major 8. So a library on
|
||||||
|
// /dev/sdb3 resolved to whatever /sys/block listed first, which is
|
||||||
|
// alphabetical, which is sda. On the machine this was found on that
|
||||||
|
// meant a 6 TB spinning disk was read as the SSD next to it and scanned
|
||||||
|
// with one worker per core. Matching on major alone cannot be right
|
||||||
|
// whenever a machine has two disks, which is the case this exists for.
|
||||||
|
func diskForPath(path string) (string, error) {
|
||||||
var st syscall.Stat_t
|
var st syscall.Stat_t
|
||||||
if err := syscall.Stat(path, &st); err != nil {
|
if err := syscall.Stat(path, &st); err != nil {
|
||||||
return "", fmt.Errorf(
|
return "", fmt.Errorf(
|
||||||
@@ -49,48 +124,50 @@ func deviceForPath(path string) (string, error) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Extract major and minor device numbers.
|
// Linux packs dev_t as 12 bits of major and 20 of minor, split
|
||||||
major := (st.Dev >> 8) & 0xff
|
// across the word. Masking the low byte of each — which is what
|
||||||
minor := st.Dev & 0xff
|
// this used to do — is right only for the first 256 of either.
|
||||||
|
major := unixMajor(uint64(st.Dev))
|
||||||
|
minor := unixMinor(uint64(st.Dev))
|
||||||
|
|
||||||
// Scan /sys/block/ for a matching device.
|
link := filepath.Join(
|
||||||
entries, err := os.ReadDir("/sys/block")
|
"/sys/dev/block",
|
||||||
|
strconv.FormatUint(major, 10)+":"+
|
||||||
|
strconv.FormatUint(minor, 10),
|
||||||
|
)
|
||||||
|
|
||||||
|
target, err := filepath.EvalSymlinks(link)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Errorf(
|
return "", fmt.Errorf(
|
||||||
"could not read /sys/block: %w", err,
|
"%w: %s (%w)", errNoBlockDevice, link, err,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
majorStr := strconv.FormatUint(major, 10)
|
// A partition's directory sits inside its disk's, and only the
|
||||||
devStr := majorStr + ":" +
|
// disk carries `queue`. Climb at most one level: sysfs nests a
|
||||||
strconv.FormatUint(minor, 10)
|
// partition exactly one deep under its disk.
|
||||||
|
name := filepath.Base(target)
|
||||||
|
|
||||||
for _, entry := range entries {
|
if _, err := os.Stat(filepath.Join(target, "queue")); err != nil {
|
||||||
devFile := filepath.Join(
|
name = filepath.Base(filepath.Dir(target))
|
||||||
"/sys/block", entry.Name(), "dev",
|
|
||||||
)
|
|
||||||
|
|
||||||
data, err := os.ReadFile(devFile)
|
|
||||||
if err != nil {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
content := strings.TrimSpace(string(data))
|
|
||||||
|
|
||||||
if content == devStr {
|
|
||||||
return entry.Name(), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// The filesystem might be on a partition (e.g. sda1)
|
|
||||||
// whose parent block device is sda. Check if the
|
|
||||||
// major number matches.
|
|
||||||
parts := strings.SplitN(content, ":", 2)
|
|
||||||
if len(parts) == 2 && parts[0] == majorStr {
|
|
||||||
return entry.Name(), nil
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return "", fmt.Errorf(
|
if name == "" || name == "." || name == string(filepath.Separator) {
|
||||||
"%w for %s", errNoBlockDevice, devStr,
|
return "", fmt.Errorf(
|
||||||
)
|
"%w for %d:%d", errNoBlockDevice, major, minor,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return name, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// unixMajor and unixMinor decode a Linux dev_t. Spelled out rather
|
||||||
|
// than taken from golang.org/x/sys/unix so this file stays readable
|
||||||
|
// beside the encoding it is undoing.
|
||||||
|
func unixMajor(dev uint64) uint64 {
|
||||||
|
return (dev>>8)&0xfff | (dev >> 32 & ^uint64(0xfff))
|
||||||
|
}
|
||||||
|
|
||||||
|
func unixMinor(dev uint64) uint64 {
|
||||||
|
return dev&0xff | (dev >> 12 & ^uint64(0xff))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,9 +2,34 @@
|
|||||||
|
|
||||||
package system
|
package system
|
||||||
|
|
||||||
|
// DiskProfile is what the scanner needs to know about the device a
|
||||||
|
// library sits on. See the Linux implementation for what each field
|
||||||
|
// means; off Linux nothing fills them, because neither macOS nor
|
||||||
|
// Windows publishes an equivalent of sysfs's `rotational` and
|
||||||
|
// `queue_depth` without going through platform APIs this package
|
||||||
|
// deliberately does not link.
|
||||||
|
type DiskProfile struct {
|
||||||
|
Device string
|
||||||
|
Rotational bool
|
||||||
|
QueueDepth int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Queues reports whether the drive can reorder outstanding commands.
|
||||||
|
// Always true here: an unknown depth is the permissive answer, and
|
||||||
|
// assuming otherwise would halve every scan on every Mac.
|
||||||
|
func (p DiskProfile) Queues() bool {
|
||||||
|
return p.QueueDepth != 1
|
||||||
|
}
|
||||||
|
|
||||||
// IsRotationalDisk reports whether the block device backing the
|
// IsRotationalDisk reports whether the block device backing the
|
||||||
// given path is a rotational (spinning) disk. On non-Linux
|
// given path is a rotational (spinning) disk. On non-Linux
|
||||||
// platforms this always returns false (assumes SSD).
|
// platforms this always returns false (assumes SSD).
|
||||||
func IsRotationalDisk(_ string) bool {
|
func IsRotationalDisk(_ string) bool {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ProfileForPath describes the device backing a filesystem path. Off
|
||||||
|
// Linux that is the zero profile, which reads as "an SSD that queues".
|
||||||
|
func ProfileForPath(_ string) DiskProfile {
|
||||||
|
return DiskProfile{}
|
||||||
|
}
|
||||||
|
|||||||
@@ -40,9 +40,21 @@ android {
|
|||||||
versionCode Integer.parseInt(System.getenv("YJ_VERSION_CODE") ?: "1")
|
versionCode Integer.parseInt(System.getenv("YJ_VERSION_CODE") ?: "1")
|
||||||
versionName System.getenv("YJ_VERSION") ?: "0.0.0"
|
versionName System.getenv("YJ_VERSION") ?: "0.0.0"
|
||||||
|
|
||||||
// Configure supported ABIs
|
// **arm64 only, and x86_64 is not a gap.** `modernc.org/libc`'s
|
||||||
|
// Xlstat64 issues a raw lstat syscall on linux/amd64, which
|
||||||
|
// Android's seccomp policy forbids (bionic never issues it), so
|
||||||
|
// the process takes SIGSYS the first time anything touches the
|
||||||
|
// database -- which for this app is startup. That is every
|
||||||
|
// x86_64 Android, emulators and x86 Chromebooks alike, not just
|
||||||
|
// some. arm64 is structurally unaffected: the architecture has
|
||||||
|
// no lstat syscall at all, so modernc routes through fstatat.
|
||||||
|
//
|
||||||
|
// So the second ABI was ~31 MB of an artifact that could not run
|
||||||
|
// anywhere. If modernc fixes it, adding 'x86_64' back here and
|
||||||
|
// to the native-code assertion in android-apk.yml is the whole
|
||||||
|
// change.
|
||||||
ndk {
|
ndk {
|
||||||
abiFilters 'arm64-v8a', 'x86_64'
|
abiFilters 'arm64-v8a'
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -71,6 +83,16 @@ android {
|
|||||||
}
|
}
|
||||||
debug {
|
debug {
|
||||||
debuggable true
|
debuggable true
|
||||||
|
// Its own application id, so it installs *beside* the release
|
||||||
|
// app rather than needing an uninstall to replace it. The two
|
||||||
|
// are signed by different certificates (the release one comes
|
||||||
|
// from a keystore CI holds), and Android's remedy for a
|
||||||
|
// certificate change is an uninstall -- which takes the
|
||||||
|
// user's library with it. This is also what makes the WebView
|
||||||
|
// inspectable on a real phone: `debuggable` is what turns on
|
||||||
|
// `setWebContentsDebuggingEnabled`, and `make android-inspect`
|
||||||
|
// drives it.
|
||||||
|
applicationIdSuffix ".dev"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -93,7 +115,6 @@ android {
|
|||||||
packagingOptions {
|
packagingOptions {
|
||||||
// Don't strip Go symbols in debug builds
|
// Don't strip Go symbols in debug builds
|
||||||
doNotStrip '*/arm64-v8a/libwails.so'
|
doNotStrip '*/arm64-v8a/libwails.so'
|
||||||
doNotStrip '*/x86_64/libwails.so'
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -15,6 +15,48 @@
|
|||||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Playback has to survive the screen locking, and that needs a
|
||||||
|
foreground service typed mediaPlayback rather than dataSync. The
|
||||||
|
type in the <service> element and the permission here must agree,
|
||||||
|
or startForeground throws at runtime.
|
||||||
|
-->
|
||||||
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Reading the user's music.
|
||||||
|
|
||||||
|
READ_MEDIA_AUDIO is the Android 13+ grant and READ_EXTERNAL_STORAGE
|
||||||
|
is its predecessor, capped so it is not requested where it no
|
||||||
|
longer applies. Both give access through **MediaStore**.
|
||||||
|
|
||||||
|
MANAGE_EXTERNAL_STORAGE is what gives access through the
|
||||||
|
*filesystem*, and this app needs it rather than merely preferring
|
||||||
|
it: `audio_files.file_path` is the primary key of ownership, the
|
||||||
|
scanner walks a directory the user chose, and every
|
||||||
|
GetFilePathsBy... query exists to hand a path to the player.
|
||||||
|
MediaStore offers no stable directory to walk and no way to write
|
||||||
|
a tag back in place, so the alternative is not "more work" but a
|
||||||
|
different application.
|
||||||
|
|
||||||
|
It is a Play-restricted permission, granted on a Settings screen
|
||||||
|
rather than in a dialog. That is acceptable *here* only because
|
||||||
|
this app is distributed as an APK through the package registry and
|
||||||
|
not through Play — see docs/android-release.md. If it ever targets
|
||||||
|
Play, this is the line that has to go, and plan 016 says what
|
||||||
|
would replace it.
|
||||||
|
-->
|
||||||
|
<uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
|
||||||
|
<uses-permission
|
||||||
|
android:name="android.permission.READ_EXTERNAL_STORAGE"
|
||||||
|
android:maxSdkVersion="32" />
|
||||||
|
<uses-permission
|
||||||
|
android:name="android.permission.WRITE_EXTERNAL_STORAGE"
|
||||||
|
android:maxSdkVersion="29" />
|
||||||
|
<uses-permission
|
||||||
|
android:name="android.permission.MANAGE_EXTERNAL_STORAGE"
|
||||||
|
tools:ignore="ScopedStorage" />
|
||||||
|
|
||||||
<queries>
|
<queries>
|
||||||
<intent>
|
<intent>
|
||||||
<action android:name="android.media.action.IMAGE_CAPTURE" />
|
<action android:name="android.media.action.IMAGE_CAPTURE" />
|
||||||
@@ -54,10 +96,16 @@
|
|||||||
android:resource="@xml/file_paths" />
|
android:resource="@xml/file_paths" />
|
||||||
</provider>
|
</provider>
|
||||||
|
|
||||||
|
<!--
|
||||||
|
mediaPlayback, not the scaffold's dataSync: this app's reason
|
||||||
|
for staying alive in the background is that a song is
|
||||||
|
playing, and Android matches the declared type against what
|
||||||
|
the service actually does.
|
||||||
|
-->
|
||||||
<service
|
<service
|
||||||
android:name=".WailsForegroundService"
|
android:name=".WailsForegroundService"
|
||||||
android:exported="false"
|
android:exported="false"
|
||||||
android:foregroundServiceType="dataSync" />
|
android:foregroundServiceType="mediaPlayback" />
|
||||||
</application>
|
</application>
|
||||||
|
|
||||||
</manifest>
|
</manifest>
|
||||||
|
|||||||
@@ -11,10 +11,13 @@ import android.net.ConnectivityManager;
|
|||||||
import android.net.Network;
|
import android.net.Network;
|
||||||
import android.net.NetworkCapabilities;
|
import android.net.NetworkCapabilities;
|
||||||
import android.net.Uri;
|
import android.net.Uri;
|
||||||
|
import android.Manifest;
|
||||||
import android.os.BatteryManager;
|
import android.os.BatteryManager;
|
||||||
import android.os.Build;
|
import android.os.Build;
|
||||||
import android.os.Bundle;
|
import android.os.Bundle;
|
||||||
|
import android.os.Environment;
|
||||||
import android.os.PowerManager;
|
import android.os.PowerManager;
|
||||||
|
import android.provider.Settings;
|
||||||
import android.content.pm.PackageManager;
|
import android.content.pm.PackageManager;
|
||||||
import android.graphics.Bitmap;
|
import android.graphics.Bitmap;
|
||||||
import android.graphics.BitmapFactory;
|
import android.graphics.BitmapFactory;
|
||||||
@@ -28,9 +31,16 @@ import android.webkit.WebSettings;
|
|||||||
import android.webkit.WebView;
|
import android.webkit.WebView;
|
||||||
import android.webkit.WebViewClient;
|
import android.webkit.WebViewClient;
|
||||||
|
|
||||||
|
import android.view.View;
|
||||||
|
|
||||||
import androidx.annotation.Nullable;
|
import androidx.annotation.Nullable;
|
||||||
import androidx.appcompat.app.AppCompatActivity;
|
import androidx.appcompat.app.AppCompatActivity;
|
||||||
import androidx.core.content.FileProvider;
|
import androidx.core.content.FileProvider;
|
||||||
|
import androidx.core.graphics.Insets;
|
||||||
|
import androidx.core.view.ViewCompat;
|
||||||
|
import androidx.core.view.WindowInsetsCompat;
|
||||||
|
import androidx.core.view.WindowCompat;
|
||||||
|
import androidx.core.view.WindowInsetsControllerCompat;
|
||||||
import androidx.webkit.WebViewAssetLoader;
|
import androidx.webkit.WebViewAssetLoader;
|
||||||
|
|
||||||
import org.json.JSONObject;
|
import org.json.JSONObject;
|
||||||
@@ -85,10 +95,18 @@ public class MainActivity extends AppCompatActivity {
|
|||||||
super.onCreate(savedInstanceState);
|
super.onCreate(savedInstanceState);
|
||||||
setContentView(R.layout.activity_main);
|
setContentView(R.layout.activity_main);
|
||||||
|
|
||||||
|
// Before anything renders: the page is laid out inside the
|
||||||
|
// window, and on Android 15 the window is the whole screen.
|
||||||
|
applyWindowInsets();
|
||||||
|
|
||||||
// Initialize the native Go library
|
// Initialize the native Go library
|
||||||
bridge = new WailsBridge(this);
|
bridge = new WailsBridge(this);
|
||||||
bridge.initialize();
|
bridge.initialize();
|
||||||
|
|
||||||
|
// Ask for access to the user's music before the frontend has
|
||||||
|
// anything to say about it. See ensureStorageAccess().
|
||||||
|
ensureStorageAccess();
|
||||||
|
|
||||||
// Set up WebView
|
// Set up WebView
|
||||||
setupWebView();
|
setupWebView();
|
||||||
|
|
||||||
@@ -96,6 +114,78 @@ public class MainActivity extends AppCompatActivity {
|
|||||||
loadApplication();
|
loadApplication();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Obtain access to the user's music.
|
||||||
|
*
|
||||||
|
* <p>This app is a library manager: its database is keyed on file
|
||||||
|
* paths, its scanner walks a directory the user chose, and its tag
|
||||||
|
* writer rewrites files in place. MediaStore offers none of those,
|
||||||
|
* so the app holds MANAGE_EXTERNAL_STORAGE — which is granted on a
|
||||||
|
* Settings screen rather than in a dialog, and therefore cannot be
|
||||||
|
* requested with requestPermissions().
|
||||||
|
*
|
||||||
|
* <p>The screen is opened on every cold start until access exists,
|
||||||
|
* because without it the app can see nothing at all and there is no
|
||||||
|
* degraded mode worth offering. Returning from it lands in
|
||||||
|
* onResume, which re-checks and tells the frontend.
|
||||||
|
*
|
||||||
|
* <p>Below Android 11 there is no all-files concept and plain
|
||||||
|
* READ_EXTERNAL_STORAGE is both sufficient and a normal dialog.
|
||||||
|
*/
|
||||||
|
private void ensureStorageAccess() {
|
||||||
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
|
||||||
|
if (Environment.isExternalStorageManager()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// The per-app screen is the one that can actually grant it.
|
||||||
|
// A few OEM builds do not implement it, so fall back to the
|
||||||
|
// global list rather than leaving the user with nothing.
|
||||||
|
try {
|
||||||
|
startActivity(new Intent(
|
||||||
|
Settings.ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION,
|
||||||
|
Uri.parse("package:" + getPackageName())));
|
||||||
|
} catch (Exception e) {
|
||||||
|
Log.w(TAG, "per-app all-files screen unavailable: " + e.getMessage());
|
||||||
|
try {
|
||||||
|
startActivity(new Intent(Settings.ACTION_MANAGE_ALL_FILES_ACCESS_PERMISSION));
|
||||||
|
} catch (Exception e2) {
|
||||||
|
Log.w(TAG, "no all-files settings screen at all: " + e2.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (checkSelfPermission(Manifest.permission.READ_EXTERNAL_STORAGE)
|
||||||
|
!= PackageManager.PERMISSION_GRANTED) {
|
||||||
|
requestPermissions(new String[]{Manifest.permission.READ_EXTERNAL_STORAGE}, 1010);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether the app can currently read the user's music, by the same
|
||||||
|
* test the Go side uses.
|
||||||
|
*/
|
||||||
|
private boolean hasStorageAccess() {
|
||||||
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
|
||||||
|
return Environment.isExternalStorageManager();
|
||||||
|
}
|
||||||
|
return checkSelfPermission(Manifest.permission.READ_EXTERNAL_STORAGE)
|
||||||
|
== PackageManager.PERMISSION_GRANTED;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Tell the frontend whether music is readable. Emitted on resume
|
||||||
|
* rather than only at startup, because the grant happens on a
|
||||||
|
* Settings screen in another task and the way back is a resume.
|
||||||
|
*/
|
||||||
|
private void emitStorageAccess() {
|
||||||
|
if (bridge == null) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
bridge.emitEvent("android:storageAccess",
|
||||||
|
"{\"granted\":" + (hasStorageAccess() ? "true" : "false") + "}");
|
||||||
|
}
|
||||||
|
|
||||||
@SuppressLint("SetJavaScriptEnabled")
|
@SuppressLint("SetJavaScriptEnabled")
|
||||||
private void setupWebView() {
|
private void setupWebView() {
|
||||||
webView = findViewById(R.id.webview);
|
webView = findViewById(R.id.webview);
|
||||||
@@ -768,6 +858,9 @@ public class MainActivity extends AppCompatActivity {
|
|||||||
if (bridge != null) {
|
if (bridge != null) {
|
||||||
bridge.onResume();
|
bridge.onResume();
|
||||||
}
|
}
|
||||||
|
// The all-files grant happens on a Settings screen in another
|
||||||
|
// task, so a resume is how the app finds out it was given.
|
||||||
|
emitStorageAccess();
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
@@ -810,8 +903,61 @@ public class MainActivity extends AppCompatActivity {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keep the web content inside the safe area.
|
||||||
|
*
|
||||||
|
* <p>targetSdk 35 is Android 15, which lays every app out
|
||||||
|
* edge-to-edge and ignores the {@code statusBarColor} and
|
||||||
|
* {@code navigationBarColor} this app's theme still sets. The
|
||||||
|
* WebView is {@code match_parent}, so the page's bottom band -- the
|
||||||
|
* transport and, on a phone, the tab bar -- was drawn underneath the
|
||||||
|
* gesture bar and reported from a device as "I can't see the
|
||||||
|
* playback controls, they seem to be off screen".
|
||||||
|
*
|
||||||
|
* <p>No web-tier test can see this: a browser viewport has no system
|
||||||
|
* bars, so the phone specs at 390x844 render a shell that fits
|
||||||
|
* while the device does not.
|
||||||
|
*
|
||||||
|
* <p>The insets are applied as padding and the window insets are
|
||||||
|
* returned rather than consumed, so the WebView is laid out inside
|
||||||
|
* them. {@code ime()} is in the mask because the same reasoning
|
||||||
|
* covers the keyboard: a focused search box that the keyboard
|
||||||
|
* covers is the same bug one surface over.
|
||||||
|
*/
|
||||||
|
private void applyWindowInsets() {
|
||||||
|
final View container = findViewById(R.id.main_container);
|
||||||
|
|
||||||
|
if (container == null) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
ViewCompat.setOnApplyWindowInsetsListener(container, (view, windowInsets) -> {
|
||||||
|
Insets insets = windowInsets.getInsets(
|
||||||
|
WindowInsetsCompat.Type.systemBars()
|
||||||
|
| WindowInsetsCompat.Type.displayCutout()
|
||||||
|
| WindowInsetsCompat.Type.ime());
|
||||||
|
|
||||||
|
view.setPadding(insets.left, insets.top, insets.right, insets.bottom);
|
||||||
|
|
||||||
|
return windowInsets;
|
||||||
|
});
|
||||||
|
|
||||||
|
// The padded band shows the window background, which is dark
|
||||||
|
// (this app's own default ramp is black), so the system's icons
|
||||||
|
// have to be the light set or they vanish into it. The theme is
|
||||||
|
// DayNight and would otherwise ask for dark icons in light mode.
|
||||||
|
WindowInsetsControllerCompat controller =
|
||||||
|
WindowCompat.getInsetsController(getWindow(), getWindow().getDecorView());
|
||||||
|
|
||||||
|
controller.setAppearanceLightStatusBars(false);
|
||||||
|
controller.setAppearanceLightNavigationBars(false);
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void onBackPressed() {
|
public void onBackPressed() {
|
||||||
|
// The frontend records every navigation as a history entry, so
|
||||||
|
// this is the app's own back stack: `canGoBack()` is false only
|
||||||
|
// at the launch entry, which is where back should leave.
|
||||||
if (webView != null && webView.canGoBack()) {
|
if (webView != null && webView.canGoBack()) {
|
||||||
webView.goBack();
|
webView.goBack();
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -81,6 +81,9 @@ public class WailsBridge {
|
|||||||
System.loadLibrary("wails");
|
System.loadLibrary("wails");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The live bridge, for in-process components that are not given one. */
|
||||||
|
private static volatile WailsBridge instance;
|
||||||
|
|
||||||
private final Activity activity;
|
private final Activity activity;
|
||||||
private final Handler mainHandler = new Handler(Looper.getMainLooper());
|
private final Handler mainHandler = new Handler(Looper.getMainLooper());
|
||||||
private WebView webView;
|
private WebView webView;
|
||||||
@@ -122,6 +125,7 @@ public class WailsBridge {
|
|||||||
|
|
||||||
public WailsBridge(Activity activity) {
|
public WailsBridge(Activity activity) {
|
||||||
this.activity = activity;
|
this.activity = activity;
|
||||||
|
instance = this;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -210,6 +214,19 @@ public class WailsBridge {
|
|||||||
if (initialized) nativeEmitEvent(name, json);
|
if (initialized) nativeEmitEvent(name, json);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Emit an event from a component that holds no bridge reference —
|
||||||
|
* {@link WailsForegroundService}, which Android constructs itself. It is a
|
||||||
|
* static hop rather than a binder because the service runs in this same
|
||||||
|
* process; before the bridge exists (or after it is gone) the event is
|
||||||
|
* dropped, which is the same thing {@link #emitEvent} does when the native
|
||||||
|
* library has not been initialized.
|
||||||
|
*/
|
||||||
|
public static void emitFromService(String name, String json) {
|
||||||
|
WailsBridge b = instance;
|
||||||
|
if (b != null) b.emitEvent(name, json);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Serve an asset from the Go asset server
|
* Serve an asset from the Go asset server
|
||||||
*/
|
*/
|
||||||
@@ -1193,7 +1210,16 @@ public class WailsBridge {
|
|||||||
i.setAction(WailsForegroundService.ACTION_START);
|
i.setAction(WailsForegroundService.ACTION_START);
|
||||||
i.putExtra("title", title);
|
i.putExtra("title", title);
|
||||||
i.putExtra("text", text);
|
i.putExtra("text", text);
|
||||||
ContextCompat.startForegroundService(activity, i);
|
// The whole document, for the media service: seven extras
|
||||||
|
// would be seven chances for the two sides to disagree about
|
||||||
|
// a key, and the service already has to parse JSON for the
|
||||||
|
// fields the scaffold's title/text pair cannot carry.
|
||||||
|
i.putExtra("payload", json);
|
||||||
|
if (WailsForegroundService.running) {
|
||||||
|
activity.startService(i);
|
||||||
|
} else {
|
||||||
|
ContextCompat.startForegroundService(activity, i);
|
||||||
|
}
|
||||||
emitEvent("android:foregroundService", "{\"running\":true}");
|
emitEvent("android:foregroundService", "{\"running\":true}");
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
Log.e(TAG, "startForegroundService failed", e);
|
Log.e(TAG, "startForegroundService failed", e);
|
||||||
|
|||||||
@@ -4,66 +4,545 @@ import android.app.Notification;
|
|||||||
import android.app.NotificationChannel;
|
import android.app.NotificationChannel;
|
||||||
import android.app.NotificationManager;
|
import android.app.NotificationManager;
|
||||||
import android.app.PendingIntent;
|
import android.app.PendingIntent;
|
||||||
|
import android.content.BroadcastReceiver;
|
||||||
|
import android.content.Context;
|
||||||
import android.content.Intent;
|
import android.content.Intent;
|
||||||
|
import android.content.IntentFilter;
|
||||||
import android.content.pm.ServiceInfo;
|
import android.content.pm.ServiceInfo;
|
||||||
|
import android.graphics.Bitmap;
|
||||||
|
import android.graphics.BitmapFactory;
|
||||||
|
import android.media.AudioAttributes;
|
||||||
|
import android.media.AudioFocusRequest;
|
||||||
|
import android.media.AudioManager;
|
||||||
|
import android.media.MediaMetadata;
|
||||||
|
import android.media.session.MediaSession;
|
||||||
|
import android.media.session.PlaybackState;
|
||||||
import android.os.Build;
|
import android.os.Build;
|
||||||
|
import android.os.Handler;
|
||||||
import android.os.IBinder;
|
import android.os.IBinder;
|
||||||
|
import android.os.Looper;
|
||||||
|
import android.util.Log;
|
||||||
|
|
||||||
import androidx.annotation.Nullable;
|
import androidx.annotation.Nullable;
|
||||||
import androidx.core.app.NotificationCompat;
|
|
||||||
|
import org.json.JSONObject;
|
||||||
|
|
||||||
|
import java.util.concurrent.ExecutorService;
|
||||||
|
import java.util.concurrent.Executors;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A minimal started foreground service. It does no work of its own — its purpose
|
* The foreground service that keeps playback alive with the screen off, and
|
||||||
* is to keep the app's process alive (with the required ongoing notification) so
|
* the app's whole media-control surface: a {@link MediaSession} for the lock
|
||||||
* the developer's Go goroutines keep running while the app is backgrounded,
|
* screen and headset buttons, a transport notification, and audio focus.
|
||||||
* which Android would otherwise be free to kill. Start it from
|
*
|
||||||
* {@link WailsBridge#startForegroundService(String)} and stop it with
|
* <p>The scaffold shipped this as a generic "keep the process alive" service
|
||||||
* {@link WailsBridge#stopForegroundService()}.
|
* typed {@code dataSync}. YellowJacket's reason for staying alive in the
|
||||||
|
* background is that a song is playing, so it is {@code mediaPlayback} — the
|
||||||
|
* manifest and {@code startForeground} must agree on that or the call throws.
|
||||||
|
*
|
||||||
|
* <p>It is driven entirely from Go. {@code backend/mediacontrols/android.go}
|
||||||
|
* pushes a JSON payload through
|
||||||
|
* {@link WailsBridge#startForegroundService(String)}, and every command the
|
||||||
|
* user gives here — a notification button, the lock screen, a headset, or the
|
||||||
|
* OS taking audio focus away — goes back the other way as a
|
||||||
|
* {@code yj:media:command} event. Nothing about playback is decided here: this
|
||||||
|
* class renders state and reports intent.
|
||||||
*/
|
*/
|
||||||
public class WailsForegroundService extends android.app.Service {
|
public class WailsForegroundService extends android.app.Service {
|
||||||
public static final String ACTION_START = "com.wails.app.FGS_START";
|
public static final String ACTION_START = "com.wails.app.FGS_START";
|
||||||
private static final String CHANNEL_ID = "wails_foreground";
|
|
||||||
|
// Transport actions, delivered to ourselves by the notification's
|
||||||
|
// PendingIntents. getService rather than a broadcast: a receiver would
|
||||||
|
// have to be exported or registered, and this service is already the
|
||||||
|
// thing that has to be running for any of them to be meaningful.
|
||||||
|
private static final String ACTION_PLAY = "com.wails.app.MEDIA_PLAY";
|
||||||
|
private static final String ACTION_PAUSE = "com.wails.app.MEDIA_PAUSE";
|
||||||
|
private static final String ACTION_NEXT = "com.wails.app.MEDIA_NEXT";
|
||||||
|
private static final String ACTION_PREVIOUS = "com.wails.app.MEDIA_PREVIOUS";
|
||||||
|
|
||||||
|
private static final String TAG = "WailsMedia";
|
||||||
|
private static final String CHANNEL_ID = "yellowjacket_playback";
|
||||||
private static final int NOTIFICATION_ID = 0x57A1; // "WAI"
|
private static final int NOTIFICATION_ID = 0x57A1; // "WAI"
|
||||||
|
private static final String COMMAND_EVENT = "yj:media:command";
|
||||||
|
|
||||||
|
/** Cover art is decoded down to this, which is larger than any lock screen. */
|
||||||
|
private static final int ART_MAX_PX = 512;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether an instance is alive. {@link WailsBridge} reads it to decide
|
||||||
|
* between startForegroundService and startService: from Android 12 an app
|
||||||
|
* in the background may not <em>start</em> a foreground service, but it
|
||||||
|
* may go on delivering intents to one it already has — and every update
|
||||||
|
* after the first (a track change with the screen off, most of them) is
|
||||||
|
* exactly that case.
|
||||||
|
*/
|
||||||
|
static volatile boolean running = false;
|
||||||
|
|
||||||
|
private final Handler mainHandler = new Handler(Looper.getMainLooper());
|
||||||
|
private final ExecutorService artExecutor = Executors.newSingleThreadExecutor();
|
||||||
|
|
||||||
|
private MediaSession session;
|
||||||
|
private AudioManager audioManager;
|
||||||
|
private AudioFocusRequest focusRequest; // API 26+ only.
|
||||||
|
private AudioManager.OnAudioFocusChangeListener focusListener;
|
||||||
|
|
||||||
|
private String title = "";
|
||||||
|
private String artist = "";
|
||||||
|
private String album = "";
|
||||||
|
private String artPath = "";
|
||||||
|
private long durationMs = 0;
|
||||||
|
private long positionMs = 0;
|
||||||
|
private boolean playing = false;
|
||||||
|
|
||||||
|
private Bitmap art;
|
||||||
|
|
||||||
|
private boolean hasFocus = false;
|
||||||
|
/**
|
||||||
|
* Whether *we* paused because focus went away. Only then does regaining it
|
||||||
|
* resume: a user who paused during a phone call did not ask us to start
|
||||||
|
* again when it ended.
|
||||||
|
*/
|
||||||
|
private boolean pausedByFocusLoss = false;
|
||||||
|
|
||||||
|
private boolean noisyRegistered = false;
|
||||||
|
|
||||||
|
/** Headphones pulled out. Anything else and the room hears the album. */
|
||||||
|
private final BroadcastReceiver noisyReceiver = new BroadcastReceiver() {
|
||||||
|
@Override
|
||||||
|
public void onReceive(Context context, Intent intent) {
|
||||||
|
if (AudioManager.ACTION_AUDIO_BECOMING_NOISY.equals(intent.getAction())) {
|
||||||
|
emitCommand("pause");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onCreate() {
|
||||||
|
super.onCreate();
|
||||||
|
running = true;
|
||||||
|
audioManager = (AudioManager) getSystemService(AUDIO_SERVICE);
|
||||||
|
createChannel();
|
||||||
|
createSession();
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public int onStartCommand(Intent intent, int flags, int startId) {
|
public int onStartCommand(Intent intent, int flags, int startId) {
|
||||||
String title = "Wails";
|
String action = intent == null ? null : intent.getAction();
|
||||||
String text = "Running in the background";
|
|
||||||
if (intent != null) {
|
if (ACTION_PLAY.equals(action)) {
|
||||||
if (intent.getStringExtra("title") != null) title = intent.getStringExtra("title");
|
emitCommand("play");
|
||||||
if (intent.getStringExtra("text") != null) text = intent.getStringExtra("text");
|
} else if (ACTION_PAUSE.equals(action)) {
|
||||||
|
emitCommand("pause");
|
||||||
|
} else if (ACTION_NEXT.equals(action)) {
|
||||||
|
emitCommand("next");
|
||||||
|
} else if (ACTION_PREVIOUS.equals(action)) {
|
||||||
|
emitCommand("previous");
|
||||||
|
} else if (intent != null) {
|
||||||
|
applyPayload(intent);
|
||||||
}
|
}
|
||||||
|
|
||||||
NotificationManager nm = (NotificationManager) getSystemService(NOTIFICATION_SERVICE);
|
// Unconditionally, on every path: a service started with
|
||||||
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
|
// startForegroundService that returns from onStartCommand without
|
||||||
NotificationChannel ch = new NotificationChannel(
|
// calling startForeground is killed with a
|
||||||
CHANNEL_ID, "Background work", NotificationManager.IMPORTANCE_LOW);
|
// ForegroundServiceDidNotStartInTimeException.
|
||||||
nm.createNotificationChannel(ch);
|
goForeground();
|
||||||
|
|
||||||
|
return START_STICKY;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Read the state Go pushed. "payload" is the whole JSON document; the
|
||||||
|
* title/text extras are the scaffold's original contract and are kept as a
|
||||||
|
* fallback so a non-media caller still gets a sensible notification.
|
||||||
|
*/
|
||||||
|
private void applyPayload(Intent intent) {
|
||||||
|
String payload = intent.getStringExtra("payload");
|
||||||
|
if (payload == null || payload.isEmpty()) {
|
||||||
|
if (intent.getStringExtra("title") != null) {
|
||||||
|
title = intent.getStringExtra("title");
|
||||||
|
}
|
||||||
|
if (intent.getStringExtra("text") != null) {
|
||||||
|
artist = intent.getStringExtra("text");
|
||||||
|
}
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
PendingIntent contentIntent = null;
|
try {
|
||||||
Intent launch = getPackageManager().getLaunchIntentForPackage(getPackageName());
|
JSONObject o = new JSONObject(payload);
|
||||||
if (launch != null) {
|
title = o.optString("title", "");
|
||||||
int piFlags = Build.VERSION.SDK_INT >= Build.VERSION_CODES.M
|
artist = o.optString("artist", "");
|
||||||
? PendingIntent.FLAG_IMMUTABLE : 0;
|
album = o.optString("album", "");
|
||||||
contentIntent = PendingIntent.getActivity(this, 0, launch, piFlags);
|
durationMs = o.optLong("durationSec", 0) * 1000L;
|
||||||
|
positionMs = o.optLong("positionSec", 0) * 1000L;
|
||||||
|
playing = "playing".equals(o.optString("state", "paused"));
|
||||||
|
|
||||||
|
String path = o.optString("artPath", "");
|
||||||
|
if (!path.equals(artPath)) {
|
||||||
|
artPath = path;
|
||||||
|
loadArt(path);
|
||||||
|
}
|
||||||
|
} catch (Exception e) {
|
||||||
|
Log.e(TAG, "bad media payload", e);
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
Notification n = new NotificationCompat.Builder(this, CHANNEL_ID)
|
if (playing) {
|
||||||
.setSmallIcon(android.R.drawable.ic_popup_sync)
|
requestFocus();
|
||||||
.setContentTitle(title)
|
registerNoisy();
|
||||||
.setContentText(text)
|
} else {
|
||||||
.setOngoing(true)
|
unregisterNoisy();
|
||||||
.setContentIntent(contentIntent)
|
}
|
||||||
|
|
||||||
|
updateSession();
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- MediaSession ------------------------------------------------------
|
||||||
|
|
||||||
|
private void createSession() {
|
||||||
|
session = new MediaSession(this, "YellowJacket");
|
||||||
|
session.setFlags(MediaSession.FLAG_HANDLES_MEDIA_BUTTONS
|
||||||
|
| MediaSession.FLAG_HANDLES_TRANSPORT_CONTROLS);
|
||||||
|
session.setCallback(new MediaSession.Callback() {
|
||||||
|
@Override
|
||||||
|
public void onPlay() {
|
||||||
|
emitCommand("play");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onPause() {
|
||||||
|
emitCommand("pause");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onStop() {
|
||||||
|
emitCommand("stop");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onSkipToNext() {
|
||||||
|
emitCommand("next");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onSkipToPrevious() {
|
||||||
|
emitCommand("previous");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onSeekTo(long pos) {
|
||||||
|
try {
|
||||||
|
JSONObject o = new JSONObject();
|
||||||
|
o.put("command", "seek");
|
||||||
|
o.put("positionSec", pos / 1000L);
|
||||||
|
WailsBridge.emitFromService(COMMAND_EVENT, o.toString());
|
||||||
|
} catch (Exception e) {
|
||||||
|
Log.e(TAG, "seek command failed", e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
session.setActive(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void updateSession() {
|
||||||
|
MediaMetadata.Builder meta = new MediaMetadata.Builder()
|
||||||
|
.putString(MediaMetadata.METADATA_KEY_TITLE, title)
|
||||||
|
.putString(MediaMetadata.METADATA_KEY_ARTIST, artist)
|
||||||
|
.putString(MediaMetadata.METADATA_KEY_ALBUM, album)
|
||||||
|
.putLong(MediaMetadata.METADATA_KEY_DURATION, durationMs);
|
||||||
|
if (art != null) {
|
||||||
|
meta.putBitmap(MediaMetadata.METADATA_KEY_ALBUM_ART, art);
|
||||||
|
}
|
||||||
|
session.setMetadata(meta.build());
|
||||||
|
|
||||||
|
// The position is an anchor, not a clock: the state carries the
|
||||||
|
// playback speed and the OS interpolates from here, which is why the
|
||||||
|
// Go side only pushes on a real state change or a seek.
|
||||||
|
PlaybackState state = new PlaybackState.Builder()
|
||||||
|
.setActions(PlaybackState.ACTION_PLAY
|
||||||
|
| PlaybackState.ACTION_PAUSE
|
||||||
|
| PlaybackState.ACTION_PLAY_PAUSE
|
||||||
|
| PlaybackState.ACTION_STOP
|
||||||
|
| PlaybackState.ACTION_SKIP_TO_NEXT
|
||||||
|
| PlaybackState.ACTION_SKIP_TO_PREVIOUS
|
||||||
|
| PlaybackState.ACTION_SEEK_TO)
|
||||||
|
.setState(playing ? PlaybackState.STATE_PLAYING : PlaybackState.STATE_PAUSED,
|
||||||
|
positionMs, playing ? 1.0f : 0.0f)
|
||||||
.build();
|
.build();
|
||||||
|
session.setPlaybackState(state);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Notification ------------------------------------------------------
|
||||||
|
|
||||||
|
private void createChannel() {
|
||||||
|
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
NotificationManager nm = (NotificationManager) getSystemService(NOTIFICATION_SERVICE);
|
||||||
|
// LOW: a transport notification is a control surface, not news, and
|
||||||
|
// IMPORTANCE_DEFAULT would make a sound on every track change.
|
||||||
|
NotificationChannel ch = new NotificationChannel(
|
||||||
|
CHANNEL_ID, "Playback", NotificationManager.IMPORTANCE_LOW);
|
||||||
|
ch.setShowBadge(false);
|
||||||
|
nm.createNotificationChannel(ch);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void goForeground() {
|
||||||
|
Notification n = buildNotification();
|
||||||
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
|
||||||
startForeground(NOTIFICATION_ID, n, ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC);
|
startForeground(NOTIFICATION_ID, n, ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK);
|
||||||
} else {
|
} else {
|
||||||
startForeground(NOTIFICATION_ID, n);
|
startForeground(NOTIFICATION_ID, n);
|
||||||
}
|
}
|
||||||
// Restart if the OS kills us while still wanted.
|
}
|
||||||
return START_STICKY;
|
|
||||||
|
@SuppressWarnings("deprecation")
|
||||||
|
private Notification buildNotification() {
|
||||||
|
Notification.Builder b = Build.VERSION.SDK_INT >= Build.VERSION_CODES.O
|
||||||
|
? new Notification.Builder(this, CHANNEL_ID)
|
||||||
|
: new Notification.Builder(this);
|
||||||
|
|
||||||
|
b.setSmallIcon(android.R.drawable.ic_media_play)
|
||||||
|
.setContentTitle(title.isEmpty() ? getString(R.string.app_name) : title)
|
||||||
|
.setContentText(artist)
|
||||||
|
.setSubText(album)
|
||||||
|
.setOngoing(playing)
|
||||||
|
.setVisibility(Notification.VISIBILITY_PUBLIC)
|
||||||
|
.setContentIntent(launchIntent());
|
||||||
|
|
||||||
|
if (art != null) {
|
||||||
|
b.setLargeIcon(art);
|
||||||
|
}
|
||||||
|
|
||||||
|
b.addAction(new Notification.Action.Builder(
|
||||||
|
android.R.drawable.ic_media_previous, "Previous",
|
||||||
|
transportIntent(ACTION_PREVIOUS, 1)).build());
|
||||||
|
b.addAction(playing
|
||||||
|
? new Notification.Action.Builder(android.R.drawable.ic_media_pause, "Pause",
|
||||||
|
transportIntent(ACTION_PAUSE, 2)).build()
|
||||||
|
: new Notification.Action.Builder(android.R.drawable.ic_media_play, "Play",
|
||||||
|
transportIntent(ACTION_PLAY, 3)).build());
|
||||||
|
b.addAction(new Notification.Action.Builder(
|
||||||
|
android.R.drawable.ic_media_next, "Next",
|
||||||
|
transportIntent(ACTION_NEXT, 4)).build());
|
||||||
|
|
||||||
|
Notification.MediaStyle style = new Notification.MediaStyle()
|
||||||
|
.setShowActionsInCompactView(0, 1, 2);
|
||||||
|
if (session != null) {
|
||||||
|
style.setMediaSession(session.getSessionToken());
|
||||||
|
}
|
||||||
|
b.setStyle(style);
|
||||||
|
|
||||||
|
return b.build();
|
||||||
|
}
|
||||||
|
|
||||||
|
private PendingIntent transportIntent(String action, int requestCode) {
|
||||||
|
Intent i = new Intent(this, WailsForegroundService.class).setAction(action);
|
||||||
|
return PendingIntent.getService(this, requestCode, i, pendingIntentFlags());
|
||||||
|
}
|
||||||
|
|
||||||
|
private PendingIntent launchIntent() {
|
||||||
|
Intent launch = getPackageManager().getLaunchIntentForPackage(getPackageName());
|
||||||
|
if (launch == null) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return PendingIntent.getActivity(this, 0, launch, pendingIntentFlags());
|
||||||
|
}
|
||||||
|
|
||||||
|
private int pendingIntentFlags() {
|
||||||
|
// Mandatory from S, unavailable before M.
|
||||||
|
return Build.VERSION.SDK_INT >= Build.VERSION_CODES.M
|
||||||
|
? PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT
|
||||||
|
: PendingIntent.FLAG_UPDATE_CURRENT;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Cover art ---------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decode the cover off the main thread and redraw when it lands. A track
|
||||||
|
* change must not wait on a JPEG, and the notification is correct without
|
||||||
|
* one — it simply has no image until this returns.
|
||||||
|
*/
|
||||||
|
private void loadArt(final String path) {
|
||||||
|
art = null;
|
||||||
|
if (path == null || path.isEmpty()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
artExecutor.execute(() -> {
|
||||||
|
Bitmap decoded = decodeScaled(path);
|
||||||
|
mainHandler.post(() -> {
|
||||||
|
// The track may have changed while we decoded.
|
||||||
|
if (!path.equals(artPath)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
art = decoded;
|
||||||
|
updateSession();
|
||||||
|
goForeground();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private Bitmap decodeScaled(String path) {
|
||||||
|
try {
|
||||||
|
BitmapFactory.Options bounds = new BitmapFactory.Options();
|
||||||
|
bounds.inJustDecodeBounds = true;
|
||||||
|
BitmapFactory.decodeFile(path, bounds);
|
||||||
|
|
||||||
|
int longest = Math.max(bounds.outWidth, bounds.outHeight);
|
||||||
|
int sample = 1;
|
||||||
|
while (longest / sample > ART_MAX_PX) {
|
||||||
|
sample *= 2;
|
||||||
|
}
|
||||||
|
|
||||||
|
BitmapFactory.Options opts = new BitmapFactory.Options();
|
||||||
|
opts.inSampleSize = sample;
|
||||||
|
return BitmapFactory.decodeFile(path, opts);
|
||||||
|
} catch (Throwable t) {
|
||||||
|
// OutOfMemoryError included: a missing cover is not a crash.
|
||||||
|
Log.w(TAG, "cover art decode failed: " + path, t);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Audio focus -------------------------------------------------------
|
||||||
|
|
||||||
|
private void requestFocus() {
|
||||||
|
if (hasFocus || audioManager == null) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (focusListener == null) {
|
||||||
|
focusListener = this::onFocusChange;
|
||||||
|
}
|
||||||
|
|
||||||
|
int result;
|
||||||
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
|
||||||
|
AudioAttributes attrs = new AudioAttributes.Builder()
|
||||||
|
.setUsage(AudioAttributes.USAGE_MEDIA)
|
||||||
|
.setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)
|
||||||
|
.build();
|
||||||
|
// No setWillPauseWhenDucked: from Oreo the framework ducks us
|
||||||
|
// itself and reports no CAN_DUCK loss, so the Go-side duck below
|
||||||
|
// is a pre-Oreo path. Asking to be told instead would mean
|
||||||
|
// pausing for every notification tone.
|
||||||
|
focusRequest = new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
|
||||||
|
.setAudioAttributes(attrs)
|
||||||
|
.setOnAudioFocusChangeListener(focusListener, mainHandler)
|
||||||
|
.build();
|
||||||
|
result = audioManager.requestAudioFocus(focusRequest);
|
||||||
|
} else {
|
||||||
|
result = requestFocusLegacy();
|
||||||
|
}
|
||||||
|
|
||||||
|
hasFocus = result == AudioManager.AUDIOFOCUS_REQUEST_GRANTED;
|
||||||
|
}
|
||||||
|
|
||||||
|
@SuppressWarnings("deprecation")
|
||||||
|
private int requestFocusLegacy() {
|
||||||
|
return audioManager.requestAudioFocus(focusListener,
|
||||||
|
AudioManager.STREAM_MUSIC, AudioManager.AUDIOFOCUS_GAIN);
|
||||||
|
}
|
||||||
|
|
||||||
|
@SuppressWarnings("deprecation")
|
||||||
|
private void abandonFocus() {
|
||||||
|
if (!hasFocus || audioManager == null) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O && focusRequest != null) {
|
||||||
|
audioManager.abandonAudioFocusRequest(focusRequest);
|
||||||
|
} else {
|
||||||
|
audioManager.abandonAudioFocus(focusListener);
|
||||||
|
}
|
||||||
|
hasFocus = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void onFocusChange(int change) {
|
||||||
|
switch (change) {
|
||||||
|
case AudioManager.AUDIOFOCUS_LOSS:
|
||||||
|
// Someone else owns the output now, for good.
|
||||||
|
hasFocus = false;
|
||||||
|
pausedByFocusLoss = false;
|
||||||
|
emitCommand("pause");
|
||||||
|
break;
|
||||||
|
case AudioManager.AUDIOFOCUS_LOSS_TRANSIENT:
|
||||||
|
// A phone call. Remember that the pause was ours to undo.
|
||||||
|
pausedByFocusLoss = playing;
|
||||||
|
emitCommand("pause");
|
||||||
|
break;
|
||||||
|
case AudioManager.AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK:
|
||||||
|
emitDuck(true);
|
||||||
|
break;
|
||||||
|
case AudioManager.AUDIOFOCUS_GAIN:
|
||||||
|
hasFocus = true;
|
||||||
|
emitDuck(false);
|
||||||
|
if (pausedByFocusLoss) {
|
||||||
|
pausedByFocusLoss = false;
|
||||||
|
emitCommand("play");
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Noisy (headphones) ------------------------------------------------
|
||||||
|
|
||||||
|
private void registerNoisy() {
|
||||||
|
if (noisyRegistered) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
registerReceiver(noisyReceiver,
|
||||||
|
new IntentFilter(AudioManager.ACTION_AUDIO_BECOMING_NOISY));
|
||||||
|
noisyRegistered = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void unregisterNoisy() {
|
||||||
|
if (!noisyRegistered) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
unregisterReceiver(noisyReceiver);
|
||||||
|
} catch (IllegalArgumentException ignored) {
|
||||||
|
// Already gone; nothing to undo.
|
||||||
|
}
|
||||||
|
noisyRegistered = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Talking to Go -----------------------------------------------------
|
||||||
|
|
||||||
|
private void emitCommand(String command) {
|
||||||
|
try {
|
||||||
|
JSONObject o = new JSONObject();
|
||||||
|
o.put("command", command);
|
||||||
|
WailsBridge.emitFromService(COMMAND_EVENT, o.toString());
|
||||||
|
} catch (Exception e) {
|
||||||
|
Log.e(TAG, "command emit failed: " + command, e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void emitDuck(boolean on) {
|
||||||
|
try {
|
||||||
|
JSONObject o = new JSONObject();
|
||||||
|
o.put("command", "duck");
|
||||||
|
o.put("on", on);
|
||||||
|
WailsBridge.emitFromService(COMMAND_EVENT, o.toString());
|
||||||
|
} catch (Exception e) {
|
||||||
|
Log.e(TAG, "duck emit failed", e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onDestroy() {
|
||||||
|
running = false;
|
||||||
|
unregisterNoisy();
|
||||||
|
abandonFocus();
|
||||||
|
if (session != null) {
|
||||||
|
session.setActive(false);
|
||||||
|
session.release();
|
||||||
|
session = null;
|
||||||
|
}
|
||||||
|
artExecutor.shutdownNow();
|
||||||
|
super.onDestroy();
|
||||||
}
|
}
|
||||||
|
|
||||||
@Nullable
|
@Nullable
|
||||||
|
|||||||
@@ -2,7 +2,12 @@
|
|||||||
<resources>
|
<resources>
|
||||||
<color name="wails_blue">#3574D4</color>
|
<color name="wails_blue">#3574D4</color>
|
||||||
<color name="wails_blue_dark">#2C5FB8</color>
|
<color name="wails_blue_dark">#2C5FB8</color>
|
||||||
<color name="wails_background">#1B2636</color>
|
<!-- The window background, which is what the launch screen shows and
|
||||||
|
what the system-bar padding leaves visible. Black rather than the
|
||||||
|
scaffold's blue-grey because this app's own default ramp is
|
||||||
|
black: a band of #1B2636 above and below it reads as the app
|
||||||
|
failing to fill the screen. -->
|
||||||
|
<color name="wails_background">#000000</color>
|
||||||
<color name="white">#FFFFFFFF</color>
|
<color name="white">#FFFFFFFF</color>
|
||||||
<color name="black">#FF000000</color>
|
<color name="black">#FF000000</color>
|
||||||
</resources>
|
</resources>
|
||||||
|
|||||||
+14
-1
@@ -177,12 +177,25 @@ func run(o opts) error {
|
|||||||
|
|
||||||
complete := svc.IndexImportComplete() && !errors.Is(err, errIncomplete)
|
complete := svc.IndexImportComplete() && !errors.Is(err, errIncomplete)
|
||||||
|
|
||||||
|
// Credits are maintenance, not part of any one mode. They come from
|
||||||
|
// a different dump, they are keyed on entities the catalog already
|
||||||
|
// holds, and a catalog built before the pass existed would otherwise
|
||||||
|
// only gain them from a rebuild — which re-downloads ~205 GB to
|
||||||
|
// re-derive rows it already has. Skipped when the import is not
|
||||||
|
// complete, because there is nothing to key them against yet.
|
||||||
|
creditsAdded := false
|
||||||
|
if complete {
|
||||||
|
creditsAdded = svc.EnsureArtistCredits(context.Background())
|
||||||
|
}
|
||||||
|
|
||||||
// "Changed" means there is something new worth publishing, so it is
|
// "Changed" means there is something new worth publishing, so it is
|
||||||
// only ever true for a finished import: a build stamps the listens
|
// only ever true for a finished import: a build stamps the listens
|
||||||
// series early, long before its rows are assembled, and reporting a
|
// series early, long before its rows are assembled, and reporting a
|
||||||
// change off that would be a lie about a half-built index.
|
// change off that would be a lie about a half-built index.
|
||||||
changed := complete &&
|
changed := complete &&
|
||||||
(svc.IndexBaselineSeries() != seriesBefore || chosen != modeRefresh)
|
(svc.IndexBaselineSeries() != seriesBefore ||
|
||||||
|
chosen != modeRefresh ||
|
||||||
|
creditsAdded)
|
||||||
|
|
||||||
report(logger, svc, chosen, complete, changed)
|
report(logger, svc, chosen, complete, changed)
|
||||||
|
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
_ "modernc.org/sqlite"
|
_ "modernc.org/sqlite"
|
||||||
|
|
||||||
"yellowjacket/backend/database"
|
"yellowjacket/backend/database"
|
||||||
|
"yellowjacket/backend/datamap"
|
||||||
"yellowjacket/backend/system"
|
"yellowjacket/backend/system"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -53,10 +54,19 @@ func TestRetireLibraryTables(t *testing.T) {
|
|||||||
CREATE TABLE recordings (id INTEGER PRIMARY KEY, title TEXT);
|
CREATE TABLE recordings (id INTEGER PRIMARY KEY, title TEXT);
|
||||||
`)
|
`)
|
||||||
|
|
||||||
// The symptom, before the repair: the schema cannot be applied over
|
// This used to assert the symptom -- that the schema cannot be
|
||||||
// a table whose shape has moved on.
|
// applied over a table whose shape has moved on -- because at the
|
||||||
if _, err := database.NewDB(logger); err == nil {
|
// time nothing repaired it and only this job did. The app-side
|
||||||
t.Fatal("expected the stale shape to fail to open; it did not")
|
// repair (backend/database/staleshape.go) now retires a stale
|
||||||
|
// non-authored table before applySchema meets it, so opening
|
||||||
|
// succeeds and the symptom no longer reproduces from here.
|
||||||
|
//
|
||||||
|
// That does not make retireLibraryTables redundant, and the rest of
|
||||||
|
// this test is why: the app-side repair only removes what is *stale*,
|
||||||
|
// while this database wants its library half gone entirely, healthy
|
||||||
|
// or not, because nothing here scans, plays or authors.
|
||||||
|
if _, err := database.NewDB(logger); err != nil {
|
||||||
|
t.Fatalf("the app-side repair should have opened this: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := retireLibraryTables(context.Background(), logger); err != nil {
|
if err := retireLibraryTables(context.Background(), logger); err != nil {
|
||||||
@@ -116,3 +126,186 @@ func count(t *testing.T, dbPath, query string) int {
|
|||||||
|
|
||||||
return n
|
return n
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestTheCatalogSurvivesAStaleShape is the accident written down.
|
||||||
|
//
|
||||||
|
// The app repairs a stale Cache table by dropping it: its catalog is
|
||||||
|
// downloaded, so a wrong shape costs about a minute of re-fetching and
|
||||||
|
// keeping it costs every Explore read. Applied here that rule is
|
||||||
|
// catastrophic — this database is what the artifact is *cut from*, so
|
||||||
|
// there is nothing to re-fetch and the only way back is the ~205 GB
|
||||||
|
// dump stream the /cache volume exists to avoid.
|
||||||
|
//
|
||||||
|
// It shipped without that distinction and dropped the real CI catalog
|
||||||
|
// on the first run:
|
||||||
|
//
|
||||||
|
// retiring a table ... table=explore_index
|
||||||
|
// reason="column entity_type is TEXT, schema declares INTEGER"
|
||||||
|
// index maintenance mode=build reason="no completed import yet"
|
||||||
|
//
|
||||||
|
// The mismatch was real: that database is deliberately kept in the
|
||||||
|
// older encoding, which `fix(indexexport): read an index older than the
|
||||||
|
// binary` exists to tolerate. So the shape will not match, every run,
|
||||||
|
// by design — and the catalog must survive it anyway.
|
||||||
|
func TestTheCatalogSurvivesAStaleShape(t *testing.T) {
|
||||||
|
logger := slog.New(slog.DiscardHandler)
|
||||||
|
|
||||||
|
t.Setenv("YJ_HOME", t.TempDir())
|
||||||
|
|
||||||
|
dataDir, err := system.GetUserDataDirPath()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve data dir: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dbPath := filepath.Join(dataDir, "yj.db")
|
||||||
|
|
||||||
|
if _, err := database.NewDB(logger); err != nil {
|
||||||
|
t.Fatalf("first open: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The shape the real index database is in: every current column,
|
||||||
|
// but the ids and the entity type still text. That is what the
|
||||||
|
// exporter's backward-compatibility fix tolerates, and it is what
|
||||||
|
// the repair saw and called stale.
|
||||||
|
exec(t, dbPath, `
|
||||||
|
DROP TABLE explore_index;
|
||||||
|
CREATE TABLE explore_index (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
entity_type TEXT NOT NULL,
|
||||||
|
mbid TEXT NOT NULL,
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
artist_name TEXT NOT NULL,
|
||||||
|
artist_mbid TEXT NOT NULL,
|
||||||
|
aliases TEXT NOT NULL DEFAULT '',
|
||||||
|
popularity INTEGER NOT NULL DEFAULT 0,
|
||||||
|
listener_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
duration INTEGER NOT NULL DEFAULT 0,
|
||||||
|
caa_release_mbid TEXT NOT NULL DEFAULT '',
|
||||||
|
release_name TEXT NOT NULL DEFAULT '',
|
||||||
|
primary_type TEXT NOT NULL DEFAULT '',
|
||||||
|
secondary_types TEXT NOT NULL DEFAULT '',
|
||||||
|
release_date TEXT NOT NULL DEFAULT '',
|
||||||
|
total_tracks INTEGER NOT NULL DEFAULT 0,
|
||||||
|
artist_type TEXT NOT NULL DEFAULT '',
|
||||||
|
country TEXT NOT NULL DEFAULT '',
|
||||||
|
disambiguation TEXT NOT NULL DEFAULT '',
|
||||||
|
sort_name TEXT NOT NULL DEFAULT '',
|
||||||
|
in_library INTEGER NOT NULL DEFAULT 0,
|
||||||
|
is_similar INTEGER NOT NULL DEFAULT 0,
|
||||||
|
local_artist_id INTEGER,
|
||||||
|
local_release_group_id INTEGER,
|
||||||
|
local_recording_id INTEGER,
|
||||||
|
discog_fetched INTEGER NOT NULL DEFAULT 0,
|
||||||
|
UNIQUE(mbid)
|
||||||
|
);
|
||||||
|
INSERT INTO explore_index
|
||||||
|
(entity_type, mbid, title, artist_name, artist_mbid)
|
||||||
|
VALUES ('artist', 'a-b-c', 'A Catalog Row', 'An Artist', 'd-e-f');
|
||||||
|
`)
|
||||||
|
|
||||||
|
if _, err := database.NewDB(logger); err != nil {
|
||||||
|
t.Fatalf("open with a stale catalog shape: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := count(t, dbPath, "SELECT COUNT(*) FROM explore_index"); got != 1 {
|
||||||
|
t.Fatalf(
|
||||||
|
"explore_index rows = %d, want 1 — the catalog was retired, "+
|
||||||
|
"which costs this database a ~205GB rebuild",
|
||||||
|
got,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNoCacheTableIsRetiredHere is the general form of the accident
|
||||||
|
// above, and it exists because the specific one is not the risk.
|
||||||
|
//
|
||||||
|
// `TestTheCatalogSurvivesAStaleShape` pins one table in one wrong shape,
|
||||||
|
// which is the failure that happened. What cost the ~205 GB was not that
|
||||||
|
// shape: it was a destructive repair added to `database.NewDB` -- the
|
||||||
|
// one chokepoint every binary in this project shares -- without asking
|
||||||
|
// which binary it was running in. The next such repair will have a
|
||||||
|
// different name and a different reason, and this database still cannot
|
||||||
|
// afford it.
|
||||||
|
//
|
||||||
|
// So the assertion is about the *outcome* rather than the mechanism: put
|
||||||
|
// every Cache table in a shape the schema has certainly moved past, open
|
||||||
|
// the database the way cmd/indexbuild does, and require that all of them
|
||||||
|
// are still there afterwards. Any future repair that drops one fails
|
||||||
|
// here regardless of how it decides to.
|
||||||
|
//
|
||||||
|
// Two things about it are deliberate.
|
||||||
|
//
|
||||||
|
// The table list comes from `datamap.ByKind(Cache)` rather than being
|
||||||
|
// written out, so a Cache table added next year is covered by this test
|
||||||
|
// on the day it is added -- the same reason `TestCatalogCoversSchema`
|
||||||
|
// reads the schema instead of a list.
|
||||||
|
//
|
||||||
|
// And `NewDB` returning an error is *accepted*, because that is the
|
||||||
|
// trade the fix documents: with Cache tables no longer rebuilt here, a
|
||||||
|
// shape the schema moved past now fails this job loudly instead of
|
||||||
|
// silently costing it a day of downloading. Loud is fine. Gone is not.
|
||||||
|
func TestNoCacheTableIsRetiredHere(t *testing.T) {
|
||||||
|
logger := slog.New(slog.DiscardHandler)
|
||||||
|
|
||||||
|
t.Setenv("YJ_HOME", t.TempDir())
|
||||||
|
|
||||||
|
dataDir, err := system.GetUserDataDirPath()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolve data dir: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dbPath := filepath.Join(dataDir, "yj.db")
|
||||||
|
|
||||||
|
if _, err := database.NewDB(logger); err != nil {
|
||||||
|
t.Fatalf("first open: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// An FTS table is four shadow tables and cannot be given a "wrong
|
||||||
|
// shape" meaningfully; the repair skips them for the same reason and
|
||||||
|
// retires them with their parent, which the parents below cover.
|
||||||
|
var cache []string
|
||||||
|
|
||||||
|
for _, table := range datamap.ByKind(datamap.Cache) {
|
||||||
|
if table.FTS {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
cache = append(cache, table.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(cache) == 0 {
|
||||||
|
t.Fatal("no Cache tables to check: the datamap or this test is wrong")
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range cache {
|
||||||
|
// A shape nothing in the current schema describes. What matters
|
||||||
|
// is only that it disagrees; the real mismatch was one column's
|
||||||
|
// type.
|
||||||
|
exec(t, dbPath, `
|
||||||
|
DROP TABLE IF EXISTS `+name+`;
|
||||||
|
CREATE TABLE `+name+` (id INTEGER PRIMARY KEY, moved_past TEXT);
|
||||||
|
INSERT INTO `+name+` (moved_past) VALUES ('irreplaceable');
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The error is not the assertion: see the note above.
|
||||||
|
_, _ = database.NewDB(logger)
|
||||||
|
|
||||||
|
for _, name := range cache {
|
||||||
|
rows := count(t, dbPath,
|
||||||
|
`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = '`+name+`'`)
|
||||||
|
if rows == 0 {
|
||||||
|
t.Errorf("%s was retired: in this database a Cache table is derived, "+
|
||||||
|
"not downloaded, and dropping one costs the ~205 GB dump stream", name)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Present but emptied is the same loss wearing a different
|
||||||
|
// shape: SQLite does an implicit DELETE before a DROP, and a
|
||||||
|
// repair that recreated the table would look identical here.
|
||||||
|
if n := count(t, dbPath, `SELECT count(*) FROM `+name); n == 0 {
|
||||||
|
t.Errorf("%s survived but was emptied", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,191 @@
|
|||||||
|
//go:build indexbuild
|
||||||
|
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
_ "modernc.org/sqlite"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The columns an index built before the completeness work has: every
|
||||||
|
// current one except total_tracks.
|
||||||
|
//
|
||||||
|
// Filtered rather than string-replaced, because the list is formatted
|
||||||
|
// across lines: `strings.Replace(catalogColumns, "total_tracks, ", …)`
|
||||||
|
// matches nothing (the name is followed by a newline, not a space) and
|
||||||
|
// silently yields the *current* list -- so the test built a modern
|
||||||
|
// source index and proved nothing while passing its own premise.
|
||||||
|
var oldColumns = withoutTotals(catalogColumns)
|
||||||
|
|
||||||
|
func withoutTotals(cols string) string {
|
||||||
|
kept := make([]string, 0, 20)
|
||||||
|
|
||||||
|
for _, part := range strings.Split(cols, ",") {
|
||||||
|
if strings.TrimSpace(part) == "total_tracks" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
kept = append(kept, strings.TrimSpace(part))
|
||||||
|
}
|
||||||
|
|
||||||
|
return strings.Join(kept, ", ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestExportFromAnIndexWithoutTotals reproduces the failure that broke
|
||||||
|
// the index-artifact job, symptom first.
|
||||||
|
//
|
||||||
|
// The job's /cache volume is a real YJ_HOME that survives between runs
|
||||||
|
// and holds ~205 GB, so its explore_index is Cache and is deliberately
|
||||||
|
// not dropped by cmd/indexbuild's schema repair -- which means a column
|
||||||
|
// added to the schema afterwards is simply absent from it. The exporter
|
||||||
|
// selected it anyway and the whole run died with
|
||||||
|
//
|
||||||
|
// indexexport: copy rows: SQL logic error: no such column: total_tracks
|
||||||
|
//
|
||||||
|
// after three minutes of work, on a job that publishes the catalog
|
||||||
|
// every user downloads.
|
||||||
|
func TestExportFromAnIndexWithoutTotals(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := openWithSource(t, oldColumns)
|
||||||
|
|
||||||
|
if got := sourceColumns(db); strings.Contains(got, "total_tracks") {
|
||||||
|
t.Fatalf("source list still names total_tracks: %s", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := copyRows(db, 10, 5, 5); err != nil {
|
||||||
|
t.Fatalf("export from an index without total_tracks: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Zero, not absent: the artifact keeps every column so an importer
|
||||||
|
// needs no second shape, and 0 is what the column already means by
|
||||||
|
// "the catalog does not say".
|
||||||
|
var total int
|
||||||
|
if err := db.QueryRow(
|
||||||
|
`SELECT total_tracks FROM core.explore_index WHERE entity_type = 2`,
|
||||||
|
).Scan(&total); err != nil {
|
||||||
|
t.Fatalf("read exported total_tracks: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if total != 0 {
|
||||||
|
t.Errorf("total_tracks = %d, want 0", total)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestExportCarriesTotalsWhenTheIndexHasThem is the other half: the
|
||||||
|
// probe must not cost the totals of an index that does have them.
|
||||||
|
func TestExportCarriesTotalsWhenTheIndexHasThem(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := openWithSource(t, catalogColumns)
|
||||||
|
|
||||||
|
if got := sourceColumns(db); !strings.Contains(got, "total_tracks") {
|
||||||
|
t.Fatalf("source list dropped total_tracks: %s", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := copyRows(db, 10, 5, 5); err != nil {
|
||||||
|
t.Fatalf("export: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var total int
|
||||||
|
if err := db.QueryRow(
|
||||||
|
`SELECT total_tracks FROM core.explore_index WHERE entity_type = 2`,
|
||||||
|
).Scan(&total); err != nil {
|
||||||
|
t.Fatalf("read exported total_tracks: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if total != 12 {
|
||||||
|
t.Errorf("total_tracks = %d, want 12", total)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// openWithSource builds a source index carrying exactly `columns`, with
|
||||||
|
// one artist and one of its release groups, and attaches a fresh
|
||||||
|
// artifact database as `core`.
|
||||||
|
func openWithSource(t *testing.T, columns string) *sql.DB {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
|
||||||
|
db, err := sql.Open("sqlite", filepath.Join(dir, "src.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open source: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
|
// The source's shape is the point of the test, so it is spelled
|
||||||
|
// out here rather than taken from the app's schema, which is
|
||||||
|
// always current by definition.
|
||||||
|
create := `CREATE TABLE explore_index (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
entity_type INTEGER NOT NULL,
|
||||||
|
mbid BLOB NOT NULL,
|
||||||
|
title TEXT NOT NULL DEFAULT '',
|
||||||
|
artist_name TEXT NOT NULL DEFAULT '',
|
||||||
|
artist_mbid BLOB NOT NULL DEFAULT x'',
|
||||||
|
aliases TEXT NOT NULL DEFAULT '',
|
||||||
|
popularity INTEGER NOT NULL DEFAULT 0,
|
||||||
|
listener_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
duration INTEGER NOT NULL DEFAULT 0,
|
||||||
|
caa_release_mbid BLOB NOT NULL DEFAULT x'',
|
||||||
|
release_name TEXT NOT NULL DEFAULT '',
|
||||||
|
primary_type TEXT NOT NULL DEFAULT '',
|
||||||
|
secondary_types TEXT NOT NULL DEFAULT '',
|
||||||
|
release_date TEXT NOT NULL DEFAULT '',
|
||||||
|
total_tracks INTEGER NOT NULL DEFAULT 0,
|
||||||
|
artist_type TEXT NOT NULL DEFAULT '',
|
||||||
|
country TEXT NOT NULL DEFAULT '',
|
||||||
|
disambiguation TEXT NOT NULL DEFAULT '',
|
||||||
|
sort_name TEXT NOT NULL DEFAULT '',
|
||||||
|
discog_fetched INTEGER NOT NULL DEFAULT 0
|
||||||
|
)`
|
||||||
|
|
||||||
|
if !strings.Contains(columns, "total_tracks") {
|
||||||
|
create = strings.Replace(
|
||||||
|
create, "total_tracks INTEGER NOT NULL DEFAULT 0,\n", "", 1,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.Exec(create); err != nil {
|
||||||
|
t.Fatalf("create source: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
seed := `INSERT INTO explore_index (` + columns + `) VALUES `
|
||||||
|
|
||||||
|
if strings.Contains(columns, "total_tracks") {
|
||||||
|
seed += `(1, x'00000000000000000000000000000001', 'A', 'A',
|
||||||
|
x'00000000000000000000000000000001', '', 100, 100, 0, x'',
|
||||||
|
'', '', '', '', 12, '', '', '', '', 0),
|
||||||
|
(2, x'00000000000000000000000000000002', 'RG', 'A',
|
||||||
|
x'00000000000000000000000000000001', '', 90, 90, 0, x'',
|
||||||
|
'', 'Album', '', '', 12, '', '', '', '', 0)`
|
||||||
|
} else {
|
||||||
|
seed += `(1, x'00000000000000000000000000000001', 'A', 'A',
|
||||||
|
x'00000000000000000000000000000001', '', 100, 100, 0, x'',
|
||||||
|
'', '', '', '', '', '', '', '', 0),
|
||||||
|
(2, x'00000000000000000000000000000002', 'RG', 'A',
|
||||||
|
x'00000000000000000000000000000001', '', 90, 90, 0, x'',
|
||||||
|
'', 'Album', '', '', '', '', '', '', 0)`
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.Exec(seed); err != nil {
|
||||||
|
t.Fatalf("seed source: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := db.Exec(
|
||||||
|
`ATTACH DATABASE ? AS core`, filepath.Join(dir, "core.db"),
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("attach core: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := createSchema(db); err != nil {
|
||||||
|
t.Fatalf("create artifact schema: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return db
|
||||||
|
}
|
||||||
+119
-2
@@ -25,6 +25,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
_ "modernc.org/sqlite"
|
_ "modernc.org/sqlite"
|
||||||
@@ -43,6 +44,41 @@ const catalogColumns = `entity_type, mbid, title, artist_name, artist_mbid,
|
|||||||
release_name, primary_type, secondary_types, release_date, total_tracks,
|
release_name, primary_type, secondary_types, release_date, total_tracks,
|
||||||
artist_type, country, disambiguation, sort_name, discog_fetched`
|
artist_type, country, disambiguation, sort_name, discog_fetched`
|
||||||
|
|
||||||
|
// sourceColumns is catalogColumns as read *from* the built index,
|
||||||
|
// which is not always shaped like the one this binary was compiled
|
||||||
|
// against.
|
||||||
|
//
|
||||||
|
// The index job's /cache volume is a real YJ_HOME that survives
|
||||||
|
// between runs and holds ~205 GB nobody can re-download casually, so
|
||||||
|
// its explore_index is classified Cache and is deliberately **not**
|
||||||
|
// dropped and recreated by cmd/indexbuild's schema repair. A column
|
||||||
|
// added to the schema after that database was built is therefore
|
||||||
|
// absent from it, and selecting it fails the whole export with
|
||||||
|
// "no such column: total_tracks" -- which is what happened the first
|
||||||
|
// time the job ran after the completeness work.
|
||||||
|
//
|
||||||
|
// So the source list is asked for rather than assumed, exactly as
|
||||||
|
// artifactHasTotals does on the importing side. Zero is what the
|
||||||
|
// column means by "the catalog does not say", and the app already
|
||||||
|
// renders that as unknown rather than as incomplete.
|
||||||
|
func sourceColumns(db *sql.DB) string {
|
||||||
|
var n int
|
||||||
|
|
||||||
|
err := db.QueryRow(
|
||||||
|
`SELECT COUNT(*) FROM pragma_table_info('explore_index', 'main')
|
||||||
|
WHERE name = 'total_tracks'`,
|
||||||
|
).Scan(&n)
|
||||||
|
if err == nil && n > 0 {
|
||||||
|
return catalogColumns
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println(
|
||||||
|
" note: this index predates total_tracks; exporting 0 for it",
|
||||||
|
)
|
||||||
|
|
||||||
|
return strings.Replace(catalogColumns, "total_tracks", "0", 1)
|
||||||
|
}
|
||||||
|
|
||||||
var errNoHome = errors.New(
|
var errNoHome = errors.New(
|
||||||
"YJ_HOME must be set to the directory holding the built index",
|
"YJ_HOME must be set to the directory holding the built index",
|
||||||
)
|
)
|
||||||
@@ -171,6 +207,26 @@ func createSchema(db *sql.DB) error {
|
|||||||
key TEXT PRIMARY KEY,
|
key TEXT PRIMARY KEY,
|
||||||
value TEXT NOT NULL
|
value TEXT NOT NULL
|
||||||
)`,
|
)`,
|
||||||
|
// Multi-artist credits. Shipped as their own tables rather than
|
||||||
|
// as an explore_index column because a credit is a variable
|
||||||
|
// number of ordered parts, and because credits are *shared* --
|
||||||
|
// an album's tracks by one artist reference one credit, which is
|
||||||
|
// what keeps this to a few hundred thousand rows.
|
||||||
|
//
|
||||||
|
// An importer that predates these reads an artifact without
|
||||||
|
// them; artifactHasCredits is what asks.
|
||||||
|
`CREATE TABLE core.artist_credit_part (
|
||||||
|
credit_id INTEGER NOT NULL,
|
||||||
|
position INTEGER NOT NULL,
|
||||||
|
artist_mbid BLOB NOT NULL,
|
||||||
|
credited_name TEXT NOT NULL,
|
||||||
|
join_phrase TEXT NOT NULL DEFAULT '',
|
||||||
|
PRIMARY KEY (credit_id, position)
|
||||||
|
) WITHOUT ROWID`,
|
||||||
|
`CREATE TABLE core.artist_credit_ref (
|
||||||
|
mbid BLOB NOT NULL PRIMARY KEY,
|
||||||
|
credit_id INTEGER NOT NULL
|
||||||
|
) WITHOUT ROWID`,
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, stmt := range stmts {
|
for _, stmt := range stmts {
|
||||||
@@ -189,6 +245,10 @@ func createSchema(db *sql.DB) error {
|
|||||||
// dumpcatalog.go — a flat global top-N would give a handful of
|
// dumpcatalog.go — a flat global top-N would give a handful of
|
||||||
// superstars everything and everyone else nothing.
|
// superstars everything and everyone else nothing.
|
||||||
func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
||||||
|
// The destination is created by this binary and always has every
|
||||||
|
// column; only the source may be older.
|
||||||
|
srcColumns := sourceColumns(db)
|
||||||
|
|
||||||
if _, err := db.Exec(`
|
if _, err := db.Exec(`
|
||||||
CREATE TEMP TABLE core_artists AS
|
CREATE TEMP TABLE core_artists AS
|
||||||
SELECT mbid FROM main.explore_index
|
SELECT mbid FROM main.explore_index
|
||||||
@@ -201,7 +261,7 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
|||||||
|
|
||||||
copied, err := insertSelect(db, `
|
copied, err := insertSelect(db, `
|
||||||
INSERT INTO core.explore_index (`+catalogColumns+`)
|
INSERT INTO core.explore_index (`+catalogColumns+`)
|
||||||
SELECT `+catalogColumns+`
|
SELECT `+srcColumns+`
|
||||||
FROM main.explore_index
|
FROM main.explore_index
|
||||||
WHERE entity_type = 1 /* artist */
|
WHERE entity_type = 1 /* artist */
|
||||||
AND mbid IN (SELECT mbid FROM core_artists)`)
|
AND mbid IN (SELECT mbid FROM core_artists)`)
|
||||||
@@ -228,7 +288,7 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
|||||||
// most `limit` rows, ranked by their own listen counts.
|
// most `limit` rows, ranked by their own listen counts.
|
||||||
n, err := insertSelect(db, `
|
n, err := insertSelect(db, `
|
||||||
INSERT INTO core.explore_index (`+catalogColumns+`)
|
INSERT INTO core.explore_index (`+catalogColumns+`)
|
||||||
SELECT `+catalogColumns+` FROM (
|
SELECT `+srcColumns+` FROM (
|
||||||
SELECT *, ROW_NUMBER() OVER (
|
SELECT *, ROW_NUMBER() OVER (
|
||||||
PARTITION BY artist_mbid ORDER BY popularity DESC
|
PARTITION BY artist_mbid ORDER BY popularity DESC
|
||||||
) AS rn
|
) AS rn
|
||||||
@@ -243,6 +303,63 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
|||||||
fmt.Printf(" %-15s %d\n", sel.label+":", n)
|
fmt.Printf(" %-15s %d\n", sel.label+":", n)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return copyCredits(db)
|
||||||
|
}
|
||||||
|
|
||||||
|
// copyCredits ships the credit decomposition for the entities that made
|
||||||
|
// it into the artifact, and only those.
|
||||||
|
//
|
||||||
|
// The refs go first and the parts follow *from* the refs, so a credit is
|
||||||
|
// carried only if something in the artifact points at it. The source
|
||||||
|
// index holds credits for every catalog entity, while the artifact is a
|
||||||
|
// windowed subset -- copying all of them would carry a large table most
|
||||||
|
// of which nothing in the artifact can reach.
|
||||||
|
//
|
||||||
|
// A source index built before the credit pass simply has no rows here,
|
||||||
|
// which is not an error: the artifact then carries the tables empty, and
|
||||||
|
// every credit falls back to its single artist exactly as before.
|
||||||
|
func copyCredits(db *sql.DB) error {
|
||||||
|
// Asked, not assumed. A source index built before the credit pass
|
||||||
|
// has no such table, and "no such table" would fail an export whose
|
||||||
|
// catalog is otherwise complete.
|
||||||
|
for _, table := range []string{"artist_credit_ref", "artist_credit_part"} {
|
||||||
|
var n int
|
||||||
|
|
||||||
|
if err := db.QueryRow(
|
||||||
|
`SELECT COUNT(*) FROM main.sqlite_master
|
||||||
|
WHERE type = 'table' AND name = ?`, table,
|
||||||
|
).Scan(&n); err != nil {
|
||||||
|
return fmt.Errorf("probe %s: %w", table, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if n == 0 {
|
||||||
|
fmt.Printf(" %-15s none in source\n", "credits:")
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
refs, err := insertSelect(db, `
|
||||||
|
INSERT INTO core.artist_credit_ref (mbid, credit_id)
|
||||||
|
SELECT r.mbid, r.credit_id
|
||||||
|
FROM main.artist_credit_ref r
|
||||||
|
WHERE r.mbid IN (SELECT mbid FROM core.explore_index)`)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
parts, err := insertSelect(db, `
|
||||||
|
INSERT INTO core.artist_credit_part
|
||||||
|
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||||
|
SELECT p.credit_id, p.position, p.artist_mbid, p.credited_name, p.join_phrase
|
||||||
|
FROM main.artist_credit_part p
|
||||||
|
WHERE p.credit_id IN (SELECT credit_id FROM core.artist_credit_ref)`)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf(" %-15s %d refs, %d parts\n", "credits:", refs, parts)
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+35
-5
@@ -1,9 +1,35 @@
|
|||||||
# Releasing the Android APK
|
# Releasing the Android APK
|
||||||
|
|
||||||
`.gitea/workflows/android-apk.yml` builds a signed fat APK
|
`.gitea/workflows/android-apk.yml` builds a signed `arm64-v8a` APK on
|
||||||
(`arm64-v8a` + `x86_64`) on every `v*` tag and publishes it to Gitea's
|
every `v*` tag and publishes it to Gitea's
|
||||||
**generic** package registry, which is readable without credentials —
|
**generic** package registry, which is readable without credentials —
|
||||||
which is what lets Obtainium poll a plain URL with no token.
|
which is what lets Obtainium poll a plain URL with no token. It also
|
||||||
|
attaches the same file to the Gitea release, which is what a person
|
||||||
|
looking at the release page downloads.
|
||||||
|
|
||||||
|
**Tags are not pushed by hand any more.** `.gitea/workflows/release.yml`
|
||||||
|
reads the Conventional Commits on every merge to `main`, decides the
|
||||||
|
version, and pushes the tag this workflow is keyed on — so releasing the
|
||||||
|
APK means merging a `fix:` or `feat:` commit, not running `git tag`. The
|
||||||
|
`workflow_dispatch` path below remains, for rebuilding a tag that already
|
||||||
|
exists.
|
||||||
|
|
||||||
|
## The 1.x installs cannot be upgraded to 0.0.x
|
||||||
|
|
||||||
|
Releases restarted at **0.0.1** when they became automatic (plan 017).
|
||||||
|
`versionCode` is computed as `maj*10000 + min*100 + pat`, so 0.0.1 is
|
||||||
|
**1** against the **10300** an installed 1.3.0 build carries — and
|
||||||
|
**Android refuses a downgrade outright**, with
|
||||||
|
`INSTALL_FAILED_VERSION_DOWNGRADE`.
|
||||||
|
|
||||||
|
The only way through is `adb uninstall app.yellowjacket` (or the
|
||||||
|
launcher's own uninstall) before installing 0.0.1, **and that takes the
|
||||||
|
device's library, playlists and play counts with it** — the same loss the
|
||||||
|
signing key section below exists to prevent, arrived at from the other
|
||||||
|
direction. This was chosen deliberately over offsetting `versionCode` by
|
||||||
|
a constant, on the grounds that the honest number is worth one reinstall
|
||||||
|
while an offset is permanent. The workflow prints a warning whenever the
|
||||||
|
code it computes is below 10600.
|
||||||
|
|
||||||
```
|
```
|
||||||
https://git.ljones.me/api/packages/yonlu/generic/yellowjacket-android/latest/yellowjacket.apk
|
https://git.ljones.me/api/packages/yonlu/generic/yellowjacket-android/latest/yellowjacket.apk
|
||||||
@@ -98,8 +124,12 @@ publish `1.100.0`**, and never move a tag that has already been built.
|
|||||||
## What the workflow checks before publishing
|
## What the workflow checks before publishing
|
||||||
|
|
||||||
- the APK exists and is non-empty;
|
- the APK exists and is non-empty;
|
||||||
- it carries **both** ABIs (`native-code: 'arm64-v8a' 'x86_64'`), or it
|
- it carries **exactly one** ABI (`native-code: 'arm64-v8a'`). x86_64
|
||||||
is not the fat APK it claims to be;
|
Android cannot run this app at all — `modernc.org/libc` issues a raw
|
||||||
|
`lstat` syscall that Android's seccomp policy forbids, on every
|
||||||
|
x86_64 device and not merely the emulator — so an x86_64 slice would
|
||||||
|
be ~31 MB that runs nowhere, and its reappearance means someone put
|
||||||
|
the ABI back in `app/build.gradle` without knowing that;
|
||||||
- its `versionCode` is the one derived from the tag;
|
- its `versionCode` is the one derived from the tag;
|
||||||
- it is **not** signed with the debug key.
|
- it is **not** signed with the debug key.
|
||||||
|
|
||||||
|
|||||||
@@ -1,194 +0,0 @@
|
|||||||
# Config Improvement Suggestions
|
|
||||||
|
|
||||||
Remaining suggestions for improving the configuration system in YellowJacket.
|
|
||||||
|
|
||||||
## 2. Thread Safety Concerns
|
|
||||||
|
|
||||||
The current `Config` struct lacks synchronization:
|
|
||||||
- `Load()` and `Save()` can race with concurrent reads
|
|
||||||
- `handleConfigUpdate()` in library mutates `l.conf.DirectoryPath` without locks
|
|
||||||
|
|
||||||
**Suggestion:** Add a `sync.RWMutex` to protect config access, especially if config is read during scans.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Config struct {
|
|
||||||
mu sync.RWMutex
|
|
||||||
ctx context.Context
|
|
||||||
logger *slog.Logger
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Config) Load() error {
|
|
||||||
c.mu.Lock()
|
|
||||||
defer c.mu.Unlock()
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 3. Nil Safety in Validation
|
|
||||||
|
|
||||||
In `config.go`, validation only runs if `c.Library != nil`, but `handleConfigPost` dereferences `postedConfig.Library` without checking for nil:
|
|
||||||
|
|
||||||
```go
|
|
||||||
if postedConfig.Library != nil {
|
|
||||||
c.Library = postedConfig.Library
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Status:** Partially addressed in the event refactor, but consider adding explicit nil checks in `Validate()` as well.
|
|
||||||
|
|
||||||
## 4. Inconsistent Error Handling on HTTP Responses
|
|
||||||
|
|
||||||
In `httphandler.go:28-31`, `WriteHeader` is called *after* rendering the error template, which won't work as expected (headers must be set before writing body):
|
|
||||||
|
|
||||||
```go
|
|
||||||
c.formSubmitError(err.Error()).Render(r.Context(), w)
|
|
||||||
w.WriteHeader(http.StatusInternalServerError) // Too late!
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix:** Set the status code before rendering:
|
|
||||||
|
|
||||||
```go
|
|
||||||
w.WriteHeader(http.StatusInternalServerError)
|
|
||||||
c.formSubmitError(err.Error()).Render(r.Context(), w)
|
|
||||||
```
|
|
||||||
|
|
||||||
## 5. Make `scanWorkerCount` Configurable
|
|
||||||
|
|
||||||
There's a TODO at `library.go:289`:
|
|
||||||
```go
|
|
||||||
// TODO: make configurable via Config.
|
|
||||||
var scanWorkerCount = goruntime.NumCPU()
|
|
||||||
```
|
|
||||||
|
|
||||||
**Suggestion:** Add this to `library.Config`:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Config struct {
|
|
||||||
DirectoryPath Directory `form:"Directory" schema:"directory,required"`
|
|
||||||
ScanWorkers int `form:"ScanWorkers" schema:"scan_workers"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Then in `NewLibrary()` or `Scan()`:
|
|
||||||
|
|
||||||
```go
|
|
||||||
workers := l.conf.ScanWorkers
|
|
||||||
if workers <= 0 {
|
|
||||||
workers = goruntime.NumCPU()
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 6. Consider Config Defaults
|
|
||||||
|
|
||||||
Currently if no config exists, an empty one is saved. Consider providing sensible defaults (e.g., common music directories like `~/Music`).
|
|
||||||
|
|
||||||
```go
|
|
||||||
func (c *Config) setDefaults() {
|
|
||||||
if c.Library == nil {
|
|
||||||
c.Library = &library.Config{}
|
|
||||||
}
|
|
||||||
if c.Library.DirectoryPath == "" {
|
|
||||||
// Try common music directories
|
|
||||||
home, _ := os.UserHomeDir()
|
|
||||||
musicDir := filepath.Join(home, "Music")
|
|
||||||
if info, err := os.Stat(musicDir); err == nil && info.IsDir() {
|
|
||||||
c.Library.DirectoryPath = library.Directory(musicDir)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 7. Config Reload/Watch Capability
|
|
||||||
|
|
||||||
The config is only loaded at startup. Consider adding:
|
|
||||||
- File watcher for external config changes (using `fsnotify`)
|
|
||||||
- Explicit reload method callable from UI
|
|
||||||
|
|
||||||
```go
|
|
||||||
func (c *Config) Watch() error {
|
|
||||||
watcher, err := fsnotify.NewWatcher()
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
go func() {
|
|
||||||
for event := range watcher.Events {
|
|
||||||
if event.Op&fsnotify.Write == fsnotify.Write {
|
|
||||||
c.Load()
|
|
||||||
// Emit event for listeners
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
return watcher.Add(c.filePath)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 8. Validation Should Return Structured Errors
|
|
||||||
|
|
||||||
Currently validation returns combined errors. Consider returning a structured validation result that the UI can map to specific fields for better user feedback.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ValidationError struct {
|
|
||||||
Field string
|
|
||||||
Message string
|
|
||||||
}
|
|
||||||
|
|
||||||
type ValidationResult struct {
|
|
||||||
Valid bool
|
|
||||||
Errors []ValidationError
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Config) ValidateStructured() ValidationResult {
|
|
||||||
var result ValidationResult
|
|
||||||
result.Valid = true
|
|
||||||
|
|
||||||
if c.Library != nil {
|
|
||||||
if err := c.Library.Validate(); err != nil {
|
|
||||||
result.Valid = false
|
|
||||||
result.Errors = append(result.Errors, ValidationError{
|
|
||||||
Field: "Library.DirectoryPath",
|
|
||||||
Message: err.Error(),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return result
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 9. Use Standard Library for Config Paths
|
|
||||||
|
|
||||||
The path construction in `system/userdata.go` doesn't respect `$XDG_CONFIG_HOME` on Linux or use the standard Go `os.UserConfigDir()`.
|
|
||||||
|
|
||||||
**Current implementation:**
|
|
||||||
```go
|
|
||||||
case "linux":
|
|
||||||
return fmt.Sprintf("/home/%s/%s/yellowjacket", username, unixSubdirs[dt]), nil
|
|
||||||
```
|
|
||||||
|
|
||||||
**Suggested improvement:**
|
|
||||||
```go
|
|
||||||
func GetUserConfigDirPath() (string, error) {
|
|
||||||
baseDir, err := os.UserConfigDir() // Respects XDG_CONFIG_HOME
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("could not get user config directory: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
path := filepath.Join(baseDir, "yellowjacket")
|
|
||||||
|
|
||||||
if err := os.MkdirAll(path, 0o755); err != nil {
|
|
||||||
return "", fmt.Errorf("could not create config directory: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This approach:
|
|
||||||
- Respects `$XDG_CONFIG_HOME` on Linux
|
|
||||||
- Uses proper macOS paths (`~/Library/Application Support`)
|
|
||||||
- Uses `%AppData%` on Windows
|
|
||||||
- Is more portable and follows platform conventions
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
# Development Overview
|
|
||||||
|
|
||||||
YellowJacket is a moderately complex application. This document gives an overview of how development of it works.
|
|
||||||
|
|
||||||
## Logical Breakdown
|
|
||||||
|
|
||||||
YellowJacket can be thought about in a heirarchy of logical modules and components. The borders of these logical sections are mostly represented in the code and directory structure as well.
|
|
||||||
|
|
||||||
- Frontend
|
|
||||||
- UI Components (see [Lit](###lit-web-components))
|
|
||||||
- Backend
|
|
||||||
- App
|
|
||||||
- Asset Handler
|
|
||||||
- Logging
|
|
||||||
- System
|
|
||||||
- Player
|
|
||||||
- Library
|
|
||||||
- Config
|
|
||||||
- Database
|
|
||||||
- Queries (see [sqlc](###sqlc))
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
YellowJacket uses many tools and libraries to provide its functionality.
|
|
||||||
This section lists each of these dependencies and explains how they are used.
|
|
||||||
|
|
||||||
### [Wails](https://wails.io)
|
|
||||||
|
|
||||||
Used to create desktop apps with Go and web technologies.
|
|
||||||
|
|
||||||
### [SQLite](https://github.com/mattn/go-sqlite3?tab=readme-ov-file#go-sqlite3)
|
|
||||||
|
|
||||||
Used for local database.
|
|
||||||
|
|
||||||
### [sqlc](https://sqlc.dev/)
|
|
||||||
|
|
||||||
Used to generate Go code from SQL.
|
|
||||||
|
|
||||||
### [Templ](https://templ.guide/)
|
|
||||||
|
|
||||||
Used to generate HTML templates with Go code.
|
|
||||||
|
|
||||||
### [Beep](https://github.com/gopxl/beep?tab=readme-ov-file#beep)
|
|
||||||
|
|
||||||
Used for audio playback.
|
|
||||||
|
|
||||||
### [Lit Web Components](https://lit.dev/)
|
|
||||||
|
|
||||||
Used for dynamic/reactive frontend components.
|
|
||||||
|
|
||||||
### [HTMX](https://htmx.org/)
|
|
||||||
|
|
||||||
Used for requesting HTML fragments from the backend and rendering them on the frontend.
|
|
||||||
-1648
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user