yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

Android: touch gestures for track lists — swipe to queue, tap to play, long-press to multi-select, double-tap for the menu #63

New Issue
Closed
opened 2026-08-18 05:59:09 +00:00 by logan · 4 comments
logan commented 2026-08-18 05:59:09 +00:00
Collaborator
Copy Link
Copy Source

Report

Browsing tracks needs a faster way to get things queued and played:

  • swipe right on a track → add to queue
  • single tap → play the track
  • long press → enter multi-select mode; subsequent taps select, and there needs to be a way to act on everything selected
  • double tap → context menu

Findings

  • utils/long-press.ts currently maps a 500ms hold to a synthetic contextmenu, installed once as a document-capture listener, with the browser's own long-press contextmenu winning where it fires (told apart by identity, not isTrusted) and the trailing click swallowed. This proposal reassigns that gesture, so that utility changes rather than gains a caller — and the double-tap must not be eaten by the same swallow logic.
  • Selection state lives in utils/selection-controller.ts; a "multi-select mode" is a new concept on top of it (a mode flag, plus a contextual action bar), and it has to reach track-list, the queue, and both playlist views.
  • Rows are inside lit-virtualizer, so gesture handlers must be delegated on the virtualizer and the rows must repaint on selection change (virtualizer.requestUpdate()).
  • Note the desktop semantics being diverged from: there, single click selects and double click plays. The mobile mapping inverts that, so the row component needs a platform-aware interaction layer rather than shared handlers.

Direction

Design the interaction model as one piece before implementing: a touch-interactions controller owning tap/double-tap/long-press/swipe, per-row swipe affordance (reveal + snap), a selection-mode action bar (queue, play next, add to playlist, remove), and an escape from the mode. Sequence it with the bottom-sheet context menu issue — they share the same gesture surface.

**Report** Browsing tracks needs a faster way to get things queued and played: - swipe right on a track → add to queue - single tap → play the track - long press → enter multi-select mode; subsequent taps select, and there needs to be a way to act on everything selected - double tap → context menu **Findings** - `utils/long-press.ts` currently maps a 500ms hold to a synthetic `contextmenu`, installed once as a document-capture listener, with the browser's own long-press `contextmenu` winning where it fires (told apart by identity, not `isTrusted`) and the trailing click swallowed. **This proposal reassigns that gesture**, so that utility changes rather than gains a caller — and the double-tap must not be eaten by the same swallow logic. - Selection state lives in `utils/selection-controller.ts`; a "multi-select mode" is a new concept on top of it (a mode flag, plus a contextual action bar), and it has to reach `track-list`, the queue, and both playlist views. - Rows are inside `lit-virtualizer`, so gesture handlers must be delegated on the virtualizer and the rows must repaint on selection change (`virtualizer.requestUpdate()`). - Note the desktop semantics being diverged from: there, single click selects and double click plays. The mobile mapping inverts that, so the row component needs a platform-aware interaction layer rather than shared handlers. **Direction** Design the interaction model as one piece before implementing: a `touch-interactions` controller owning tap/double-tap/long-press/swipe, per-row swipe affordance (reveal + snap), a selection-mode action bar (queue, play next, add to playlist, remove), and an escape from the mode. Sequence it with the bottom-sheet context menu issue — they share the same gesture surface.
logan removed the
Status
Blocked
1
 label 2026-08-21 07:26:44 +00:00
logan commented 2026-08-21 07:26:56 +00:00
Author
Collaborator
Copy Link
Copy Source

Unblocked: #60 merged in #176, so the sole dependency is closed and
Status/Blocked is off.

What landed that you inherit rather than have to build:
components/menu-surface/ is the one decision about where a menu is
drawn — a wa-popup above 600px, a wa-dialog bottom sheet below it —
and all fourteen menu call sites go through it, including the playlist
submenu, which stacks as a second sheet. contextMenuStyles carries
the 48px rows for all of them, so a new menu gets thumb-sized targets
by using the same .context-menu-panel markup and nothing else.

Two things worth knowing before you add a menu:

  • A bare <wa-popup> will fail the build, deliberately.
    frontend/test/components/menu-surface.test.ts sweeps the source and
    allows three files, each with a stated reason. A popup is clipped on
    the reference device by .main-panel's paint containment and no
    tier here can see that    , so the guard is a source sweep rather than a
    rendered assertion.
  • The device measurements are in .planning/NOTES.md, dated
    2026-08-21 — including the two that would otherwise cost a rebuild: a
    top-layer <dialog> really does escape paint containment from inside
    a view, and a native dialog's UA max-width is what makes a naive
    sheet come out 354px wide on a 424px screen.
**Unblocked**: #60 merged in #176, so the sole dependency is closed and `Status/Blocked` is off. What landed that you inherit rather than have to build: `components/menu-surface/` is the one decision about *where* a menu is drawn — a `wa-popup` above 600px, a `wa-dialog` bottom sheet below it — and all fourteen menu call sites go through it, including the playlist submenu, which stacks as a second sheet. `contextMenuStyles` carries the 48px rows for all of them, so a new menu gets thumb-sized targets by using the same `.context-menu-panel` markup and nothing else. Two things worth knowing before you add a menu: - **A bare `<wa-popup>` will fail the build**, deliberately. `frontend/test/components/menu-surface.test.ts` sweeps the source and allows three files, each with a stated reason. A popup is clipped on the reference device by `.main-panel`'s paint containment and *no tier here can see that*, so the guard is a source sweep rather than a rendered assertion. - **The device measurements are in `.planning/NOTES.md`**, dated 2026-08-21 — including the two that would otherwise cost a rebuild: a top-layer `<dialog>` really does escape paint containment from inside a view, and a native dialog's UA `max-width` is what makes a naive sheet come out 354px wide on a 424px screen.
logan self-assigned this 2026-08-22 03:50:15 +00:00
logan added the
Status
In Progress
 label 2026-08-22 03:50:15 +00:00
logan commented 2026-08-22 03:50:15 +00:00
Author
Collaborator
Copy Link
Copy Source

Taking this. Branch: 63-android-touch-model. The Direction asks for
the interaction model to be designed as one piece first, so that is
.planning/plans/active/019-android-touch-model.md and this comment is
the part that needs agreeing before any of it is built.

One of the four mappings in the report has a cost I can put a number
on, and I think it should be dropped.

The report asks for single tap → play and double tap → context
menu. Those two cannot both be honoured without deferring every play
by the double-tap interval: the first tap of a double tap is
indistinguishable from a single tap until the interval expires, so
"tap plays" becomes "tap waits to see whether you meant something
else, then plays". This app already has that constant —
utils/explore-link.ts holds a navigation for DOUBLE_CLICK_GRACE_MS = 250 for exactly this reason.

Measured on the reference device just now, six runs, time from the
play command to the backend's TrackChanged:

155, 123, 85, 56, 91 ms      (median ~100)

So the primary interaction in the app is ~100ms today, and a
double-tap discriminator in front of it makes it ~350 — 3.5x, of
which 250ms is spent deliberately doing nothing. Paid on every
track anyone ever plays, to reach a menu.

It is also against the platform's own convention, which matters more
than usual here because this is the phone build: on Android,
long-press is how you select (Gmail, Files, Photos), double-tap is
zoom or nothing, and a list's menu is either the long-press sheet or a
per-row overflow.

The alternative is that the menu and the selection action bar are
the same surface, which is the convention and also removes a
component rather than adding one:

  • single tap → play (instant, no discriminator)
  • long press → selection mode, with that row selected
  • the action bar is the context menu, contextualised to whatever
    is selected — one row or forty. #60's bottom sheet stays as the
    "more actions" overflow behind it, so contextMenuStyles,
    MenuKeyboard and menu-surface are all reused rather than
    reimplemented.
  • swipe right → add to queue, unchanged from the report
  • no double-tap at all

That keeps every action reachable, costs the primary action nothing,
and means there is no window in which multi-select is unavailable —
which a naive phase order would create, since today a touch user
selects by tapping and acts through the long-press menu. Move tap to
play without shipping selection mode in the same change and that
capability disappears in between.

And the predicate is the pointer, not the platform or the
viewport. pointerType === 'touch', decided per event, which is
already how long-press.ts decides. This is #64's argument
(SystemOwnsVolume is named after the capability, not the platform)
and its warning: keyed on width, an Android tablet at 600px gets
desktop semantics on a touchscreen, and a touchscreen laptop cannot
have both mice at once. Per-event, a mouse click keeps click-selects
/ double-click-plays on the very same row, and nothing needs a
"platform-aware interaction layer" as a concept.

Raising it here rather than just doing it, because it is a deliberate
divergence from what the report asked for. If the double-tap is wanted
anyway I will build it and eat the 250ms, but the number should be on
the record first.

Taking this. Branch: `63-android-touch-model`. The Direction asks for the interaction model to be designed as one piece first, so that is `.planning/plans/active/019-android-touch-model.md` and this comment is the part that needs agreeing before any of it is built. **One of the four mappings in the report has a cost I can put a number on, and I think it should be dropped.** The report asks for *single tap → play* and *double tap → context menu*. Those two cannot both be honoured without deferring every play by the double-tap interval: the first tap of a double tap is indistinguishable from a single tap until the interval expires, so "tap plays" becomes "tap waits to see whether you meant something else, then plays". This app already has that constant — `utils/explore-link.ts` holds a navigation for `DOUBLE_CLICK_GRACE_MS = 250` for exactly this reason. Measured on the reference device just now, six runs, time from the play command to the backend's `TrackChanged`: ``` 155, 123, 85, 56, 91 ms (median ~100) ``` So the primary interaction in the app is ~100ms today, and a double-tap discriminator in front of it makes it ~350 — **3.5x, of which 250ms is spent deliberately doing nothing**. Paid on every track anyone ever plays, to reach a menu. It is also against the platform's own convention, which matters more than usual here because this is the phone build: on Android, long-press is *how you select* (Gmail, Files, Photos), double-tap is zoom or nothing, and a list's menu is either the long-press sheet or a per-row overflow. **The alternative is that the menu and the selection action bar are the same surface**, which is the convention and also removes a component rather than adding one: - **single tap → play** (instant, no discriminator) - **long press → selection mode**, with that row selected - **the action bar is the context menu**, contextualised to whatever is selected — one row or forty. #60's bottom sheet stays as the "more actions" overflow behind it, so `contextMenuStyles`, `MenuKeyboard` and `menu-surface` are all reused rather than reimplemented. - **swipe right → add to queue**, unchanged from the report - **no double-tap at all** That keeps every action reachable, costs the primary action nothing, and means there is no window in which multi-select is unavailable — which a naive phase order *would* create, since today a touch user selects by tapping and acts through the long-press menu. Move tap to play without shipping selection mode in the same change and that capability disappears in between. **And the predicate is the pointer, not the platform or the viewport.** `pointerType === 'touch'`, decided per event, which is already how `long-press.ts` decides. This is #64's argument (`SystemOwnsVolume` is named after the capability, not the platform) and its warning: keyed on width, an Android *tablet* at 600px gets desktop semantics on a touchscreen, and a touchscreen laptop cannot have both mice at once. Per-event, a mouse click keeps click-selects / double-click-plays on the very same row, and nothing needs a "platform-aware interaction layer" as a concept. Raising it here rather than just doing it, because it is a deliberate divergence from what the report asked for. If the double-tap is wanted anyway I will build it and eat the 250ms, but the number should be on the record first.
logan commented 2026-08-22 04:36:58 +00:00
Author
Collaborator
Copy Link
Copy Source

Phase 1 landed (PR #199, merged as ff3875b): tap plays, hold
selects. The design is
.planning/plans/active/019-android-touch-model.md; #63 stays open and
claimed for phases 2–4.

The two divergences from the report are argued in the plan and both
came out of measurement. There is no double-tap — the report's tap
plays and double-tap opens the menu cannot both be honoured, because
the first tap of a double tap is a single tap until the interval
expires. Measured on the device the play path is ~100ms
(155/123/85/56/91), the app's own DOUBLE_CLICK_GRACE_MS is 250, so
the discriminator is 3.5× the primary interaction to reach a menu the
hold already reaches. The menu and the selection bar are one surface
instead, which is also the platform's convention.

And the predicate is pointerType, per event, rather than a platform
or a viewport — #64's rule, with #64's warning: on a width, an Android
tablet at 600px gets desktop semantics on a touchscreen.

The finding worth carrying into every future gesture: Chrome 113's
Android WebView fires its own contextmenu on a long press.
long-press.ts stood down when a trusted one arrived, which was
correct only while both paths ended in a context menu. Once a hold can
mean something else, standing down means the gesture silently does the
old thing — and no browser tier can see it, because dispatched
pointer events never make a browser synthesise one. All 26 tests passed
on the broken build. The native event is a trigger now, not a
competitor.

A tier changed shape here too. adb shell input tap and
input swipe x y x y 700 reach the WebView as real pointer events, so
the Android tier can now perform the thing under test rather than
describe the page afterwards — which is how the above was found, and
how the whole model was verified (tap plays; hold raises the bar at 1;
taps toggle to 2 and back; the mode ends with the last row; an album
card still opens its menu). The pixel mapping is in the plan.

Remaining, in order:

  • Phase 2 — swipe right to queue. One measured hazard is already
    written down: the rows and the virtualizer are touch-action: auto,
    so the browser can claim a horizontal drag as a scroll and end the
    gesture in pointercancel. A row that wants a horizontal swipe has
    to declare touch-action: pan-y. This is the most likely way for
    swipe-to-queue to work in Chromium and not on the phone.
  • Phase 3 — the other three surfaces (queue-panel, both playlist
    detail views). Mostly wiring: all four already share
    SelectionController, and selection mode is a flag on it.
  • Phase 4 — what this leaves for #67. The inline explore-links
    are a single-click target inside a row whose tap now plays. Tap-to-play
    wins on touch today; #67 is where the links get their own answer.

Two open questions, in the plan rather than guessed:

  1. Should selection mode push a history entry, so back is the escape?
    The shell owns the stack (#6/#55) and #55 settled the shape for the
    overlaid queue — but the queue is a place and a mode is not, so it
    wants its own argument.
  2. Answered already for one case, and recorded because the next gesture
    will meet it: a control inside a row keeps its own tap (the
    favourite icon, 44px since #56), by the rule the shortcut service
    uses for a focused control that owns a key.
**Phase 1 landed** (PR #199, merged as `ff3875b`): tap plays, hold selects. The design is `.planning/plans/active/019-android-touch-model.md`; #63 stays open and claimed for phases 2–4. **The two divergences from the report are argued in the plan and both came out of measurement.** There is no double-tap — the report's *tap plays* and *double-tap opens the menu* cannot both be honoured, because the first tap of a double tap is a single tap until the interval expires. Measured on the device the play path is ~100ms (155/123/85/56/91), the app's own `DOUBLE_CLICK_GRACE_MS` is 250, so the discriminator is 3.5× the primary interaction to reach a menu the hold already reaches. The menu and the selection bar are one surface instead, which is also the platform's convention. And the predicate is `pointerType`, per event, rather than a platform or a viewport — #64's rule, with #64's warning: on a width, an Android tablet at 600px gets desktop semantics on a touchscreen. **The finding worth carrying into every future gesture:** Chrome 113's Android WebView **fires its own `contextmenu` on a long press**. `long-press.ts` stood down when a trusted one arrived, which was correct only while both paths ended in a context menu. Once a hold can mean something else, standing down means the gesture silently does the *old* thing — and no browser tier can see it, because dispatched pointer events never make a browser synthesise one. All 26 tests passed on the broken build. The native event is a trigger now, not a competitor. **A tier changed shape here too.** `adb shell input tap` and `input swipe x y x y 700` reach the WebView as real pointer events, so the Android tier can now *perform* the thing under test rather than describe the page afterwards — which is how the above was found, and how the whole model was verified (tap plays; hold raises the bar at 1; taps toggle to 2 and back; the mode ends with the last row; an album card still opens its menu). The pixel mapping is in the plan. **Remaining, in order:** - **Phase 2 — swipe right to queue.** One measured hazard is already written down: the rows and the virtualizer are `touch-action: auto`, so the browser can claim a horizontal drag as a scroll and end the gesture in `pointercancel`. A row that wants a horizontal swipe has to declare `touch-action: pan-y`. This is the most likely way for swipe-to-queue to work in Chromium and not on the phone. - **Phase 3 — the other three surfaces** (`queue-panel`, both playlist detail views). Mostly wiring: all four already share `SelectionController`, and selection mode is a flag on it. - **Phase 4 — what this leaves for #67.** The inline `explore-link`s are a single-click target inside a row whose tap now plays. Tap-to-play wins on touch today; #67 is where the links get their own answer. **Two open questions**, in the plan rather than guessed: 1. Should selection mode push a history entry, so back is the escape? The shell owns the stack (#6/#55) and #55 settled the shape for the overlaid queue — but the queue is a *place* and a mode is not, so it wants its own argument. 2. Answered already for one case, and recorded because the next gesture will meet it: a control inside a row keeps its own tap (the favourite icon, 44px since #56), by the rule the shortcut service uses for a focused control that owns a key.
logan commented 2026-08-22 05:43:40 +00:00
Author
Collaborator
Copy Link
Copy Source

Phases 2, 3 and 4 are in PR #201, which finishes this. Phase 1 was
#199. The plan has moved to
.planning/plans/completed/019-android-touch-model.md.

Three things worth having on the issue rather than only in the plan.

The touch-action finding in the plan was half the answer. Chrome
113's WebView cancels the pointer stream ~16px into any drag whatever
touch-action says — measured at auto, pan-y and none alike,
while touchmove kept firing throughout. So the swipe is recognised
from touchmove, and a claimed one calls preventDefault() on a
non-passive listener. Both halves are required: with the
preventDefault in place but touch-action back at auto, the
gesture died after one move.

draggable="true", which the plan flagged as a possible competitor,
is not one — no dragstart fires from a touch drag on this WebView.

Phase 1 had a race, and the device found it. The native
contextmenu arrives in either order and only one was handled: our
500ms timer firing first and Chrome delivering its own menu 50–70ms
later was suppressed by nothing, so the context menu opened over the
selection bar in two holds out of four. Six clean after the fix.
Neither browser tier can see it, since neither synthesises a
contextmenu from a dispatched press.

Two things this deliberately did not do.

The queue panel has no swipe. A right swipe means add to the queue
everywhere it exists and a queue row is already in the queue; the only
thing it could mean there is remove, which is the same gesture with
the opposite effect one screen away. Removing a queue row is on the row
itself, on its sheet since #60, and now on its selection bar.

And selection mode does not answer the back gesture. Escape leaves it,
from selection-bar so all four lists share one implementation, but
back belongs to the shell — four lists reaching for history is four
stacks, which is the fault navStack was deleted for. Filed as #200.

**Phases 2, 3 and 4 are in PR #201**, which finishes this. Phase 1 was #199. The plan has moved to `.planning/plans/completed/019-android-touch-model.md`. Three things worth having on the issue rather than only in the plan. **The `touch-action` finding in the plan was half the answer.** Chrome 113's WebView cancels the *pointer* stream ~16px into any drag whatever `touch-action` says — measured at `auto`, `pan-y` and `none` alike, while `touchmove` kept firing throughout. So the swipe is recognised from `touchmove`, and a claimed one calls `preventDefault()` on a non-passive listener. Both halves are required: with the `preventDefault` in place but `touch-action` back at `auto`, the gesture died after **one** move. `draggable="true"`, which the plan flagged as a possible competitor, is not one — no `dragstart` fires from a touch drag on this WebView. **Phase 1 had a race, and the device found it.** The native `contextmenu` arrives in either order and only one was handled: our 500ms timer firing first and Chrome delivering its own menu 50–70ms later was suppressed by nothing, so the context menu opened over the selection bar in two holds out of four. Six clean after the fix. Neither browser tier can see it, since neither synthesises a `contextmenu` from a dispatched press. **Two things this deliberately did not do.** The queue panel has no swipe. A right swipe means *add to the queue* everywhere it exists and a queue row is already in the queue; the only thing it could mean there is *remove*, which is the same gesture with the opposite effect one screen away. Removing a queue row is on the row itself, on its sheet since #60, and now on its selection bar. And selection mode does not answer the back gesture. Escape leaves it, from `selection-bar` so all four lists share one implementation, but back belongs to the shell — four lists reaching for `history` is four stacks, which is the fault `navStack` was deleted for. Filed as **#200**.
logan closed this issue 2026-08-22 05:54:48 +00:00
gitea-actions bot removed the
Status
In Progress
 label 2026-08-22 05:54:59 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Area/Design
Area/Downloads
Area/Explore
Area/Library-UI
Area/Metadata
Area/Packaging
Area/Player
Area/Queue
Area/Settings
Area/Shell-Nav
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
 Platform/Android
Platform/Desktop
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Medium
3
The priority is medium
Priority
Low
4
The priority is low
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
In Progress
Somebody is actively working on this right now
No labels Area/Library-UI Kind/Feature Platform/Android
Priority
High
2
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
logan yonlu
No Assignees logan
1 Participants
Notifications
Due Date
No due date set.
Depends on
#60 Android: context menus should be a bottom sheet, not a popover that gets clipped
yonlu/yellowjacket
Reference: yonlu/yellowjacket#63