main
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tha.Les
|
9b655d5a07 |
Windows: leaner portable package and opt-in in-app updater (#421) (#423)
* feat(windows): leaner portable package and opt-in in-app updater (#421) Issue #421 asked for Python embedded in a single EXE so updating would not mean copying ~20k loose files over an existing install. A onefile EXE is not viable for this stack (onefile modes re-extract the whole multi-GB payload on every launch, and torch/onnxruntime fight frozen-import hooks), so this addresses the root cause instead: ship less, and stop making users hand-copy a full zip for a release that only changed app code. Leaner package (make-portable.ps1): - Stripping is now unconditional. The -StripVenv opt-in gate was a silent regression risk: nothing stopped a future workflow edit from shipping the unstripped venv with no error. - Also strips stdlib base/Lib/test and per-package test/tests dirs. - Deliberately does NOT strip .dist-info/RECORD. pip needs it to replace a package, and install_cuda_torch pip-installs into this venv on every NVIDIA machine at first run; removing it yields "Failed to uninstall ... missing RECORD file". - Adds a post-strip import check so an over-aggressive strip fails the build rather than a release. Updater (main.rs, catalog.js): - New commands installed_runtime_id, download_app_update, apply_app_update. - Opt-in: the check on launch is unchanged, but download and apply are each an explicit click. It never auto-applies and never interrupts a running job. - Replaces StemDeck.exe and backend/ only. python/ is never touched, because an NVIDIA install rewrites it with CUDA torch at first run and torchDeviceSettled skips ensure_torch_device once the device is cuda, so swapping the directory would silently drop that machine to CPU with no recovery. - The runtime id (uv.lock + interpreter major.minor) is a compatibility gate, not a download trigger: if a release changed the Python dependency set the updater stands down and points at the full download. Only 19 of the last 200 commits touch uv.lock, so the fast path covers most releases. - apply_app_update stages and validates everything before any destructive rename, stops the backend synchronously first (the existing stop_backend returns before the process dies, which would have made every update fail on Windows), and retries renames past transient AV/indexer handles. - Known gap, documented in code: the two exe renames are not atomic. A hard crash in that window leaves StemDeck.exe.old needing a manual rename. Closing it needs a bootstrap launcher that is never itself replaced. CI publishes -app.zip, its .sha256 and -runtime-version.json alongside the unchanged full zips. Fresh installs are unaffected. i18n: the 5 new strings are translated into all 7 language tables, not just English. t() falls back to English silently, so an English-only key looks correct in testing and ships untranslated to six locales. Verified: Windows and Linux (WSL) both compile clean with no new clippy warnings, 39 Rust tests pass on both, JS suites pass, ruff clean. Two new unit tests pin the JSON contract between the PowerShell writer and the Rust reader. Not yet verified: no end-to-end run against a real release. * fix(updater): make the in-app update actually work, verified end to end (#421) Built both packages on a real Windows box and drove the whole flow. Four bugs that only surfaced by running it, none of which static checks could see. 1. Stale version after updating. app_version() read installed dist metadata, which lives in python/ -- the directory the updater deliberately never replaces. A self-updated install kept reporting the old version and would re-offer an update it had already applied, forever. It now prefers the app layer's static/version.json, which moves with backend/. Gitignored, so Docker and source checkouts still fall through to the hatch-vcs metadata. Proven: after a real update, python/ dist-info says 0.13.0 while /api/health reports 0.13.1. 2. The page CSP blocked the whole feature. The UI is served over http by the Python backend, so its connect-src applies: api.github.com is allowed, github.com and objects.githubusercontent.com are not, and that is where release assets live. Fetching the checksum and runtime id from JS was refused, so the pill would simply never appear. Those two reads moved into Rust (check_app_update), whose HTTP client is not bound by the page CSP, so the policy from #171 stays exactly as tight as it was. 3. plugin:event|listen refused by the Tauri ACL. App-defined commands are not ACL-gated but plugin commands are, and the capability does not cover the remote http origin the UI is served from. The progress bar is now indeterminate instead of granting a remote origin event permissions to put a percentage on a 5 MB download. 4. The post-strip import check re-bloated the package. Running Python regenerated 1,912 files / 39 MB of __pycache__ that the strip had just removed, cancelling nearly all of it: the net saving was 180 files. Swept once after the last interpreter run, and backend/ no longer ships a developer's local __pycache__ either. Also: the *.old sweep now runs on every launch rather than only on a version change. apply_app_update relaunches then exits, so on the first launch of the new build Windows still holds StemDeck.exe.old open, the delete fails silently, and gated on a change that already happened it would never retry. Observed for real: 15.7 MB stranded. Verified swept on the next launch. UI: "Update now" is an accent pill BESIDE Download, not a replacement, so the zip stays one click away and is the escape hatch if an update fails. Measured against the published v0.13.0 package: 18,143 -> 16,056 files (-2,087, -11.5%) and 883 -> 850 MB. The real win for #421 is the update path itself: 5 MB and 123 files instead of 284 MB and 16,056. Verified on this machine: a real 6-stem Demucs separation through the stripped package; the full notify -> Update now -> download -> restart -> relaunch cycle, after which user data (job, 7 stems, 130 MB of models), portable.txt, cpu-only and python/ were all untouched; and the safety gate correctly declining, with no download attempted, when the release's runtime id differs. Not covered: the NVIDIA package was not built, though the risk that motivated the gate is structurally gone now that python/ is never swapped. * fix: address code-quality review on the version-source change (#421) Both findings from the automated review were fair. Narrow the bare `except Exception: pass` in app_version() to (OSError, ValueError, AttributeError). That is bandit B110, which this repo's own security conventions call out. The three cover every real failure here -- absent or unreadable file, invalid JSON or bad encoding, and valid JSON that is not an object so has no .get -- while letting an actual bug in the function surface instead of silently degrading the reported version. Bandit now reports no issues for the file. Use one import style in test_health_api.py so app.main is no longer imported both as `import app.main as main` and `from app.main import app` in the same module. Also added a "[]" case: JSON that parses but is not an object, which is the AttributeError branch the narrowed except now names explicitly. * feat(updater): extend the in-app update to Linux (#421) Linux ships the same shape as Windows -- executable, backend/ and python/ side by side -- so the updater generalises rather than needing a second design. The platform-specific parts are now three small seams: the archive format, the executable name, and one new gate. Rust: - widen the updater's cfg gates from `windows` to `any(windows, linux)`, and replace extract_zip_archive with extract_update_archive, which uses zip on Windows and the existing extract_tar_archive on Linux - APP_EXE_NAME so the swap and the leftover sweep stop hardcoding StemDeck.exe - stop_backend_and_wait now sends SIGTERM and waits before escalating on unix, matching what stop_backend already does on window close - new app_root_is_writable gate: packaging/linux/install.sh offers a global install into /opt/stemdeck, which is root-owned while the app runs as the user. check_app_update declines up front rather than failing part way through a swap. Windows portable installs are user-writable by construction, but the probe is cheap and honest on both. tar rather than zip on Linux is deliberate: it preserves the executable bit. A zip would land StemDeck without +x and the relaunch after an update would fail with a permission error. Packaging (scripts/linux/make-portable.sh): - write python/runtime-version.json using the same uv.lock + interpreter major.minor formula as the Windows script, so the compatibility gate behaves identically on both - bring the strip to parity: stdlib test/, per-package test/tests, a post-strip import check, and a final __pycache__ sweep after the last interpreter run - PUBLISH_UPDATER_ASSETS=1 emits the slim app-layer tarball, its checksum and the runtime marker; wired into the CPU build in linux-release.yml since StemDeck and backend/ are identical between both variants Frontend: updaterAssetNames() maps the platform to its asset names, and the wiring is gated on that rather than on os === "windows". macOS is deliberately still excluded, and the comments now say why rather than just that it is: backend_dir() resolves the backend inside the downloaded runtime pack rather than the .app, so its app layer is a different thing and the existing runtime-pack updater already covers most of it. Verified: both platforms compile clean with no new clippy warnings, 42 tests on Windows and 43 on Linux (the extra one is the read-only-root gate, which is meaningless on Windows). The app-layer archive was round-tripped on Linux to confirm it contains exactly StemDeck + backend/, that python/ does not leak into it, that the executable bit survives, and that replacing a running binary works. Not yet run end to end against a real Linux release. * fix(updater): see pre-releases, and compile the Rust in CI (#421) Two gaps that would each have undermined the update flow on release day. The update check polled /releases/latest, which GitHub defines as the most recent NON-PRERELEASE, non-draft release. Ship a version with the pre-release box ticked and it becomes invisible: no notification, no update button, on any platform, with nothing in the logs to explain it. StemDeck has always published even its alphas as normal releases (v0.8.0-alpha.17 has prerelease=false), which is the only reason this has not bitten yet -- it was a trap waiting on someone ticking a box. Now polls the releases list and takes the newest non-draft, so it is correct either way. Drafts stay excluded: they are already invisible unauthenticated, and a maintainer should not be offered a release whose assets do not exist yet. windows-check.yml and macos-check.yml now also run on pull requests that touch desktop/src-tauri/**, not workflow_dispatch only. This PR added roughly 600 lines of mostly cfg-gated Rust across two commits and every CI check passed without compiling a single line of it; the comment at the top of windows-check.yml notes that exact gap already shipped a broken Windows build in v0.11.1's first release attempt. Scoped by path so the self-hosted runners see no extra load from the majority of PRs, which never go near src-tauri. This also gets the macOS branch compiled for the first time. Local verification covered Windows and Linux, so the cfg(not(any(windows, linux))) arm of the three updater commands has never been near a compiler. * test(e2e): match the releases-list shape the app now polls (#421) The update-check stub returned a single release object, which was right for /releases/latest. The app now polls the releases list so a pre-release is still seen, so the fixture has to return an array or checkForUpdate bails and the release card never appears. Caught by frontend-e2e on the previous commit, which is the suite doing exactly its job: the only assertion that covers this path is report-failure.spec.mjs:98, and it went red immediately. * feat(i18n): add French, and make the runtime id line-ending independent French is a complete table, not a partial one: 435 keys, the same set German and Portuguese carry (English's 443 minus the ten Polish-only .few/.many forms and the bare upload.skippedFiles, plus singular forms for the three playlist.skip.* families). French takes the one/other buckets, so plural() needs no change. Verified with the checks from .claude/rules/i18n.md: the drift check reports clean, and separately there are zero {placeholder} mismatches and zero HTML tag mismatches against English. The 27 strings identical to English are genuinely identical in French (Piano, Solo, Transport, Position, LUFS, Standard, Port, the brand names, CUDA (NVIDIA), MPS (Apple Silicon)). Separately: the runtime id was being computed from the raw bytes of uv.lock, so a Windows checkout with core.autocrlf=true hashed CRLF and Linux hashed LF, and the same lockfile produced two different ids -- caught by building the Linux package and seeing py3.12-d74d6ef80c5e9d1f where Windows had produced py3.12-dbda45e38e1044cf. Each platform stayed self-consistent so the gate still worked, but the id would shift spuriously if a runner's autocrlf ever changed, silently declining app-only updates that were in fact compatible. Both scripts now hash the content with newlines normalised; PowerShell, bash and a reference Python implementation all agree on d74d6ef80c5e9d1f. * chore: pin Unraid template to 0.14.0 Per .claude/rules/unraid-template-version.md this is an explicit decision each time, not a default. Confirmed for this release. The 0.14.0 GHCR image is published by docker-publish.yml when the release is created, so the tag exists shortly after this lands. --------- Co-authored-by: Thales <> |
||
|
Tha.Les
|
b50e221ac0 |
fix: pin librosa <1 and add audioread explicitly (#407) (#408)
librosa 1.0.0 dropped its audioread dependency, but audio-separator (vocal split, #275) still imports audioread directly. uv.lock already resolved librosa 0.11.0, so dev/CI never hit this, but the packaging scripts install from pyproject.toml rather than the lockfile, so an unbounded upper bound let the shipped macOS/Windows/Linux runtime packs silently pick up librosa 1.0.0 and lose audioread, breaking only the vocal-split subprocess in the packaged artifact. Also add audio_separator/onnxruntime to all three packaging scripts' import verification, platform-guarded on Intel macOS, so a broken vocal-split dependency chain fails the build instead of shipping. Co-authored-by: Thales <> |
||
|
Tha.Les
|
5565b216ba |
feat(linux): add an optional installer for desktop integration (#364)
Implements #342. StemDeck stays portable: extract the tarball, run ./StemDeck, and none of this is required. install.sh is there for people who would rather launch from their applications menu. It installs the package it sits in and never downloads anything, so the version and the CPU/NVIDIA variant come from the package itself (backend/static/version.json and the cpu-only marker) and cannot drift from the build being installed. That also removes any need to verify a second download. Design notes, mostly things the reference installer in #342 got wrong: - Install is atomic. The new copy goes to <target>.new and is verified before the old one is moved aside, so a failure partway leaves the working install untouched. Removing the old copy first is what made a failed upgrade in that fork leave the machine with no StemDeck, no launcher and no manifest recording where it had been. - A failed copy cleans up its own staging directory rather than leaving a package-sized partial on disk. - Exec is quoted, so an install path containing a space still launches. - Version comparison is semver-aware. sort -V ranks 0.8.0-alpha.17 above 0.8.0, which would tell every pre-release user they were current the day a stable release shipped. - Reading a missing manifest key yields empty rather than killing the script, which under set -euo pipefail is what a grep|head|cut pipeline does. - Global installs put the launcher in /usr/share/applications and the icon in /usr/share/pixmaps, so other users on the machine can see it. - Installing from inside the destination is refused rather than moving the running script out from under bash. - Non-x86_64 machines get a clear refusal instead of a binary that cannot run. User data is never touched. Stems live in ~/Documents/StemDeck and the runtime, models and logs in $XDG_DATA_HOME/stemdeck, both outside the install directory. Legacy data/ from pre-migration builds is carried across an upgrade, and uninstall refuses to delete it, leaving the folder and saying why. tests/linux/test_install_sh.sh runs the real installer against a synthetic package in a throwaway HOME: 52 checks covering install, upgrade, the failed-upgrade case, uninstall, corrupt manifests, paths with spaces, legacy data, self-install, arch refusal and the semver table. CI runs it on Linux with shellcheck and desktop-file-validate. Closes #361 |
||
|
Tha.Les
|
f9d7182f4b |
build(linux): stage desktop-entry assets into the portable tarball (#363)
Prep for the optional Linux installer (#342). Carrying the icon and a .desktop template inside the package is what lets the installer be self-contained: no second download, and no asset URL that can drift from the release being installed. The Tauri icon is already square at 1024x1024, so it doubles as the desktop icon with no separate artwork to keep in sync. Exec= is quoted in the template. The freedesktop spec splits Exec on whitespace, so the unquoted form used by the reference installer in #342 produces an entry that tries to run ".../My" when the user installs to a path like ~/My Apps. It is invisible until someone picks a custom directory, which is why it is pinned by a test. Both variants pick this up: the CPU and NVIDIA packages run the same script with CPU_ONLY toggled. install.sh itself, and the README-LINUX.txt change documenting it, land with #361 -- this commit deliberately ships nothing that references a file which does not exist yet. Closes #360 |
||
|
Tha.Les
|
655b0d0e95 |
fix(linux): install CUDA runtime deps with the GPU torch wheel (#324) (#325)
The Linux NVIDIA package could not start its backend at all. Setup detected
the GPU and pip-installed torch==X+cuXXX with --no-deps, mirroring Windows.
But Linux CUDA wheels do not bundle the CUDA runtime -- they dlopen
libcublas/libcudnn/... out of the nvidia-* PyPI packages at import time, and
make-portable.sh strips exactly those packages to keep the tarball under
GitHub's 2 GiB asset cap. The result was a CUDA torch with no CUDA runtime:
ValueError: libcublas.so.*[0-9] not found in the system path
app/main.py imports torch at module scope, so this killed the backend
outright ("backend did not become healthy within 90 seconds") on every
launch, not just GPU work.
- install_cuda_torch now runs a second, dependency-resolving pip pass on
Linux only (cuda_wheel_needs_runtime_deps). Same specs and index, without
--no-deps/--ignore-installed, so pip sees torch as satisfied and installs
only the missing nvidia-* wheels. Windows keeps the single --no-deps swap:
its wheels carry the DLLs in torch/lib and pulling ~2.5 GB of nvidia-*
there would be pure waste. macOS is untouched (MPS path).
- Restore CPU torch when verify_cuda_torch fails. Recording torchDevice=cpu
was never enough -- an unloadable CUDA wheel stays on disk and keeps the
backend from importing torch at all. New reason
"cuda-verify-failed-cpu-restore-failed" when even that fails.
- Extract run_pip_install (PID tracking, 20 min timeout, stderr logging) so
both installs and the restore share one path.
- Add the missing "Linux tar.gz" option to the bug report template, as the
reporter noted.
Co-authored-by: Thales <>
|
||
|
Tha.Les
|
da93c5ee44 |
feat: export as MP4 (karaoke video) for MP4 uploads and YouTube (#226)
* feat: export as MP4 (karaoke video) for MP4 uploads and YouTube (#219) Add an MP4 export that muxes the current mixer state (e.g. vocals muted) with the source video, producing a karaoke-style video. Backend: - Preserve a silent video.mp4 from .mp4 uploads (stream-copy, no re-encode). - YouTube jobs do a best-effort video-only download (H.264/avc1, <=720p) to video.mp4, decoupled from the audio source so failures degrade to audio-only. New STEMDECK_VIDEO_MAX_HEIGHT config. - GET /api/jobs/{id}/video.mp4 streams a fragmented MP4: the amix audio graph encoded as AAC, video stream-copied. - has_video flag on Job, surfaced in state and persisted to metadata. Frontend: - MP4 added as a fourth export format (WAV/MP3/FLAC/MP4), shown only for jobs with a preserved video track. In MP4 mode, Export Mix produces the karaoke video and the audio-only Stems/Region rows are hidden. SoundCloud and plain audio uploads are audio-only (no MP4 option). * feat: bundle FFmpeg on Linux via first-launch download Linux no longer requires `sudo apt install ffmpeg`. The desktop shell now downloads a static FFmpeg build into the user data dir on first launch (like Windows/macOS), falling back to a system ffmpeg on PATH when present. This also fixes Demucs failing to decode compressed sources, since the download lands in data_dir/ffmpeg which config.json already adds to PATH. - ensure_ffmpeg: prefer a system ffmpeg, else download_linux_ffmpeg. - download_linux_ffmpeg: fetch the .tar.xz, extract with system tar, copy ffmpeg + ffprobe into data_dir/ffmpeg. STEMDECK_FFMPEG_URL overrides. - Widen download_file and make_executable from macos to unix so Linux reuses them. - Not bundled in the tarball, so we don't redistribute FFmpeg. - Update Linux README/notices/packaging comment to drop the ffmpeg apt step. * style: apply ruff format to MP4 export code --------- Co-authored-by: Thales <> |
||
|
Tha.Les
|
3995e9236f |
fix: remove orphaned nvidia-* CUDA packages from Linux bundle (#225)
Both Linux tarballs were 2.5 GB (over GitHub's 2 GiB asset limit) even with CPU torch. Root cause: 'uv pip install <project>' pulls the default Linux torch, which is the CUDA build, dragging in nvidia-* runtime packages (cuDNN/cuBLAS/NCCL/...) and triton (~2.5 GB). The CPU torch swap uses --force-reinstall --no-deps, so torch becomes CPU but those CUDA packages stay installed and orphaned, bloating the tarball. Uninstall the nvidia-* packages and triton after the swap. CPU torch does not use them and the NVIDIA variant re-downloads CUDA at first run. Co-authored-by: Thales <> |
||
|
Tha.Les
|
421f2b344c |
fix: ship CPU torch in Linux NVIDIA variant; download CUDA at first run (#222)
The Linux NVIDIA tarball baked the full CUDA torch wheel, producing an asset >2 GiB that GitHub release uploads reject (size must be < 2147483648). On Linux the default PyPI torch wheel bundles the CUDA runtime (~2.5 GB), unlike Windows where the default wheel is CPU-only. The Windows NVIDIA package therefore never baked CUDA -- it ships CPU torch and downloads the CUDA wheel at first run via the desktop shell (install_cuda_torch, which is cfg(not(macos)) and already covers Linux). Mirror that on Linux: bake the small CPU torch in both variants; the NVIDIA variant differs only by omitting the cpu-only marker, so the shell detects the GPU and downloads CUDA on first launch. Keeps both tarballs well under the 2 GiB limit. Co-authored-by: Thales <> |
||
|
Tha.Les
|
8131900d0a |
feat: Linux portable builds (CPU + NVIDIA) and release workflow (#220)
* feat: add CPU-only Linux portable build and release workflow
Adds a Linux .tar.gz portable package mirroring the existing Windows/macOS
build paths. Bundles a python-build-standalone runtime (CPU torch + demucs)
plus the Tauri binary so users extract and run ./StemDeck.
- scripts/linux/make-portable.sh: stages PBS Python, force-installs CPU-only
torch, builds the Tauri binary, and produces StemDeck-Linux-x64.tar.gz with
the backend/app + python/ layout find_repo_root resolves at runtime.
- .github/workflows/linux-release.yml: builds on hosted ubuntu-latest on
release publish; installs Tauri v2 apt deps + uv, ClamAV-scans, uploads.
- packaging/linux/{README-LINUX,THIRD_PARTY_NOTICES}.txt: extract-and-run
instructions noting ffmpeg + WebKitGTK are system (apt) prerequisites.
FFmpeg is not bundled: the Linux shell expects ffmpeg on PATH. NVIDIA/CUDA
and AppImage variants are intentionally deferred to later phases.
* fix: don't set PYTHONHOME on Linux (breaks PBS stdlib resolution)
The Linux backend failed to start with 'ModuleNotFoundError: No module
named encodings'. PYTHONHOME was being set to python/bin instead of the
prefix python/, so CPython looked for its stdlib under python/bin/lib and
could not boot.
Linux bundles python-build-standalone exactly like macOS, which detects
its own prefix by walking up from bin/ and must NOT have PYTHONHOME set.
The two PYTHONHOME sites were gated #[cfg(not(target_os = "macos"))],
wrongly including Linux alongside Windows. Only Windows -- whose portable
venv keeps the stdlib under base/Lib -- needs PYTHONHOME, so gate both
sites (start_backend and python_stdlib_ok) to #[cfg(windows)].
This also fixes the latent inconsistency where probe_runtime reported
Python ready (python_stdlib_ok set PYTHONHOME=python, the correct prefix)
while start_backend set PYTHONHOME=python/bin and failed.
* feat: add NVIDIA/CUDA Linux portable variant
Adds a second Linux package, StemDeck-Linux-x64.NVIDIA.tar.gz, with
CUDA-enabled torch baked in (mirrors the Windows NVIDIA variant).
- make-portable.sh: CPU_ONLY toggle (default 1). CPU_ONLY=0 keeps the
project's default torch wheel, which on Linux x86_64 is the CUDA build,
and omits the cpu-only marker so the desktop shell detects the GPU and
uses CUDA at runtime. No app-side changes needed -- the CUDA detection/
install path in main.rs is already cfg(not(macos)) and covers Linux.
- linux-release.yml: builds both variants in one job. CPU first (full Tauri
build), then NVIDIA with SKIP_TAURI_BUILD=1 reusing the same binary. Adds
a free-disk-space step (CUDA bundle is several GB) and drops each
uncompressed stage after taring to stay within the hosted runner's disk.
- README-LINUX.txt: documents both variants and the NVIDIA driver
prerequisite (nvidia-smi must work; CUDA runtime is bundled, no toolkit
install needed; falls back to CPU when no GPU).
* ci: run Linux release on self-hosted linux/x64 runner
Targets the org's self-hosted wsl2 runner ([self-hosted, linux, x64])
instead of hosted ubuntu-latest, matching the Windows/macOS release
jobs. Drops the free-disk-space step: it was a hosted-runner workaround
and would needlessly rm system directories on a persistent self-hosted
box (WSL2's virtual disk has ample room for the CUDA bundle).
* ci: add workflow_dispatch test build for Linux release
Lets you run the full two-variant build + ClamAV scan on the self-hosted
runner without publishing a release, to validate the runner toolchain and
the CUDA build. Resolves the version from a manual input (default 0.0.0,
must be valid PEP 440) instead of the branch ref, and skips the upload
step on non-release events.
---------
Co-authored-by: Thales <>
|