Fixed photon-node loading for Docker and multiple

Summary

Closed, Completed

Understand
Investigated photon-node usage and binary build.
Understand
Verify
Tested compatibility in Node.js, Bun binary, and tsx.
Verify
Build
Implemented lazy-loaded wrapper for photon.ts.
Build
Verify
Verified fix in all environments including Docker.
Verify

Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/784 test in docker locally, temp container

For each issue:

Read the issue in full, including all comments and linked issues/PRs.
For bugs:
- Ignore any root cause analysis in the issue (likely wrong)
- Read all related code files in full (no truncation)
- Trace the code path and identify the actual root cause
- Propose a fix
For feature requests:
- Read all related code files in full (no truncation)
- Propose the most concise implementation approach
- List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

This is a bug report. The standalone Pi binary is broken on x86 Linux since v0.46.0. The error shows it's trying to load a WASM file from a hardcoded path that was used during build time on GitHub Actions.

The issue is with the @silvia-odwyer/photon-node package. Let me investigate:

When was this dependency added?
How is it being loaded?
Why does it have a hardcoded path?

Let me look at the relevant code files.

Let me investigate the issue. I'll look at the photon-node usage and how the binary is built.

Readimage-convert.ts

Readimage-resize.ts

The issue is clear now. Looking at the last lines of photon_rs.js:

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

This is trying to load the WASM file using require('fs').readFileSync(). When Bun bundles the binary, this path gets hardcoded to the absolute path during build time on GitHub Actions: /home/runner/work/pi-mono/pi-mono/node_modules/@silvia-odwyer/photon-node/photon_rs_bg.wasm.

But Bun can handle this if the wasm file is embedded. Let me look at how the binary is built.

Readpackage.json

I see the issue now. The build command is:

This bundles the JS code, but the WASM file from @silvia-odwyer/photon-node isn't embedded properly. The photon_rs.js file tries to read the WASM file at runtime with:

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

But when bundled, __dirname becomes the build machine's path: /home/runner/work/pi-mono/pi-mono/node_modules/@silvia-odwyer/photon-node/.

Let me check when photon was added to the project and how other wasm modules might be handled.

Let me check what changed between v0.45.7 and v0.46.0 when the issue started:

The issue was introduced in v0.46.0 with commit 6bf073f1 that replaced wasm-vips with @silvia-odwyer/photon-node. Let me see what issue #710 was about and if wasm-vips had a special handling.

The root cause is clear: photon_rs.js from @silvia-odwyer/photon-node uses:

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

const path = require('path').join(__dirname, 'photon_rs_bg.wasm');const bytes = require('fs').readFileSync(path);

When Bun compiles the binary, this path becomes hardcoded to the build machine path (/home/runner/work/...). The WASM file isn't embedded in the binary.

Let me verify Docker can reproduce this and then explore potential solutions.

Now let me test locally in Docker to confirm the issue:

>cd /Users/badlogic/workspaces/pi-mono && docker run --rm -it debian:bookworm-slim bash -c "
apt-get update && apt-get install -y wget ca-certificates >/dev/null 2>&1 && \
mkdir /tmp/pi-test && cd /tmp/pi-test && \
wget -q https://github.com/badlogic/pi-mono/releases/download/v0.47.0/pi-linux-x64.tar.gz && \
tar -xzf pi-linux-x64.tar.gz && \
./pi/pi --version 2>&1 || echo '--- test failed as expected ---'
"

Rosetta issue (running x86 on ARM Mac). Let me try a different approach. I should test on a proper x86 platform but since I can confirm from the issue report that the error is real, let me focus on analyzing the root cause and proposing a fix.

The root cause is clear:

@silvia-odwyer/photon-node has a photon_rs.js file that loads the WASM file at runtime using fs.readFileSync(__dirname + '/photon_rs_bg.wasm')
When Bun compiles the standalone binary, __dirname becomes the absolute path from the build machine (/home/runner/work/...)
At runtime on the user's machine, that path doesn't exist, so it fails

The solution needs to ensure the WASM file is embedded in the binary.

Possible solutions:

Use Bun's --asset-naming or file embedding: Bun can embed assets but needs explicit configuration.
Create a wrapper module: Create our own module that uses a different loading approach:
- Load the WASM as a base64 string embedded at build time
- Or use Bun's special import syntax for binary files
Dynamic import with URL: Some WASM modules support loading via URL/fetch
Patch at build time: Modify the photon_rs.js during build to use an embedded approach
Use a different image library: One that doesn't have this WASM loading issue

Let me investigate how Bun handles WASM files in compiled binaries:

Let me investigate how Bun handles WASM in compiled binaries and check for potential solutions:

Interesting! The package has two entry points:

photon_rs.js (main): Uses CommonJS style with require('fs').readFileSync(path) to load WASM at runtime
photon_rs_bg.js: Uses ESM-style import with import * as wasm from './photon_rs_bg.wasm'

This is important. When using ESM (photon_rs_bg.js), Bun can potentially handle the WASM import correctly. But when using CJS (photon_rs.js), it uses filesystem access which breaks in compiled binaries.

The current code in image-resize.ts and image-convert.ts uses:tsimport photon from "@silvia-odwyer/photon-node";

This resolves to the main field in package.json, which is photon_rs.js (the problematic one).

Solution approaches:

Import from photon_rs_bg.js directly: This might work better with Bun's bundler since it uses ESM imports for WASM.
Create a wrapper that loads WASM differently: Load the WASM bytes directly and instantiate manually.
Use Bun's asset embedding: Use bun: special imports or embed the WASM as base64.

Let me check if we can use the ESM entry point directly.

Let me check how the imports work and if there's an ESM variant we could use:

The ESM version (photon_rs_bg.js) uses import * as wasm from './photon_rs_bg.wasm' which should allow Bun to properly embed the WASM file when compiling.

However, looking at the current code:ts import photon from "@silvia-odwyer/photon-node";

This imports from the main field (photon_rs.js) which uses fs.readFileSync().

Looking at photon_rs_bg.js, it exports individual functions, not a default photon object. The structure is different from photon_rs.js.

Let me check what photon_rs.js exports vs photon_rs_bg.js.

Both PhotonImage and SamplingFilter are exported from photon_rs_bg.js. The fix should be:

Import from @silvia-odwyer/photon-node/photon_rs_bg.js instead of @silvia-odwyer/photon-node
Change the import style to use named imports since it's an ESM module

The current code does:tsimport photon from "@silvia-odwyer/photon-node";Then uses:- photon.PhotonImage.new_from_byteslice(...)- photon.resize(...)- photon.SamplingFilter.Lanczos3

With the ESM variant, it would be:tsimport { PhotonImage, resize, SamplingFilter } from "@silvia-odwyer/photon-node/photon_rs_bg.js"; Then:- PhotonImage.new_from_byteslice(...)- resize(...)- SamplingFilter.Lanczos3This should allow Bun's bundler to properly handle the WASM import (import * as wasm from './photon_rs_bg.wasm') and embed it in the compiled binary.Let me create a summary of the analysis.

Analysis Summary

Bug: Standalone Pi binary broken on x86 Linux since v0.46.0

Root Cause:

The issue was introduced in commit 6bf073f1 (v0.46.0) when wasm-vips was replaced with @silvia-odwyer/photon-node for image processing.

The @silvia-odwyer/photon-node package has two entry points:

photon_rs.js (main, CJS): Loads WASM at runtime via fs.readFileSync(__dirname + '/photon_rs_bg.wasm')
photon_rs_bg.js (ESM): Uses import * as wasm from './photon_rs_bg.wasm'

The current code imports from the main entry point:

When Bun compiles the binary, __dirname in photon_rs.js becomes the absolute build-time path (/home/runner/work/pi-mono/pi-mono/node_modules/@silvia-odwyer/photon-node/), which doesn't exist on the user's machine.

Proposed Fix

Change the import to use the ESM entry point (photon_rs_bg.js) which uses ESM-style WASM imports that Bun can properly embed:

Files to modify:

packages/coding-agent/src/utils/image-resize.ts:

// Before:import photon from "@silvia-odwyer/photon-node";
// After:import { PhotonImage, resize, SamplingFilter } from "@silvia-odwyer/photon-node/photon_rs_bg.js";
// Update usages:// photon.PhotonImage.new_from_byteslice(...) → PhotonImage.new_from_byteslice(...)// photon.resize(...) → resize(...)// photon.SamplingFilter.Lanczos3 → SamplingFilter.Lanczos3

// Before:import photon from "@silvia-odwyer/photon-node";
// After:import { PhotonImage, resize, SamplingFilter } from "@silvia-odwyer/photon-node/photon_rs_bg.js";
// Update usages:// photon.PhotonImage.new_from_byteslice(...) → PhotonImage.new_from_byteslice(...)// photon.resize(...) → resize(...)// photon.SamplingFilter.Lanczos3 → SamplingFilter.Lanczos3

packages/coding-agent/src/utils/image-convert.ts:

// Before:import photon from "@silvia-odwyer/photon-node";
// After:import { PhotonImage } from "@silvia-odwyer/photon-node/photon_rs_bg.js";
// Update usages:// photon.PhotonImage.new_from_byteslice(...) → PhotonImage.new_from_byteslice(...)

// Before:import photon from "@silvia-odwyer/photon-node";
// After:import { PhotonImage } from "@silvia-odwyer/photon-node/photon_rs_bg.js";
// Update usages:// photon.PhotonImage.new_from_byteslice(...) → PhotonImage.new_from_byteslice(...)

This approach should allow Bun to properly embed the WASM file during compilation since the ESM import syntax is recognized by bundlers.

Risk: The photon_rs_bg.js file uses ESM-style import for the WASM file. This is the standard approach that bundlers like Bun understand and can embed properly. The fix is minimal and doesn't change functionality, only the module resolution path.

make the change

Readimage-resize.ts

Readimage-convert.ts

Editedimage-resize.ts

Editedimage-convert.ts

Editedimage-resize.ts

Editedimage-convert.ts

Readphoton_rs.d.ts

I need to add a TypeScript declaration for the ESM entry point. Let me create one:

Check passes. Let me verify the changes:

Readimage-resize.ts20 lines

Readimage-convert.ts

Now let me build and test locally:

Readtsconfig.build.json

Readtsconfig.json

Readtsconfig.base.json

The issue is that tsconfig.build.json explicitly excludes .d.ts files. This is typical to avoid emitting declaration files during build. I need to make the declaration file visible to TypeScript regardless of the build config.The correct approach is to use TypeScript's typeRoots or add the types directory to the config. But since the file is being excluded, I should probably:1. Add a reference in typeRoots 2. Or include it explicitlyActually, looking at the problem more carefully: the --noEmit check worked because it doesn't exclude .d.ts files. But the build config does. The solution is to either:1. Not exclude .d.ts files from the build config (they shouldn't be compiled anyway since they're declarations) 2. Or put the declaration in a global types directory that gets included

Let me check if there's a global types directory used across the monorepo:

Let me check if other packages have declarations and how they're handled. Actually, the simplest fix here is to modify the build config to not exclude .d.ts files since they aren't compiled anyway, just used for type checking. Or I could move the declaration to a location that isn't excluded.Alternatively, since the type declaration is just adding a module augmentation, I could put it in a file that TypeScript will read. Let me check what options exist.

Actually the cleaner solution is to remove the exclusion of .d.ts files from the build config, since TypeScript doesn't emit them during compilation anyway (they're just declarations). Or I could explicitly include my declarations.

The build config excludes .d.ts files. Let me fix the config to include the type declarations:

Editedtsconfig.build.json

Build succeeded (warnings are expected for WASM imports). Let me test the binary:

All checks pass. Here's a summary of the changes:

Files modified:

packages/coding-agent/src/utils/image-resize.ts: Changed import from CJS entry to ESM entry point (@silvia-odwyer/photon-node/photon_rs_bg.js), using named imports instead of default
packages/coding-agent/src/utils/image-convert.ts: Same change as above
packages/coding-agent/src/types/photon-node.d.ts (new): Added TypeScript declaration for the ESM entry point
packages/coding-agent/tsconfig.build.json: Updated to include type declarations from src/types/

Why this fixes the issue:

The CJS entry (photon_rs.js) loads WASM via fs.readFileSync(__dirname + '/photon_rs_bg.wasm'), which bakes the build machine's absolute path into the binary
The ESM entry (photon_rs_bg.js) uses import * as wasm from './photon_rs_bg.wasm', which Bun can properly embed during compilation

The fix should be tested on actual x86 Linux to confirm the WASM is correctly embedded in the standalone binary. The local macOS binary works, but the real test is the GitHub Actions build running on Linux.

we need a script in scripts/ that builds the distribution like we do in .github/workflows/build-binaries.yml so i can test locally

Readbuild-binaries.yml

Created scripts/build-binaries.sh. Usage:

bash

# Build all platforms./scripts/build-binaries.sh
# Build only linux-x64 (for testing the fix)./scripts/build-binaries.sh --platform linux-x64
# Skip reinstalling cross-platform deps (faster rebuild)./scripts/build-binaries.sh --skip-deps --platform linux-x64

# Build all platforms./scripts/build-binaries.sh
# Build only linux-x64 (for testing the fix)./scripts/build-binaries.sh --platform linux-x64
# Skip reinstalling cross-platform deps (faster rebuild)./scripts/build-binaries.sh --skip-deps --platform linux-x64

Output goes to packages/coding-agent/binaries/.

To test the fix in Docker:

bash

# Build linux-x64./scripts/build-binaries.sh --platform linux-x64
# Test in Dockerdocker run --rm -v $(pwd)/packages/coding-agent/binaries/linux-x64:/app debian:bookworm-slim /app/pi --version

# Build linux-x64./scripts/build-binaries.sh --platform linux-x64
# Test in Dockerdocker run --rm -v $(pwd)/packages/coding-agent/binaries/linux-x64:/app debian:bookworm-slim /app/pi --version

test it

Build complete. Now testing in Docker:

where are the dependencies installed to? does this override the node_modules and package-lock.json in this cwd?

The user is asking about the cross-platform dependencies installed by the build script. Let me check what the script does - it uses npm install --no-save --force which:

--no-save means it won't modify package.json
But it WILL install to node_modules
It won't modify package-lock.json because of --no-save

However, this could still pollute the local node_modules with packages for other platforms. Let me check what's happening.

Good catch. The npm install --no-save --force installs to the local node_modules/ directory. It won't modify package.json or package-lock.json, but it does add packages to node_modules/ that shouldn't be there for your current platform.

Let me check what got installed:

Yes, the cross-platform dependencies were installed to the local node_modules. This can pollute the user's development environment.

To clean this up, the user should run npm ci to restore the correct state. But ideally, the build script should either:

Work in a temp directory
Or clean up after itself
Or warn the user to run npm ci afterwards

Let me clean up for the user and fix the docker test command first.

Yes, it pollutes node_modules/. The script installed cross-platform bindings that don't belong on your Mac. To restore:

The proper solution would be to run the build in a Docker container to avoid polluting the host. But for quick local testing, npm ci after the build restores the correct state.

Let me fix the Docker test command (path was wrong):

Hmm, the error is now about the clipboard native binding, not the photon WASM. This suggests the photon fix worked, but now we're hitting another issue with the clipboard native bindings.

Actually wait, looking at the error more carefully - the build was done on macOS and didn't have the linux-x64-gnu clipboard binding installed during the build. The --no-save --force install should have installed them, but maybe it didn't work correctly or the build happened before those deps were installed.

Let me check what the build order was in the script. Looking at the script:

npm ci - installs only macOS deps
Install cross-platform deps
npm run build - builds TypeScript
Build binaries

The issue is that npm run build happens AFTER cross-platform deps are installed, so the bindings should be there. But wait - the user said this overrides node_modules. Maybe during the build, the clipboard module checks if the binding exists and chooses one. Let me check.

Actually, looking at the error - the error is NOT about photon anymore. The photon fix worked! The error is now about @mariozechner/clipboard-linux-x64-gnu.

The problem is that the clipboard package has platform-specific native bindings. Looking at the error path existsSync20(join23(__dirname, "clipboard.linux-x64-gnu.node")), it seems like it's trying to find a local .node file first, then falling back to the package.

The issue might be that during cross-compilation, Bun needs the native bindings to be available. Let me check if the clipboard packages were actually installed.

Actually, looking at the earlier output, only some clipboard packages were installed:

drwxr-xr-x@  11 badlogic  staff    352 Jan 16 20:43 clipboarddrwxr-xr-x@   5 badlogic  staff    160 Jan 16 20:43 clipboard-darwin-arm64

drwxr-xr-x@   5 badlogic  staff    160 Jan 16 20:43 clipboard-darwin-universal

drwxr-xr-x@  11 badlogic  staff    352 Jan 16 20:43 clipboarddrwxr-xr-x@   5 badlogic  staff    160 Jan 16 20:43 clipboard-darwin-arm64

drwxr-xr-x@   5 badlogic  staff    160 Jan 16 20:43 clipboard-darwin-universal

The linux ones aren't showing up! The script tried to install them with --force but maybe they failed silently. The installation command might not have worked as expected, or the packages weren't actually added to node_modules despite the force flag.

Progress! The photon WASM error is gone, but now there's a clipboard native binding error. The cross-platform clipboard deps weren't installed properly. Let me check:

The Linux clipboard bindings weren't installed. Let me try installing them manually:

Now rebuild and test:

The error is still there. The issue is that Bun's --compile is including code that tries to dynamically require the clipboard native binding at runtime. This is a similar problem to the photon one - the clipboard module uses conditional requires based on platform, and Bun's bundler is having issues embedding the native .node files.

Looking at this, the clipboard issue is separate from the photon WASM issue we fixed. The photon fix is working (we're not seeing the photon error anymore). But the clipboard bindings have their own bundling problems.

This is actually a known issue with Bun and native Node addons. The .node files need special handling.

For now, let me verify the photon fix worked by checking if we get past the photon initialization. The current error is about clipboard, not photon. Let me check if this is a pre-existing issue (was also present in v0.45.7 or earlier versions that "worked").

Actually wait - the user's original report said v0.45.7 worked fine. So the clipboard bindings must have worked before. Let me check what changed...

Actually, looking more carefully at the error - the code is checking localFileExisted and then trying to require both a local file AND the package. The logic seems to be:

Check if local .node file exists
If yes, try to require local file
If that fails, try to require the package

But in Bun's compiled binary, both of these fail because:

The local .node file isn't bundled
The package require can't find the module

This is actually a separate bug from the photon one. The photon fix is working (we got past the photon error). The clipboard issue might have been there before too but was masked, or there's something else going on.

Let me check when clipboard was added and if it worked in the old binary.

The photon fix worked (that error is gone). The clipboard native binding error is a separate issue.

Let me check if this was present in v0.45.7 (which "worked" per the issue):

v0.45.7 works. Let me check if the clipboard issue started in v0.46.0:

v0.46.0 has the original photon error (the one reported in the issue). So the clipboard issue we're seeing now is NEW - it's something that our fix introduced, or there's something different about how we're building.Let me check what might be different. The clipboard issue might be because:1. During cross-compilation on the user's macOS, the linux native binding isn't being bundled properly 2. Or the way Bun handles native modules changedLet me check if the clipboard module version changed or if something else in the build process is different. Actually, wait - let me re-read the GitHub Actions workflow to see how it handles the cross-platform bindings. The workflow installs them explicitly and then builds. But maybe the node_modules structure on CI is different...Actually, I think the issue is that we built locally with --skip-deps which means the linux binding was installed but perhaps after a npm ci that removed it, or during the build the wrong binding was used.Let me try a cleaner test - install all cross-platform deps, then rebuild.

v0.46.0 shows the original photon error (expected). The clipboard error is new in our build.

Let me clean up and do a fresh build with all cross-platform deps: