Tested image resizing in npm release 0.49.0

Summary

Closed, Completed

Reproduce
Tested image resizing with large PNG on npm 0.49.0.
Reproduce
Understand
Analyzed possible causes for user-reported issue.
Understand
Verify
Confirmed resizing works correctly in tests.
Verify

https://github.com/badlogic/pi-mono/issues/824

issue got closed, we just fixed shit for bun. but user reports the last npm release 0.49.0 has this issue. we have an image in /tmp/large_test.png, can you test the last release please?

Let me check the image and test with the released npm version:

Let me find where image processing happens and write a test:

Readimage-resize.ts

Readphoton.ts

Let me write a test script to check if photon loads in 0.49.0:

Let me check what's exported and find another way to test:

Interesting! The resizing works correctly in the npm version when I test the module directly. The image went from 23.55 MB to 0.66 MB. So the issue isn't in the npm package itself.

Wait, the user's screenshot shows the issue happening. Maybe it's about how the image is being read in the tool result context. Let me check where images are processed in tool results.

Let me look at where images are processed when being sent to Claude in tool results or the Read tool.

The resizing works in 0.49.0! The image went from 23.55 MB to 0.66 MB. Let me check if there's a path where images might bypass the resize logic:

Readread.ts

Readfile-processor.ts

continue

Both have photon-node 0.3.4. The resizing works fine when I test it directly. Let me check if there's maybe something in the tool result processing that could bypass resizing.

Actually, let me test the full pi CLI with the image to see if something happens there. But that's not easily scriptable.

Wait, the user's screenshot shows they're adding an image via @-file argument. Let me check if maybe there's a different issue. Let me also check if there might be an issue with how the image gets processed when the model makes a read tool call vs when the user provides an image.

Actually, let me re-read the issue comments more carefully. The user brunohaid says it's still happening with npm 0.49.0 and they show a screenshot. The error message shows it's in a tool_result context (messages.169.content.0.tool_result.content.1.image).

So the issue might be in how tool results with images are processed. But I just checked read.ts and it does use resizeImage...

Let me check if there might be cases where resizeImage returns early without actually resizing - like when photon can't load.

Let me check if photon could silently fail to load. Looking at the resize code, when photon is null it returns wasResized: false:

>cd /tmp/pi-test-049 && cat > test-photon-load.cjs << 'EOF'
const path = require("path");

async function main() {
  const photonPath = path.join(__dirname, "node_modules/@mariozechner/pi-coding-agent/dist/utils/photon.js");
  const { loadPhoton } = await import(photonPath);
  
  const photon = await loadPhoton();
  console.log("Photon loaded:", photon !== null);
  
  if (photon) {
    console.log("Has PhotonImage:", typeof photon.PhotonImage);
    console.log("Has resize:", typeof photon.resize);
  }
}

main().catch(console.error);
EOF
node test-photon-load.cjs 2>&1

Photon loads fine. Let me check if there could be an issue with the way images are being handled when they come from different sources (like clipboard or screenshots). Let me also check if maybe the user has an older cached version or something, or if there's a race condition.Wait, let me think about this differently. The test I wrote works. The user reports it doesn't work for them. What could be different?

They might be using bun binary (but they said npm 0.49.0)
There could be a caching issue
They might be on an older Node.js version
The image might be getting processed somewhere else

Let me check if there are other places where images might be passed without resizing.

Readimage-convert.ts

Readclipboard.ts

Readclipboard-image.ts

Readcustom-editor.ts100 lines

OK so processFileArguments in main.ts does resize images. But looking at the issue again, the user's error shows messages.169.content.0.tool_result, so the issue is when the image comes from a tool result (the Read tool). And the Read tool does resize images...

Wait, let me look at the error message more carefully: messages.169.content.0.tool_result.content.1.image.source.base64. This is a tool result. Let me look at the 0.49.0 read.ts to see if there's a difference.

Readread.js

The Read tool works correctly in 0.49.0. The 24MB image is resized to 0.66MB. So the npm version works fine...The user brunohaid claims npm 0.49.0 doesn't work for them. Let me think about what could be different:

Maybe they have an older cached version?
Maybe their Node.js is too old and has issues with the ESM/CJS interop?
Maybe they're actually using Bun without realizing it?
Maybe there's an issue specific to their environment?

The issue was closed after fixing the Bun binary issue. Looking at the comments again:

badlogic said "can't repro with latest pi"
badlogic said "doesn't work with self-contained Bun executable"
brunohaid then said "Still happening with npm 0.49.0" with a screenshot

Let me check if there's any condition under which photon loading might fail silently. Looking at the code, when photon fails to load, it returns wasResized: false and returns the original image unchanged.

Maybe the user has some issue where photon fails to load but they don't know it? Let me check if there's any way to test this.Actually, let me confirm by checking the screenshot context more carefully. The user's screenshot shows the image being over 5MB. Let me re-read the comments.

In brunohaid's screenshot, it shows an error about the base64 image exceeding the limit. This could happen if:

resizeImage returns early because photon couldn't load
The image is being added somewhere that doesn't go through resize

Looking at the resize code:

javascript

if (!photon) {  // Photon not available, return original image  return {    data: img.data,    mimeType: img.mimeType,    ...    wasResized: false,  };}

if (!photon) {  // Photon not available, return original image  return {    data: img.data,    mimeType: img.mimeType,    ...    wasResized: false,  };}

So if photon fails to load, the original image is returned.

If photon isn't loading properly, the uncompressed image would exceed the 5MB limit. I need to figure out what could prevent photon from initializing in their npm setup.

The npm version works correctly in my tests. Let me check what could cause photon to fail silently:

continue

I see. The photon module loads synchronously using fs.readFileSync(path) at the module level. This should work fine with npm install. If the wasm file is missing, it would throw an error.

Let me think about what the user might be experiencing. Looking at the issue again, the user brunohaid says it's still happening with npm 0.49.0. But my tests show it works fine.

Possible scenarios:

User has a corrupted install or cached old version
User is using a global install that hasn't been updated
User's environment has something preventing photon from loading
There's a bug we haven't found

Let me check if there's any difference between global and local installs. Also, let me ask the user to provide more details. But wait, I'm just testing, not responding to the user directly.

Actually, looking at the issue more carefully, the issue was closed. The user was told to use npm instead of bun binary. Then they reported it still happens with npm. But the fix mentioned in the last comment was about loading photon wasm in compiled binaries - which wouldn't affect the npm version.

Let me check if there's a scenario where the npm version could fail. Maybe when running with npx?

Let me test with npx which is how users commonly run it:

>cd /tmp/pi-npx-test && cat > test.js << 'EOF'
// Simulate what happens when pi reads an image via the Read tool
const { createReadTool } = require("@mariozechner/pi-coding-agent/dist/core/tools/read.js");

async function test() {
  const readTool = createReadTool(process.cwd(), { autoResizeImages: true });
  const result = await readTool.execute("test", { path: "/tmp/large_test.png" });
  
  const img = result.content.find(c => c.type === "image");
  if (img) {
    const size = Buffer.from(img.data, "base64").length;
    console.log(`Image size: ${(size / 1024 / 1024).toFixed(2)} MB`);
    console.log(`Under 5MB: ${size < 5 * 1024 * 1024}`);
  }
}

test().catch(e => console.error(e));
EOF
node test.js 2>&1

All my tests confirm 0.49.0 works correctly:

Original image size: 23.55 MBWas resized: trueOriginal dimensions: 4000x4000Final dimensions: 2000x2000Final size: 0.66 MB

Original image size: 23.55 MBWas resized: trueOriginal dimensions: 4000x4000Final dimensions: 2000x2000Final size: 0.66 MB

The Read tool properly resizes the 24MB PNG to 0.66MB JPEG, well under the 5MB limit.

Possible explanations for the user's issue:

Cached global install - they might have an older version globally installed that takes precedence
Photon fails silently - if photon can't load (missing native deps?), resizeImage() returns the original image unchanged with wasResized: false
Different code path - clipboard paste writes to temp file, then the path is typed into the editor, but if processed differently than @-file arguments...

Let me check one more thing - how clipboard-pasted images flow through: