The first version of our screenshot-triage feature worked great in every demo. A support agent could paste a screenshot of whatever a customer was seeing, hit "Analyze," and a draft bug summary would show up in the ticket form a couple seconds later. Then we rolled it out to the actual support team, most of whom run 5K or 4K monitors, and within a day someone reported that "analyze" just spins forever and then fails.

The feature was built on top of deepseek-v4-flash-vision-exp, the DeepSeek model that accepts images alongside text in the same chat completion call you'd already be using for text. That part is genuinely simple to wire up. What isn't obvious until you hit it is that there are three different ways to hand the model a picture, and the one that's easiest to reach for from a browser is also the one most likely to blow up on you once real users with real screenshots show up.

This post is the version of the integration I wish I'd read before shipping the first one — what actually broke, why the size limits are shaped the way they are, why sending a giant retina screenshot doesn't get you a better analysis, and the component we ended up with.

The model caps every image at 384 tokens after an internal resize to roughly 800×800 pixels, so a 5000×5000 screenshot and an 800×800 one cost exactly the same — the extra resolution you're uploading is pure waste.

The Request Body Wall I Didn't See Coming

The naive version of the upload path was about as straightforward as it gets: grab the pasted screenshot as a File, base64-encode it in the browser, and POST it straight to our backend proxy, which forwarded it to DeepSeek as a data: URL inside an image_url content block. On my laptop, testing with a cropped 800px-wide screenshot, this worked every time.

The problem is that base64 inflates binary data by roughly a third, and DeepSeek's inline request body limit is 48 MiB total. A 12MB PNG from a 5K display becomes a 16MB base64 string before you've even added the rest of the JSON payload — and support agents were routinely pasting two or three screenshots into one ticket. It didn't take much for that to sail past the limit, and separately, any single image over 32 MiB when sent inline gets rejected regardless of the total body size.

Here's what the first version actually looked like — base64-encoding whatever came out of the clipboard with no thought given to its dimensions:

// screenshotUploader.js — v1, worked fine on my laptop
async function analyzeScreenshot(file) {
  const buffer = await file.arrayBuffer();
  const base64 = btoa(
    new Uint8Array(buffer).reduce((data, byte) => data + String.fromCharCode(byte), '')
  );

  return fetch('/api/analyze-screenshot', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ image: `data:${file.type};base64,${base64}` }),
  });
}

The fix wasn't a bigger limit on our proxy — it was realizing there was no reason to send full resolution in the first place. Once I knew the model resizes everything down to roughly 800×800 before it even looks at the pixels, downscaling on the client became the obvious move:

// screenshotUploader.js — v2, downscale before touching base64
async function analyzeScreenshot(file, maxDim = 1600) {
  const bitmap = await createImageBitmap(file);
  const scale = Math.min(1, maxDim / Math.max(bitmap.width, bitmap.height));

  const canvas = document.createElement('canvas');
  canvas.width = Math.round(bitmap.width * scale);
  canvas.height = Math.round(bitmap.height * scale);
  canvas.getContext('2d').drawImage(bitmap, 0, 0, canvas.width, canvas.height);

  const resizedBlob = await new Promise((resolve) => canvas.toBlob(resolve, 'image/jpeg', 0.85));
  const base64 = await blobToBase64(resizedBlob);

  return fetch('/api/analyze-screenshot', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ image: base64 }),
  });
}

1600px on the long edge is still well above what the model actually keeps, so nothing about the analysis quality changed. What changed was that even a stack of three retina screenshots now sits comfortably inside the body limit, and uploads on a slow office wifi connection got noticeably snappier as a side effect.

Three Ways to Hand DeepSeek a Picture

Once I stopped assuming "base64 in the request" was the only option, the rest of the integration got easier to reason about. DeepSeek's Chat Completions endpoint (and the Anthropic-compatible and Responses API endpoints, for that matter) accept an image in one of three shapes, all living inside the same array-of-blocks content field you'd already be using for a text-only message.

The first is the inline data: URL approach shown above — simplest to reason about, but it's the one that counts fully against the 48 MiB body limit and caps any single image at 32 MiB. The second is a plain external http(s) URL, where you just point image_url.url at a publicly reachable image and the model fetches it itself; that URL string is capped at 8192 characters and the download has to finish within 60 seconds, so it's a poor fit for anything sitting behind auth. The third is the Files API: upload the image once, get back a file_id that looks like file-api-xxxxxxxxxxxxxxxx, and reference it in a file content block instead of re-sending the bytes every time. Files uploaded this way get a more generous 64 MiB per-image ceiling and skip the 32 MiB inline check entirely.

For our support tool, the external-URL option was mostly useless — screenshots live behind our app's auth, not on a public CDN — which left the real decision as inline-versus-Files-API, and that turned out to depend almost entirely on whether the image gets reused.

Picking Between Inline and the Files API

For a one-off screenshot attached to a single support ticket, inline base64 is simpler — there's no extra round trip to upload the file first, and the code path is one function instead of two. Where the Files API earns its complexity is a feature we didn't expect going in: a "compare this screenshot against the reference UI mockup" check, where the same reference mockup gets sent alongside dozens of different user screenshots over the course of a day.

Re-encoding and re-uploading that same reference image as base64 on every request was pure waste, since it never changes — we upload it once, keep the returned file_id, and reference it with a lightweight {"type": "file", "file_id": "..."} block from then on. The same path is mandatory, not just convenient, for anything too big to inline in the first place: a single image over 32 MiB has to go through the Files API regardless of reuse, since that's the only route that allows up to 64 MiB per image.

The exact shape of the Files API upload call itself — multipart fields, response schema — lives on DeepSeek's separate Files API guide rather than the vision docs. Treat any Files API snippet here as "roughly this shape," not a complete reference for that endpoint.

Detail Level and Why Resolution Barely Matters

Once I understood the resizing behavior, the detail field on an image_url block started to make a lot more sense. It's not really a "quality" knob so much as a way to tell the model whether to bother keeping the image at all before running it through the same pixel-budget resize everything else goes through.

There are four values, and right now the differences between three of them are smaller than the names suggest:

ValueWhat actually happens
lowDownscaled to 512×512 before inference — cheapest and fastest, fine when you just need a rough read on what's in the frame.
highKeeps the original image; currently just an alias for original.
originalKeeps the original image going into the model's own resize step.
autoAutomatic selection; currently behaves the same as original.

Given that everything larger than roughly 384×384 gets scaled down to an ~800×800 pixel budget anyway, and every image tops out at 384 tokens regardless of its original dimensions, I ended up defaulting our tool to low for the initial triage pass — where all we need is "what kind of error dialog is this" — and only switching to original for a follow-up request when an agent explicitly asks the model to read small text out of a screenshot, since low's 512×512 downscale can blur tiny fonts past the point of being legible.

Restrictions That Will Get You a 400

A couple of the harder failures we hit weren't about size at all — they were about where in the conversation the image lived. It's tempting, once you've got a multi-turn conversation going, to stash a reference screenshot in a system message so it "persists" across turns without re-sending it in every user turn.

That doesn't work, and it fails loudly rather than silently, which honestly I was grateful for once I understood what was happening.

Images are only accepted in user messages. Put one in a system or assistant message and the API returns a 400 — and if you accidentally send an image to a non-vision model, you'll get a 400 with the message "This model does not support image."

The second gotcha is easier to hit than it sounds: if your app lets a request fall back to a cheaper text-only model under load, or you're routing by conversation type and misclassify one, any image blocks in that request just get rejected outright rather than silently dropped. We ended up adding a small guard in our request builder that refuses to construct a payload with image content unless the target model string is exactly deepseek-v4-flash-vision-exp, specifically so that kind of routing mistake fails at build time in our own code instead of as a confusing API error downstream.


Wiring It Into a Backend Route

The API key obviously can't live in the browser, so the React side of this is just a thin upload form that posts the resized blob to our own Next.js route, which is where the actual DeepSeek call and the inline-vs-Files-API decision happen. Keeping that decision server-side also means we can tune the size threshold without shipping a new frontend build.

Here's the route that decides which of the two paths to take, based on the resized image's byte length:

import OpenAI from "openai";

const deepseek = new OpenAI({
  apiKey: process.env.DEEPSEEK_API_KEY,
  baseURL: "https://api.deepseek.com",
});

const INLINE_LIMIT = 20 * 1024 * 1024; // stay well under the 48 MiB body cap

export async function analyzeScreenshot(imageBuffer: Buffer, mimeType: string) {
  let imageBlock;

  if (imageBuffer.byteLength > INLINE_LIMIT) {
    const uploaded = await deepseek.files.create({
      file: new File([imageBuffer], "screenshot.jpg", { type: mimeType }),
      purpose: "vision",
    });
    imageBlock = { type: "file", file_id: uploaded.id };
  } else {
    const base64 = imageBuffer.toString("base64");
    imageBlock = {
      type: "image_url",
      image_url: { url: `data:${mimeType};base64,${base64}`, detail: "low" },
    };
  }

  return deepseek.chat.completions.create({
    model: "deepseek-v4-flash-vision-exp",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Summarize the bug shown in this screenshot for a support ticket." },
          imageBlock,
        ],
      },
    ],
  });
}

20 MiB is an arbitrary line I picked to leave headroom for the rest of the request body alongside base64's overhead, not a number DeepSeek publishes anywhere — the only hard numbers you actually have to respect are the 48 MiB total body and the 32 MiB per-image inline ceiling. Everything else here is a judgment call about how much margin you want.

If You're Coming In Through a Different SDK

We happened to build our proxy on the OpenAI-compatible Chat Completions shape, but DeepSeek also exposes the same vision model through an Anthropic-compatible /messages endpoint and through an OpenAI-style Responses API. If your app already has an Anthropic SDK client wired up for a different feature, it's worth knowing the content block shape changes even though the underlying image-handling rules don't.

The three ways of sending an image still map cleanly across all three interfaces, but the field names shift depending on which one you're calling:

  • Base64 inline becomes an Anthropic image block with source.type: "base64", plus a required media_type field like image/jpeg.
  • External URL becomes source.type: "url", same 8192-character cap as the OpenAI shape.
  • Files API reference becomes source.type: "file", and needs an anthropic-beta: files-api-2025-04-14 header on top of the usual auth.

The Responses API keeps the OpenAI-style naming but renames the block to input_image, and it's the one place I'd double check carefully if you're passing images back through tool calls — image content is also allowed inside the output of function_call_output and custom_tool_call_output items, not just in user messages, which isn't something you'd guess from the Chat Completions shape alone.

What I'd Check First Next Time

If I were starting this integration over, the very first thing I'd do differently is read the token-usage math before writing a single line of upload code, because it would have saved me a day of debugging a body-size error that had a one-line fix. The size limits and the resizing behavior aren't arbitrary API friction — they're a pretty direct signal about how much resolution the model is actually using, and building the client around that number instead of around "whatever the user's monitor produced" is the whole trick.

None of this matters much if you're building something that only ever handles small, pre-cropped images — a logo checker, an icon classifier, whatever — where nobody's going to paste in a 5K screenshot. But the moment your upload path is exposed to real users with real hardware, assume someone's clipboard is going to hand you something bigger than your test fixtures ever were, and design the resize step first instead of bolting it on after the first outage report.

Sources: Hacker News — DeepSeek-v4-flash-vision-exp