Multimodal input - APIONE AI

Multimodal input structure depends on the protocol and model. Query model parameters first to confirm supported input types, image count, size, format, and delivery method.

OpenAI-compatible image example

{
  "model": "YOUR_VISION_MODEL_ID",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Summarize the main information in the image"},
        {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}}
      ]
    }
  ]
}

This is a common OpenAI-compatible structure. Anthropic and Gemini use different content blocks; do not copy request bodies across protocols directly.

URL, Base64, and file references

Method	Advantage	Considerations
HTTPS URL	Small request body	URL must be reachable by the service; avoid long-term public exposure of sensitive files
Base64 / Data URL	No external download dependency	Request body increases significantly; watch size limits and log leakage
File ID or upload interface	Easy to reuse	Only use when explicitly supported by the model or platform

Input preprocessing

Remove EXIF, location, and unnecessary personal information.
Scale oversized images to the resolution required by your application.
Verify MIME type matches the actual file type.
Prepare test samples for scanned documents, tables, and screenshots separately.
Do not log full Base64 or temporary signed URLs.

Common errors

Symptom	Check
Model does not recognize the image	Whether the model supports vision, and whether the protocol structure is correct
URL download fails	Whether it is publicly reachable, expired, or blocking server access
413	Image count, resolution, or Base64 body size too large
Response ignores an image	Image order, text instructions, and model image count limits
Unstable OCR	Image clarity, rotation, language, and layout complexity

Next: Model selection, Structured output, Image generation, Request reliability.