[BUG] Reading image from artifact fails with tag mismatch error (Claude 4.6) and base64 parsing error (Claude 4.5)

[umishra1504]

Description

When attempting to read an image from an artifact in the indexer_worker, the system throws different errors depending on the Claude model being used. With Claude Sonnet 4.6, the error indicates an unexpected 'thinking' tag that doesn't match expected content tags. With Claude Sonnet 4.5, there's a base64 image parsing error in the litellm library that causes the request to fail.

This prevents users from processing image artifacts in agent/pipeline executions, breaking image-related workflows.

Steps to Reproduce

1. Create an artifact toolkit and upload any image.
2. Navigate to conversation chat page & create new conversation by adding the artifact as toolkit.
3. Make sure you have chosen LLM as Claude 4.5 or Claude 4.6
4. Ensure atleast image should present in the selected artifacts
5. Try to read that image through conversation chat and observe the response

Actual Result

With Claude Sonnet 4.6:

Traceback (most recent call last):
  File "/data/plugins/indexer_worker/methods/indexer_predict_agent.py", line 299, in _indexer_predict_agent_task_inner
    raise alita_callback.llm_error
plugins.indexer_worker.utils.exceptions.InternalSDKError: status code: 400, message: Input tag 'thinking' found using 'type' does not match any of the expected tags: 'document', 'image', 'search_result', 'text'

Error Analysis:

• Status code: 400 (Bad Request)
• The system is sending a 'thinking' tag to the LLM API
• Claude 4.6 API expects only: 'document', 'image', 'search_result', 'text'
• The 'thinking' tag is not recognized by the API

With Claude Sonnet 4.5:

Traceback (most recent call last):
  File "/data/plugins/indexer_worker/methods/indexer_predict_agent.py", line 299, in _indexer_predict_agent_task_inner
    raise alita_callback.llm_error
plugins.indexer_worker.utils.exceptions.InternalSDKError: status code: 500, message: Error calling litellm.acompletion for non-Anthropic model: litellm.APIConnectionError: too many values to unpack (expected 2)
Traceback (most recent call last):
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/main.py", line 595, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 117, in async_streaming
    request_data = await litellm.AmazonConverseConfig()._async_transform_request(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 953, in _async_transform_request
    await BedrockConverseMessagesProcessor._bedrock_converse_messages_pt_async(
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 3276, in _bedrock_converse_messages_pt_async
    _part = await BedrockImageProcessor.process_image_async(  # type: ignore
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 2748, in process_image_async
    img_bytes, mime_type, image_format = cls._parse_base64_image(image_url)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/litellm/venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 2585, in _parse_base64_image
    image_metadata, img_without_base_64 = image_url.split(",")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)

Error Analysis:

• Status code: 500 (Internal Server Error)
• Error occurs in litellm library during image processing
• _parse_base64_image() expects format: data:image/png;base64,<base64_data>
• The split(",") operation expects exactly 2 parts but receives more
• Suggests the image URL contains multiple commas or incorrect format

Expected Result

• The system should successfully read the image from the artifact
• The image should be properly formatted and sent to the LLM API
• For Claude 4.6: Should use correct content tags (e.g., 'image', not 'thinking')
• For Claude 4.5: Should properly parse base64 image data regardless of comma count
• Agent/pipeline execution should complete successfully
• No InternalSDKError should be raised

Test data:

• https://next.elitea.ai/app/12664/chat/151?name=claude+sonnet+4.6&shared_chat=1
• https://next.elitea.ai/app/12664/chat/149?name=sonnet+4.5&shared_chat=1

[umishra1504]

With Claude Sonnet 4.5, it works in Agent Chat, but fails in conversation