Plugin can't find an installed package

[mstfldmr]

Describe the bug
The plugin can't find a package which is already installed in the virtual environment.

To Reproduce
Steps to reproduce the behavior:

1. activate a virtual environment
2. install easyocr package pip install easyocr
3. enable image2text plugin
4. start Taskweaver
5. ask to convert an image to text
6. The plugin is called, but it can't find easyocr package

``Plugin image2text failed to load: Plugin image2text failed to register: failed to load plugin image2text Please install easyocr with pip install easyocr.`

Expected behavior
use the easyocr package installed in the virtual environment.

Environment Information (please complete the following information):

• OS: Ubuntu
• Python Version 3.10
• LLM that you're using: GPT-4
• code_verification_on: false

[ShilinHe]

By default, the code will be executed in the container mode. Therefore, please make sure the package install environment is the docker. Or, you can try with the local mode first as your packages are usually installed there.

[mstfldmr]

I added RUN pip install --no-cache-dir --no-warn-script-location yfinance easyocr to ~/taskweaver/docker/ces_container/Dockerfile

and then, I built the image ~/taskweaver/scripts$ source build_executor.sh

~/taskweaver/scripts$ source build_executor.sh 
The script directory is: /home/mstfldmr/taskweaver/scripts
Found module files from /home/mstfldmr/taskweaver/scripts/../taskweaver
Dockerfile path: /home/mstfldmr/taskweaver/scripts/../docker/ces_container/Dockerfile
Context path: /home/mstfldmr/taskweaver/scripts/../
[+] Building 24.6s (16/16) FINISHED                                                                                                                                                  docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                           0.0s
 => => transferring dockerfile: 762B                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim                                                                                                                            0.6s
 => [internal] load .dockerignore                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                0.0s
 => [ 1/11] FROM docker.io/library/python:3.10-slim@sha256:38ced5e623fb8a6b256826a9f1bbcf5247a05123d6b6cb3847c9cd6ba46ef9ac                                                                    0.0s
 => [internal] load build context                                                                                                                                                              0.0s
 => => transferring context: 3.10kB                                                                                                                                                            0.0s
 => CACHED [ 2/11] WORKDIR /app                                                                                                                                                                0.0s
 => CACHED [ 3/11] COPY requirements.txt .                                                                                                                                                     0.0s
 => CACHED [ 4/11] RUN pip install --no-cache-dir --no-warn-script-location -r requirements.txt                                                                                                0.0s
 => [ 5/11] RUN pip install --no-cache-dir --no-warn-script-location --user yfinance                                                                                                          11.2s
 => [ 6/11] COPY taskweaver/ces /app/taskweaver/ces                                                                                                                                            0.1s 
 => [ 7/11] COPY taskweaver/plugin /app/taskweaver/plugin                                                                                                                                      0.1s 
 => [ 8/11] COPY taskweaver/module /app/taskweaver/module                                                                                                                                      0.1s 
 => [ 9/11] COPY taskweaver/__init__.py /app/taskweaver/__init__.py                                                                                                                            0.1s 
 => [10/11] COPY docker/ces_container/entrypoint.sh /app/entrypoint.sh                                                                                                                         0.1s 
 => [11/11] RUN chmod +x /app/entrypoint.sh                                                                                                                                                    0.4s 
 => exporting to image                                                                                                                                                                        11.4s
 => => exporting layers                                                                                                                                                                       11.4s
 => => writing image sha256:ca18227dfb7f802b32aa36fca28b777e1f435332fa089c1d1efbba3867f4bb36                                                                                                   0.0s
 => => naming to docker.io/taskweavercontainers/taskweaver-executor:0.2                                                                                                                        0.0s

 2 warnings found (use docker --debug to expand):
 - UndefinedVar: Usage of undefined variable '$PYTHONPATH' (line 21)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 21)

I still get the same error. Should I register the newly built image somewhere?

=========================================================
 _____         _     _       __
|_   _|_ _ ___| | _ | |     / /__  ____ __   _____  _____
  | |/ _` / __| |/ /| | /| / / _ \/ __ `/ | / / _ \/ ___/
  | | (_| \__ \   < | |/ |/ /  __/ /_/ /| |/ /  __/ /
  |_|\__,_|___/_|\_\|__/|__/\___/\__,_/ |___/\___/_/
=========================================================
--- new session starts ---
 TaskWeaver ▶  I am TaskWeaver, an AI assistant. To get started, could you please enter your request?
    Human   ▶  convert image to text
 ╭───< Planner >
 ├─► [init_plan] 1. Ask the user for the image path
 ├─► [plan] 1. Ask the user for the image path
 ├─► [current_plan_step] 1. Ask the user for the image path
 ├──● Could you please provide the path to the image you want to convert to text?
 ├─► [board]
 │   I have drawn up a plan: 
 │   1. Ask the user for the image path
 │   
 │   Please proceed with this step of this plan: Could you please provide the path to the image you want to convert to text?
 ╰──● sending message to User
 TaskWeaver ▶  Could you please provide the path to the image you want to convert to text?
    Human   ▶  https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png
 ╭───< Planner >
 ├─► [init_plan] 1. Convert the image at the provided URL to text using OCR
 ├─► [plan] 1. Instruct CodeInterpreter to convert the image at the provided URL to text using OCR
 ├─► [current_plan_step] 1. Instruct CodeInterpreter to convert the image at the provided URL to text using OCR
 ├──● Please convert the image to text from the following URL: [googlelogo_color_272x92dp.png](https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png)
 ├─► [board]
 │   I have drawn up a plan: 
 │   1. Instruct CodeInterpreter to convert the image at the provided URL to text using OCR
 │   
 │   Please proceed with this step of this plan: Please convert the image to text from the following URL: 
 │   [googlelogo_color_272x92dp.png](https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png)
 ╰──● sending message to CodeInterpreter
 ╭───< CodeInterpreter >
 ├─► [thought] ProgramApe will use the image2text plugin function to convert the image from the provided URL to text using OCR. Since the plugin functions can be directly called without importing,
 │    there's no need to import or install any modules.
 ├─► [reply_type] python
 ├─► [reply_content]
 │   image_url = "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png"
 │   # Since the plugin function image2text can be called directly, we will use it to convert the image to text.
 │   ocr_result = image2text(image_url)
 │   ocr_result
 ├─► [verification] NONE
 ├─► [code_error] No code verification is performed.
Plugin image2text failed to load: Plugin image2text failed to register: failed to load plugin image2text Please install easyocr with `pip install easyocr`.
Plugin stock_price failed to load: Plugin stock_price failed to register: failed to load plugin stock_price No module named 'yfinance'
 ├─► [execution_status] FAILURE

[ShilinHe]

The new image should be in your local env, which can be checked by docker images. You don't need to register it anywhere.

[mstfldmr]

OK, I see the newly built image

taskweavercontainers/taskweaver-executor   0.2       e1d9aff32d19   30 minutes ago   6.08GB
<none>                                     <none>    a5708e2ac27a   47 minutes ago   6.08GB
<none>                                     <none>    ca18227dfb7f   52 minutes ago   748MB
<none>                                     <none>    b1c0abd6ac88   55 minutes ago   5.93GB
taskweavercontainers/taskweaver-executor   latest    a9f5c9f0d958   3 months ago     716MB
hello-world                                latest    d2c94e258dcb   15 months ago    13.3kB

but the plugin still can't find the package.

[ShilinHe]

can you try to use the tag "latest" instead of 0.2 for your image?

[mstfldmr]

I deleted all containers and images, then built it with "latest" tag. Now I have only 1 image:

~/taskweaver$ docker images
REPOSITORY                                 TAG       IMAGE ID       CREATED         SIZE
taskweavercontainers/taskweaver-executor   latest    8de32ae5ad19   6 minutes ago   6.08GB

When I start Taskweaver, it downloads the old image again and removes the name of the image I built:

~/taskweaver$ docker images
REPOSITORY                                 TAG       IMAGE ID       CREATED         SIZE
<none>                                     <none>    8de32ae5ad19   8 minutes ago   6.08GB
taskweavercontainers/taskweaver-executor   latest    a9f5c9f0d958   3 months ago    716MB

and Taskweaver gives the same error.

[mstfldmr]

The lines 148-159 in taskweaver/ces/environment.py are responsible of downloading the old image from the repo if it's not the same as the local image. Thus, I commented out these lines

            try:
                local_image = self.docker_client.images.get(self.image_name)
                registry_image = self.docker_client.images.get_registry_data(self.image_name)
                if local_image.id != registry_image.id:
                    logger.info(f"Local image {local_image.id} does not match registry image {registry_image.id}.")
                    raise docker.errors.ImageNotFound("Local image is outdated.")
            except docker.errors.ImageNotFound:
                logger.info("Pulling image from docker.io.")
                try:
                    self.docker_client.images.pull(self.image_name)
                except docker.errors.DockerException as e:
                    raise docker.errors.DockerException(f"Failed to pull image: {e}. ")

This prevented Taskweaver from downloading the old image, now it's using the image I built and the plugin works well.

[ShilinHe]

okay. @liqul Can you take a look?

[liqul]

=> [ 5/11] RUN pip install --no-cache-dir --no-warn-script-location --user yfinance

I see this log from your previous reply. Did you miss adding easyocr?

[mstfldmr]

@liqul no, I later split that line to multiple lines. I confirm that it is redownloading the old docker image instead of the newly built one.

[liqul]

So you are saying that, after commenting out the lines in environment.py, the code still downloads the old image instead of loading your newly built one? This sounds weird and can you check if you are running the modified code? for instance, did you somehow installed TaskWeaver as a package? I'm pretty sure that the code you commented out is the only place determining which image is going to be loaded.

[mstfldmr]

No, it's the other way around. It was always re-downloading the image instead of using the one I built. I commented out these lines, now it doesn't download anything.