Replies: 2 comments 6 replies
-
|
One addition. I just checked the layers of the docker image: So I guess, the lines starting with |
Beta Was this translation helpful? Give feedback.
-
|
Thomas - I was running into the same thing. For the maintainers, here is my quick analysis of what may have happened (note, I play a developer on TV, but am doing this quickly and this may be completely inaccurate): LocalAI: In the images, which appear from about 1.40.0 forward, it appears that the /opt/conda/envs/*/lib directories are being duplicated across all the different conda install groupings. Each one is about 5GB - which I believe is causing the bloat, although I believe many of the dependancies are duplicated across the different extra build settings, I believe this is somewhat cascading and causing the unnecessary growth of the image:
I think while the concept was good to try to remove having a huge layer, this section from the Dockerfiles (introduced in a couple of commits), is contributing to the build size being huge:
For reference, these appear to be the two commits to Dockerfile that may be causing some of the issues, in an effort to get folks back to the potential issues fastest: |
Beta Was this translation helpful? Give feedback.
-
|
Exactly the same journey! I was surprized by the huge size of image. I had to modify the Dockerfile, build from scratch. I finally got a smaller image with size around 12 GB. It's still too big, I would expect you guys can create a PR to fix the issue. Thanks in advance. |
Beta Was this translation helpful? Give feedback.
-
|
core images don't contain conda environments. See also https://localai.io/basics/getting_started/#container-images and https://localai.io/basics/news/#04-12-2023-__v200__ I think probably we should make core the defaults, and have images with python dependencies separately - however I think that would just make things just worse, indeed I'm more keen towards #1411 . |
Beta Was this translation helpful? Give feedback.
-
|
@mudler Bringing up an old thread. Is there an appetite for creating smaller Docker images that contain fewer intermediate results and fewer layers, and are therefore smaller?
|
Beta Was this translation helpful? Give feedback.
-
|
Well, I'll take my L here. There's a lot more use of good builder layers than I initially read. There's a few optimizations (building CMake from source can be done cleaner) but it's solid. I was building with some of the extras, so that's why I was seeing it whiz by, I think. It still baffles me why the images are so big though... I'll see if there's anything that can be done. |
Beta Was this translation helpful? Give feedback.
-
hey - yes please any optimization is very much welcome, and it's an ongoing effort time to time which I focus too, but not entirely my primary focus. CPU images are smaller, but extras images take much more space due to the venv that each backend brings. I've been trying lately to merge some of these to minimize space. Please open up a PR if you have improvements and we'll pick it up from there. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Was digging through the layers, had the idea that a non-rebuildable image might be a lot smaller. Probably not something you can distribute given the current state of how LocalAI works overall, but it could be something that someone (maybe me) could trigger when building to keep their built image small, and maybe eventually it would become more generally useful? Let me know if I should consider that in a PR as well (probably separate). |
Beta Was this translation helpful? Give feedback.
-
Dear community
First off: Thank you for your effort. I really like this software!
When trying to run different versions of LocalAI on my computer, i always run into the problem, that image pull takes very long. The images are huge! Deploying it with portainer will even fail, because a 70GB image is not pulled in a minute or so. Using some tag like
:latestwill often pull a new image. It is at a point, where testing is tedious, because oftentimes I cannot pull the image.So I was wondering, if someone can explain: Why is it so huge? Are all these layers required?
Thank you and best regards
Thomas
Below you see a list of some of my images. I also added some other images for reference: 68GB, 70GB?
Beta Was this translation helpful? Give feedback.
All reactions