Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import stop on any failure, FR: add a retry option #699

Open
licryle opened this issue Feb 11, 2025 · 3 comments
Open

Import stop on any failure, FR: add a retry option #699

licryle opened this issue Feb 11, 2025 · 3 comments

Comments

@licryle
Copy link

licryle commented Feb 11, 2025

Hi there,

I'm on my way to upload my 177k (!) pictures, and unfortunately at the first sign of failure... the whole program stops and doesn't go forward. Between the "immich content" step and the "google photo puzzle" steps, it takes already more than 1h to go back to the same step.

Sounds like there should be an option to either:

  • retry right now
  • resume from where it failed
  • skip the file.

In particular now, I'm facing an Internal Server Error on a file and not sure I can ever go beyond that. (Filing a separate issue right now)

@licryle licryle changed the title Cannot finish upload, 1 failure stops the import Import stop on any failure, FR: add a retry option Feb 11, 2025
@simulot
Copy link
Owner

simulot commented Feb 12, 2025

Apparently there easy "Internal Errors" where the server replies immediately, and immich-go may do something smart with the error.
And they are errors that hurt the server. All connections are frozen until a timeout run off after 20 minutes...

A long timeout is needed to give to the server enough time to process large files. The server remains quite during long minutes.

At the moment, I try to find conditions (or logical errors in the immich-go code) that lead this kind of freezing errors.
In many cases, it's sufficient to run immich-go to clear the error. So it's difficult to get accurate logs.

@licryle
Copy link
Author

licryle commented Feb 13, 2025

Absolutely, they appear to be quite random, and that would make sense from your point of view if they are server-side.

Thing is, in my case, I run it on 180k medias, so it always fails over time, and I run into some edge cases like #700 #701 #703 and actually #706 (sorry!).

#700 / #699 (mines) and #706 seem to be the same issue though.

The UI isn't condusive of whether immich-go is hanging waiting for the server, or simply had stopped, so I killed immich-go and restarted over afer a while. Regardless, 2 nights I went to bed with immich-go running to wake up to the error, meaning it probably hanged for hours, instead of timing out and retrying.

FYI in #699 & #706:

2025-02-11 20:09:01 ERR upload error file=takeout-20250206T162211Z-044:Takeout/Google Photos/Photos from 2018/4016.png error=updateAsset, PUT, http://localhost:2283/api/assets/6151d447-cccf-4a8b-8f78-431d988a0251, 500 Internal Server Error
Failed to update asset

│2025-02-12 06:18:13 ERR upload error file=takeout-20250209T204248Z-005:Takeout/Google Photos/XXXXXXXXX/IMG_5616.HEIC error=updateAsset, PUT, http://192.168.1.164:2283/api/assets/b77cb63e-e26e-4cae-ae11-9fac6ad0083f │

I'm restarting now with --log-level=DEBUG --api-trace

@simulot
Copy link
Owner

simulot commented Feb 13, 2025

Grab the server log as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants