[BUG] New Scans

[Merijeek]

So, this is interesting. I had to do some middle of the night work, so I was poking at this at about 3AM PST this morning. I ran a scan and I swear, it was almost (apart from various mis-hearings of "Craig Alanson") perfect

But there were a couple dupes and since I wasn't at 100% brain power I decided to just rerun the whole thing. And hoo boy, it's bad. I don't know if something could have changed. I'm on 121.

I've got 45/90 that are "all processing layers exhausted".

And, things like this which I'm guessing are the non-audio stuff not satisfying for some reason so it's moving onto audio and getting intro text and trusting it too much:

And these are pendings that I don't think even fall under the "attention" badge:

[Merijeek]

Looking at this, I had something occur to me. Have you considered doing something where weighting is affected by other books that are in the same queue to make a match more likely?

For example, if I'm importing 20 books and a few are positively identified as Robert Jordan, it seems that other books in that same queue are more likely to also be Robert Jordan. No, not a huge factor, but if the algorithm is thinking it could be Robert Jordan, or it could be Stephen King, well, several others already matched Robert Jordan, so it's more likely to be him.

Just a thought.

[deucebucket]

I've thought of this, I just didn't want to add more weight to the current logic puzzle. Currently it's an issue of the system not being happy with not having an answer. So if it returns NULL in the database the AIs will hallucinate their answers. So currently trying to figure out a way to juggle database being empty, but also having the backend spot search/enrichment with searxng to give correct information even when there isn't any in the database. So currently what the back end looks like is.

Database lookup through narrator ID, folder path names(which was messed up by series detection as there's not enough series info compiled but we are working on it)> if it's in the database, great. If not the transcription goes through filtering to remove garbage as much as we can, before handing the text to an AI to parse the narrator, author, and book name from the transcription. Then queries the database with this information> if nothing in the database, what's supposed to happen, is it's supposed to query using searxng to intelligently pull the correct information usually available in searches|

what's happening is, the AI is just sometimes passing back it's garbled unfixed, phonetically transcribed names and titles which explains your many different versions of the same authors name. The system is working, it's just got flow issues.

On top of that, I have very bad data from good reads and early data collection that has author and book swapped. And just goofiness that doesn't make sense in general.

I will definitely mess around with maybe a weighted collection parser? Maybe it could even track your series if you don't have the whole thing. As the backend database information grows, I promise the skaldleita end of it will get loads better.

I did get excited when I read the first part of your message where you said it was perfect then you restarted it. From my end the backend wasn't receiving the version key from your lm instance and you were getting rate limited, so it is possible a lot of the good ones you got were from skaldleita backend before the rate limit started pushing your requests aside and using the other api in library manager flow.

[Merijeek]

When you say I got rate limited, was that locally because of the max allowed per hour by LM, or is it rate limiting from the host end of the AI APIs? If it is at the host end...since you can use API calls for doing the analysis, can you use API calls to determine how many tokens a user has left in their daily bucket? If so, maybe some quick math that says "X remaining, Y needed, if X > Y, ask user if they want to burn that many tokens, and if X < Y, inform them that AI analysis will start going off the rails before they're done so do they want to proceed with that, or put things on hold til more calls are available"? If a book can't get a full analysis, stick it in a "To-Be-Analyzed" state that will try again when more calls are available rather than making poor guesses. Of course, if they do have enough tokens, I'd probably even keep that first question answered with a default "yes" silently in the background. Again, yet another possibly no good idea.

[Merijeek]

OK, I've let things sit long enough I think my APIs have calmed down.

What I see as "issues" are:

1. Craig Alanson and his multiple aliases
2. (Side note - when reading, rather than speed-skimming, a list of 50 Pendings, it's annoying to have it refresh itself so often)
3. This again, we can be pretty confident there's no such book, and since the source file is James S.A. Corey - The Expanse Series - The Expanse 01 Leviathan Wakes, it seems like it wouldn't be that hard to ID :

4. Something I'm unclear on. When I look at my badges, I see:

But when I look in /Unknown I see:

Shouldn't those things in /Unknown be indicated somewhere? (note -when I apply all pending it leaves behind just:

Which still seems off to me.

Final result on errors:

[deucebucket]

Hey Merijeek - wanted to circle back on this. A few things landed on develop that should address some of what you hit:

The rate limiting situation: So what happened on your end was a perfect storm. The API key wasn't actually being sent on /search requests (fixed in #155), which meant the backend was treating your instance as unauthenticated and throttling it way harder. On top of that, the server was genuinely getting hammered - way more people ended up using this than I planned for, and the rate limiting wasn't graceful about it. It would just start rejecting requests and the app would fall back to AI-only identification, which is where all those hallucinated author names came from (the many flavors of Craig Alanson).

That's been fixed - proper API key auth, exponential backoff with Retry-After header support, and the frontend now shows you when you're being rate limited instead of silently falling back to garbage results (#154).

The Leviathan Wakes type issues: Path completion landed (#127) - when Skaldleita returns a truncated result, the app now uses the folder path to fill in the gaps. So if the folder says "James S.A. Corey - The Expanse Series - The Expanse 01 Leviathan Wakes" and the API comes back with a partial match, it'll complete it from the path info instead of sending it off to an AI to hallucinate.

Still open from your list:

• The auto-refresh frequency on the pending list - haven't touched that yet
• The /Unknown folder not showing in badges - still need to look at this
• The "all processing layers exhausted" pile - should be way smaller now that auth is working, but curious to see what you get

Would be great to know how it looks on your end with the latest develop. The rate limiting + path completion combo should knock out a good chunk of what you saw.

[Merijeek]

So, one small note right off the bat. I love the "submit bug" widget on the bottom right. That's great. However, having that widget do different things based on whether you're in dashboard or other tabs seems like a major UI no-no.

[Merijeek]

Overall, a generally decent mixed bag.

Fixed, I see a good list except for one extraneous number:

Verified, a couple things, some of course involving the Most Difficult Author Ever:

and some noise:

Pending, we invented a new word:

Attention:

We've actually got something wrong here. Because we've got a pile of "All processing layers exhausted - could not identi..." and they're all things that have been identified before, successfully. All the McClellans, lots of Alanson, Of Monsters and Mainframes, etc.

And finally, if we look at dashboard, we seem to get very few series and book numbers, even if they get seen as part of a title:

[Merijeek]

Unless I missed something (I did update) I seem to be seeing more or less the same thing: