[FR] Settings (Names, Censorship, Language)

[seniorm0ment]

A few reccomendations, and possibly bugs?

• Not sure how possible this would be to implement, but basically would like to see settings to allow a preference in idol names. So R18 uses First Name, Last Name (I may be wrong, don't quote me) and Jav Library uses Last Name, First Name which is more true to JAV. So if There are no matches for JavLibrary, and it falls back to R18, then it will probably fetch the name as First Name, Last Name creating an inconsistency if majority of stuff pulled from JavLibrary.

• Also R18 seems to censor lots of words in their titles for some reason, I'm unsure if there is anyway to figure out what the word is and uncensor it if it's not able to be found in JavLibrary/JavBus. But it would be nice.

• Idol photos don't always seem to be found, or sometimes are extremely low quality ones). Also sometimes boxarts.

• I have noticed (possibly?) a few incorrect titles such as for S2MBD-046 which is supposed to be Uehara Ai's Encore vol.46 - Lifting of the ban, is set to have the title of "Shinobu's strap-on dildo diary" lol.

• Also the option to select between a preference of English or Japanese titles (including tags, bio, etc) would be very nice.

In case it matters, I currently have my metadata preferences in order as follows: JavLibrary > R18 > JavBus

[imaginary-upside]

I didn't see this issue for whatever reason. I only got a notification for your other issue.

1. I have thought about this issue before and am planning on fixing that before announcing the project. I have a couple different thoughts on if I should just do a simple fix or something a little more grandiose. I'll probably ask your opinion on that later once I do the simple fix I have in mind.

2. The censoring is a new issue which is really weird. When I started writing the plugin which wasn't even that long ago, they weren't censoring to my knowledge. It should be pretty simple to just create a dictionary of the censored words and replace them. But it'll be pretty boring work so no guarantee when I get around to doing that.

3. Do you know of any actress metadata sites which are good? Right now I only have asianscreens.com setup as a proper actress metadata provider. So it's really up to just finding another website which has more actresses with pictures.

4. The really weird thing is that S2MBD-046 doesn't even show up on r18 or javlibrary. But the r18 provider is returning that random result. I also noticed that the javbus provider isnt finding S2MBD-046 even though the javbus website does have metadata for that video, which I'm just writing down to remember in the future.

5. This is something that has been on the backburner for a while. I'd also like to implement proper chinese support, though it seems like chinese with javbus is working fine enough. It shouldn't be hard since I assume the html structure of the websites stay mostly the same no matter the language. But its another thing that I just haven't gotten around to doing yet.

[seniorm0ment]

3. Not sure off hand of actress metadata sites off hand. You could probably make a post in r/jav or something though tbh.

4. https://www.javbus.com/S2MBD-046
   I don't know why it's not on any other site, I was trying to figure that out myself a while ago.

Also thank you

[imaginary-upside]

Other than these issues, is the plugin mostly functional for you? Is it able to successfully match the majority of videos and give you decent useful metadata?

And no problem. It's good to get feedback before I announce the project anywhere.

[seniorm0ment]

Other than these issues, is the plugin mostly functional for you?

Yeah it seems to be working well. It seems to be able to match most just fine. There are a few that have issues.
Uh one of them was 'KV-130', It's worth noting I still have them split up into 3 parts so that could've fucked with it. But I do remember (I was using JavLibraryScraper) before and it grabbed some gay JAV like Cock Heroes or something lmao. And your script grabbed the same. But if I "skipped" to the next scraper on JavLibraryScraper the proper KV-130 came up. I'm not sure why it was grabbing the gay scene. I haven't tested the identify for the jav films yet, I actually didn't know that was a thing in your addon. I had just changed the metadata names and image of them.

Yeah and no worries, a lot of those I'd call them feature improvements. But overall it seems to grab them. I'd definitely try to implement the language preference options first though if you can. Because quite a few came up in Japanese and quite a few in English, it kinda falls out of place and makes it hard to search things. Maybe an option to select what provider you want to use specifically for each component like titles, tags etc. Because I tend to like R18's english titles more than JavLibrary's, but then again R18 does censor. So it's that tradeoff rn. But yeah it's better than having stuff I can't even read lol

[imaginary-upside]

lol yeah I just noticed that KV-130 is matched the same way in my library. It's doing that because that video doesn't exist on r18, so it's just grabbing the first related result that the search returns. It's a pretty easy fix that I knew would probably become a problem at some point. I just need to make sure the javcodes actually match when returning a result.

Can you give me one that shows up in Japanese for you? I think its probably ones which aren't being matched on either javlibrary or r18, so are getting metadata from javbus. Which right now was only setup to scrap the chinese version of javbus, which uses japanese titles and chinese for genres / actresses. I'm also pretty sure that 90% of the problem is because the javlibrary metadata provider only has regex matching on master not in the release yet.

I'll look into if Jellyfin has the built in ability to rank providers for each individual metadata field. If they don't, then I know the dumb simple solution that should probably work. But I also may be able to write a small library which does it in a cleaner way. So I'm going to leave that a bit on the backburner right now. I'll just uncensor the genres that I notice to make r18 functional, and then later once I get ranked fields working, you can just rescan your library.

I'll probably release a new build in a week to 2 weeks with the important changes and yet you know when I do.

[seniorm0ment]

Julia's RKI-111 was one of the japanese titles, I didn't check r18 or anything yet so thats probably whats happening.

I know with the anime plugin theres an option to set names to romaji so I figured maybe you can accomplish it that way through plugin settings. So if it pulls a Japanese title and theres another available then it skips the title to the next metadata provider but keeps everything else on the first if nothing if found for those fields. Something like that probably.

[seniorm0ment]

Just wanted to give an update, As my library grows, I find MANY get scraped that take Japanese titles/bios/names instead of English ones.
One example is, I have DOKS-506, and it pulled it as MMUS-038 for some reason. Also the star is Eimi Fukada, but it pulled her name in Japanese, so it essentially added a new Jav Star to the DB instead of merging it with Eimi. I notice a handful of wrong titles. My filenames are similar to one another, so I don't see it being an issue to my filenames.Most fetch fine, but there's a few here and there that have JAVLibrary or R18 pages and the codes are no differently setup than my other files, so they should be pulling properly, but they're not.
Not sure but this (artist Japanese naming issue) could possibly be solved if there was some way to create some kind of nickname or sym-link. But it seems it'd be easier to try to get it to pull English names, titles, and bios.

The other thing I notice, is the genres and tags it pulls are pretty shit.
Again, if it pulls a japanese title, it won't grab the English tags, not merging. But also there's lots of genres with redundant or stupid names like "R18 Special", "R18 Discount", "Kiss Kiss", "JULIA", "JAV", "DMM", "Toy", "StaffPicks03", etc. Also a lot of the censored tags I mentioned in the past posts above still remain present (to be expected seeing you haven't released an update, just figured I'd point out because I'm seeing it a lot more as my library grows).
I also find it misses a lot of genres that relate to the films.

Another issue I notice is it doesn't grab stuff with codes such as "[FC2-PPV-1049588]", and "[Heydouga] [4140-025]" but I don't believe these are considered JAV on DB sites so that may be why. I could be wrong, I'm not sure what this code is, but if there is a way to have it implemented to fetch pictures/metadata it would be nice. I see them posted constantly alongside JAV videos for download, and have a few myself.

I have noticed theres a few other JAV plugins for Jellyfin, but yours seems to be the easiest as it is literally just a plugin. The others seem to be stuff that you download and it sorts or writes metadata, etc. Personally I much prefer the ease of yours, but I mention this because there might be some possibility that you could grab code from those to help make building your plugin easier, more efficient, and work better.
I do hope for the ability to add preferences/plugins as I mentioned in the previous posts above.
I think out of everything, the biggest issue that should be addressed is the language issue imo.

But again, I see you're slowly making commits, my library would be a disaster without your plugin though, so I hope to see it worked on more and can't wait for future updates and improvements! Thank you so much!

[imaginary-upside]

Oh hey, I honestly totally forgot about these these issues. I just checked my email for this account and noticed an email from github. I'm just going to reply to the easy stuff then go take a 2nd look at everything.

So about the Japanese actress names thing. That should be from the javbus metadata provider which was only built to work for Chinese people. And since I noticed that someone made a different jav plugin specifically for Chinese people, I just fully removed that code. So that shouldn't be an issue anymore. But there still would be an issue when an actress has multiple names or different romaji translations. But honestly that is too much of a pain to fix. My idea would be though to do like you said and have one "real name" with a bunch of nicknames.

Lol yeah the tags from r18 specifically are pretty bad. I did though try to remove some of the auto generated ones like "2019 sale", but there's really only so much I can do. I might though be able to get a list of "legit" tags, and then just discard everything that isn't in that list. But idk if I want to spend the tedious time doing that lol. Maybe if you want to go through all the r18 and javlibrary tags to get only the legit ones, I'd code something up to use that list.

Yeah those codes aren't really considered jav on the metadata sites that I know of. But if you find a consistent site that I could scrape for those, I'd probably code something up. I just don't want to make a million different scrapers for something I probably wouldn't use.

Yeah honestly some of the other jav programs are just objectively better than mine, but I'm the same as you about perfecting a simple drop in plugin. And hey it's cool that you're actually still using this. I wasn't sure if it was just me.

[seniorm0ment]

Oh hey, I honestly totally forgot about these these issues. I just checked my email for this account and noticed an email from github. I'm just going to reply to the easy stuff then go take a 2nd look at everything.

No worries!

Yeah those codes aren't really considered jav on the metadata sites that I know of. But if you find a consistent site that I could scrape for those, I'd probably code something up. I just don't want to make a million different scrapers for something I probably wouldn't use.

Not sure what they're from, just searching the ID's generally bring up streaming sites and download sites. I'm not sure though because they are IDed, so they gotta be coming from somewhere. I'll keep a look out and if I figure it out will post.

Maybe if you want to go through all the r18 and javlibrary tags to get only the legit ones, I'd code something up to use that list.

Two ideas here, implement a settings area for the plugin in JF where users can just blacklist whatever tags they want from being scraped, or I can try to help compile a list of a handful of the more common bad tags I see. I think that'd be easier than doing all of the legit ones unless you found a list of them from R18 which may be possible idk. But I think the settings area where users can just blacklist whatever tags would be the easier option.

Yeah honestly some of the other jav programs are just objectively better than mine, but I'm the same as you about perfecting a simple drop in plugin. And hey it's cool that you're actually still using this. I wasn't sure if it was just me.

Yeah no I just use your plugin, I haven't really bothered with the other ones. I was using some Java program at one point I forget the name where it just created .nfo files and used a GUI, but it was a total PITA tbh.
But I had mainly suggested the other JAV plugins as there might be some code you could use in yours to improve yours. Maybe one of them compiled a list of tags to blacklist for example, or stuff like that.

So about the Japanese actress names thing. That should be from the javbus metadata provider which was only built to work for Chinese people. And since I noticed that someone made a different jav plugin specifically for Chinese people, I just fully removed that code. So that shouldn't be an issue anymore. But there still would be an issue when an actress has multiple names or different romaji translations. But honestly that is too much of a pain to fix. My idea would be though to do like you said and have one "real name" with a bunch of nicknames.

I do have JAVBus enabled, but it is set to the lowest priority in my fetch settings (Javlibrary > R18 > Javbus). And again, it seemed to be grabbing stuff from Javbus before Javlibrary when I confirmed Javlibrary has them listed so Idk what it was doing that. As for the actor/actress names, yeah I think nicknames would be the best. Maybe if possible implement a setting where users can select the preference of names (Last name, First name | First name, last name) (English, Japanese, Romaji), etc. would be great too.

Appreciate your efforts greatly. Haven't seen you announce this anywhere, but maybe if you posted it in a few places you could get more people giving ideas and contributing. Doesn't seem like there's many people using it, but again I think many people would like this as it's more plug and play compared to the other options, it just needs a bit of polishing.
I don't really have any programming knowledge or skill so I really can't help with that side of things :/

[imaginary-upside]

Two ideas here, implement a settings area for the plugin in JF where users can just blacklist whatever tags they want from being scraped, or I can try to help compile a list of a handful of the more common bad tags I see. I think that'd be easier than doing all of the legit ones unless you found a list of them from R18 which may be possible idk. But I think the settings area where users can just blacklist whatever tags would be the easier option.

Yeah, my first thought was a user defined blacklist as well, but then I thought that honestly there may be more shit tags than actual legit ones. A user defined blacklist list is simple enough though, and doesn't really add much code that I'd need to support. That's my main thing, I don't want to add too much bloat and make the project unmaintainable. The less code I write, the less chance of totally fucking the code up there is lol.

I do have JAVBus enabled, but it is set to the lowest priority in my fetch settings (Javlibrary > R18 > Javbus). And again, it seemed to be grabbing stuff from Javbus before Javlibrary when I confirmed Javlibrary has them listed so Idk what it was doing that.

So the thing with Javlibrary is that it has to bypass Cloudflare's bot detection page, and the library I use (the only real c# library there is), needs to be updated every time Cloudflare changes their bot detection challenge. So right now javlibrary just isn't working for you at all. It's actually a huge pain. If this wasn't just a drag and drop plugin, then it would be so much easier to bypass that using something like cfscrape.

Appreciate your efforts greatly. Haven't seen you announce this anywhere, but maybe if you posted it in a few places you could get more people giving ideas and contributing. Doesn't seem like there's many people using it, but again I think many people would like this as it's more plug and play compared to the other options, it just needs a bit of polishing.
I don't really have any programming knowledge or skill so I really can't help with that side of things :/

I was going to do that a while ago back when I sent you a link to this repo, but then just got lazy lol. The thing is that most people don't actually contribute code back. Most people just consume, not that it's a huge problem, but I don't want to go out of my way to increase my workload. If you ever want to tell people about this though, I'm fine with that.

[seniorm0ment]

Yeah, my first thought was a user defined blacklist as well, but then I thought that honestly there may be more shit tags than actual legit ones. A user defined blacklist list is simple enough though, and doesn't really add much code that I'd need to support. That's my main thing, I don't want to add too much bloat and make the project unmaintainable. The less code I write, the less chance of totally fucking the code up there is lol.

Maybe, this is fair. I'm not sure but R18 may list out all their tags somewhere in like a advanced search or genre selection page or something. You might be able to just scrape that and remove whatever. Can maybe post it as a text file on here and let people like me mark things for you to blacklist, and have it fetch every now and then automatically for new tag additions, and reapply our blacklist so things stay updated. If this area on R18 doesn't exist you may be able to email them and just ask for a list of tags for "organizing your local library" or something.

I was going to do that a while ago back when I sent you a link to this repo, but then just got lazy lol. The thing is that most people don't actually contribute code back. Most people just consume, not that it's a huge problem, but I don't want to go out of my way to increase my workload. If you ever want to tell people about this though, I'm fine with that.

Is this why my Jellyfin is pinging Cloudflare constantly and I also see it constantly in the logs? Is it only Javlibrary that is using Cloudflare? I actually meant to edit and ask this in my last post. I really didn't want my server constantly pinging Cloudflare, I assume it's going to be fetching everything through Cloudflare? Is there any way around this? Or how "anonymous" or "secure" really is my traffic? I know in general Cloudflare essentially breaks encryption by decrypting then re-encrypting traffic, it also isn't great at all when it comes to privacy.
I'm ok with not using Javlibrary but I feel I'd miss a lot of stuff if R18 doesn't have.
Well I guess actually it's not even working right now.. but yeah I guess that's whyI have some non English titles. Hmm..

I was going to do that a while ago back when I sent you a link to this repo, but then just got lazy lol. The thing is that most people don't actually contribute code back. Most people just consume, not that it's a huge problem, but I don't want to go out of my way to increase my workload. If you ever want to tell people about this though, I'm fine with that.

Yeah I kinda figured this is why you haven't, and that's understandable. I mean you don't have to make it a priority in your life though. I'd say once it's faaairly stable maybe announce it though. Again, the issues are there but you don't have to make it a priority.

[imaginary-upside]

Maybe, this is fair. I'm not sure but R18 may list out all their tags somewhere in like a advanced search or genre selection page or something. You might be able to just scrape that and remove whatever. Can maybe post it as a text file on here and let people like me mark things for you to blacklist, and have it fetch every now and then automatically for new tag additions, and reapply our blacklist so things stay updated. If this area on R18 doesn't exist you may be able to email them and just ask for a list of tags for "organizing your local library" or something.

When I wrote that message I was looking through my library's genres list and forgot about all the genres that Javlibrary also makes. It's actually just pretty easy to remove all the useless genres from R18. I already was removing the sale ones, but I can remove them all easily now.

Is this why my Jellyfin is pinging Cloudflare constantly and I also see it constantly in the logs? Is it only Javlibrary that is using Cloudflare? I actually meant to edit and ask this in my last post. I really didn't want my server constantly pinging Cloudflare,

Yeah it's actually a pretty terrible situation. But it is pretty easy to figure out that Javlibrary is using Cloudflare. If you just open up their site, you can see Cloudflare's 5 second delay page show up.

I assume it's going to be fetching everything through Cloudflare? Is there any way around this? Or how "anonymous" or "secure" really is my traffic? I know in general Cloudflare essentially breaks encryption by decrypting then re-encrypting traffic, it also isn't great at all when it comes to privacy.

And yeah they do MITM traffic, but you can just use tor to keep yourself unkown to them. Honestly though you shouldn't trust any of the metadata sites either. They are all going to be pretty shady. So if you do care about privacy, definitely setup Jellyfin to use tor.

I'm ok with not using Javlibrary but I feel I'd miss a lot of stuff if R18 doesn't have.

The thing is that there really is no other good English metadata site. Before coding up a Javlibrary scraper, my first thought was to go look for a better one that would be easier to scrape, but there just really isn't one that I could see. Every other English focused JAV organizer gets the bulk of their English metadata from Javlibrary and R18 too.

And so overall just going back to your original post:

• I did implement the name ordering stuff.
• I did add in support for decensoring r18 titles / genres. Though a few probably did slip through the cracks, I just used my own massive library as reference.
• I added a second actress scraper using http://warashi-asian-pornstars.fr/
• Incorrect video matching is still an issue. But less of an issue than before. Now it's more of a semi-tricky problem where you have a difference between something like EBOD-50 and EBOD-050 which sometimes are totally different videos.
• I'm honestly probably not going to code up different language support. It's not that difficult, but it's not something I'm going to use or ever really continually test. But the codebase isn't total shit, so another developer that's interested in that should be able to implement the feature.

[seniorm0ment]

And yeah they do MITM traffic, but you can just use tor to keep yourself unkown to them. Honestly though you shouldn't trust any of the metadata sites either. They are all going to be pretty shady. So if you do care about privacy, definitely setup Jellyfin to use tor.

Is it possible to have Jellyfin use Tor for metadata fetching only?
I kinda figured this, I mean I'd be ok with running it through a VPN as well if that's possible and less work to setup. But I wasn't sure, currently I use Caddy as my reverse proxy, but besides that I don't have much incorporated with Jellyfin. I don't want to run Jellyfin entirely through a VPN or TOR because that'd definitely cause issues with playback speed. But just grabbing metadata if that's possible that'd be great to be able to just run it through a VPN or TOR.

* I did implement the name ordering stuff.

Awesome!

* I did add in support for decensoring r18 titles / genres. Though a few probably did slip through the cracks, I just used my own massive library as reference.

Glad to hear this!

* I added a second actress scraper using http://warashi-asian-pornstars.fr/

Glad to hear this!

* Incorrect video matching is still an issue. But less of an issue than before. Now it's more of a semi-tricky problem where you have a difference between something like EBOD-50 and EBOD-050 which sometimes are totally different videos.

Curiously, why would this be an issue? Are metadata sites not labeled the same? Like is it EBOD-50 on one site and EBOD-050 on another? If not, then there shouldn't be issues here given people label their files with the tags exactly how they are on the metadata sites (Usually EBOD-050)

[imaginary-upside]

Is it possible to have Jellyfin use Tor for metadata fetching only?
I kinda figured this, I mean I'd be ok with running it through a VPN as well if that's possible and less work to setup. But I wasn't sure, currently I use Caddy as my reverse proxy, but besides that I don't have much incorporated with Jellyfin. I don't want to run Jellyfin entirely through a VPN or TOR because that'd definitely cause issues with playback speed. But just grabbing metadata if that's possible that'd be great to be able to just run it through a VPN or TOR.

You should be able to just set the environment variables http_proxy and https_proxy to http://127.0.0.1:9050 (or whatever address your proxy is listening on), and then pass them to Jellyfin. I don't do this, so I don't know if there's any weird things to lookout for, but it should be just that straight forward.

Curiously, why would this be an issue? Are metadata sites not labeled the same? Like is it EBOD-50 on one site and EBOD-050 on another? If not, then there shouldn't be issues here given people label their files with the tags exactly how they are on the metadata sites (Usually EBOD-050)

I don't remember any codes off the top of my head which caused issues, but I definitely remember running into this issue before for a few. It's not really any big issue, just some minor thing that I'll probably just ignore for auto identifying.

[seniorm0ment]

You should be able to just set the environment variables http_proxy and https_proxy to http://127.0.0.1:9050 (or whatever address your proxy is listening on), and then pass them to Jellyfin. I don't do this, so I don't know if there's any weird things to lookout for, but it should be just that straight forward.

Seems like this would pass Jellyfin in entirely through the proxy which I didn't wanted as it'd probably limit streaming speeds. But no worries, I'll look into it more. Thank you!

I don't remember any codes off the top of my head which caused issues, but I definitely remember running into this issue before for a few. It's not really any big issue, just some minor thing that I'll probably just ignore for auto identifying.

Ok fair, on a side note, is it possible to scrape "Carribbeancom" metadata?
I noticed a few of those, Doesn't seem like they share ID system with JAVLibrary/R18/etc, yet have their own ID's.

[imaginary-upside]

Seems like this would pass Jellyfin in entirely through the proxy which I didn't wanted as it'd probably limit streaming speeds. But no worries, I'll look into it more. Thank you!

It doesn't, I just spent under a minute setting it up to test. And just logically it wouldn't make sense for inbound connections to go through through the proxy anyway. It's easy to try it out yourself. Just try it with an invalid proxy address and you can see that streaming still works.

Ok fair, on a side note, is it possible to scrape "Carribbeancom" metadata?
I noticed a few of those, Doesn't seem like they share ID system with JAVLibrary/R18/etc, yet have their own ID's.

Yeah I seen some other porgrams scrape from there, but I don't look at much uncensored jav, so I have no interest in coding / maintaining it myself. But writing scrapers isn't challenging, so you or anyone else that's interested in it could code something up without too much work.

[seniorm0ment]

Late response but Gotcha will look into that, and as for wrong metadata fetched, another option could be allowing to Identify (search) like you can with regular media. If I try to search JavID nothing is found.

Also is 0.3 the latest or should I build for a more updated version, just curious.
Thanks