Config error or bug? I'm guessing config?

[mediocreatmybest]

Hi!

I'm trying to enable the generic extractor to enable gallery-dl to attempt downloads for non-supported sites. (Not a huge amount of documentation that I could find on this feature)

I've flagged the generic extractor to be enabled (within the gallery conf file) with:

"generic":
{
        "enabled": true
    },

Is this all I should need to do? Or am I missing something else? with this enabled I'm unable to match generic unsupported URLS
but if I change the syntax in the config to create an error (for example too many commas or missing a comma) I get an error stating it's unable to parse the config, but then the generic downloader works. So not sure if there is something I'm missing within the config. I'm using the default one on GitHub.

Totally lost. Thanks!

Edited for clarity (hopefully)

[github-account1111]

Have you tried prepending generic: to the URL?

[mediocreatmybest]

Hi!,

Yes sorry I did forgot to include my download command.

Yep I did the following. The command was:

gallery-dl --write-metadata --range 1-5 generic:"https://www.website.com/search/?page=1&sort=curated&tags=%24john&topic=%24doe&topic_group=%24publicdomain"

I also tried:

gallery-dl --write-metadata --range 1-5 "generic:https://www.website.com/search/?page=1&sort=curated&tags=%24john&topic=%24doe&topic_group=%24publicdomain"

both fail with "No suitable extractor found"

but if I put the comma in the wrong spot in the config, it does seem to force the download with both of those download commands, and I get this message on the console:
[generic][info] Forcing use of generic information extractor.
Then continues to try download.

Thanks :)

[mikf]

$ cat cfg.json 
{
    "extractor": {
        "generic": {
            "enabled": true
        }
    }
}

$ gallery-dl -c cfg.json "https://www.website.com/search/?page=1&sort=curated&tags=%24john&topic=%24doe&topic_group=%24publicdomain"
[generic][info] Falling back on generic information extractor.
[generic][error] HttpError: '404 Not Found' for 'https://www.website.com/search/?page=1&sort=curated&tags=%24john&topic=%24doe&topic_group=%24publicdomain'

[mediocreatmybest]

Sorry, was just avoiding posting the URL directly to rawpixel, even though its public domain images I was looking at.

Thanks for the fast post, I just went through config and I think it would be because of the YTDL config.
Switching it to false, seem to get the downloading working correctly, is there something I should change or switch to get them working together?

from my config:

"ytdl": { "enabled": true, "format": null, "generic": false, "logging": true, "module": "youtube_dl", "raw-options": null },

[mikf]

because of the YTDL config.

Yeah, that's it. You can realistically only have one of them "enabled", since both try to match everything.

[mediocreatmybest]

Perfect :) thanks! Glad to have it figured out.

[rautamiekka]

Sorry, was just avoiding posting the URL

What purpose does that serve ? Especially when considering, but even then regardless of, the latter half of the sentence.

[mediocreatmybest]

Sorry, was just avoiding posting the URL

What purpose does that serve ? Especially when considering, but even then regardless of, the latter half of the sentence.

Hi!

Well, it wasn't an issue with the URL specifically, as it was the same with every website, I just included part of it due to the additional symbols. I also have seen arguments over scraping and website admins putting in measures to block scraping, as this is my primary github account I figured it was easier to just drop the URL to prevent the 1 in a million chance someone would get upset.

¯\_(ツ)_/¯