Status Code 403 for Some Watch URLs

[Lord0fBytes]

I tried looking around for a fix to this, but came up short. I setup a couple URL's I'd like to watch, but when they are checked it get an error: EmptyReply: Status Code 403

I have tried with both the Basic HTTP fetch and the Webdrivers Fetchers. From what I can tell this is due to the site not loading for the changedetector to view the changes, but I thought that was what the Webdrivers are for.

I am using Docker Compose running on Ubuntu 20.04.
URLs that arent working:

https://support.wyze.com/hc/en-us/articles/360015979872-Service-Status-Known-Issues
https://downdetector.com/status/twitter

Hopefully this is just something I overlooked in the docs.

Here is my settings:

[dgtlmoon]

they both work for me, looks like you ended up on their blacklist somehow (maybe by your IP address or country)

[Lord0fBytes]

Interesting, I am able to view those websites normally from the same IP address.

[dgtlmoon]

yeah that is odd, can you do some tcpdump and see what the difference in the requests are?

[Lord0fBytes]

I am still new to this to feel free to let me know exactly what tcpdump you are looking for.

Here is a tcpdump from port 443: https://pastebin.com/TsJTuSdC
This one is from port 5000: https://pastebin.com/kgSs3nq0

[dgtlmoon]

I'm able to reproduce it locally - i can see that the error is actually a cloudflare page coming up - so unfortunately - i dont have an answer, maybe they know the browser runs some user-agent associated with testing

and also, the 'basic low level' fetcher uses the python user agent.. so i dont know what todo here

i tried adding a user agent (with a quick hack) to the 'basic low level' one and i still got 403

their cloudflare anti-robot stuff is VERY good

[Lord0fBytes]

Well that is good to know lol. I guess some more hackery will be needed then.

[Lord0fBytes]

Any chance you could test with these User Agent values:
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
I do see they are using certain User Agent blocks and firing a captcha when it doesnt show those values.

[dgtlmoon]

with Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 locally (europe) and using the raw http fetcher, it gives 403

[dgtlmoon]

try the subscription service :(

[Lord0fBytes]

OK, thanks for your help looking into this. I appreciate it.

[dgtlmoon]

@Webtron18 it just occured to me that you can use the 'http header' section of the "Requests" for to set user-agent too

[carlogarro]

Hello, I am having a very similar error with this website Error - 403 (Access denied) received. Could you help me? https://www.upf.edu/es/web/personal/ofertes-pdi/-/asset_publisher/QSQJdCTI6BOh/content/lloc-de-treball-escala-de-funcionari-categoria-de-pdi-141/213063326/
Thank you so much.