feature: add robots.txt

[web3Gurung]

Additional context
Here's what needs to be added in robots.txt so that web crawlers can crawl + we have good seo:

User-agent: *
allow: /
Sitemap: https://strkfarm.com/sitemap.xml
Sitemap: https://blog.strkfarm.com/sitemap.xml
Sitemap: https://docs.strkfarm.com/p/sitemap.xml

The file should be as https://strkfarm.com/robots.txt

[web3Gurung]

this file exists at https://github.com/strkfarm/starkfarm-client/blob/dev/src/app/robots.txt, but can't be accessible on the internet.

[TobieTom]

Hola I'm Tobias. I would like to take over this issue. I've already contribute to some projects recently since I just learned Rust and Cairo

[web3Gurung]

hey folks, this issue has been taken cared off. thanks for reaching out though!

…

On Tue, 18 Feb 2025 at 02:32, Adamu Jethro ***@***.***> wrote: Hello, I am a frontend/mobile app developer with six years of experience. I have previously handled robot.txt for best seo optimization practices This is an issue I can confidently handle. — Reply to this email directly, view it on GitHub <https://github.com/strkfarm/starkfarm-client/issues/217#issuecomment-2664070368>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARFSCGSFWOYI6URFCQKHTGD2QJE7HAVCNFSM6AAAAABXEG6IC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUGA3TAMZWHA> . You are receiving this because you authored the thread.Message ID: ***@***.***> [image: JayWebtech]*JayWebtech* left a comment (strkfarm/starkfarm-client#217) <https://github.com/strkfarm/starkfarm-client/issues/217#issuecomment-2664070368> Hello, I am a frontend/mobile app developer with six years of experience. I have previously handled robot.txt for best seo optimization practices This is an issue I can confidently handle. — Reply to this email directly, view it on GitHub <https://github.com/strkfarm/starkfarm-client/issues/217#issuecomment-2664070368>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARFSCGSFWOYI6URFCQKHTGD2QJE7HAVCNFSM6AAAAABXEG6IC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUGA3TAMZWHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[techpilot]

Can I take this from here?