Feature: Add Output Format Selection to Crawl and Search Robots

[RohitR311]

Each robot type (Scrape, Crawl, Search) outputs data in different formats, but currently only the Scrape robot has an "Output Formats" configuration option. The same output format selection should be added to the Crawl robot and the Search robot's "Extract Data from Results" mode along with the functionality.

[Rutetid]

Hi , @RohitR311 , I would like to work in this issue , can you please assign this to me?

[Rutetid]

Hi , @RohitR311 I have a doubt regarding the implementation , like We have the option to select Screenshot (Visible viewport) and Screenshot Full page in the Scrape Robot , but how am I supposed to implement those in Crawl Robot and search robot , like should the robot be attaching screenshots of every link it crawls to in the crawl robot? and what about the search robot??

I have already implemented the Text , Markdown and HTML in Both Crawl and Search Robot , but confused about the screenshot part , kindly pls help with them!

[RohitR311]

Hey @Rutetid, great job implementing the Text, Markdown, and HTML formats! What we need for the crawl robot screenshots is that every page that is crawled should capture either a visible screenshot, full-page screenshot or both depending on the configuration. Similarly, for the search robot mode "Extract Data from Results" each page opened from the search results should also support capturing both types of screenshots. I hope that clears it out. If you have any further queries or need clarification on anything, feel free to let me know.