The first open-source Artificial Narrow Intelligence generalist agentic framework Computer-Using-Agent that fully operates graphical-user-interfaces (GUIs) by using only natural language. Uses Visualization-of-Thought and Chain-of-Thought reasoning to elicit spatial reasoning and perception, emulates, plans and simulates synthetic HID interactions.

This is the repo for the paper "OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use".

Mark web pages for use with vision-language models