October 23, 2024

Anthropic publicly releases AI tool that can take over the user’s mouse cursor

In this article:

Anthropic, an AI software company, has announced a new tool called “Computer Use” that can take control of the user’s mouse cursor and perform basic tasks on their computer.
The tool is available exclusively with Anthropic’s mid-range 3.5 Sonnet model and allows users to give multi-step instructions for the AI to accomplish tasks on the user’s computer.
The tool operates by taking rapid successive screengrabs and counting pixels to move the cursor and click in the correct places.
While the tool has limitations and can be error-prone at times, it is seen as a step towards automating many menial tasks in office jobs.
However, there are valid concerns about the widespread deployment of this technology and its potential impact on employment.
Right now, Anthropic is putting Computer Use out there for testing to see what problems arise and to work with developers to improve its capabilities and find positive uses.

Contact Expert Request Form