Definition
Computer use
Computer use is when an AI looks at a screen and operates it like a person would, by moving a cursor, clicking and typing.
- The AI sees the screen, decides what to do, and acts with a cursor and keyboard.
- It works in apps that were never built for it, because it uses them the way a person does.
- For risky steps, like sending or paying, a careful system asks first.
Computer use means an AI operating a computer through its screen. Instead of calling a program's hidden functions, it looks at what is displayed, works out what each part is, and then moves a cursor, clicks, scrolls and types, the same way you would. Because it only needs the screen, it can work in almost any app.
Example: opening a page and clicking a button
Say you ask an AI to download this month's statement from your bank's website. It follows a loop:
- It looks at the screen and sees a browser with a home page.
- It decides the next move: open the statements page. It moves its cursor to the link and clicks.
- It looks again, finds the button labelled "Download", and clicks it.
- It checks that a file appeared, and reports back.
Each cycle of look, decide, act is one step. If the page asks you to sign in, a careful system hands that to you. DuckyHelper, for example, is built not to type your passwords or card numbers, and you can stop it or take control at any moment. Its agent work is in the Mac app.
Why it matters
Most software has no way for another program to control it, but all of it has a screen. Computer use opens that software to automation, at the cost of being slower than calling a function directly. Because the AI can act, it needs limits: asking before steps that cannot be undone, which is the idea of a human in the loop. Read what computer use means for AI assistants for more.
Related terms
Further reading
Reviewed . Next review .
Owner: DuckyHelper