The AI Agent Factory

7.5 Prefer connectors to browsers, and browsers to screen control

Status
stable
5 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. To pay a bill, you can use the company's app, log in on its website, or hand someone your unlocked laptop.

An AI Worker can reach another system, such as an accounting system, in three ways.

A connector is a link to one system that offers named actions through the system's own interface. Each named action is a tool. Reading and writing are separate tools. Each call to a tool can be approved and recorded one by one.

A browser uses web pages the way a person does. It can reach any site. If it is signed in, its session carries the person's full rights: everything that person may do on the site. And its clicks are harder to review than named tool calls.

Screen control uses desktop apps by looking at the screen, moving the pointer and typing. Anthropic calls it computer use. It reaches anything visible, including windows you did not mean to show.

Order them by how narrow the reach is, how clear the record is, and how the permissions are enforced. Connectors usually come first, but not always. A connector with every tool switched on can reach more than a browser signed in to an account with few rights. So use the narrowest well-controlled path. That is usually a connector, then the browser. Use screen control only for a trusted app that has no other way in, with sensitive windows closed.

Browser and screen actions skip a connector's own limits, such as which tools are on and which ask first. The application still checks the rights of the signed-in account. But those are a person's rights, usually wider than the worker needs. So rebuild the limits around them yourself, with approval for each app or site and a record of what was done.

The title reads "Choose the narrowest path with effective controls." The line below it reads "Usually: connector first, browser next, screen control last." A table compares three routes in four rows: reach, action record, permissions, and how to use it well. Connector, usually first: it reaches the named actions the connector exposes. Its action record is structured tool calls, and logging and approval depend on setup. Its permissions are tool restrictions plus the connected account's rights. To use it well, enable only the tools and data the task needs. Browser, when a connector cannot do the job: it reaches allowed sites and accessible signed-in sessions. Its record is page interactions and clicks, so inspect the available logs. Account and browser controls apply, but connector limits do not. To use it well, restrict sites, use a limited account and require needed approvals. Screen control, when no narrower route works: it reaches accessible apps, windows and desktop content. Its record is clicks, typing and screen observations, which are often harder to audit. Account, operating-system and app controls apply, but connector limits do not. To use it well, use trusted apps, close sensitive windows and record actions. A gold note says to compare the actual setup, because a broadly privileged connector can reach more than a tightly restricted browser account. A dark bar reads: check reach, action records and enforceable permissions before choosing.

Figure 7.5. Three ways to reach a system, compared by reach, record and permissions.

Check yourself

Question 1 / 8 · current

0 answered

In what order should a worker usually reach a system?

7.4 Permissions set the blast radius

Why product settings decide the worst case, three rules that keep what a worker can do inside what it may do, and what to do when no setting fits.

7.6 Untrusted input plus the power to act outward

The one combination that lets a stranger's text turn into an action, and three ways to break it for each task, with two workers from other jobs.

On this page