AI agents get a finger: New macOS CLI lets Claude point at your screen
A new Swift tool lets AI agents draw arrows on your Mac to show you exactly where to click, bridging the gap between terminal and GUI.
Tools · Source: Hacker News
What happened
AI agents can write complex code but fail when they need you to click a button. They dump text in a terminal you are ignoring. A new macOS command-line tool called bigarrow fixes this. It gives AI agents a literal finger to point at your screen. The project gained nearly four hundred stars in its first two days. Engineers from Apple, NVIDIA, and SAP are already taking notice.
The tool draws transparent arrows, boxes, and text over any window or display. It is a pure Swift binary built for macOS. There is no background daemon, no menu bar icon, no telemetry, and absolutely no AI inside the tool itself. It runs entirely from the command line. It integrates directly as a skill for Claude Code and Codex. The agent uses simple commands to target coordinates, windows, or specific UI elements.
It solves the notorious macOS focus-stealing problem. The arrow sits at the screen-saver level above all other windows. You can click right through the arrow shaft to interact with your apps. Your keyboard focus stays exactly where it is. When the job is done, or a set time limit expires, the arrow removes itself automatically. The agent cleans up its own mess.
Key facts
- 400 — GitHub stars in the first two days
- 182 — Tokens required for the base skill description
- 1.4% — CPU usage on a CI runner
Why it matters
Builders no longer have to guess what their AI agent wants them to do. When an agent hits a wall with OAuth consent, passkeys, or system permissions, it stops guessing. It draws a massive zigzag arrow on your screen. It can even add a copy button to the label for quick pasting of PIN codes or URLs. The human stays in the loop for critical actions without losing context or digging through terminal logs. This bridges the physical gap between terminal outputs and graphical interfaces.
This shifts how we design agent-human handoffs. Right now, agents are mostly confined to the terminal or browser DOM. They break when forced to interact with native OS dialogs. By giving them a lightweight, zero-dependency way to interact with the host OS layer visually, we open up hybrid workflows. Agents handle the heavy lifting in the background. Humans handle the final physical click. It proves that sometimes the best AI feature is a dumb visual pointer.
For builders
Zero permission drawing
Drawing the arrow requires zero macOS permissions. Finding specific UI elements requires Accessibility or Screen Recording, but these are granted to your terminal app like Ghostty or iTerm, never the binary itself. You maintain strict security control.
Low token overhead
The base skill description costs only 182 tokens. The full instructions load dynamically only when the agent decides to point. This saves Anthropic API costs for developers while keeping the agent context window clean.
Built for exit codes
The CLI returns standard exit codes for bad inputs, missing targets, or missing permissions. Agents thrive on explicit exit codes to correct their own mistakes. If a target is covered, the arrow waits until it is visible.
My take
We keep trying to build fully autonomous agents, but humans still hold the keys to the operating system. Giving an agent a simple visual pointer is a brilliant, low-tech hack for a high-tech problem. I love tools that do exactly one thing perfectly and get out of the way. Stop building complex screen annotators and just let the machine point.
Original reporting: Hacker News. This is my rewrite and opinion.