AI agents are good at generating text and calling APIs. But many real-world tasks still require interacting with graphical applications: editing a video, checking a spreadsheet, using legacy Windows software, or operating an Android phone.
rdp-cli provides a simple command-line workflow for letting an AI agent control RDP and VNC desktops—and verify every action through screenshots.
1. Install rdp-cli
On Linux x86_64, download the executable and make it runnable:
chmod u+x ./rdp-cli-linux-x86_64
./rdp-cli-linux-x86_64 --help
No additional runtime or administrator access is required. For Windows and macOS, see the quick start guide.
2. Connect to a Windows RDP session
Make sure RDP is enabled on the target computer. Then connect with your username and password:
IFS= read -r -s -p 'Password: ' rdp_password
printf '\n'
printf '%s' "$rdp_password" | \
./rdp-cli-linux-x86_64 connect \
--host 192.0.2.20 \
--user USER \
--password-stdin
unset rdp_password
The command returns a session identifier. Save it as SESSION for the next steps. The password is passed through standard input rather than being written directly into the command line.
3. Let the agent see the desktop
Before taking an action, capture the current screen:
./rdp-cli-linux-x86_64 screenshot \
--session SESSION \
--out ./before.png
An AI agent can inspect before.png, locate a button or input field, and decide what to do next. This observe → act → observe pattern helps the agent verify the current state after every action.
4. Click, type, and verify
After locating an input field at coordinates X, Y:
./rdp-cli-linux-x86_64 click \
--session SESSION \
--desktop \
--x X \
--y Y \
--observation OBSERVATION
./rdp-cli-linux-x86_64 screenshot \
--session SESSION \
--out ./focused.png
./rdp-cli-linux-x86_64 type \
--session SESSION \
--text "Remote task completed." \
--observation NEW_OBSERVATION
./rdp-cli-linux-x86_64 screenshot \
--session SESSION \
--out ./after.png
The agent can inspect the final screenshot instead of assuming the command succeeded.
5. Watch or take over when needed
A person can open the same session in a native window or browser:
./rdp-cli-linux-x86_64 watch \
--session SESSION \
--viewer window
The viewer starts in read-only mode and supports pausing agent input, taking control, and handing the session back.
Why this workflow matters
With rdp-cli, an AI agent can work with existing desktop applications and remote environments without requiring a new API integration for every tool.
The basic loop is simple:
Connect → Screenshot → Act → Screenshot → Verify
Try the complete setup at rdp-cli.com.
Always use authorized computers and trusted networks. Review the connection and certificate settings before using rdp-cli in production.
