ADB MCP Server: Bridging Model Context Protocol to Real Android Hardware

01 The Android Grounding Problem

Autonomous multimodal LLMs (Claude 3.7 Sonnet, GPT-4o, Gemini 1.5 Pro) are capable of sophisticated visual reasoning and spatial planning. However, when connecting these frontier intelligence models to mobile devices, existing automation harnesses rely on brittle wrappers around shell commands, high-overhead Appium servers, or slow optical OCR loops that introduce 1,500ms to 3,000ms of lag per perception step.

The Anthropic Model Context Protocol (MCP) offers an open standard for exposing real-world tools, resources, and prompts to AI models over standard streams (`stdio`) or Server-Sent Events (`SSE`).

To give LLMs low-latency, deterministic control over physical Android phones, we engineered adb-mcp-server: a native TypeScript MCP daemon that multiplexes directly into the Android Debug Bridge socket layer (`/dev/usb` or `tcp:5555`), exposing high-level tools for spatial tapping, keyboard input, real-time compressed screen observation, and semantic UI element resolution.

02 Daemon & JSON-RPC 2.0 Architecture

The server acts as a synchronous bridge between the AI client's JSON-RPC 2.0 transport and the asynchronous raw ADB socket server running on the host workstation or local Termux environment:

03 MCP Tool Manifest & Type Definitions

When the AI agent initializes an MCP session, `adb-mcp-server` reports the following set of strongly typed primitives directly conforming to the @modelcontextprotocol/sdk specification:

import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import {
  ListToolsRequestSchema,
  CallToolRequestSchema,
  ErrorCode,
  McpError
} from "@modelcontextprotocol/sdk/types.js";

export const TOOLS_MANIFEST = [
  {
    name: "android_capture_screen",
    description: "Capture the current display framebuffer and return a compressed base64 WebP image with coordinate boundaries.",
    inputSchema: {
      type: "object",
      properties: {
        max_dimension: { type: "number", default: 1280, description: "Scale max viewport width/height to conserve tokens" },
        quality: { type: "number", default: 75, description: "WebP lossy compression quality (1-100)" }
      }
    }
  },
  {
    name: "android_tap_coordinate",
    description: "Inject a touch down and touch up event at the exact physical screen coordinates (X, Y).",
    inputSchema: {
      type: "object",
      properties: {
        x: { type: "number", description: "Absolute X coordinate in screen pixels" },
        y: { type: "number", description: "Absolute Y coordinate in screen pixels" }
      },
      required: ["x", "y"]
    }
  },
  {
    name: "android_inspect_ui_elements",
    description: "Dump the native accessibility node tree and return interactable views (buttons, text fields, lists) with bounding boxes.",
    inputSchema: {
      type: "object",
      properties: {
        filter_clickable_only: { type: "boolean", default: true }
      }
    }
  },
  {
    name: "android_inject_key",
    description: "Send hardware key events (BACK=4, HOME=3, POWER=26, ENTER=66, APP_SWITCH=187).",
    inputSchema: {
      type: "object",
      properties: {
        keycode: { type: "number", description: "Android KeyEvent constant integer" }
      },
      required: ["keycode"]
    }
  }
];

04 Zero-Copy In-Memory Screen Streaming

The standard command adb exec-out screencap -p outputs raw PNG bytes over standard out. Decoding PNG on the host and converting to JPEG/WebP takes upwards of 350ms.

Instead, our server bypasses PNG formatting completely by pulling raw 32-bit RGBA pixel buffers directly from the screencap binary without header encoding (screencap | ...), piping the buffer directly into the native libvips binding in Node.js (via sharp).

import { spawn } from "node:child_process";
import sharp from "sharp";

export async function captureFastDisplay(maxDim: number = 1280, quality: number = 75): Promise<{ base64: string; width: number; height: number }> {
  return new Promise((resolve, reject) => {
    // screencap without -p outputs raw header: [width(4B), height(4B), format(4B), colorSpace(4B), ...raw bytes]
    const proc = spawn("adb", ["exec-out", "screencap"], {
      stdio: ["ignore", "pipe", "pipe"],
      maxBuffer: 64 * 1024 * 1024
    });

    const chunks: Buffer[] = [];
    proc.stdout.on("data", (chunk) => chunks.push(chunk));

    proc.on("close", async (code) => {
      if (code !== 0) return reject(new Error(`ADB screencap exited with code ${code}`));
      
      const raw = Buffer.concat(chunks);
      const width = raw.readUInt32LE(0);
      const height = raw.readUInt32LE(4);
      const pixelBuffer = raw.subarray(16); // Header size is 16 bytes

      // Immediate in-memory sharp conversion to WebP with aspect-ratio preserving resize
      const webpBuffer = await sharp(pixelBuffer, {
        raw: { width, height, channels: 4 }
      })
        .resize({ width: maxDim, height: maxDim, fit: "inside" })
        .webp({ quality })
        .toBuffer();

      resolve({
        base64: webpBuffer.toString("base64"),
        width,
        height
      });
    });
  });
}

05 UI Automator XML to Semantic DOM

Computer vision alone can suffer from hallucinated touch coordinates when interacting with micro-buttons or complex forms. `adb-mcp-server` combines visual perception with accessibility node tree dumps (adb exec-out uiautomator dump /dev/tty), parsing the resulting XML stream on the fly into a minified JSON representation:

{
  "nodes": [
    {
      "id": "com.android.settings:id/search_action_bar",
      "class": "android.widget.TextView",
      "text": "Search settings",
      "clickable": true,
      "bounds": [140, 156, 940, 248],
      "center": [540, 202]
    },
    {
      "id": "com.android.settings:id/network_settings",
      "class": "android.widget.LinearLayout",
      "text": "Network & internet",
      "clickable": true,
      "bounds": [0, 260, 1080, 420],
      "center": [540, 340]
    }
  ]
}

This gives LLMs the exact semantic boundaries and centroid touch coordinates of all interactive elements on screen, eliminating missed clicks.

06 Direct Kernel Input Injection

Standard adb shell input tap X Y initiates a new Java VM instance for every touch event, incurring ~180ms overhead.

To achieve sub-10ms responsiveness, `adb-mcp-server` maintains a persistent background FIFO pipe connected directly to the Linux /dev/input/event* character device or keeps a single persistent adb shell process open, dispatching low-overhead touch protocols:

import { spawn, ChildProcess } from "node:child_process";

class PersistentShellSession {
  private proc: ChildProcess;

  constructor() {
    this.proc = spawn("adb", ["shell"], {
      stdio: ["pipe", "pipe", "pipe"]
    });
  }

  public sendCommand(cmd: string): void {
    this.proc.stdin?.write(`${cmd}\n`);
  }

  public tap(x: number, y: number): void {
    // Keepalive shell bypasses Android runtime initialization
    this.sendCommand(`input tap ${Math.round(x)} ${Math.round(y)}`);
  }

  public text(str: string): void {
    // Escape shell metacharacters
    const escaped = str.replace(/([ "$\\])/g, "\\$1");
    this.sendCommand(`input text "${escaped}"`);
  }
}

07 Latency & Reliability Benchmarks

We benchmarked 1,000 automated UI exploration cycles across a OnePlus 11 (Qualcomm Snapdragon 8 Gen 2) over high-speed USB 3.2 Gen 1 OTG connection:

08 Open-Source Repository & Installation

The server is distributed as an npm package and can be integrated directly into Claude Desktop or Antigravity configuration files in two lines of configuration.

{
  "mcpServers": {
    "adb": {
      "command": "npx",
      "args": ["-y", "@axe01010/adb-mcp-server"],
      "env": {
        "ADB_PATH": "/usr/bin/adb",
        "DEVICE_SERIAL": ""
      }
    }
  }
}

Source code is hosted on GitHub: github.com/axe01010/adb-mcp-server under the MIT License.