How to Build 3D & AR Apps with AI — Cursor, Windsurf, Claude Code

AI coding assistants are great at generating UI code. But ask them to build a 3D scene or an AR experience, and they usually hallucinate APIs that don't exist.

That changes with MCP servers (Model Context Protocol). MCP gives your AI assistant access to real SDK documentation and validated code patterns.

Here's how to set it up for 3D/AR development.

The Problem

Ask any AI to "display a 3D model in Jetpack Compose" and you'll get:

• References to deprecated Sceneform (dead since 2021)
• Three.js code when you asked for Android
• Made-up Compose APIs that don't compile
• Unity boilerplate when you wanted a lightweight solution

The Solution: MCP Servers

MCP servers give AI assistants access to specialized tools. sceneview-mcp provides:

• Complete API reference for SceneView (Android, iOS, Web)
• Code generation with validation
• Model search (Sketchfab integration)
• Project analysis

Setup

Claude Code / Claude Desktop

claude mcp add sceneview -- npx sceneview-mcp

Cursor

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "sceneview": {
      "command": "npx",
      "args": ["-y", "sceneview-mcp"]
    }
  }
}

Windsurf

Add to Windsurf MCP settings:

{
  "mcpServers": {
    "sceneview": {
      "command": "npx",
      "args": ["-y", "sceneview-mcp"]
    }
  }
}

What You Can Build

Once configured, ask your AI:

• "Build me a 3D model viewer for Android"
• "Create an AR furniture placement app"
• "Add a 3D product configurator with color switching"
• "Build an anatomy viewer for medical education"

The MCP server feeds correct API docs to the AI, so the generated code actually compiles.

Specialized MCP Servers

For domain-specific development, use these specialized servers:

Quick Example

After setup, here's what happens when you ask Claude Code to "build an AR app":

// AI generates this — and it compiles on first try
@Composable
fun ARScreen() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    var anchor by remember { mutableStateOf<Anchor?>(null) }

    ARSceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        planeRenderer = true,
        onSessionUpdated = { _, frame ->
            if (anchor == null) {
                anchor = frame.getUpdatedPlanes()
                    .firstOrNull { it.type == Plane.Type.HORIZONTAL_UPWARD_FACING }
                    ?.let { frame.createAnchorOrNull(it.centerPose) }
            }
        }
    ) {
        anchor?.let { a ->
            rememberModelInstance(modelLoader, "models/chair.glb")?.let { m ->
                AnchorNode(anchor = a) {
                    ModelNode(modelInstance = m, scaleToUnits = 0.5f)
                }
            }
        }
    }
}

No hallucinated APIs. No deprecated libraries. Just working code.

Resources

• GitHub: github.com/sceneview/sceneview
• npm: npmjs.com/package/sceneview-mcp
• API Reference: sceneview.github.io/llms.txt

SceneView is the only Compose-native 3D & AR SDK for Android, with additional support for iOS (SwiftUI), Web, Flutter, and React Native.