Leveraging the Google Agent Development Kit (ADK) and the underlying Gemini LLM to build Multi-Agent Applications with A2A protocol support using the Python programming language deployed to Azure AKS.
Aren’t There a Billion Python ADK Demos?
Yes there are.
Python has traditionally been the main coding language for ML and AI tools. The goal of this article is to provide a multi-agent test bed for building, debugging, and deploying multi-agent applications.
Hold the Pickles, Hold the Lettuce!
So what is different about this lab compared to all the others out there?
This is one of the first deep dives into a Multi-Agent application leveraging the advanced tooling of Gemini CLI. The starting point for the demo was an existing Codelab- which was updated and re-engineered with Gemini CLI.
The original Codelab- is here:
Building a Multi-Agent System | Google Codelabs
Python Version Management
One of the downsides of the wide deployment of Python has been managing the language versions across platforms and maintaining a supported version.
The pyenv tool enables deploying consistent versions of Python:
GitHub - pyenv/pyenv: Simple Python version management
As of writing — the mainstream python version is 3.13. To validate your current Python:
python --version
Python 3.13.13
Azure Kubernates Service
Azure Kubernetes Service (AKS) is a fully managed, serverless Kubernetes service on Microsoft Azure that simplifies deploying, scaling, and managing containerized applications. It handles critical tasks like health monitoring, maintenance, and automated upgrades, reducing operational complexity. AKS is used for microservices, DevOps, and cloud-native app development.
More details are available here:
https://azure.microsoft.com/en-us/products/kubernetes-service
Isn’t that Overkill? A whole Cluster Just for some Agents?!
An entire cluster is a large deployment for just a basic ADK server. The goal was to validate that ADK servers can be deployed — and that opens the door for more complex deployments that can take advantage of the full services in the cluster.
Why would I want Gemini CLI with Azure? Isn’t that a Google Thing?
Yes- Gemini CLI leverages the Google Cloud console and Gemini models but it is also open source and platform agnostic. Many applications are already cross-cloud so this enables familiar tools to be run natively on Microsoft Azure.
Gemini CLI
If not pre-installed you can download the Gemini CLI to interact with the source files and provide real-time assistance:
npm install -g @google/gemini-cli
Testing the Gemini CLI Environment
Once you have all the tools and the correct Node.js version in place- you can test the startup of Gemini CLI. You will need to authenticate with a Key or your Google Account:
▝▜▄ Gemini CLI v0.33.1
▝▜▄
▗▟▀ Logged in with Google /auth
▝▀ Gemini Code Assist Standard /upgrade no sandbox (see /docs) /model Auto (Gemini 3) | 239.8 MB
Node Version Management
Gemini CLI needs a consistent, up to date version of Node. The nvm command can be used to get a standard Node environment:
Agent Development Kit
The Google Agent Development Kit (ADK) is an open-source, Python-based framework designed to streamline the creation, deployment, and orchestration of sophisticated, multi-agent AI systems. It treats agent development like software engineering, offering modularity, state management, and built-in tools (like Google Search) to build autonomous agents.
The ADK can be installed from here:
Agent Skills
Gemini CLI can be customized to work with ADK agents. Both an Agent Development MCP server, and specific Agent skills are available.
More details are here:
To get the Agent Skills in Gemini CLI:
> /skills list
Available Agent Skills:
and the ADK documentation:
> /mcp list
Configured MCP servers:
🟢 adk-docs-mcp (from adk-docs-ext) - Ready (2 tools)
Tools:
- mcp_adk-docs-mcp_fetch_docs
- mcp_adk-docs-mcp_list_doc_sources
Where do I start?
The strategy for starting multi agent development is a incremental step by step approach.
First, the basic development environment is setup with the required system variables, and a working Gemini CLI configuration.
Then, ADK Multi-Agent is built, debugged, and tested locally. Finally — the entire solution is deployed to Google Cloud Run.
Setup the Basic Environment
At this point you should have a working Python environment and a working Gemini CLI installation. All of the relevant code examples and documentation is available in GitHub.
The next step is to clone the GitHub repository to your local environment:
cd ~
git clone https://github.com/xbill9/gemini-cli-azure
cd multi-aks
Then run init2.sh from the cloned directory.
The script will attempt to determine your shell environment and set the correct variables:
source init2.sh
If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:
source set_env.sh
Variables like PROJECT_ID need to be setup for use in the various build scripts- so the set_env script can be used to reset the environment if you time-out.
Finally install the packages and dependencies:
make install
Verify The ADK Installation
To verify the setup, run the ADK CLI locally with the researcher agent:
xbill@penguin:~/gemini-cli-azure/multi-aks/agents$ adk run researcher
/home/xbill/.local/lib/python3.13/site-packages/google/adk/features/_feature_decorator.py:72: UserWarning: [EXPERIMENTAL] feature FeatureName.PLUGGABLE_AUTH is enabled.
check_feature_enabled()
Log setup complete: /tmp/agents_log/agent.20260412_151927.log
To access latest log: tail -F /tmp/agents_log/agent.latest.log
{"asctime": "2026-04-12 15:19:27,144", "name": "root", "levelname": "INFO", "message": "Logging initialized for researcher", "filename": "logging_config.py", "lineno": 54, "service": "researcher", "log_level": "INFO"}
{"asctime": "2026-04-12 15:19:27,145", "name": "researcher.agent", "levelname": "INFO", "message": "Initialized researcher agent with model: gemini-2.5-flash", "filename": "agent.py", "lineno": 85}
{"asctime": "2026-04-12 15:19:27,146", "name": "google_adk.google.adk.cli.utils.envs", "levelname": "INFO", "message": "Loaded .env file for researcher at /home/xbill/gemini-cli-azure/multi-aks/.env", "filename": "envs.py", "lineno": 83}
{"asctime": "2026-04-12 15:19:27,146", "name": "google_adk.google.adk.cli.utils.local_storage", "levelname": "INFO", "message": "Using per-agent session storage rooted at /home/xbill/gemini-cli-azure/multi-aks/agents", "filename": "local_storage.py", "lineno": 84}
{"asctime": "2026-04-12 15:19:27,146", "name": "google_adk.google.adk.cli.utils.local_storage", "levelname": "INFO", "message": "Using file artifact service at /home/xbill/gemini-cli-azure/multi-aks/agents/researcher/.adk/artifacts", "filename": "local_storage.py", "lineno": 110}
{"asctime": "2026-04-12 15:19:27,146", "name": "google_adk.google.adk.cli.utils.service_factory", "levelname": "INFO", "message": "Using in-memory memory service", "filename": "service_factory.py", "lineno": 266}
{"asctime": "2026-04-12 15:19:27,152", "name": "google_adk.google.adk.cli.utils.local_storage", "levelname": "INFO", "message": "Creating local session service at /home/xbill/gemini-cli-azure/multi-aks/agents/researcher/.adk/session.db", "filename": "local_storage.py", "lineno": 60}
Running agent researcher, type exit to exit.
[user]
Test The ADK Web Interface
This tests the ADK agent interactions with a browser:
xbill@penguin:~/gemini-cli-azure/multi-aks$ adk web --host 0.0.0.0
/home/xbill/.local/lib/python3.13/site-packages/google/adk/features/_feature_decorator.py:72: UserWarning: [EXPERIMENTAL] feature FeatureName.PLUGGABLE_AUTH is enabled.
check_feature_enabled()
2026-04-12 15:20:23,207 - INFO - service_factory.py:266 - Using in-memory memory service
2026-04-12 15:20:23,208 - INFO - local_storage.py:84 - Using per-agent session storage rooted at /home/xbill/gemini-cli-azure/multi-aks
2026-04-12 15:20:23,208 - INFO - local_storage.py:110 - Using file artifact service at /home/xbill/gemini-cli-azure/multi-aks/.adk/artifacts
/home/xbill/.local/lib/python3.13/site-packages/google/adk/cli/fast_api.py:198: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
credential_service = InMemoryCredentialService()
/home/xbill/.local/lib/python3.13/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
super(). __init__ ()
INFO: Started server process [13962]
INFO: Waiting for application startup.
+-----------------------------------------------------------------------------+
| ADK Web Server started |
| |
| For local testing, access at http://0.0.0.0:8000. |
+-----------------------------------------------------------------------------+
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Then use the web interface — either on the local interface 127.0.0.1 or the catch-all web interface 0.0.0.0 -depending on your environment:
Special note for Google Cloud Shell Deployments- add a CORS allow_origins configuration exemption to allow the ADK agent to run:
adk web --host 0.0.0.0 --allow_origins 'regex:.*'
Multi Agent Design
The multi-agent deployment consists of 5 agents:
- Researcher
- Judge
- Orchestrator
- Content Builder
- Course Builder
For a detailed analysis of the multi-agent architecture- this article provides the background information:
Multi-Agent A2A with the Agent Development Kit(ADK), Cloud Run, and Gemini CLI
Running/Testing/Debugging Locally
The main Makefile has been extended with extensive targets for managing the agents on the local development environment.
The key targets include:
xbill@penguin:~/multi-agent$ make help
Available commands:
install - Install all dependencies for root, agents, and app
start - Start all services locally (alias for start-local)
stop - Stop all local services (alias for stop-local)
run - Start all services locally (alias for start-local)
local - Show local service URLs
start-local - Start all local services in background
stop-local - Stop all local processes
test - Run all tests (pytest)
test-researcher - Test the Researcher agent directly
test-judge - Test the Judge agent directly
test-orchestrator - Test the Orchestrator logic
lint - Run linting checks (ruff)
deploy - Deploy all services to Cloud Run
destroy - Delete all Cloud Run services
clean - Remove caches and logs
First check for local running agents:
xbill@penguin:~/multi-agent$ make status
Checking status of locally running agents and servers...
--- Network Status ---
No services listening on expected ports (8000-8004, 5173).
--- Process Status ---
No matching processes found.
Then all the agents can be started together:
xbill@penguin:~/multi-agent$ make start
Stopping any existing agent and server processes...
Starting all agents in background...
Waiting for sub-agents to start...
All agents started. Logs: researcher.log, judge.log, content_builder.log, orchestrator.log
Starting App Backend in background...
Starting Frontend dev server in background...
All services started. Logs: researcher.log, judge.log, content_builder.log, orchestrator.log, backend.log, frontend.log
Frontend: http://localhost:5173
Backend: http://localhost:8000
make status
--- Local Service URLs ---
Frontend: [http://localhost:5173](http://localhost:5173)
Backend: [http://localhost:8000](http://localhost:8000) (main app)
Agents:
Researcher: [http://localhost:8001](http://localhost:8001)
Judge: [http://localhost:8002](http://localhost:8002)
Content Builder: [http://localhost:8003](http://localhost:8003)
Orchestrator: [http://localhost:8004](http://localhost:8004)
Then connect to the front end locally:
And the entire agent system will run in the local environment:
Local Logging / Debugging
Gemini CLI has full access to the local agent logs for debugging and troubleshooting:
> review the local logs for the multi-agent run
✦ I will check run_local.sh and the project structure to determine where logs are stored or how they are generated.
This will verify the local operation of the code:
✦ The local logs confirm a successful multi-agent run for the topic "the history of the internet." Here is a summary of the execution flow:
* Orchestrator (orchestrator.log): Successfully managed the pipeline. It captured the topic, coordinated the Research-Judge loop, and escalated to
content generation after the first iteration.
* Researcher (researcher.log): Gathered approximately 9.2KB of research data using the google_search tool. It completed its task with a 200 OK
response.
* Judge (judge.log): Evaluated the findings and gave a "pass" status. The judge noted that the research was "comprehensive, accurate, and
well-structured," making it an "excellent foundation for educational content."
* Content Builder (content_builder.log): Successfully transformed the validated research into a structured Markdown course module and finished
execution.
* Backend & Frontend (backend.log, frontend.log): The FastAPI backend (port 8000) and Vite frontend (port 5173) are running and communicating with
the orchestrator as expected.
Notable Observation:
* A minor warning appeared in researcher.log: Researcher callback: 'response' not found or has no parts. However, this did not impact the
successful completion of the research or the overall pipeline.
All services are currently active and logged at the DEBUG level as specified in run_local.sh.
Deploying to Azure Kubernates Service(AKS)
The project level Makefile has targets for managing the Agent deployment to serverless endpoints. A utility script checks the deployment status:
xbill@penguin:~/gemini-cli-azure/multi-aks$ make status
Checking AKS Deployment status...
You can submit the build for GKE deployment:
make deploy
│ deployment "orchestrator" successfully rolled out │
│ deployment "course-creator" successfully rolled out │
│ Deployment complete! │
│ Course Creator External IP (may take a moment to appear): │
│ 20.99.250.177
✦ I have successfully deployed the AI Course Creator system to Azure AKS.
All services are now running and healthy:
- Researcher: Running
- Judge: Running
- Content Builder: Running
- Orchestrator: Running
- Course Creator (Web App): Running at http://20.99.250.177 (http://20.99.250.177)
Key fixes applied during deployment:
- Tracing: Disabled mandatory Cloud Trace in app/main.py and removed the --trace_to_cloud flag from all agent Dockerfiles to prevent startup
crashes caused by missing Google Cloud credentials on Azure.
- Rollout: Forced a container restart to ensure the latest Docker images with the above fixes were pulled by the AKS cluster.
You can now access the AI Course Creator at the external IP provided above.
Once the containers are deployed- you can then get the endpoint:
✦ I have retrieved the external IP of the AI Course Creator system on Azure AKS.
The application is now accessible at: http://20.99.250.177 (http://20.99.250.177)
The status can then be checked:
> make status
xbill@penguin:~/gemini-cli-azure/multi-aks$ make status
Checking AKS Deployment status...
NAME READY UP-TO-DATE AVAILABLE AGE
content-builder 0/1 1 0 119m
course-creator 1/1 1 1 119m
judge 0/1 1 0 119m
orchestrator 0/1 1 0 119m
researcher 0/1 1 0 119m
NAME READY STATUS RESTARTS AGE
content-builder-7b5c47d4ff-4r7xt 0/1 CrashLoopBackOff 6 (3m35s ago) 9m54s
course-creator-85769bfb77-9dn7t 1/1 Running 0 9m53s
judge-84d84b55f-6hkv2 0/1 CrashLoopBackOff 6 (3m39s ago) 9m54s
orchestrator-69cfb5c6dd-cv8kh 0/1 CrashLoopBackOff 6 (3m32s ago) 9m53s
researcher-6545fbcb8d-wsnbx 0/1 CrashLoopBackOff 6 (3m49s ago) 9m54s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
content-builder ClusterIP 10.0.214.225 <none> 8000/TCP 119m
course-creator LoadBalancer 10.0.85.76 20.99.250.177 80:32394/TCP 119m
judge ClusterIP 10.0.106.30 <none> 8000/TCP 119m
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 123m
orchestrator ClusterIP 10.0.165.216 <none> 8000/TCP 119m
researcher ClusterIP 10.0.32.188 <none> 8000/TCP 119m
The service will be visible in the Azure console:
And the entire system can be tested:
✦ I will run the end-to-end test against the AKS cluster to ensure the deployment is functional.
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ✓ Shell make e2e-test-aks [current working directory /home/xbill/gemini-cli-azure/multi-aks] (Execute the end-to-end test against the AKS c… │
│ │
│ ... first 6 lines hidden (Ctrl+O to show) ... │
│ {"type": "progress", "text": "\ud83d\udd0d Researcher is gathering information..."} │
│ {"type": "progress", "text": "\u2696\ufe0f Judge is evaluating findings..."} │
│ {"type": "progress", "text": "\u2696\ufe0f Judge is evaluating findings..."} │
│ {"type": "progress", "text": "\u270d\ufe0f Building the final course content..."} │
│ {"type": "progress", "text": "\u270d\ufe0f Content Builder is writing the course..."} │
│ {"type": "result", "text": "# The History of the Internet\n\n## Module 1: The Dawn of Networking (Pre-1960s to 1970s)\n\n### Early Concepts │
│ and Cold War Context\n* **Cold War Imperatives** : The internet's origins are rooted in 1950s Cold War anxieties, specifically after │
│ Sputnik's launch in 1957. This prompted the U.S. to establish ARPA (Advanced Research Projects Agency) with a key driver being the desire for │
│ a communication network resilient to attack.\n* **Visionaries and Packet Switching** :\n * **Paul Baran** at RAND Corporation proposed │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
✦ The make e2e-test-aks command was executed successfully.
I first switched the kubectl context to the correct Azure AKS cluster (adk-aks-penguin). Then, I verified that the course-creator service was
running with a public LoadBalancer IP. Finally, I ran the end-to-end test, which successfully communicated with the AKS-deployed microservices
(Researcher, Judge, and Content Builder) to generate a course on "The History of the Internet."
Running the Web Interface
Start a connection to the Cloud Run deployed app:
http://20.99.250.177
Then connect to the app :
Then use online course generator:
Final Gemini CLI Code Review
As a final step — Gemini CLI was used for a full code review of the project:
✦ This multi-agent project is a well-engineered application of ADK 2.5 and the A2A protocol. The separation of specialized agents (Researcher,
Judge, Content Builder) coordinated by a central Orchestrator demonstrates a mature microservice-oriented design.
Key Strengths
* Coordinated Orchestration: The use of SequentialAgent and LoopAgent creates a robust, iterative research-judge cycle.
* Resilient State Flow: The strategy of passing data through event content and "recovering" it via before_agent_callback heuristics is a clever
way to handle state across distributed, independent session stores.
* Polished Streaming: The web app's SSE implementation, specifically the greedy overlap deduplication (merge_strings) and system message
cleanup, ensures a high-quality user experience despite the inherent noise in multi-agent LLM streams.
* Cloud-Native Readiness: Using Identity Tokens for authenticated service-to-service communication and middleware for dynamic A2A URL rewriting
makes the system ready for production deployment on Cloud Run.
Summary
The Agent Development Kit (ADK) was used to build a multi-agent system with A2A support using the Gemini Flash LLM Model. This application was tested locally with Gemini CLI and then deployed to Azure AKS. Finally, Gemini CLI was used for a complete project code review.
