# Local AI Agents

Builds AI agents that run fully on a workstation with Foundry Local and Qwen models.

- Repository: https://github.com/microsoft/ai-agents-for-beginners (78611 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-10-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add microsoft/ai-agents-for-beginners --skill local-ai-agents -g -a claude-code -y`
Codex: `npx skills add microsoft/ai-agents-for-beginners --skill local-ai-agents -g -a codex -y`
Cursor: `npx skills add microsoft/ai-agents-for-beginners --skill local-ai-agents -g -a cursor -y`
Gemini CLI: `npx skills add microsoft/ai-agents-for-beginners --skill local-ai-agents -g -a gemini-cli -y`

## Description

Build local-first AI agents that run entirely on a developer workstation with Microsoft Foundry Local and Qwen function-calling models. Covers Small Language Models (SLMs), the OpenAI-compatible local endpoint, sandboxed local tools, local RAG with Chroma, local MCP servers, hybrid cloud/local routing, and the privacy/cost/offline trade-offs. Based on Lesson 17 of AI Agents for Beginners. USE FOR: run an agent locally, offline agent, on-device agent, Foundry Local, Qwen function calling, local tool calling, local RAG, Chroma vector database, local MCP server, privacy-preserving agent, hybrid local and cloud agent, small language model agent, engineering assistant on my machine. DO NOT USE FOR: deploying agents to the cloud at scale (use deploying-scalable-agents / Lesson 16), building your first agent concept (Lesson 01), Foundry (cloud) hosted agents, GPU cluster / server-side inference provisioning.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Creating Local AI Agents with Foundry Local and Qwen

> Companion skill for [Lesson 17, Creating Local AI Agents](../../../17-creating-local-ai-agents/README.md).
> Use it to help a learner build an agent that reasons, calls tools, and searches
> documentation entirely on their own machine, no cloud inference. Ground every
> recommendation in the lesson content and the runnable notebook.

## Triggers

Activate this skill when a learner wants to:
- Run an agent **fully on-device** for privacy, cost, or offline reasons.
- Serve a model locally with **Foundry Local** and connect via the OpenAI-compatible endpoint.
- Use a **Qwen function-calling** model to drive reliable local tool calls.
- Add **local RAG** (Chroma) or a **local MCP server**.
- Design a **hybrid** local/cloud routing strategy.

## Core mental model

An SLM trades breadth for privacy, cost, and offline operation. The winning
strategy: **let the SLM orchestrate and let tools do the heavy lifting.** The
model does not need to *know* the codebase, it needs to know when to call
`read_file` and `search_docs`. That plays to an SLM's strength (bounded decisions
like tool selection) and away from its weakness (broad knowledge, long multi-hop
reasoning).

## Why these specific pieces

- **Foundry Local** exposes an **OpenAI-compatible HTTP endpoint**, so cloud agent code transfers by changing only `base_url` (and using a local placeholder API key). It also auto-selects the best build (CPU/GPU/NPU) for the machine.
- **Qwen** models are trained for function calling and emit well-formed tool calls consistently, this is what turns a local *chat* model into a local *agent*.
- **Chroma** runs in-process and stores vectors on disk, so the whole RAG pipeline (embed → store → retrieve → reason) stays local.
- **MCP** is a transport, not a cloud service: an MCP server can run locally over `stdio`.

## Setup essentials

```bash
foundry model run qwen2.5-7b-instruct
foundry service status
```

The rest: https://github.com/microsoft/ai-agents-for-beginners/blob/HEAD/.agents/skills/local-ai-agents/SKILL.md