Optimizing AI Coding Assistant Workflows with AI Engineer Coach

Practical Summary and Purpose

AI Engineer Coach is an open-source tool designed to provide actionable insights into how developers interact with AI coding assistants. By analyzing local session logs, the tool generates a comprehensive dashboard that tracks progress, identifies anti-patterns, and measures output volume. It is built to operate entirely on your local machine, ensuring that no session data or proprietary information is transmitted externally. The tool is intended for developers who wish to refine their agentic engineering practices, improve prompt quality, and maintain better session hygiene across various AI harnesses. It serves as a centralized hub for monitoring your development lifecycle and enhancing your technical proficiency with AI tools.

Prerequisites and Installation

To utilize AI Engineer Coach, you must build the extension from source as it is not distributed via public marketplaces. Depending on your environment, you can choose between a Dev Container approach or a local build. For a local build, you require VS Code, Node.js, and npm. The installation process involves cloning the repository from https://github.com/microsoft/AI-Engineering-Coach and executing the build commands. Once the .vsix file is generated, you can install it directly into VS Code using the command palette or the CLI. This manual build process ensures you maintain full control over the extension code running in your environment.

For local builds, use the following sequence:

git clone https://github.com/microsoft/ai-engineering-coach.git
cd ai-engineering-coach
npm ci
npm run package

Documented Workflow and Integration

Once installed, you can access the dashboard by running the AI Engineer Coach: Open Dashboard command from the VS Code command palette. The tool organizes data into several categories: Observe, Measure, Improve, and Level Up. The Observe section provides a timeline and coding moments, while the Measure section tracks code volume and activity patterns. The Improve section is particularly useful for identifying anti-patterns through 45 predefined rules, which can be edited or tuned via the Rule Editor. Additionally, the tool can run as a canvas within the GitHub Copilot app, allowing for interactive side-panel UI without requiring a full VS Code window. This integration allows for seamless monitoring while you continue your primary development tasks.

Reading and Interpreting Output

The dashboard provides a multi-faceted view of your AI usage. The Dashboard page offers practice scores and weekly trends, while the Anti-Patterns page highlights specific areas for improvement with severity ratings and example prompts. If you have the GitHub Copilot app installed, you gain access to the GitHub App section, which includes productivity metrics and issue credits. Note that issue credit percentages are rough relative estimates used to normalize AI usage across linked issues and should not be used for formal billing reconciliation. The tool also includes a Skill Finder to help you discover and reuse effective prompt patterns, and a Context Health page to audit your workspace configuration and instruction files.

Limitations and Privacy Considerations

Privacy is a core design principle of AI Engineer Coach. The tool is read-only, meaning it never modifies your session files, and all analysis is performed locally. There is no proprietary telemetry, and the tool does not phone home. However, some advanced features—specifically the rule compiler, skill finder, and context review—utilize the VS Code built-in Copilot language model API when you explicitly invoke them. Furthermore, certain features like the Burndown chart are currently disabled, and token breakdowns are temporarily hidden. Users should also be aware that some features, such as the Learning Center and Context Health AI review, are unavailable when running the tool in canvas mode within the GitHub Copilot app.

Target Audience and Usage Scenarios

This tool is intended for developers and engineers who are actively using AI coding assistants and wish to move beyond basic usage to optimize their workflows. It is particularly useful for those who want to identify recurring anti-patterns in their prompting style or those who need to measure the impact of AI on their overall coding output. By providing a structured way to view session logs, it helps users transition from passive AI usage to a more intentional, agentic engineering approach. Whether you are working in a large enterprise or on individual projects, the tool provides the visibility needed to refine your interaction with AI models and improve your long-term productivity and code quality.