Moving engineering tools into src/tests format.

[conniey]

What does this PR do?

• Moves ToolDescriptionEvaluator into a src/test folder for future tools.
• Creates a Tool.sln to open tools at once.

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

• Required for All PRs
  • Read contribution guidelines
  • PR title clearly describes the change
  • Commit history is clean with descriptive messages (cleanup guide)
  • Added comprehensive tests for new/modified functionality
  • Updated servers/Azure.Mcp.Server/CHANGELOG.md and/or servers/Fabric.Mcp.Server/CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
• For MCP tool changes:
  • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
  • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
  • Validate README.md changes using script at eng/scripts/Process-PackageReadMe.ps1. See Package README
  • Updated command list in /servers/Azure.Mcp.Server/docs/azmcp-commands.md and/or /docs/fabric-commands.md
  • Run .\eng\scripts\Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
  • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
  • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
• Extra steps for Azure MCP Server tool changes:
  • Updated test prompts in /servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
  • 👉 For Community (non-Microsoft team member) PRs:
    • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
    • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

[vcolin7]

Maybe I don't know enough about .NET, but I had originally envisioned having the /src directory be inside ToolDescriptionEvaluator and not the other way around. If we want to not have the latter right under eng/tools we could create a directory named eng/tools/util or something like that.

[Copilot]

Pull Request Overview

This PR introduces a Tool Description Evaluator, a comprehensive testing and analysis tool that validates Azure MCP Server tool descriptions against user prompts using Azure OpenAI embeddings. The tool helps ensure proper tool selection by measuring semantic similarity between tool descriptions and test prompts using cosine similarity scoring.

Key Changes

• Added a complete ToolDescriptionEvaluator application with vector database implementation
• Implemented embedding service using Azure OpenAI's text-embedding-3-large model
• Created supporting infrastructure including PowerShell scripts for execution, prompt generation, and tools.json updates
• Added comprehensive documentation (README, Quickstart guide)

Reviewed Changes

Copilot reviewed 1 out of 24 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
eng/tools/src/ToolDescriptionEvaluator/Program.cs	Main application logic with command-line argument parsing, tool/prompt loading, vector database population, and analysis execution
eng/tools/src/ToolDescriptionEvaluator/VectorDb/VectorDB.cs	Vector database implementation with cosine similarity support and thread-safe operations
eng/tools/src/ToolDescriptionEvaluator/Services/EmbeddingService.cs	Azure OpenAI embedding service wrapper for generating text embeddings
eng/tools/src/ToolDescriptionEvaluator/Models/*.cs	Data models for tools, prompts, embeddings, and metrics
eng/tools/src/ToolDescriptionEvaluator/*.ps1	PowerShell scripts for running evaluator, updating tools.json, and generating prompts
eng/tools/src/ToolDescriptionEvaluator/README.md	Comprehensive documentation covering usage modes, setup, and configuration
eng/tools/src/ToolDescriptionEvaluator/Quickstart.md	Quick start guide for new users
eng/tools/src/ToolDescriptionEvaluator/*.json	Example data files for tools, prompts, and namespace mappings
eng/tools/Tools.sln	Solution file for the Tools project

[hallipr]

The common "testable project" folder structure is:

<Project Name>/
  src/
    <Project Name>.csproj
    ProjectFile.cs
  tests/
     <Project Name>.UnitTests/       <-  this should probably be shortened to just unitTests/ or unit/
       <Project Name>.UnitTests.csproj
        TestFile.cs
     <Project Name>.LiveTests/       <-  this should probably be shortened to just liveTests/ or live/
       <Project Name>.LiveTests.csproj
        TestFile.cs
      test-resources.bicep
      test-resources-post.ps1
  <Project Name>.sln   <-  we don't do this yet, but probably should start with tools and servers

[joshfree]

Let's get @vcolin7 and @anannya03 to review this PR as the tool owner

[vcolin7]

@conniey and I talked about this yesterday on Teams and we landed on following the structure we use for all our other projects in the repo, having one /src directory per tool/server/utility

[vcolin7]

I think removing the results files is the right move. Would you mind adding them to our .gitignore or creating a new one for this directory?