Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
773f7d9
Added a `--namespaces` switch to the `azmcp tools list` command
vcolin7 Sep 18, 2025
69103a3
Added count values for "results" and "subcommands"
vcolin7 Sep 18, 2025
d2a676f
Update core/Azure.Mcp.Core/src/Areas/Tools/Commands/ToolsListCommand.cs
vcolin7 Sep 18, 2025
ce3868c
Mapped prompts to namespaces
vcolin7 Sep 18, 2025
62bfb5c
Added a script to generate a prompts JSON file from a markdown file
vcolin7 Sep 18, 2025
fab5a04
Applied PR feedback
vcolin7 Sep 18, 2025
da06e1d
Merge remote-tracking branch 'origin/main' into tools-list-as-namespaces
vcolin7 Sep 18, 2025
8ea32cd
Updated tools.json
vcolin7 Sep 18, 2025
6c2d31a
Ran dotnet format
vcolin7 Sep 18, 2025
8425d4a
Updated prompts and tools JSON files
vcolin7 Sep 22, 2025
44ad7f4
Updated the Best Practices tool name and ran evaluations. Also remove…
vcolin7 Sep 22, 2025
85ed09d
Added the --output-file-name and --top switches. Also updated documen…
vcolin7 Sep 22, 2025
c564fec
Renamed namespaces JSON
vcolin7 Sep 22, 2025
bf2fd73
Fixed wrong entries in prompts files and re-ran utility
vcolin7 Sep 22, 2025
21e95f7
Added a way of processing consolidated tools and added a "total execu…
vcolin7 Sep 22, 2025
28998e1
Small updates to Program.cs
vcolin7 Sep 23, 2025
a4a17ab
Added consolidated tools and prompts files, as well as a script to ma…
vcolin7 Sep 23, 2025
27b9e5b
Merge remote-tracking branch 'origin/main' into tools-list-as-namespaces
vcolin7 Sep 23, 2025
15e2232
Updated tools JSON files
vcolin7 Sep 23, 2025
2211b55
Updated scripts for prompt JSON generation
vcolin7 Sep 23, 2025
109c8cb
Re-ran the utility for all modes
vcolin7 Sep 23, 2025
5ee4d30
Fixed failing unit test
vcolin7 Sep 23, 2025
77d48b7
Update consolidated tools
fanyang-mono Sep 23, 2025
3375f15
Update consolidated tools round 2
fanyang-mono Sep 24, 2025
c0d8acc
Mapped some new tools
fanyang-mono Sep 26, 2025
477fae7
Merge branch 'main' into tools-list-as-namespaces
vcolin7 Sep 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 109 additions & 1 deletion core/Azure.Mcp.Core/src/Areas/Tools/Commands/ToolsListCommand.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System.CommandLine;
using Azure.Mcp.Core.Commands;
using Azure.Mcp.Core.Models.Option;
using Microsoft.Extensions.Logging;
Expand All @@ -11,6 +12,10 @@ namespace Azure.Mcp.Core.Areas.Tools.Commands;
public sealed class ToolsListCommand(ILogger<ToolsListCommand> logger) : BaseCommand<EmptyOptions>
{
private const string CommandTitle = "List Available Tools";
private static readonly Option<bool> NamespacesOption = new("--namespaces")
{
Description = "If specified, returns a list of top-level service namespaces instead of individual commands.",
};

public override string Name => "list";

Expand All @@ -33,23 +38,124 @@ arguments. Use this to explore the CLI's functionality or to build interactive c
Secret = false
};

protected override void RegisterOptions(Command command)
{
command.Options.Add(NamespacesOption);
}

protected override EmptyOptions BindOptions(ParseResult parseResult) => new();

public override async Task<CommandResponse> ExecuteAsync(CommandContext context, ParseResult parseResult)
{
try
{
var factory = context.GetService<CommandFactory>();

// If the --namespaces flag set, return distinct top-level namespaces (area group names beneath root 'azmcp')
var namespacesOnly = parseResult.CommandResult.HasOptionResult(NamespacesOption);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though this is a core tool, I don't think we should break design patterns that are expected to be followed by Azure tools.

if (namespacesOnly)
{
// Exclude internal or special namespaces. 'extension' is flattened as top-level leaf commands.
var ignored = new HashSet<string>(StringComparer.OrdinalIgnoreCase) { "server", "tools" };
var rootGroup = factory.RootGroup; // azmcp

// Build lookup for namespace descriptions from the existing registered groups
var namespaceDescriptionMap = rootGroup.SubGroup.ToDictionary(commandGroup => commandGroup.Name, g => g.Description, StringComparer.OrdinalIgnoreCase);

// Single pass over all visible commands to bucket by namespace
var namespaceBuckets = new Dictionary<string, List<CommandInfo>>(StringComparer.OrdinalIgnoreCase);
var extensionLeafCommands = new List<CommandInfo>();

foreach (var kvp in CommandFactory.GetVisibleCommands(factory.AllCommands))
{
var key = kvp.Key; // Tokenized e.g. azmcp_storage_account_get
var firstSeparatorIndex = key.IndexOf(CommandFactory.Separator); // Expect at least root + namespace + verb

if (firstSeparatorIndex < 0)
continue; // Malformed, skip

var secondSeparatorIndex = key.IndexOf(CommandFactory.Separator, firstSeparatorIndex + 1);

if (secondSeparatorIndex < 0)
continue; // Not enough tokens
Comment on lines +71 to +80
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that so much of this logic could be completely removed if we changed how CommandFactory retained information. Given this is literally just a search for rootCommand.SubGroup.Select(sub => sub.Name), right?

Copy link
Contributor

@fanyang-mono fanyang-mono Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should get all the matching namespaces. Please simplify the logic here.
rootGroup.SubGroup.Select(group => !ignored.Contains(group.Name, StringComparer.OrdinalIgnoreCase))


var namespaceToken = key.Substring(firstSeparatorIndex + 1, secondSeparatorIndex - firstSeparatorIndex - 1);

if (ignored.Contains(namespaceToken))
{
// Skip internal groups entirely
continue;
}

var cmdInfo = CreateCommand(key, kvp.Value);

if (namespaceToken.Equals("extension", StringComparison.OrdinalIgnoreCase))
{
// Flatten: treat each extension command as top-level leaf
extensionLeafCommands.Add(cmdInfo);

continue;
}

if (!namespaceBuckets.TryGetValue(namespaceToken, out var list))
{
list = new List<CommandInfo>();
namespaceBuckets[namespaceToken] = list;
}

list.Add(cmdInfo);
}

// Build namespace CommandInfo objects
var namespaceCommands = namespaceBuckets
.Select(kvp =>
{
var namespaceName = kvp.Key;
var subcommands = kvp.Value
.OrderBy(ci => ci.Command, StringComparer.OrdinalIgnoreCase)
.ToList();
namespaceDescriptionMap.TryGetValue(namespaceName, out var desc);
return new CommandInfo
{
Name = namespaceName,
Description = desc ?? string.Empty,
Command = $"azmcp {namespaceName}",
Subcommands = subcommands,
Options = null,
SubcommandsCount = subcommands.Count
};
})
.OrderBy(ci => ci.Name, StringComparer.OrdinalIgnoreCase)
.ToList();

// Order extension leaf commands
extensionLeafCommands = extensionLeafCommands
.OrderBy(ci => ci.Command, StringComparer.OrdinalIgnoreCase)
.ToList();

// Combine and sort: namespaces first, then extension leaves by Name
namespaceCommands.AddRange(extensionLeafCommands);
namespaceCommands = namespaceCommands
.OrderByDescending(ci => ci.SubcommandsCount > 0)
.ThenBy(ci => ci.Name, StringComparer.OrdinalIgnoreCase)
.ToList();

context.Response.Results = ResponseResult.Create(namespaceCommands, ModelsJsonContext.Default.ListCommandInfo);
context.Response.ResultsCount = namespaceCommands.Count;
return context.Response;
}

var tools = await Task.Run(() => CommandFactory.GetVisibleCommands(factory.AllCommands)
.Select(kvp => CreateCommand(kvp.Key, kvp.Value))
.ToList());

context.Response.Results = ResponseResult.Create(tools, ModelsJsonContext.Default.ListCommandInfo);
context.Response.ResultsCount = tools.Count;
return context.Response;
}
catch (Exception ex)
{
logger.LogError(ex, "An exception occurred processing tool.");
logger.LogError(ex, "An exception occurred while processing tool listing.");
HandleException(context, ex);

return context.Response;
Expand All @@ -73,6 +179,8 @@ private static CommandInfo CreateCommand(string tokenizedName, IBaseCommand comm
Description = commandDetails.Description ?? string.Empty,
Command = tokenizedName.Replace(CommandFactory.Separator, ' '),
Options = optionInfos,
// Leaf commands have no subcommands.
SubcommandsCount = 0,
};
}
}
6 changes: 6 additions & 0 deletions core/Azure.Mcp.Core/src/Models/Command/CommandInfo.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,10 @@ public class CommandInfo
[JsonPropertyName("option")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public List<OptionInfo>? Options { get; set; }

// Number of immediate subcommands for grouping/namespace entries.
// Leaf commands will always have 0.
[JsonPropertyName("subcommandsCount")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
public int SubcommandsCount { get; set; }
}
6 changes: 6 additions & 0 deletions core/Azure.Mcp.Core/src/Models/Command/CommandResponse.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ public class CommandResponse

[JsonPropertyName("duration")]
public long Duration { get; set; }

// Number of top-level result items (length of the deserialized results collection if a collection is returned).
// Set explicitly by commands that know the concrete collection size.
[JsonPropertyName("resultsCount")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
public int ResultsCount { get; set; }
}

[JsonConverter(typeof(ResultConverter))]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ public async Task ExecuteAsync_WithValidContext_ReturnsCommandInfoList()
Assert.NotNull(result);
Assert.NotEmpty(result);

Assert.Equal(result.Count, response.ResultsCount);

foreach (var command in result)
{
Assert.False(string.IsNullOrWhiteSpace(command.Name), "Command name should not be empty");
Expand All @@ -82,6 +84,9 @@ public async Task ExecuteAsync_WithValidContext_ReturnsCommandInfoList()

Assert.StartsWith("azmcp ", command.Command);

// Leaf commands: subcommandsCount should always be 0
Assert.Equal(0, command.SubcommandsCount);

if (command.Options != null && command.Options.Count > 0)
{
foreach (var option in command.Options)
Expand Down Expand Up @@ -118,6 +123,7 @@ public async Task ExecuteAsync_JsonSerializationStressTest_HandlesLargeResults()
// Verify JSON round-trip preserves all data
var serializedJson = JsonSerializer.Serialize(result);
Assert.Equal(json, serializedJson);
Assert.Equal(result.Count, response.ResultsCount);
}

/// <summary>
Expand Down Expand Up @@ -145,6 +151,8 @@ public async Task ExecuteAsync_WithValidContext_FiltersHiddenCommands()

Assert.Contains(result, cmd => !string.IsNullOrEmpty(cmd.Name));

Assert.Equal(result.Count, response.ResultsCount);

}

/// <summary>
Expand Down Expand Up @@ -244,6 +252,8 @@ public async Task ExecuteAsync_ReturnsSpecificKnownCommands()
Assert.NotNull(result);
Assert.NotEmpty(result);

Assert.Equal(result.Count, response.ResultsCount);

Assert.True(result.Count >= MinimumExpectedCommands, $"Expected at least {MinimumExpectedCommands} commands, got {result.Count}");

var allCommands = result.Select(cmd => cmd.Command).ToList();
Expand Down Expand Up @@ -300,6 +310,8 @@ public async Task ExecuteAsync_CommandPathFormattingIsCorrect()

Assert.NotNull(result);

Assert.Equal(result.Count, response.ResultsCount);

foreach (var command in result)
{
// Command paths should not start or end with spaces
Expand All @@ -311,6 +323,104 @@ public async Task ExecuteAsync_CommandPathFormattingIsCorrect()
}
}

/// <summary>
/// Verifies that the --namespaces switch returns only distinct top-level namespaces.
/// </summary>
[Fact]
public async Task ExecuteAsync_WithNamespaceSwitch_ReturnsNamespacesOnly()
{
// Arrange
var args = _commandDefinition.Parse(new[] { "--namespaces" });

// Act
var response = await _command.ExecuteAsync(_context, args);

// Assert
Assert.NotNull(response);
Assert.NotNull(response.Results);

// Serialize then deserialize as list of CommandInfo
var json = JsonSerializer.Serialize(response.Results);
var namespaces = JsonSerializer.Deserialize<List<CommandInfo>>(json);

Assert.NotNull(namespaces);
Assert.NotEmpty(namespaces);

Assert.Equal(namespaces!.Count, response.ResultsCount);

// Should include some well-known namespaces (matching Name property)
Assert.Contains(namespaces, ci => ci.Name.Equals("subscription", StringComparison.OrdinalIgnoreCase));
Assert.Contains(namespaces, ci => ci.Name.Equals("storage", StringComparison.OrdinalIgnoreCase));
Assert.Contains(namespaces, ci => ci.Name.Equals("keyvault", StringComparison.OrdinalIgnoreCase));

bool foundNamespaceWithSubcommands = false;
bool foundSubcommandWithOptions = false;

foreach (var ns in namespaces!)
{
Assert.False(string.IsNullOrWhiteSpace(ns.Name));
Assert.False(string.IsNullOrWhiteSpace(ns.Command));
Assert.StartsWith("azmcp ", ns.Command, StringComparison.OrdinalIgnoreCase);
Assert.Equal(ns.Name, ns.Name.Trim());
Assert.DoesNotContain(" ", ns.Name);
// Namespace should not itself have options
Assert.Null(ns.Options);
// Count should equal number of subcommands for namespaces
Assert.Equal(ns.Subcommands?.Count ?? 0, ns.SubcommandsCount);

if (ns.Subcommands is { Count: > 0 })
{
foundNamespaceWithSubcommands = true;
// Validate a few subcommands
foreach (var sub in ns.Subcommands.Take(5))
{
Assert.False(string.IsNullOrWhiteSpace(sub.Name));
Assert.False(string.IsNullOrWhiteSpace(sub.Command));
Assert.StartsWith($"azmcp {ns.Name} ", sub.Command, StringComparison.OrdinalIgnoreCase);
// Subcommand entries are leaf commands; count must be 0
Assert.Equal(0, sub.SubcommandsCount);

if (sub.Options != null && sub.Options.Count > 0)
{
foundSubcommandWithOptions = true;
}
}
}
}

Assert.True(foundNamespaceWithSubcommands, "Expected at least one namespace to contain subcommands.");
Assert.True(foundSubcommandWithOptions, "Expected at least one subcommand to include options.");
}

/// <summary>
/// Explicitly verifies that count reflects subcommand counts for namespaces and 0 for their leaf subcommands.
/// </summary>
[Fact]
public async Task ExecuteAsync_Namespaces_CountMatchesSubcommandCounts()
{
var args = _commandDefinition.Parse(new[] { "--namespaces" });
var response = await _command.ExecuteAsync(_context, args);

Assert.NotNull(response.Results);

var namespaces = DeserializeResults(response.Results);

Assert.NotEmpty(namespaces);
Assert.Equal(namespaces.Count, response.ResultsCount);

foreach (var ns in namespaces.Take(10))
{
Assert.Equal(ns.Subcommands?.Count ?? 0, ns.SubcommandsCount);
if (ns.Subcommands != null)
{
foreach (var sub in ns.Subcommands)
{
Assert.Equal(0, sub.SubcommandsCount);
}
}
}
}

/// <summary>
/// Verifies that the command handles empty command factory gracefully
/// and returns empty results when no commands are available.
Expand Down Expand Up @@ -350,6 +460,7 @@ public async Task ExecuteAsync_WithEmptyCommandFactory_ReturnsEmptyResults()

Assert.NotNull(result);
Assert.Empty(result); // Should be empty when no commands are available
Assert.Equal(result.Count, response.ResultsCount);
}

/// <summary>
Expand Down
18 changes: 18 additions & 0 deletions docs/azmcp-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -869,6 +869,9 @@ azmcp bestpractices get --resource <resource> --action <action>
```bash
# List all available tools in the Azure MCP server
azmcp tools list

# List only the available top-level service namespaces
azmcp tools list --namespaces
```

### Azure Monitor Operations
Expand Down Expand Up @@ -1397,6 +1400,21 @@ All responses follow a consistent JSON format:
}
```

### Tool & Namespace Result Objects
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Tool & Namespace Result Objects
### Tool and Namespace Result Objects

To follow writing guidelines.


When invoking `azmcp tools list` (with or without `--namespaces`), each returned object now includes a `count` field:

| Field | Description |
|-------|-------------|
| `name` | Command or namespace name |
| `description` | Human-readable description |
| `command` | Fully qualified CLI invocation path |
| `subcommands` | (Namespaces only) Array of leaf command objects |
| `option` | (Leaf commands only) Array of options supported by the command |
| `count` | Namespaces: number of subcommands; Leaf commands: always 0 (options not counted) |

This quantitative field enables quick sizing of a namespace without traversing nested arrays. Leaf command complexity should be inferred from its option list, not the `count` field.

## Error Handling

The CLI returns structured JSON responses for errors, including:
Expand Down
2 changes: 0 additions & 2 deletions docs/e2eTestPrompts.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,8 +290,6 @@ This file contains prompts used for end-to-end testing to ensure each tool is in
| azmcp_bestpractices_get | Get the latest Azure Functions best practices |
| azmcp_bestpractices_get | Get the latest Azure Static Web Apps best practices |
| azmcp_bestpractices_get | What are azure function best practices? |
| azmcp_bestpractices_get | Create the plan for creating a simple HTTP-triggered function app in javascript that returns a random compliment from a predefined list in a JSON response. And deploy it to azure eventually. But don't create any code until I confirm. |
| azmcp_bestpractices_get | Create the plan for creating a to-do list app. And deploy it to azure as a container app. But don't create any code until I confirm. |

## Azure Monitor

Expand Down
Loading