.NET DLLs for Azure Machine Learning Studio management

Overview

Description

This repository contains a proof of concept project for Azure Machine Learning Studio management.

The solution includes .NET Standard 2.0 Class Libraries (DLLs), which can be incorporated in many .NET applications and be used for scale and automate Azure ML Studio workspaces and experiments.

🚧 At the moment, the project is in the pre-release phase - be aware of possible errors and exceptions before using it! 🚧

Solution Base Overview

The idea of creating a tool to manage Azure Machine Learning Studio was born during the meeting with our partner Soneta/Enova365.

Our partner was looking for solutions that could help automate and scale experiments in Azure ML Studio portal. After the initial analysis of the available tools, I discovered the only one that I used later as a base for the current solution (source: PowerShell Module for Azure Machine Learning Studio & Web Services) - this solution was developed for PowerShell users, and the C# code implementation (SDK) helped me as a base to create this project.

Prerequisites

Azure Machine Learning Studio

This solution was created for the use of Azure Machine Learning Studio resources - Workspace, Experiment, Database, Modules, etc. We need a proper account to be able to work by using this library.

We can create different types of workspaces, because our library should deal with any type of account.

Free Workspace

Visit the main website of Azure Machine Learning Studio and simply create a new account - https://studio.azureml.net/

You should notice this:

Azure Subscription

Alternatively, create an Azure Account with a subscription.

And create a workspace withing the portal:

Software

AzureML.Studio.dll and AzureML.Studio.Core.dll are class libraries written in .NET Standard 2.0 .

"The .NET Standard is a formal specification of .NET APIs that are intended to be available on all .NET implementations. The motivation behind the .NET Standard is establishing greater uniformity in the .NET ecosystem. ECMA 335 continues to establish uniformity for .NET implementation behavior, but there is no similar spec for the .NET Base Class Libraries (BCL) for .NET library implementations."

You can use these DLLs in particluar, however before using concider the following table of all versions of .NET Standard and the platforms supported (source: .NET Standard):

.NET Standard	1.0	1.1	1.2	1.3	1.4	1.5	1.6	2.0
.NET Core	1.0	1.0	1.0	1.0	1.0	1.0	1.0	2.0
.NET Framework	4.5	4.5	4.5.1	4.6	4.6.1	4.6.1	4.6.1	4.6.1
Mono	4.6	4.6	4.6	4.6	4.6	4.6	4.6	5.4
Xamarin.iOS	10.0	10.0	10.0	10.0	10.0	10.0	10.0	10.14
Xamarin.Mac	3.0	3.0	3.0	3.0	3.0	3.0	3.0	3.8
Xamarin.Android	7.0	7.0	7.0	7.0	7.0	7.0	7.0	8.0
Universal Windows Platform	10.0	10.0	10.0	10.0	10.0	10.0.16299	10.0.16299	10.0.16299

Tools & Libraries

You may need such tools as code-editors, IDEs or notepads, and libraries through which you can create a .NET application. In that case, I used Visual Studio with proper frameworks to run a Console Application as an example.

Visual Studio IDE

Usage

Workspace ID
- This value can be found in Workspace Settings in ML Studio.
Workspace Authorization Token
- This value can be found in Workspace Settings in ML Studio. Note you must be an Owner of the Workspace in order to have access to this token.
Location
- This value can be found in the Workspace drop-down. It is the Azure region the Workspace is provisioned in. Currently supported values for this configuration are:
  - South Central US (use this value for all Free Workspaces)
  - Southeast Asia
  - Japan East
  - West Europe
  - West Central US
  - ...

Configuration

Add reference or install as a NuGet package to your project.

Create an instance of StudioClient class and use it.

using AzureML.Studio.Core.Models;

namespace AzureML.Studio.ConsoleApplicationExample
{
    class Program
    {
        static void Main(string[] args)
        {
            var studioClient = new StudioClient();
            
            var workspaceSettings = WorkspaceSettings();
            workspaceSettings.WorkspaceId = "XYZ";
            workspaceSettings.AuthorizationToken = "######";
            workspaceSettings.Location = "";

            var workspace = studioClient.GetWorkspace(workspaceSettings);
        ...
    ...
}

Operations

Get Workspace metadata (GetWorkspace)
Get selected Workspaces metadata (GetWorkspaces)
Get Users of the Workspace (GetWorkspaceUsers)
Get Users from selected Workspaces (GetWorkspacesUsers)
Add a new User to Workspace (AddUserToWorkspace)
Add a new User to selected Workspaces (AddUserToWorkspaces)
Add new users to Workspace (AddUsersToWorkspace)
Add new users to selected Workspaces (AddUsersToWorkspaces)
Get Datasets from a Workspace (GetDatasetsFromWorkspace)
Get Datasets for selected Workspaces (GetDatasetsFromWorkspaces)
Delete a Dataset from a Workspace (DeleteDatasetFromWorkspace)
Delete Datasets from a Workspace (DeleteDatasetsFromWorkspace)
Delete all Datasets from a Workspace (DeleteAllDatasetsFromWorkspace)
Delete all Datasets from selected Workspaces (DeleteAllDatasetsFromWorkspaces)
Download a Dataset from a Workspace (DownloadDatasetFromWorkspace)
Download selected Datasets from a Workspace (DownloadDatasetsFromWorkspace)
Download all Datasets from a Workspace (DownloadAllDatasetsFromWorkspace)
Download all Datasets from selected Workspaces (DownloadAllDatasetsFromWorkspaces)
Upload resource file to a Workspace (UploadResourceToWorkspace)
Upload resource files to a Workspace (UploadResourcesToWorkspace)
Upload resource file to Workspaces (UploadResourceToWorkspaces)
Upload resource files to Workspaces (UploadResourcesToWorkspaces)
Get an Experiment by id (GetExperiment)
Get Experiments by ids (GetExperiments)
Get all Experiments from a Workspace (GetAllExperiments)
Run an Experiment (RunExperiment)
Run Experiments (RunExperiments)
Run all Experiments (RunAllExperiments)
Save an Experiment (SaveExperiment)
Save Experiments (SaveExperiments)
Save all Experiments (SaveAllExperiments)
Save an Experiment as (SaveExperimentAs)
Delete an Experiment (DeleteExperiment)
Delete Experiments (DeleteExperiments)
Delete all Experiments (DeleteAllExperiments)
Export an Experiment as JSON (ExportExperiment)
Export specific Experiments as JSON (ExportExperiments)
Export all Experiments as JSON (ExportAllExperiments)
Import experiment as JSON (ImportExperiment)
Copy an Experiment from Workspace to another Workspace (CopyExperiment)
Copy specific Experiments from Workspace to another Workspace (CopyExperiments)
Copy all Experiments from Workspace to another Workspace (CopyAllExperiments)
Get a Trained Model (GetTrainedModel)
Get Trained Models (GetTrainedModels)
Get a Transform (GetTransform)
Get Transforms (GetTransforms)
Modify node parameter by changing its value (ModifyNodeParameter)
Modify edge connection between two modules (ModifyNodeEdge)
Add a new module to the Experiment (AddModule)

GetWorkspace

var workspace = studioClient.GetWorkspace(workspaceSettings);

var workspace = studioClient.GetWorkspace("XYZ", "######", "");

GetWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;

var workspaces = studioClient.GetWorkspaces(workspacesSettings);

GetWorkspaceUsers

var users = studioClient.GetWorkspaceUsers(workspaceSettings);

var users = studioClient.GetWorkspaceUsers("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var users = studioClient.GetWorkspaceUsers(workspace);

GetWorkspacesUsers

IEnumerable<WorkspaceSettings> workspacesSettings;
var workspaceUsersdictionary = studioClient.GetWorkspacesUsers(workspacesSettings);

IEnumerable<Workspace> workspaces;
var workspaceUsersdictionary = studioClient.GetWorkspacesUsers(workspaces);

AddUserToWorkspace

var workspaceUser = new WorkspaceUser(
            new WorkspaceUserInternal() {
                User = new UserDetailInternal() {
                    Email = "email", Role = "role"}}));

studioClient.AddUserToWorkspace(workspaceSettings, workspaceUser);

studioClient.AddUserToWorkspace(workspaceSettings, "email", "role");

studioClient.AddUserToWorkspace("XYZ", "######", "", workspaceUser);

studioClient.AddUserToWorkspace("XYZ", "######", "", "email", "role");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.AddUserToWorkspace(workspace, workspaceUser);

studioClient.AddUserToWorkspace(workspace, "email", "role");

AddUserToWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;
var workspaceUser = new WorkspaceUser(
            new WorkspaceUserInternal() {
                User = new UserDetailInternal() {
                    Email = "email", Role = "role"}}));

studioClient.AddUserToWorkspaces(workspacesSettings, workspaceUser);

studioClient.AddUserToWorkspaces(workspacesSettings, "email", "role");

IEnumerable<Workspace> workspaces;

studioClient.AddUserToWorkspaces(workspaces, workspaceUser);

studioClient.AddUserToWorkspaces(workspaces, "email", "role");

AddUsersToWorkspace

IEnumerable<WorkspaceUser> workspaceUsers;

studioClient.AddUsersToWorkspace(workspaceSettings, workspaceUsers);

studioClient.AddUsersToWorkspace("XYZ", "######", "", workspaceUsers);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.AddUsersToWorkspace(workspace, workspaceUsers);

AddUsersToWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;
IEnumerable<WorkspaceUser> workspaceUsers;

studioClient.AddUsersToWorkspaces(workspacesSettings, workspaceUsers);

IEnumerable<Workspace> workspaces;

studioClient.AddUsersToWorkspaces(workspaces, workspaceUsers);

GetDatasetsFromWorkspace

var datasets = studioClient.GetDatasetsFromWorkspace(workspaceSettings);

var datasets = studioClient.GetDatasetsFromWorkspace("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var datasets = studioClient.GetDatasetsFromWorkspace(workspace);

GetDatasetsFromWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;

var workspaceDatasetsDictionary = studioClient.GetDatasetsFromWorkspaces(workspacesSettings);

IEnumerable<Workspace> workspaces;

var workspaceDatasetsDictionary = studioClient.GetDatasetsFromWorkspaces(workspaces);

DeleteDatasetFromWorkspace

studioClient.DeleteDatasetFromWorkspace(workspaceSettings, "datasetFamilyId");

var dataset = new Dataset();
dataset.FamilyId = "datasetFamilyId";

studioClient.DeleteDatasetFromWorkspace(workspaceSettings, dataset);

studioClient.DeleteDatasetFromWorkspace("XYZ", "######", "", dataset);

studioClient.DeleteDatasetFromWorkspace("XYZ", "######", "", "datasetFamilyId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteDatasetFromWorkspace(workspace, "datasetFamilyId")

studioClient.DeleteDatasetFromWorkspace(workspace, dataset);

DeleteDatasetsFromWorkspace

IEnumerable<Dataset> datasets;

studioClient.DeleteDatasetsFromWorkspace(workspaceSettings, datasets);

studioClient.DeleteDatasetsFromWorkspace("XYZ", "######", "", datasets);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteDatasetsFromWorkspace(workspace, datasets);

DeleteAllDatasetsFromWorkspace

studioClient.DeleteAllDatasetsFromWorkspace(workspaceSettings);

studioClient.DeleteAllDatasetsFromWorkspace("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteAllDatasetsFromWorkspace(workspace);

DeleteAllDatasetsFromWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.DeleteAllDatasetsFromWorkspaces(workspacesSettings);

IEnumerable<Workspace> workspaces;

studioClient.DeleteAllDatasetsFromWorkspaces(workspaces);

DownloadDatasetFromWorkspace

studioClient.DownloadDatasetFromWorkspace(workspaceSettings, "datasetId", "fileName");

studioClient.DownloadDatasetFromWorkspace("XYZ", "######", "", "datasetId", "fileName");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DownloadDatasetFromWorkspace(workspace, "datasetId", "fileName");

var dataset = new Dataset();
dataset.Id = "datasetId";

studioClient.DownloadDatasetFromWorkspace(workspaceSettings, dataset, "fileName");

studioClient.DownloadDatasetFromWorkspace("XYZ", "######", "", dataset, "fileName");

studioClient.DownloadDatasetFromWorkspace(workspace, dataset, "fileName");

DownloadDatasetsFromWorkspace

IEnumerable<Dataset> datasets;

studioClient.DownloadDatasetsFromWorkspace(workspaceSettings, datasets);

studioClient.DownloadDatasetsFromWorkspace("XYZ", "######", "", datasets);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DownloadDatasetsFromWorkspace(workspace, datasets);

DownloadAllDatasetsFromWorkspace

studioClient.DownloadAllDatasetsFromWorkspace(workspaceSettings);

studioClient.DownloadAllDatasetsFromWorkspace("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DownloadAllDatasetsFromWorkspace(workspace);

DownloadAllDatasetsFromWorkspaces

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.DownloadAllDatasetsFromWorkspaces(workspacesSettings);

IEnumerable<Workspace> workspaces;

studioClient.DownloadAllDatasetsFromWorkspaces(workspaces);

UploadResourceToWorkspace

Enum: ResourceFileFormat.cs

ResourceFileFormat resourceFileFormat;

async studioClient.UploadResourceToWorkspace(workspaceSettings, resourceFileFormat, "filePath");

async studioClient.UploadResourceToWorkspace("XYZ", "######", "", resourceFileFormat, "filePath");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

async studioClient.UploadResourceToWorkspace(workspace, resourceFileFormat, "filePath");

UploadResourcesToWorkspace

Enum: ResourceFileFormat.cs

IDictionary<string, ResourceFileFormat> filePathResourceFileFormatDict;

studioClient.UploadResourcesToWorkspace(workspaceSettings, filePathResourceFileFormatDict);

studioClient.UploadResourcesToWorkspace("XYZ", "######", "", filePathResourceFileFormatDict);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.UploadResourcesToWorkspace(workspace, filePathResourceFileFormatDict);

UploadResourceToWorkspaces

Enum: ResourceFileFormat.cs

IEnumerable<WorkspaceSettings> workspacesSettings;
ResourceFileFormat resourceFileFormat;

studioClient.UploadResourceToWorkspaces(workspacesSettings, resourceFileFormat, "filePath");

IEnumerable<Workspace> workspaces;

studioClient.UploadResourceToWorkspaces(workspaces, resourceFileFormat, "filePath");

UploadResourcesToWorkspaces

Enum: ResourceFileFormat.cs

IEnumerable<WorkspaceSettings> workspacesSettings;
IDictionary<string, ResourceFileFormat> filePathResourceFileFormatDict;

studioClient.UploadResourcesToWorkspaces(workspacesSettings, filePathResourceFileFormatDict);

IEnumerable<Workspace> workspaces;

studioClient.UploadResourcesToWorkspaces(workspaces, filePathResourceFileFormatDict);

GetExperiment

var experiment = studioClient.GetExperiment(workspaceSettings, "experimentId");

var experiment = studioClient.GetExperiment("XYZ", "######", "", "experimentId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var experiment = studioClient.GetExperiment(workspace, "experimentId");

GetExperiments

IEnumerable<string> experimentsIds;

var experiments = studioClient.GetExperiments(workspaceSettings, experimentsIds);

var experiments = studioClient.GetExperiments("XYZ", "######", "", experimentsIds);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var experiments = studioClient.GetExperiments(workspace, experimentsIds);

GetAllExperiments

IEnumerable<WorkspaceSettings> workspacesSettings;

var workspaceExperimentsDict = studioClient.GetAllExperiments(workspacesSettings);

IEnumerable<Workspace> workspaces;

var workspaceExperimentsDict = studioClient.GetAllExperiments(workspaces);

var experiments = studioClient.GetAllExperiments(workspaceSettings);

var experiments = studioClient.GetAllExperiments("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var experiments = studioClient.GetAllExperiments(workspace);

RunExperiment

studioClient.RunExperiment(workspaceSettings, "experimentId");

studioClient.RunExperiment("XYZ", "######", "", "experimentId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.RunExperiment(workspace, "experimentId");

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.RunExperiment(workspaceSettings, experiment);

studioClient.RunExperiment("XYZ", "######", "", experiment);

studioClient.RunExperiment(workspace, experiment);

RunExperiments

IEnumerable<string> experimentsIds;

studioClient.RunExperiments(workspaceSettings, experimentsIds);

studioClient.RunExperiments("XYZ", "######", "", experimentsIds);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.RunExperiments(workspace, experimentsIds);

IEnumerable<Experiment> experiments;

studioClient.RunExperiments(workspaceSettings, experiments);

studioClient.RunExperiments("XYZ", "######", "", experiments);

studioClient.RunExperiments(workspace, experiments);

RunAllExperiments

studioClient.RunAllExperiments(workspaceSettings);

studioClient.RunAllExperiments("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.RunAllExperiments(workspace);

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.RunAllExperiments(workspacesSettings);

IEnumerable<Workspace> workspaces;

studioClient.RunAllExperiments(workspaces);

SaveExperiment

studioClient.SaveExperiment(workspaceSettings, "experimentId");

studioClient.SaveExperiment("XYZ", "######", "", "experimentId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.SaveExperiment(workspace, "experimentId");

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.SaveExperiment(workspaceSettings, experiment);

studioClient.SaveExperiment("XYZ", "######", "", experiment);

studioClient.SaveExperiment(workspace, experiment);

SaveExperiments

IEnumerable<string> experimentsIds;

studioClient.SaveExperiments(workspaceSettings, experimentsIds);

studioClient.SaveExperiments("XYZ", "######", "", experimentsIds);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.SaveExperiments(workspace, experimentsIds);

IEnumerable<Experiment> experiments;

studioClient.SaveExperiments(workspaceSettings, experiments);

studioClient.SaveExperiments("XYZ", "######", "", experiments);

studioClient.SaveExperiments(workspace, experiments);

SaveAllExperiments

studioClient.SaveAllExperiments(workspaceSettings);

studioClient.SaveAllExperiments("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.SaveAllExperiments(workspace);

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.SaveAllExperiments(workspacesSettings);

IEnumerable<Workspace> workspaces;

studioClient.SaveAllExperiments(workspaces);

SaveExperimentAs

studioClient.SaveExperimentAs(workspaceSettings, "experimentId", "newName");

studioClient.SaveExperimentAs("XYZ", "######", "", "experimentId", "newName");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.SaveExperimentAs(workspace, "experimentId", "newName");

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.SaveExperimentAs(workspaceSettings, experiment, "newName");

studioClient.SaveExperimentAs("XYZ", "######", "", experiment, "newName");

studioClient.SaveExperimentAs(workspace, experiment, "newName");

DeleteExperiment

studioClient.DeleteExperiment(workspaceSettings, "experimentId");

studioClient.DeleteExperiment("XYZ", "######", "", "experimentId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteExperiment(workspace, "experimentId");

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.DeleteExperiment(workspaceSettings, experiment);

studioClient.DeleteExperiment("XYZ", "######", "", experiment);

studioClient.DeleteExperiment(workspace, experiment);

DeleteExperiments

IEnumerable<string> experimentsIds;

studioClient.DeleteExperiments(workspaceSettings, experimentsIds);

studioClient.DeleteExperiments("XYZ", "######", "", experimentsIds);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteExperiments(workspace, experimentsIds);

IEnumerable<Experiment> experiments;

studioClient.DeleteExperiments(workspaceSettings, experiments);

studioClient.DeleteExperiments("XYZ", "######", "", experiments);

studioClient.DeleteExperiments(workspace, experiments);

DeleteAllExperiments

studioClient.DeleteAllExperiments(workspaceSettings);

studioClient.DeleteAllExperiments("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.DeleteAllExperiments(workspace);

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.DeleteAllExperiments(workspacesSettings);

IEnumerable<Workspace> workspaces;

studioClient.DeleteAllExperiments(workspaces);

ExportExperiment

studioClient.ExportExperiment(workspaceSettings, "experimentId");

studioClient.ExportExperiment("XYZ", "######", "", "experimentId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ExportExperiment(workspace, "experimentId");

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.ExportExperiment(workspaceSettings, experiment);

studioClient.ExportExperiment("XYZ", "######", "", experiment);

studioClient.ExportExperiment(workspace, experiment);

studioClient.ExportExperiment(workspaceSettings, "experimentId", "outputFilePath");

studioClient.ExportExperiment("XYZ", "######", "", "experimentId", "outputFilePath");

studioClient.ExportExperiment(workspace, "experimentId", "outputFilePath");

studioClient.ExportExperiment(workspaceSettings, experiment, "outputFilePath");

studioClient.ExportExperiment("XYZ", "######", "", experiment, "outputFilePath");

studioClient.ExportExperiment(workspace, experiment, "outputFilePath");

ExportExperiments

IEnumerable<string> experimentsIds;

studioClient.ExportExperiments(workspaceSettings, experimentsIds);

studioClient.ExportExperiments("XYZ", "######", "", experimentsIds);

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ExportExperiments(workspace, experimentsIds);

IEnumerable<Experiment> experiments;

studioClient.ExportExperiments(workspaceSettings, experiments);

studioClient.ExportExperiments("XYZ", "######", "", experiments);

studioClient.ExportExperiments(workspace, experiments);

ExportAllExperiments

studioClient.ExportAllExperiments(workspaceSettings);

studioClient.ExportAllExperiments("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ExportAllExperiments(workspace);

ImportExperiment

studioClient.ImportExperiment(workspaceSettings, "inputFilePath");

studioClient.ImportExperiment("XYZ", "######", "", "inputFilePath");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ImportExperiment(workspace, "inputFilePath");

IEnumerable<WorkspaceSettings> workspacesSettings;

studioClient.ImportExperiment(workspacesSettings, "inputFilePath");

IEnumerable<Workspace> workspaces;

studioClient.ImportExperiment(workspaces, "inputFilePath");

studioClient.ImportExperiment(workspaceSettings, "inputFilePath", "newName");

studioClient.ImportExperiment("XYZ", "######", "", "inputFilePath", "newName");

studioClient.ImportExperiment(workspace, "inputFilePath", "newName");

studioClient.ImportExperiment(workspacesSettings, "inputFilePath", "newName");

studioClient.ImportExperiment(workspaces, "inputFilePath", "newName");

CopyExperiment

studioClient.CopyExperiment(sourceWorkspaceSettings, "experimentId", destinationWorkspaceSettings);

var experiment = new Experiment;
experiment.ExperimentId = "experimentId";

studioClient.CopyExperiment(sourceWorkspaceSettings, experiment, destinationWorkspaceSettings);

CopyExperiments

IEnumerable<string> experimentsIds;

studioClient.CopyExperiments(sourceWorkspaceSettings, experimentsIds, destinationWorkspaceSettings);

IEnumerable<Experiment> experiments;

studioClient.CopyExperiments(sourceWorkspaceSettings, experiments, destinationWorkspaceSettings);

CopyAllExperiments

studioClient.CopyAllExperiments(sourceWorkspaceSettings, destinationWorkspaceSettings);

GetTrainedModel

var userAsset = studioClient.GetTrainedModel(workspaceSettings, "userAssetId");

var userAsset = studioClient.GetTrainedModel("XYZ", "######", "", "userAssetId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var userAsset = studioClient.GetTrainedModel(workspace, "userAssetId");

GetTrainedModels

var userAssets = studioClient.GetTrainedModels(workspaceSettings);

var userAssets = studioClient.GetTrainedModels("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var userAssets = studioClient.GetTrainedModels(workspace);

IEnumerable<WorkspaceSettings> workspacesSettings;

var workspaceUserAssetsDict = studioClient.GetTrainedModels(workspacesSettings);

IEnumerable<Workspace> workspaces;

var workspaceUserAssetsDict = studioClient.GetTrainedModels(workspaces);

GetTransform

var userAsset = studioClient.GetTransform(workspaceSettings, "userAssetId");

var userAsset = studioClient.GetTransform("XYZ", "######", "", "userAssetId");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var userAsset = studioClient.GetTransform(workspace, "userAssetId");

GetTransforms

var userAssets = studioClient.GetTransforms(workspaceSettings);

var userAssets = studioClient.GetTransforms("XYZ", "######", "");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

var userAssets = studioClient.GetTransforms(workspace);

IEnumerable<WorkspaceSettings> workspacesSettings;

var workspaceUserAssetsDict = studioClient.GetTransforms(workspacesSettings);

IEnumerable<Workspace> workspaces;

var workspaceUserAssetsDict = studioClient.GetTransforms(workspaces);

ModifyNodeParameter

studioClient.ModifyNodeParameter(workspaceSettings, "experimentId", "nodeNameComment", "nodeParameterName", "value", "saveAs");

studioClient.ModifyNodeParameter("XYZ", "######", "", "experimentId", "nodeNameComment", "nodeParameterName", "value", "saveAs");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ModifyNodeParameter(workspace, "experimentId", "nodeNameComment", "nodeParameterName", "value", "saveAs");

ModifyNodeEdge

studioClient.ModifyNodeEdge(workspaceSettings, "experimentId", "sourceNodeComment", "destinationNodeComment", "saveAs");

studioClient.ModifyNodeEdge("XYZ", "######", "", "experimentId", "sourceNodeComment", "destinationNodeComment", "saveAs");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.ModifyNodeEdge(workspace, "experimentId", "sourceNodeComment", "destinationNodeComment", "saveAs");

AddModule

studioClient.AddModule(workspaceSettings, "experimentId", "nameOfNewModule", "saveAs");

studioClient.AddModule("XYZ", "######", "", "experimentId", "nameOfNewModule", "saveAs");

var workspace = new Workspace();
workspace.WorkspaceId = "XYZ";
workspace.AuthorizationToken.PrimaryToken = "######";
workspace.Region = "";

studioClient.AddModule(workspace, "experimentId", "nameOfNewModule", "saveAs");

Many lines of code are missing, however at this state you will be able to do many basic operations.

Testing

There is a Unit Test project included but it is empty and it will be finished in next iterration of development. So far I have tested few important operations in Console Application.

Copying an Experiment from one Workspace to another, both are having same pricing tier and same region:

Copy Experiment: 'Import Data - Experiment'
Source Workspace: 'FakeWestEuropeCommandCenterS1'
Destination Workspace: 'FakeWestEuropeCustomerS1'
Before copying experiment should be saved and has finished running status.

static void CopyExperimentFromWorkspaceToWorkspaceSamePricingSameRegion(StudioClient studioClient)
{
    var sourceWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    var destinationWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(sourceWorkspace, experimentId);

    studioClient.CopyExperiment(sourceWorkspace, experiment, destinationWorkspace);
}

Copying an Experiment from one Workspace to another, different pricing tier but same region:

Copy Experiment: 'Import Data - Experiment'
Source Workspace: 'FakeWestEuropeCommandCenterS1'
Destination Workspace: 'FakeWestEuropeCustomerDEVTEST'
Before copying experiment should be saved and has finished running status.

static void CopyExperimentFromWorkspaceToWorkspaceDifferentPricingSameRegion(StudioClient studioClient)
{
    var sourceWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    var destinationWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(sourceWorkspace, experimentId);

    studioClient.CopyExperiment(sourceWorkspace, experiment, destinationWorkspace);
}

Copying an Experiment from one Workspace to another, both are having same pricing tier but different region:

Copy Experiment: 'Import Data - Experiment'
Source Workspace: 'FakeWestEuropeCommandCenterS1'
Destination Workspace: 'FakeSouthCentralUSCustomerS1'
Before copying experiment should be saved and has finished running status.

static void CopyExperimentFromWorkspaceToWorkspaceSamePricingDifferentRegion(StudioClient studioClient)
{
    var sourceWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    var destinationWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(sourceWorkspace, experimentId);

    studioClient.ExportExperiment(sourceWorkspace, experiment);

    var inputFilePath = @"C:\...\experimentFileName";
    studioClient.ImportExperiment(destinationWorkspace, inputFilePath, "Copied from other region");
}

Copying an Experiment from one Workspace to another, different pricing tier and different region:

Copy Experiment: 'Import Data - Experiment'
Source Workspace: 'FakeWestEuropeCommandCenterS1'
Destination Workspace: 'FakeSouthCentralUSDEVTEST'
Before copying experiment should be saved and has finished running status.

static void CopyExperimentFromWorkspaceToWorkspaceDifferentPricingDifferentRegion(StudioClient studioClient)
{
    var sourceWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    var destinationWorkspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };
    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(sourceWorkspace, experimentId);

    studioClient.ExportExperiment(sourceWorkspace, experiment);

    var inputFilePath = @"C:\...\experimentFileName";
    studioClient.ImportExperiment(destinationWorkspace, inputFilePath, "Copied from other region");
}

That's what it looked like from the portal's view:

Modify an experiment's node parameter value and overwriting the experiment:

Modify Experiment: 'Import Data - Experiment'
Modified Node Name: 'Import Data' //Found by comment "Import Data Comment"
Modified Parameter Name: 'Database Query'

static void ModifyExperimentNodeAndOverwrite(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    studioClient.ModifyNodeParameter(workspace, experimentId, "Import Data Comment", "Database Query", "SELECT Name, ProductNumber, CAST(Weight AS float) Weight\r\nFROM SalesLT.Product");
}

Modify an experiment's node parameter value and saving as a new experiment:

Modify Experiment: 'Import Data - Experiment'
Modified Node Name: 'Import Data' //Found by comment "Import Data Comment"
Modified Parameter Name: 'Database Query'

static void ModifyExperimentNodeAndSaveAsAnotherExperiment(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = "West Europe"
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    studioClient.ModifyNodeParameter(workspace, experimentId, "Import Data Comment", "Database Query", "SELECT Name, Color, CAST(Weight AS float) Weight\r\nFROM SalesLT.Product", "Import Data - Experiment 2");
}

From the portal's view:

Modify the connection within the modules (nodes) and overwriting the experiment:

Modify Experiment: *'Connect Modules - Experiment'

static void ModifyConnectionWithinTheModulesAndOverwrite(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = ""
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    studioClient.ModifyNodeEdge(workspace, experimentId, "CSV", "Dataset");
}

Modify the connection within the modules (nodes) and saving as a new experiment:

Modify Experiment: *'Connect Modules - Experiment'

static void ModifyConnectionWithinTheModulesAndSaveAsAnotherExperiment(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = ""
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    studioClient.ModifyNodeEdge(workspace, experimentId, "CSV", "Dataset", "Connect Modules - Experiment 2");
}

Aaand from the portal's view:

Adding a new module to the experiment and overwriting it:

Modify Experiment: 'Add Module - Experiment'

static void AddModuleToTheExperimentAndOverwrite(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = ""
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    var nameOfNewModule = "";
    //nameOfNewModule hard-coded
    //TODO: make a dictionary of <Module Names, Module IDs>
    //EXAMPLES:
    //Convert to TSV: 506153734175476c4f62416c57734963.1cdbcda42ece49088b87e6b636258d3d.v1-default-1644
    //Convert to Dataset: 506153734175476c4f62416c57734963.72bf58e0fc874bb19704f1805003b975.v1-default-1642

    studioClient.AddModule(workspace, experimentId, nameOfNewModule);
}

Adding a new module to the experiment and saving as a new one:

Modify Experiment: 'Add Module - Experiment'

static void AddModuleToTheExperimentAndSaveAsAnotherExperiment(StudioClient studioClient)
{
    var workspace = new WorkspaceSettings()
    {
        WorkspaceId = "",
        AuthorizationToken = "",
        Location = ""
    };

    //var experiments = studioClient.GetExperiments(sourceWorkspace);
    var experimentId = "";
    var experiment = studioClient.GetExperiment(workspace, experimentId);

    var nameOfNewModule = "";
    //nameOfNewModule hard-coded
    //TODO: make a dictionary of <Module Names, Module IDs>
    //EXAMPLES:
    //Convert to TSV: 506153734175476c4f62416c57734963.1cdbcda42ece49088b87e6b636258d3d.v1-default-1644
    //Convert to Dataset: 506153734175476c4f62416c57734963.72bf58e0fc874bb19704f1805003b975.v1-default-1642

    studioClient.AddModule(workspace, experimentId, nameOfNewModule, "Connect Modules - Experiment 2");
}

Source code of the website and portal's view:

Learnings

There are many thoughts that I have had during coding this solution. Most of all it was hard to be consistent regarding naming methods, classes, or even describing summaries of each instance. I tried to be sure that the code that I have been creating would not be a spaghetti one. The solution is still in development, and continuous support is needed! For sure refactoring is a must in many aspects and few principles should be implemented. Moreover, few methods are still a mystery for me, and I left them behind in the SDK (ManagementService.cs) - you need to find them!

What is more, I encountered several errors of the portal itself. These bugs are hidden somewhere in those copy/saving operations.

As I wrote at the begging - it is a pre-release version that contains many errors and not-handle exceptions (implementation of these is on my TODO list). Be aware!!!

Feel free to contribute, fork, modify, cooperate. I hope this library will be helpful.

Credits

Borys Rybak - Software Development Engineer, Microsoft:
- AzureMLSDK (source: azuremlps by Hai Ning) small refactor into AzureML.Studio.Core
- Entire solution of AzureMachineLearningStudio
  - AzureML.Studio
  - AzureML.Studio.ConsoleApplicationExample
  - AzureML.Studio.Core
  - [In Progress] AzureML.Studio.Test

Name		Name	Last commit message	Last commit date
Latest commit History 76 Commits
AzureML.Studio.ConsoleApplicationExample		AzureML.Studio.ConsoleApplicationExample
AzureML.Studio.Core		AzureML.Studio.Core
AzureML.Studio.Tests		AzureML.Studio.Tests
AzureML.Studio		AzureML.Studio
assets		assets
.gitattributes		.gitattributes
.gitignore		.gitignore
AzureMachineLearningStudio.sln		AzureMachineLearningStudio.sln
LICENSE		LICENSE
README.md		README.md

License

borysrybak/azure-machine-learning-studio

Folders and files

Latest commit

History

Repository files navigation

.NET DLLs for Azure Machine Learning Studio management

Overview

Description

Solution Base Overview

Table of Contents

Prerequisites

Azure Machine Learning Studio

Free Workspace

Azure Subscription

Software

Tools & Libraries

Usage

NuGet

Release

Requirements

Configuration

Operations

GetWorkspace

GetWorkspaces

GetWorkspaceUsers

GetWorkspacesUsers

AddUserToWorkspace

AddUserToWorkspaces

AddUsersToWorkspace

AddUsersToWorkspaces

GetDatasetsFromWorkspace

GetDatasetsFromWorkspaces

DeleteDatasetFromWorkspace

DeleteDatasetsFromWorkspace

DeleteAllDatasetsFromWorkspace

DeleteAllDatasetsFromWorkspaces

DownloadDatasetFromWorkspace

DownloadDatasetsFromWorkspace

DownloadAllDatasetsFromWorkspace

DownloadAllDatasetsFromWorkspaces

UploadResourceToWorkspace

UploadResourcesToWorkspace

UploadResourceToWorkspaces

UploadResourcesToWorkspaces

GetExperiment

GetExperiments

GetAllExperiments

RunExperiment

RunExperiments

RunAllExperiments

SaveExperiment

SaveExperiments

SaveAllExperiments

SaveExperimentAs

DeleteExperiment

DeleteExperiments

DeleteAllExperiments

ExportExperiment

ExportExperiments

ExportAllExperiments

ImportExperiment

CopyExperiment

CopyExperiments

CopyAllExperiments

GetTrainedModel

GetTrainedModels

GetTransform

GetTransforms

ModifyNodeParameter

ModifyNodeEdge

AddModule

Testing

Copying an Experiment from one Workspace to another, both are having same pricing tier and same region:

Copying an Experiment from one Workspace to another, different pricing tier but same region:

Copying an Experiment from one Workspace to another, both are having same pricing tier but different region:

Copying an Experiment from one Workspace to another, different pricing tier and different region:

Learnings

Credits

Helpful Materials

Packages