automl.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Automated ML\n",
    "\n",
    "TODO: Import Dependencies. In the cell below, import all the dependencies that you will need to complete the project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "gather": {
     "logged": 1598423888013
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "from azureml.core import Workspace, Experiment\n",
    "from azureml.core import Model \n",
    "from azureml.train.automl import AutoMLConfig\n",
    "from azureml.widgets import RunDetails\n",
    "from azureml.core.compute import ComputeTarget, AmlCompute\n",
    "from pprint import pprint \n",
    "import joblib\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "### Overview\n",
    "TODO: In this markdown cell, give an overview of the dataset you are using. Also mention the task you will be performing.\n",
    "\n",
    "\n",
    "TODO: Get data. In the cell below, write code to access the data you will be using in this project. Remember that the dataset needs to be external."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#getting data: the dataset was uploaded to and is accesible through my github repo\n",
    "from azureml.data.dataset_factory import TabularDatasetFactory\n",
    "\n",
    "rawdata_homic2018 = \"https://raw.githubusercontent.com/hftamayo/azuremlproj03/main/cad2018.csv\"\n",
    "dshomic2018 = TabularDatasetFactory.from_delimited_files(path=rawdata_homic2018, separator=',')\n",
    "pdhomic2018 = dshomic2018.to_pandas_dataframe().dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "gather": {
     "logged": 1598423890461
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "#setting up workspace and experiment\n",
    "ws = Workspace.from_config()\n",
    "\n",
    "# choose a name for experiment\n",
    "experiment_name = 'expp03ht'\n",
    "\n",
    "experiment=Experiment(ws, experiment_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating\n",
      "Succeeded\n",
      "AmlCompute wait for completion finished\n",
      "\n",
      "Minimum number of nodes requested have been provisioned\n"
     ]
    }
   ],
   "source": [
    "# setting up IaaS where the experiment will run on it\n",
    "ccname = \"ccp03aml\"\n",
    "try:\n",
    "    ccluster = ComputeTarget(ws, ccname)\n",
    "    print(f\"{ccname} exists or is in use!, choose a different name\")\n",
    "except:\n",
    "    ccconfig = AmlCompute.provisioning_configuration(vm_size=\"Standard_D2_V2\", max_nodes=5)\n",
    "    ccluster = ComputeTarget.create(ws, ccname, ccconfig)\n",
    "ccluster.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AutoML Configuration\n",
    "\n",
    "TODO: Explain why you chose the automl settings and cofiguration you used below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "gather": {
     "logged": 1598429217746
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# TODO: Put your automl settings here\n",
    "automl_settings = {\n",
    "    \"experiment_timeout_minutes\": 20,\n",
    "    \"max_concurrent_iterations\": 5,\n",
    "    \"primary_metric\" : 'accuracy'\n",
    "}\n",
    "\n",
    "# TODO: Put your automl config here\n",
    "automl_config = AutoMLConfig(\n",
    "        task='classification',\n",
    "        compute_target=ccluster,\n",
    "        training_data=dshomic2018,\n",
    "        label_column_name='sexo',\n",
    "        n_cross_validations=5,\n",
    "        **automl_settings\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "gather": {
     "logged": 1598431107951
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on remote.\n"
     ]
    }
   ],
   "source": [
    "# TODO: Submit your experiment\n",
    "exprun = experiment.submit(automl_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run Details\n",
    "\n",
    "OPTIONAL: Write about the different models trained and their performance. Why do you think some models did better than others?\n",
    "\n",
    "TODO: In the cell below, use the `RunDetails` widget to show the different experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "gather": {
     "logged": 1598431121770
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "08866ff1e16e4857826fbfa1bec1833a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "_AutoMLWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', 's…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/experiments/expp03ht/runs/AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5?wsid=/subscriptions/1b944a9b-fdae-4f97-aeb1-b7eea0beac53/resourcegroups/aml-quickstarts-136496/workspaces/quick-starts-ws-136496\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"run_properties\": {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"created_utc\": \"2021-01-29T12:15:44.54137Z\", \"properties\": {\"num_iterations\": \"1000\", \"training_type\": \"TrainFull\", \"acquisition_function\": \"EI\", \"primary_metric\": \"accuracy\", \"train_split\": \"0\", \"acquisition_parameter\": \"0\", \"num_cross_validation\": \"5\", \"target\": \"ccp03aml\", \"AMLSettingsJsonString\": \"{\\\"path\\\":null,\\\"name\\\":\\\"expp03ht\\\",\\\"subscription_id\\\":\\\"1b944a9b-fdae-4f97-aeb1-b7eea0beac53\\\",\\\"resource_group\\\":\\\"aml-quickstarts-136496\\\",\\\"workspace_name\\\":\\\"quick-starts-ws-136496\\\",\\\"region\\\":\\\"southcentralus\\\",\\\"compute_target\\\":\\\"ccp03aml\\\",\\\"spark_service\\\":null,\\\"azure_service\\\":\\\"remote\\\",\\\"many_models\\\":false,\\\"pipeline_fetch_max_batch_size\\\":1,\\\"iterations\\\":1000,\\\"primary_metric\\\":\\\"accuracy\\\",\\\"task_type\\\":\\\"classification\\\",\\\"data_script\\\":null,\\\"validation_size\\\":0.0,\\\"n_cross_validations\\\":5,\\\"y_min\\\":null,\\\"y_max\\\":null,\\\"num_classes\\\":null,\\\"featurization\\\":\\\"auto\\\",\\\"_ignore_package_version_incompatibilities\\\":false,\\\"is_timeseries\\\":false,\\\"max_cores_per_iteration\\\":1,\\\"max_concurrent_iterations\\\":5,\\\"iteration_timeout_minutes\\\":null,\\\"mem_in_mb\\\":null,\\\"enforce_time_on_windows\\\":false,\\\"experiment_timeout_minutes\\\":20,\\\"experiment_exit_score\\\":null,\\\"whitelist_models\\\":null,\\\"blacklist_algos\\\":[\\\"TensorFlowLinearClassifier\\\",\\\"TensorFlowDNN\\\"],\\\"supported_models\\\":[\\\"LightGBM\\\",\\\"AveragedPerceptronClassifier\\\",\\\"LogisticRegression\\\",\\\"SGD\\\",\\\"BernoulliNaiveBayes\\\",\\\"MultinomialNaiveBayes\\\",\\\"KNN\\\",\\\"RandomForest\\\",\\\"ExtremeRandomTrees\\\",\\\"SVM\\\",\\\"XGBoostClassifier\\\",\\\"GradientBoosting\\\",\\\"TensorFlowLinearClassifier\\\",\\\"TensorFlowDNN\\\",\\\"DecisionTree\\\",\\\"LinearSVM\\\"],\\\"auto_blacklist\\\":true,\\\"blacklist_samples_reached\\\":false,\\\"exclude_nan_labels\\\":true,\\\"verbosity\\\":20,\\\"_debug_log\\\":\\\"azureml_automl.log\\\",\\\"show_warnings\\\":false,\\\"model_explainability\\\":true,\\\"service_url\\\":null,\\\"sdk_url\\\":null,\\\"sdk_packages\\\":null,\\\"enable_onnx_compatible_models\\\":false,\\\"enable_split_onnx_featurizer_estimator_models\\\":false,\\\"vm_type\\\":\\\"STANDARD_D2_V2\\\",\\\"telemetry_verbosity\\\":20,\\\"send_telemetry\\\":true,\\\"enable_dnn\\\":false,\\\"scenario\\\":\\\"SDK-1.13.0\\\",\\\"environment_label\\\":null,\\\"force_text_dnn\\\":false,\\\"enable_feature_sweeping\\\":true,\\\"enable_early_stopping\\\":false,\\\"early_stopping_n_iters\\\":10,\\\"metrics\\\":null,\\\"enable_ensembling\\\":true,\\\"enable_stack_ensembling\\\":true,\\\"ensemble_iterations\\\":15,\\\"enable_tf\\\":false,\\\"enable_subsampling\\\":null,\\\"subsample_seed\\\":null,\\\"enable_nimbusml\\\":false,\\\"enable_streaming\\\":false,\\\"force_streaming\\\":false,\\\"track_child_runs\\\":true,\\\"allowed_private_models\\\":[],\\\"label_column_name\\\":\\\"sexo\\\",\\\"weight_column_name\\\":null,\\\"cv_split_column_names\\\":null,\\\"enable_local_managed\\\":false,\\\"_local_managed_run_id\\\":null,\\\"cost_mode\\\":1,\\\"lag_length\\\":0,\\\"metric_operation\\\":\\\"maximize\\\",\\\"preprocess\\\":true}\", \"DataPrepJsonString\": \"{\\\\\\\"training_data\\\\\\\": \\\\\\\"{\\\\\\\\\\\\\\\"blocks\\\\\\\\\\\\\\\": [{\\\\\\\\\\\\\\\"id\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"d781c78a-7bc1-4170-8771-ec781cb006f0\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"Microsoft.DPrep.GetFilesBlock\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"arguments\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"isArchive\\\\\\\\\\\\\\\": false, \\\\\\\\\\\\\\\"path\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"target\\\\\\\\\\\\\\\": 4, \\\\\\\\\\\\\\\"resourceDetails\\\\\\\\\\\\\\\": [{\\\\\\\\\\\\\\\"path\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"https://raw.githubusercontent.com/hftamayo/azuremlproj03/main/cad2018.csv\\\\\\\\\\\\\\\"}]}}, \\\\\\\\\\\\\\\"localData\\\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\\\": true, \\\\\\\\\\\\\\\"name\\\\\\\\\\\\\\\": null, \\\\\\\\\\\\\\\"annotation\\\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\\\"id\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"e0ac160c-c9ef-40fb-9a8e-6bc8635d5908\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"Microsoft.DPrep.ParseDelimitedBlock\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"arguments\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"columnHeadersMode\\\\\\\\\\\\\\\": 3, \\\\\\\\\\\\\\\"fileEncoding\\\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\\\"handleQuotedLineBreaks\\\\\\\\\\\\\\\": false, \\\\\\\\\\\\\\\"preview\\\\\\\\\\\\\\\": false, \\\\\\\\\\\\\\\"separator\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"skipRows\\\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\\\"skipRowsMode\\\\\\\\\\\\\\\": 0}, \\\\\\\\\\\\\\\"localData\\\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\\\": true, \\\\\\\\\\\\\\\"name\\\\\\\\\\\\\\\": null, \\\\\\\\\\\\\\\"annotation\\\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\\\"id\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"66a5e446-94f3-4f37-b0f0-4404b27c2559\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"Microsoft.DPrep.DropColumnsBlock\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"arguments\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"columns\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumns\\\\\\\\\\\\\\\": [\\\\\\\\\\\\\\\"Path\\\\\\\\\\\\\\\"]}}}, \\\\\\\\\\\\\\\"localData\\\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\\\": true, \\\\\\\\\\\\\\\"name\\\\\\\\\\\\\\\": null, \\\\\\\\\\\\\\\"annotation\\\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\\\"id\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"be0bb52d-5419-4c3a-880a-6086d37b8cfd\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"Microsoft.DPrep.SetColumnTypesBlock\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"arguments\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"columnConversion\\\\\\\\\\\\\\\": [{\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"id\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"regfalle\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"edad\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"sexo\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"deptoocuhe\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"tipoarma\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\\\"column\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"type\\\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\\\"details\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"pracaut\\\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\\\": 2}]}, \\\\\\\\\\\\\\\"localData\\\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\\\": true, \\\\\\\\\\\\\\\"name\\\\\\\\\\\\\\\": null, \\\\\\\\\\\\\\\"annotation\\\\\\\\\\\\\\\": null}], \\\\\\\\\\\\\\\"inspectors\\\\\\\\\\\\\\\": [], \\\\\\\\\\\\\\\"meta\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"savedDatasetId\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"83126fd9-f456-48a6-ae73-466351bfaa69\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"datasetType\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"tabular\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"subscriptionId\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"1b944a9b-fdae-4f97-aeb1-b7eea0beac53\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"workspaceId\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"2bc4379e-e9ea-4b1a-9fb9-b7297d2b4a58\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"workspaceLocation\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"southcentralus\\\\\\\\\\\\\\\"}}\\\\\\\", \\\\\\\"activities\\\\\\\": 0}\", \"EnableSubsampling\": null, \"runTemplate\": \"AutoML\", \"azureml.runsource\": \"automl\", \"display_task_type\": \"classification\", \"dependencies_versions\": \"{\\\"azureml-widgets\\\": \\\"1.20.0\\\", \\\"azureml-train\\\": \\\"1.20.0\\\", \\\"azureml-train-restclients-hyperdrive\\\": \\\"1.20.0\\\", \\\"azureml-train-core\\\": \\\"1.20.0\\\", \\\"azureml-train-automl\\\": \\\"1.20.0\\\", \\\"azureml-train-automl-runtime\\\": \\\"1.20.0\\\", \\\"azureml-train-automl-client\\\": \\\"1.20.0\\\", \\\"azureml-tensorboard\\\": \\\"1.20.0\\\", \\\"azureml-telemetry\\\": \\\"1.20.0\\\", \\\"azureml-sdk\\\": \\\"1.20.0\\\", \\\"azureml-samples\\\": \\\"0+unknown\\\", \\\"azureml-pipeline\\\": \\\"1.20.0\\\", \\\"azureml-pipeline-steps\\\": \\\"1.20.0\\\", \\\"azureml-pipeline-core\\\": \\\"1.20.0\\\", \\\"azureml-opendatasets\\\": \\\"1.20.0\\\", \\\"azureml-model-management-sdk\\\": \\\"1.0.1b6.post1\\\", \\\"azureml-mlflow\\\": \\\"1.20.0.post1\\\", \\\"azureml-interpret\\\": \\\"1.20.0\\\", \\\"azureml-explain-model\\\": \\\"1.20.0\\\", \\\"azureml-defaults\\\": \\\"1.20.0\\\", \\\"azureml-dataset-runtime\\\": \\\"1.20.0\\\", \\\"azureml-dataprep\\\": \\\"2.7.3\\\", \\\"azureml-dataprep-rslex\\\": \\\"1.5.0\\\", \\\"azureml-dataprep-native\\\": \\\"27.0.0\\\", \\\"azureml-datadrift\\\": \\\"1.20.0\\\", \\\"azureml-core\\\": \\\"1.20.0\\\", \\\"azureml-contrib-services\\\": \\\"1.20.0\\\", \\\"azureml-contrib-server\\\": \\\"1.20.0\\\", \\\"azureml-contrib-reinforcementlearning\\\": \\\"1.20.0\\\", \\\"azureml-contrib-pipeline-steps\\\": \\\"1.20.0\\\", \\\"azureml-contrib-notebook\\\": \\\"1.20.0\\\", \\\"azureml-contrib-interpret\\\": \\\"1.20.0\\\", \\\"azureml-contrib-gbdt\\\": \\\"1.20.0\\\", \\\"azureml-contrib-fairness\\\": \\\"1.20.0\\\", \\\"azureml-contrib-dataset\\\": \\\"1.20.0\\\", \\\"azureml-cli-common\\\": \\\"1.20.0\\\", \\\"azureml-automl-runtime\\\": \\\"1.20.0\\\", \\\"azureml-automl-core\\\": \\\"1.20.0\\\", \\\"azureml-accel-models\\\": \\\"1.20.0\\\"}\", \"_aml_system_scenario_identification\": \"Remote.Parent\", \"ClientType\": \"SDK\", \"environment_cpu_name\": \"AzureML-AutoML\", \"environment_cpu_label\": \"prod\", \"environment_gpu_name\": \"AzureML-AutoML-GPU\", \"environment_gpu_label\": \"prod\", \"root_attribution\": \"automl\", \"attribution\": \"AutoML\", \"Orchestrator\": \"AutoML\", \"CancelUri\": \"https://southcentralus.experiments.azureml.net/jasmine/v1.0/subscriptions/1b944a9b-fdae-4f97-aeb1-b7eea0beac53/resourceGroups/aml-quickstarts-136496/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-136496/experimentids/7229f264-e173-4e00-b190-95618964ceaf/cancel/AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"ClientSdkVersion\": \"1.20.0.post1\", \"snapshotId\": \"00000000-0000-0000-0000-000000000000\", \"SetupRunId\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_setup\", \"SetupRunContainerId\": \"dcid.AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_setup\", \"FeaturizationRunJsonPath\": \"featurizer_container.json\", \"FeaturizationRunId\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_featurize\", \"ProblemInfoJsonString\": \"{\\\"dataset_num_categorical\\\": 0, \\\"is_sparse\\\": true, \\\"subsampling\\\": false, \\\"dataset_classes\\\": 2, \\\"dataset_features\\\": 34, \\\"dataset_samples\\\": 3346, \\\"single_frequency_class_detected\\\": false}\", \"ModelExplainRunId\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_ModelExplain\"}, \"tags\": {\"model_explain_run\": \"best_run\", \"_aml_system_azureml.automlComponent\": \"AutoML\", \"pipeline_id\": \"\", \"score\": \"\", \"predicted_cost\": \"\", \"fit_time\": \"\", \"training_percent\": \"\", \"iteration\": \"\", \"run_preprocessor\": \"\", \"run_algorithm\": \"\", \"dynamic_whitelisting_iterations\": \"<25>;<30>;<35>;<40>;<45>;<50>;<55>;<60>;<65>;<70>;<75>;\", \"automl_best_child_run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80\", \"model_explain_best_run_child_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80\"}, \"end_time_utc\": \"2021-01-29T12:47:19.976252Z\", \"status\": \"Completed\", \"log_files\": {}, \"log_groups\": [], \"run_duration\": \"0:31:35\"}, \"child_runs\": [{\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_0\", \"run_number\": 74, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:25:13.08877Z\", \"end_time\": \"2021-01-29T12:26:10.799446Z\", \"created_time\": \"2021-01-29T12:24:09.021217Z\", \"created_time_dt\": \"2021-01-29T12:24:09.021217Z\", \"duration\": \"0:02:01\", \"iteration\": \"0\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87238605, \"best_metric\": 0.87238605}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_4\", \"run_number\": 75, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:25:52.67948Z\", \"end_time\": \"2021-01-29T12:29:25.55995Z\", \"created_time\": \"2021-01-29T12:24:09.117095Z\", \"created_time_dt\": \"2021-01-29T12:24:09.117095Z\", \"duration\": \"0:05:16\", \"iteration\": \"4\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.65601767, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_3\", \"run_number\": 76, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:25:52.891457Z\", \"end_time\": \"2021-01-29T12:29:19.263786Z\", \"created_time\": \"2021-01-29T12:24:09.133273Z\", \"created_time_dt\": \"2021-01-29T12:24:09.133273Z\", \"duration\": \"0:05:10\", \"iteration\": \"3\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_2\", \"run_number\": 77, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:24:19.860691Z\", \"end_time\": \"2021-01-29T12:25:10.168324Z\", \"created_time\": \"2021-01-29T12:24:09.148047Z\", \"created_time_dt\": \"2021-01-29T12:24:09.148047Z\", \"duration\": \"0:01:01\", \"iteration\": \"2\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_1\", \"run_number\": 78, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:25:52.777591Z\", \"end_time\": \"2021-01-29T12:29:10.854635Z\", \"created_time\": \"2021-01-29T12:24:09.143682Z\", \"created_time_dt\": \"2021-01-29T12:24:09.143682Z\", \"duration\": \"0:05:01\", \"iteration\": \"1\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, XGBoostClassifier\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87985499, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_5\", \"run_number\": 79, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:25:57.556676Z\", \"end_time\": \"2021-01-29T12:29:20.551688Z\", \"created_time\": \"2021-01-29T12:25:13.941131Z\", \"created_time_dt\": \"2021-01-29T12:25:13.941131Z\", \"duration\": \"0:04:06\", \"iteration\": \"5\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.62792629, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_6\", \"run_number\": 80, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:26:24.610497Z\", \"end_time\": \"2021-01-29T12:27:23.905865Z\", \"created_time\": \"2021-01-29T12:26:14.524794Z\", \"created_time_dt\": \"2021-01-29T12:26:14.524794Z\", \"duration\": \"0:01:09\", \"iteration\": \"6\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7ffac92df4a8\", \"primary_metric\": 0.87776499, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_7\", \"run_number\": 81, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:27:40.971107Z\", \"end_time\": \"2021-01-29T12:28:35.647344Z\", \"created_time\": \"2021-01-29T12:27:27.319317Z\", \"created_time_dt\": \"2021-01-29T12:27:27.319317Z\", \"duration\": \"0:01:08\", \"iteration\": \"7\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LogisticRegression\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.8780626, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_8\", \"run_number\": 82, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:28:48.812045Z\", \"end_time\": \"2021-01-29T12:29:42.063813Z\", \"created_time\": \"2021-01-29T12:28:37.853609Z\", \"created_time_dt\": \"2021-01-29T12:28:37.853609Z\", \"duration\": \"0:01:04\", \"iteration\": \"8\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_9\", \"run_number\": 83, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:29:22.968461Z\", \"end_time\": \"2021-01-29T12:30:16.184143Z\", \"created_time\": \"2021-01-29T12:29:13.119508Z\", \"created_time_dt\": \"2021-01-29T12:29:13.119508Z\", \"duration\": \"0:01:03\", \"iteration\": \"9\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7fdc00525710\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_10\", \"run_number\": 84, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:29:32.900129Z\", \"end_time\": \"2021-01-29T12:30:24.349131Z\", \"created_time\": \"2021-01-29T12:29:21.116605Z\", \"created_time_dt\": \"2021-01-29T12:29:21.116605Z\", \"duration\": \"0:01:03\", \"iteration\": \"10\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, RandomForest\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fc28b807a90\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_11\", \"run_number\": 85, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:29:35.395029Z\", \"end_time\": \"2021-01-29T12:30:28.599849Z\", \"created_time\": \"2021-01-29T12:29:25.08398Z\", \"created_time_dt\": \"2021-01-29T12:29:25.08398Z\", \"duration\": \"0:01:03\", \"iteration\": \"11\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7ff10bfceac8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_12\", \"run_number\": 86, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:29:38.476093Z\", \"end_time\": \"2021-01-29T12:30:34.820144Z\", \"created_time\": \"2021-01-29T12:29:28.557882Z\", \"created_time_dt\": \"2021-01-29T12:29:28.557882Z\", \"duration\": \"0:01:06\", \"iteration\": \"12\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f4f201e3518\", \"primary_metric\": 0.87925842, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_13\", \"run_number\": 87, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:29:53.755779Z\", \"end_time\": \"2021-01-29T12:30:49.979022Z\", \"created_time\": \"2021-01-29T12:29:43.63657Z\", \"created_time_dt\": \"2021-01-29T12:29:43.63657Z\", \"duration\": \"0:01:06\", \"iteration\": \"13\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, RandomForest\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f4cbbd1bbe0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_14\", \"run_number\": 88, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:30:30.326197Z\", \"end_time\": \"2021-01-29T12:31:30.989119Z\", \"created_time\": \"2021-01-29T12:30:17.899233Z\", \"created_time_dt\": \"2021-01-29T12:30:17.899233Z\", \"duration\": \"0:01:13\", \"iteration\": \"14\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_15\", \"run_number\": 89, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:30:38.090133Z\", \"end_time\": \"2021-01-29T12:31:29.652386Z\", \"created_time\": \"2021-01-29T12:30:25.993695Z\", \"created_time_dt\": \"2021-01-29T12:30:25.993695Z\", \"duration\": \"0:01:03\", \"iteration\": \"15\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_16\", \"run_number\": 90, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:30:40.49927Z\", \"end_time\": \"2021-01-29T12:31:30.705083Z\", \"created_time\": \"2021-01-29T12:30:30.332896Z\", \"created_time_dt\": \"2021-01-29T12:30:30.332896Z\", \"duration\": \"0:01:00\", \"iteration\": \"16\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f70ac687f28\", \"primary_metric\": 0.87925842, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_17\", \"run_number\": 91, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:30:46.058193Z\", \"end_time\": \"2021-01-29T12:31:36.321201Z\", \"created_time\": \"2021-01-29T12:30:36.443058Z\", \"created_time_dt\": \"2021-01-29T12:30:36.443058Z\", \"duration\": \"0:00:59\", \"iteration\": \"17\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, GradientBoosting\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_18\", \"run_number\": 92, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:31:01.41307Z\", \"end_time\": \"2021-01-29T12:31:54.558345Z\", \"created_time\": \"2021-01-29T12:30:51.526557Z\", \"created_time_dt\": \"2021-01-29T12:30:51.526557Z\", \"duration\": \"0:01:03\", \"iteration\": \"18\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LogisticRegression\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_19\", \"run_number\": 93, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:31:43.204753Z\", \"end_time\": \"2021-01-29T12:32:34.679312Z\", \"created_time\": \"2021-01-29T12:31:31.281512Z\", \"created_time_dt\": \"2021-01-29T12:31:31.281512Z\", \"duration\": \"0:01:03\", \"iteration\": \"19\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LogisticRegression\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.8780626, \"best_metric\": 0.87985499}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_20\", \"run_number\": 94, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:31:46.589517Z\", \"end_time\": \"2021-01-29T12:32:44.002319Z\", \"created_time\": \"2021-01-29T12:31:34.999939Z\", \"created_time_dt\": \"2021-01-29T12:31:34.999939Z\", \"duration\": \"0:01:09\", \"iteration\": \"20\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f1a49c43a58\", \"primary_metric\": 0.88134931, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_21\", \"run_number\": 95, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:31:46.094016Z\", \"end_time\": \"2021-01-29T12:32:42.47469Z\", \"created_time\": \"2021-01-29T12:31:35.011767Z\", \"created_time_dt\": \"2021-01-29T12:31:35.011767Z\", \"duration\": \"0:01:07\", \"iteration\": \"21\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f553451cbe0\", \"primary_metric\": 0.88015483, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_22\", \"run_number\": 96, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:31:49.946316Z\", \"end_time\": \"2021-01-29T12:32:41.378251Z\", \"created_time\": \"2021-01-29T12:31:38.377883Z\", \"created_time_dt\": \"2021-01-29T12:31:38.377883Z\", \"duration\": \"0:01:03\", \"iteration\": \"22\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f3082037e10\", \"primary_metric\": 0.87746514, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_23\", \"run_number\": 97, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:32:06.189996Z\", \"end_time\": \"2021-01-29T12:33:02.319358Z\", \"created_time\": \"2021-01-29T12:31:56.212466Z\", \"created_time_dt\": \"2021-01-29T12:31:56.212466Z\", \"duration\": \"0:01:06\", \"iteration\": \"23\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, ExtremeRandomTrees\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f88a42027f0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_24\", \"run_number\": 98, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:32:47.76737Z\", \"end_time\": \"2021-01-29T12:33:42.342095Z\", \"created_time\": \"2021-01-29T12:32:36.260317Z\", \"created_time_dt\": \"2021-01-29T12:32:36.260317Z\", \"duration\": \"0:01:06\", \"iteration\": \"24\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, GradientBoosting\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f82919a0be0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_25\", \"run_number\": 99, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:32:55.832125Z\", \"end_time\": \"2021-01-29T12:33:41.372609Z\", \"created_time\": \"2021-01-29T12:32:44.275695Z\", \"created_time_dt\": \"2021-01-29T12:32:44.275695Z\", \"duration\": \"0:00:57\", \"iteration\": \"25\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f3a3ec627b8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_26\", \"run_number\": 100, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:32:57.202416Z\", \"end_time\": \"2021-01-29T12:33:53.409457Z\", \"created_time\": \"2021-01-29T12:32:47.249351Z\", \"created_time_dt\": \"2021-01-29T12:32:47.249351Z\", \"duration\": \"0:01:06\", \"iteration\": \"26\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f8ba5948160\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_27\", \"run_number\": 101, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:32:59.355204Z\", \"end_time\": \"2021-01-29T12:33:44.966322Z\", \"created_time\": \"2021-01-29T12:32:47.252751Z\", \"created_time_dt\": \"2021-01-29T12:32:47.252751Z\", \"duration\": \"0:00:57\", \"iteration\": \"27\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f6115f44eb8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_28\", \"run_number\": 102, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:33:15.471921Z\", \"end_time\": \"2021-01-29T12:34:01.114735Z\", \"created_time\": \"2021-01-29T12:33:03.923635Z\", \"created_time_dt\": \"2021-01-29T12:33:03.923635Z\", \"duration\": \"0:00:57\", \"iteration\": \"28\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7ff3f6909f60\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_29\", \"run_number\": 103, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:33:52.959204Z\", \"end_time\": \"2021-01-29T12:34:40.284217Z\", \"created_time\": \"2021-01-29T12:33:42.900083Z\", \"created_time_dt\": \"2021-01-29T12:33:42.900083Z\", \"duration\": \"0:00:57\", \"iteration\": \"29\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_30\", \"run_number\": 104, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:33:58.817555Z\", \"end_time\": \"2021-01-29T12:34:47.283269Z\", \"created_time\": \"2021-01-29T12:33:46.762788Z\", \"created_time_dt\": \"2021-01-29T12:33:46.762788Z\", \"duration\": \"0:01:00\", \"iteration\": \"30\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f62eb1e8898\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_31\", \"run_number\": 105, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:34:00.288273Z\", \"end_time\": \"2021-01-29T12:34:47.63722Z\", \"created_time\": \"2021-01-29T12:33:50.061424Z\", \"created_time_dt\": \"2021-01-29T12:33:50.061424Z\", \"duration\": \"0:00:57\", \"iteration\": \"31\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, RandomForest\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_32\", \"run_number\": 106, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:34:06.556358Z\", \"end_time\": \"2021-01-29T12:34:55.020521Z\", \"created_time\": \"2021-01-29T12:33:54.912779Z\", \"created_time_dt\": \"2021-01-29T12:33:54.912779Z\", \"duration\": \"0:01:00\", \"iteration\": \"32\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, ExtremeRandomTrees\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_33\", \"run_number\": 107, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:34:14.470811Z\", \"end_time\": \"2021-01-29T12:35:08.808419Z\", \"created_time\": \"2021-01-29T12:34:02.977468Z\", \"created_time_dt\": \"2021-01-29T12:34:02.977468Z\", \"duration\": \"0:01:05\", \"iteration\": \"33\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fb02ec76c88\", \"primary_metric\": 0.87088816, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_34\", \"run_number\": 108, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:34:53.209472Z\", \"end_time\": \"2021-01-29T12:35:41.541661Z\", \"created_time\": \"2021-01-29T12:34:41.906994Z\", \"created_time_dt\": \"2021-01-29T12:34:41.906994Z\", \"duration\": \"0:00:59\", \"iteration\": \"34\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f77ef5b05f8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_35\", \"run_number\": 109, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:34:59.841458Z\", \"end_time\": \"2021-01-29T12:35:47.196613Z\", \"created_time\": \"2021-01-29T12:34:49.69419Z\", \"created_time_dt\": \"2021-01-29T12:34:49.69419Z\", \"duration\": \"0:00:57\", \"iteration\": \"35\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f55d603d5f8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_36\", \"run_number\": 110, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:35:02.383689Z\", \"end_time\": \"2021-01-29T12:35:52.698839Z\", \"created_time\": \"2021-01-29T12:34:52.615565Z\", \"created_time_dt\": \"2021-01-29T12:34:52.615565Z\", \"duration\": \"0:01:00\", \"iteration\": \"36\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7fd51c0d1908\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_37\", \"run_number\": 111, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:35:08.746795Z\", \"end_time\": \"2021-01-29T12:36:06.744942Z\", \"created_time\": \"2021-01-29T12:34:56.61952Z\", \"created_time_dt\": \"2021-01-29T12:34:56.61952Z\", \"duration\": \"0:01:10\", \"iteration\": \"37\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fa68e1041d0\", \"primary_metric\": 0.85744372, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_38\", \"run_number\": 112, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:35:22.611461Z\", \"end_time\": \"2021-01-29T12:36:09.312386Z\", \"created_time\": \"2021-01-29T12:35:10.642072Z\", \"created_time_dt\": \"2021-01-29T12:35:10.642072Z\", \"duration\": \"0:00:58\", \"iteration\": \"38\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f5d8902b7f0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_39\", \"run_number\": 113, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:35:53.855757Z\", \"end_time\": \"2021-01-29T12:36:56.150547Z\", \"created_time\": \"2021-01-29T12:35:43.645145Z\", \"created_time_dt\": \"2021-01-29T12:35:43.645145Z\", \"duration\": \"0:01:12\", \"iteration\": \"39\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f2bab991198\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_40\", \"run_number\": 114, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:36:02.104881Z\", \"end_time\": \"2021-01-29T12:36:51.239862Z\", \"created_time\": \"2021-01-29T12:35:50.107929Z\", \"created_time_dt\": \"2021-01-29T12:35:50.107929Z\", \"duration\": \"0:01:01\", \"iteration\": \"40\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f1675f494a8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_41\", \"run_number\": 115, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:36:06.390838Z\", \"end_time\": \"2021-01-29T12:37:00.772282Z\", \"created_time\": \"2021-01-29T12:35:54.393232Z\", \"created_time_dt\": \"2021-01-29T12:35:54.393232Z\", \"duration\": \"0:01:06\", \"iteration\": \"41\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f77443b7cc0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_42\", \"run_number\": 116, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:36:20.455829Z\", \"end_time\": \"2021-01-29T12:37:15.886124Z\", \"created_time\": \"2021-01-29T12:36:08.40019Z\", \"created_time_dt\": \"2021-01-29T12:36:08.40019Z\", \"duration\": \"0:01:07\", \"iteration\": \"42\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f44601f9e80\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_43\", \"run_number\": 117, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:36:21.208511Z\", \"end_time\": \"2021-01-29T12:37:09.167838Z\", \"created_time\": \"2021-01-29T12:36:11.452297Z\", \"created_time_dt\": \"2021-01-29T12:36:11.452297Z\", \"duration\": \"0:00:57\", \"iteration\": \"43\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, GradientBoosting\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_44\", \"run_number\": 118, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:37:03.501934Z\", \"end_time\": \"2021-01-29T12:37:50.844546Z\", \"created_time\": \"2021-01-29T12:36:52.845293Z\", \"created_time_dt\": \"2021-01-29T12:36:52.845293Z\", \"duration\": \"0:00:57\", \"iteration\": \"44\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7ff28199ce48\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_45\", \"run_number\": 119, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:37:10.146433Z\", \"end_time\": \"2021-01-29T12:38:07.537327Z\", \"created_time\": \"2021-01-29T12:36:58.623486Z\", \"created_time_dt\": \"2021-01-29T12:36:58.623486Z\", \"duration\": \"0:01:08\", \"iteration\": \"45\", \"goal\": \"accuracy_max\", \"run_name\": \"TruncatedSVDWrapper, XGBoostClassifier\", \"run_properties\": \"n_components=0.20789473684210527, random_state=None\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_46\", \"run_number\": 120, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:37:14.938227Z\", \"end_time\": \"2021-01-29T12:38:15.461181Z\", \"created_time\": \"2021-01-29T12:37:02.563764Z\", \"created_time_dt\": \"2021-01-29T12:37:02.563764Z\", \"duration\": \"0:01:12\", \"iteration\": \"46\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, GradientBoosting\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7fcfb0ce1e48\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_47\", \"run_number\": 121, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:37:23.011345Z\", \"end_time\": \"2021-01-29T12:38:11.511026Z\", \"created_time\": \"2021-01-29T12:37:10.975881Z\", \"created_time_dt\": \"2021-01-29T12:37:10.975881Z\", \"duration\": \"0:01:00\", \"iteration\": \"47\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f44f0753b70\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_48\", \"run_number\": 122, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:37:28.694392Z\", \"end_time\": \"2021-01-29T12:38:21.879283Z\", \"created_time\": \"2021-01-29T12:37:17.530534Z\", \"created_time_dt\": \"2021-01-29T12:37:17.530534Z\", \"duration\": \"0:01:04\", \"iteration\": \"48\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fccd2532e48\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_49\", \"run_number\": 123, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:38:06.61799Z\", \"end_time\": \"2021-01-29T12:38:59.994814Z\", \"created_time\": \"2021-01-29T12:37:52.807309Z\", \"created_time_dt\": \"2021-01-29T12:37:52.807309Z\", \"duration\": \"0:01:07\", \"iteration\": \"49\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f838c84c518\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_50\", \"run_number\": 124, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:38:23.357638Z\", \"end_time\": \"2021-01-29T12:39:09.107825Z\", \"created_time\": \"2021-01-29T12:38:10.272579Z\", \"created_time_dt\": \"2021-01-29T12:38:10.272579Z\", \"duration\": \"0:00:58\", \"iteration\": \"50\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7fc7fc0d8978\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_51\", \"run_number\": 125, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:38:27.496527Z\", \"end_time\": \"2021-01-29T12:39:17.812088Z\", \"created_time\": \"2021-01-29T12:38:14.331594Z\", \"created_time_dt\": \"2021-01-29T12:38:14.331594Z\", \"duration\": \"0:01:03\", \"iteration\": \"51\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f5e92b63b70\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_52\", \"run_number\": 126, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:38:29.658335Z\", \"end_time\": \"2021-01-29T12:39:27.081788Z\", \"created_time\": \"2021-01-29T12:38:17.73552Z\", \"created_time_dt\": \"2021-01-29T12:38:17.73552Z\", \"duration\": \"0:01:09\", \"iteration\": \"52\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, GradientBoosting\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fb560546438\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_53\", \"run_number\": 127, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:38:33.750055Z\", \"end_time\": \"2021-01-29T12:39:21.098514Z\", \"created_time\": \"2021-01-29T12:38:23.489821Z\", \"created_time_dt\": \"2021-01-29T12:38:23.489821Z\", \"duration\": \"0:00:57\", \"iteration\": \"53\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LogisticRegression\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.65660308, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_54\", \"run_number\": 128, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:39:12.281072Z\", \"end_time\": \"2021-01-29T12:39:59.135997Z\", \"created_time\": \"2021-01-29T12:39:01.874708Z\", \"created_time_dt\": \"2021-01-29T12:39:01.874708Z\", \"duration\": \"0:00:57\", \"iteration\": \"54\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_55\", \"run_number\": 129, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:39:23.65322Z\", \"end_time\": \"2021-01-29T12:40:18.159386Z\", \"created_time\": \"2021-01-29T12:39:11.516501Z\", \"created_time_dt\": \"2021-01-29T12:39:11.516501Z\", \"duration\": \"0:01:06\", \"iteration\": \"55\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f9ac93809e8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_56\", \"run_number\": 130, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:39:30.6922Z\", \"end_time\": \"2021-01-29T12:40:29.981581Z\", \"created_time\": \"2021-01-29T12:39:19.801137Z\", \"created_time_dt\": \"2021-01-29T12:39:19.801137Z\", \"duration\": \"0:01:10\", \"iteration\": \"56\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_57\", \"run_number\": 131, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:39:35.218699Z\", \"end_time\": \"2021-01-29T12:40:23.670292Z\", \"created_time\": \"2021-01-29T12:39:23.462813Z\", \"created_time_dt\": \"2021-01-29T12:39:23.462813Z\", \"duration\": \"0:01:00\", \"iteration\": \"57\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_58\", \"run_number\": 132, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:39:38.57216Z\", \"end_time\": \"2021-01-29T12:40:28.886089Z\", \"created_time\": \"2021-01-29T12:39:28.599344Z\", \"created_time_dt\": \"2021-01-29T12:39:28.599344Z\", \"duration\": \"0:01:00\", \"iteration\": \"58\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_59\", \"run_number\": 133, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:40:13.182847Z\", \"end_time\": \"2021-01-29T12:40:58.740845Z\", \"created_time\": \"2021-01-29T12:40:01.634222Z\", \"created_time_dt\": \"2021-01-29T12:40:01.634222Z\", \"duration\": \"0:00:57\", \"iteration\": \"59\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_60\", \"run_number\": 134, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:40:32.006138Z\", \"end_time\": \"2021-01-29T12:41:27.806215Z\", \"created_time\": \"2021-01-29T12:40:20.679475Z\", \"created_time_dt\": \"2021-01-29T12:40:20.679475Z\", \"duration\": \"0:01:07\", \"iteration\": \"60\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fb3f4321978\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_61\", \"run_number\": 135, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:40:37.271177Z\", \"end_time\": \"2021-01-29T12:41:31.640292Z\", \"created_time\": \"2021-01-29T12:40:25.236281Z\", \"created_time_dt\": \"2021-01-29T12:40:25.236281Z\", \"duration\": \"0:01:06\", \"iteration\": \"61\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f678704cac8\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_62\", \"run_number\": 136, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:40:40.815054Z\", \"end_time\": \"2021-01-29T12:41:33.778337Z\", \"created_time\": \"2021-01-29T12:40:30.561178Z\", \"created_time_dt\": \"2021-01-29T12:40:30.561178Z\", \"duration\": \"0:01:03\", \"iteration\": \"62\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7ff9f12ad278\", \"primary_metric\": 0.87925842, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_63\", \"run_number\": 137, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:40:45.991799Z\", \"end_time\": \"2021-01-29T12:41:34.031883Z\", \"created_time\": \"2021-01-29T12:40:33.351092Z\", \"created_time_dt\": \"2021-01-29T12:40:33.351092Z\", \"duration\": \"0:01:00\", \"iteration\": \"63\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f8169e89588\", \"primary_metric\": 0.88015483, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_64\", \"run_number\": 138, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:41:13.896223Z\", \"end_time\": \"2021-01-29T12:42:02.834562Z\", \"created_time\": \"2021-01-29T12:41:00.984621Z\", \"created_time_dt\": \"2021-01-29T12:41:00.984621Z\", \"duration\": \"0:01:01\", \"iteration\": \"64\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_65\", \"run_number\": 139, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:41:41.882524Z\", \"end_time\": \"2021-01-29T12:42:35.041741Z\", \"created_time\": \"2021-01-29T12:41:30.399132Z\", \"created_time_dt\": \"2021-01-29T12:41:30.399132Z\", \"duration\": \"0:01:04\", \"iteration\": \"65\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f0a68378fd0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_66\", \"run_number\": 140, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:41:48.040197Z\", \"end_time\": \"2021-01-29T12:42:36.593443Z\", \"created_time\": \"2021-01-29T12:41:35.213713Z\", \"created_time_dt\": \"2021-01-29T12:41:35.213713Z\", \"duration\": \"0:01:01\", \"iteration\": \"66\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f090e964f60\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_67\", \"run_number\": 141, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:41:51.670608Z\", \"end_time\": \"2021-01-29T12:42:50.811066Z\", \"created_time\": \"2021-01-29T12:41:39.260214Z\", \"created_time_dt\": \"2021-01-29T12:41:39.260214Z\", \"duration\": \"0:01:11\", \"iteration\": \"67\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fbda58e49b0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_68\", \"run_number\": 142, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:41:53.119933Z\", \"end_time\": \"2021-01-29T12:42:44.90151Z\", \"created_time\": \"2021-01-29T12:41:39.403016Z\", \"created_time_dt\": \"2021-01-29T12:41:39.403016Z\", \"duration\": \"0:01:05\", \"iteration\": \"68\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, RandomForest\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7fb68505f7f0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_69\", \"run_number\": 143, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:42:15.522327Z\", \"end_time\": \"2021-01-29T12:43:00.075576Z\", \"created_time\": \"2021-01-29T12:42:04.957232Z\", \"created_time_dt\": \"2021-01-29T12:42:04.957232Z\", \"duration\": \"0:00:55\", \"iteration\": \"69\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_70\", \"run_number\": 144, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:42:49.595713Z\", \"end_time\": \"2021-01-29T12:43:48.872079Z\", \"created_time\": \"2021-01-29T12:42:37.934384Z\", \"created_time_dt\": \"2021-01-29T12:42:37.934384Z\", \"duration\": \"0:01:10\", \"iteration\": \"70\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f064787c710\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_71\", \"run_number\": 145, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:42:53.306789Z\", \"end_time\": \"2021-01-29T12:43:37.924767Z\", \"created_time\": \"2021-01-29T12:42:41.864137Z\", \"created_time_dt\": \"2021-01-29T12:42:41.864137Z\", \"duration\": \"0:00:56\", \"iteration\": \"71\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f9fa955e748\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_72\", \"run_number\": 146, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:42:57.938187Z\", \"end_time\": \"2021-01-29T12:43:51.499088Z\", \"created_time\": \"2021-01-29T12:42:46.506751Z\", \"created_time_dt\": \"2021-01-29T12:42:46.506751Z\", \"duration\": \"0:01:04\", \"iteration\": \"72\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, GradientBoosting\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_73\", \"run_number\": 147, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:43:02.950422Z\", \"end_time\": \"2021-01-29T12:43:53.720911Z\", \"created_time\": \"2021-01-29T12:42:52.266396Z\", \"created_time_dt\": \"2021-01-29T12:42:52.266396Z\", \"duration\": \"0:01:01\", \"iteration\": \"73\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f3ed014ea90\", \"primary_metric\": 0.87866051, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_74\", \"run_number\": 148, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:43:15.429568Z\", \"end_time\": \"2021-01-29T12:44:04.269437Z\", \"created_time\": \"2021-01-29T12:43:01.703779Z\", \"created_time_dt\": \"2021-01-29T12:43:01.703779Z\", \"duration\": \"0:01:02\", \"iteration\": \"74\", \"goal\": \"accuracy_max\", \"run_name\": \"StandardScalerWrapper, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.StandardScalerWrapper object at 0x7f52a1ce82b0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_75\", \"run_number\": 149, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:43:52.94324Z\", \"end_time\": \"2021-01-29T12:44:44.864069Z\", \"created_time\": \"2021-01-29T12:43:40.202169Z\", \"created_time_dt\": \"2021-01-29T12:43:40.202169Z\", \"duration\": \"0:01:04\", \"iteration\": \"75\", \"goal\": \"accuracy_max\", \"run_name\": \"TruncatedSVDWrapper, XGBoostClassifier\", \"run_properties\": \"n_components=0.10894736842105263, random_state=None\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_76\", \"run_number\": 150, \"metric\": null, \"status\": \"Canceled\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:44:04.283745Z\", \"end_time\": \"2021-01-29T12:44:52.389092Z\", \"created_time\": \"2021-01-29T12:43:50.60199Z\", \"created_time_dt\": \"2021-01-29T12:43:50.60199Z\", \"duration\": \"0:01:01\", \"iteration\": \"76\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, XGBoostClassifier\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f9b92306588\", \"primary_metric\": 0.88015528, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_77\", \"run_number\": 151, \"metric\": null, \"status\": \"Canceled\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:44:08.910105Z\", \"end_time\": \"2021-01-29T12:44:54.586394Z\", \"created_time\": \"2021-01-29T12:43:55.589785Z\", \"created_time_dt\": \"2021-01-29T12:43:55.589785Z\", \"duration\": \"0:00:58\", \"iteration\": \"77\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_78\", \"run_number\": 152, \"metric\": null, \"status\": \"Canceled\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:44:11.28769Z\", \"end_time\": \"2021-01-29T12:44:55.527346Z\", \"created_time\": \"2021-01-29T12:43:59.515454Z\", \"created_time_dt\": \"2021-01-29T12:43:59.515454Z\", \"duration\": \"0:00:56\", \"iteration\": \"78\", \"goal\": \"accuracy_max\", \"run_name\": \"SparseNormalizer, LightGBM\", \"run_properties\": \"<azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7ff46c4374e0\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_79\", \"run_number\": 153, \"metric\": null, \"status\": \"Canceled\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:44:17.602904Z\", \"end_time\": \"2021-01-29T12:44:53.776724Z\", \"created_time\": \"2021-01-29T12:44:06.494001Z\", \"created_time_dt\": \"2021-01-29T12:44:06.494001Z\", \"duration\": \"0:00:47\", \"iteration\": \"79\", \"goal\": \"accuracy_max\", \"run_name\": \"MaxAbsScaler, LightGBM\", \"run_properties\": \"copy=True\", \"primary_metric\": 0.87895946, \"best_metric\": 0.88134931}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80\", \"run_number\": 154, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:45:07.774099Z\", \"end_time\": \"2021-01-29T12:46:54.800581Z\", \"created_time\": \"2021-01-29T12:44:56.675427Z\", \"created_time_dt\": \"2021-01-29T12:44:56.675427Z\", \"duration\": \"0:01:58\", \"iteration\": \"80\", \"goal\": \"accuracy_max\", \"run_name\": \"VotingEnsemble\", \"run_properties\": \"classification_labels=None,\\n                              estimators=[('20',\\n                                           Pipeline(memory=None,\\n                                                    steps=[('sparsenormalizer',\\n                                                            <azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7f1ed075cb00>\", \"primary_metric\": 0.88194766, \"best_metric\": 0.88194766}, {\"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_81\", \"run_number\": 155, \"metric\": null, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": \"100\", \"start_time\": \"2021-01-29T12:45:09.934167Z\", \"end_time\": \"2021-01-29T12:47:05.296992Z\", \"created_time\": \"2021-01-29T12:44:56.692788Z\", \"created_time_dt\": \"2021-01-29T12:44:56.692788Z\", \"duration\": \"0:02:08\", \"iteration\": \"81\", \"goal\": \"accuracy_max\", \"run_name\": \"StackEnsemble\", \"run_properties\": \"base_learners=[('20',\\n                                        Pipeline(memory=None,\\n                                                 steps=[('sparsenormalizer',\\n                                                         <azureml.automl.runtime.shared.model_wrappers.SparseNormalizer object at 0x7feefaf28d30>\", \"primary_metric\": 0.8810508, \"best_metric\": 0.88194766}], \"children_metrics\": {\"categories\": [0], \"series\": {\"precision_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"precision_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.878959462775807, 0.878959462775807, 0.6560176695000335, 0.6279262878432947, 0.877764986725565, 0.8780626017892599, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.8780626017892599, 0.8813493072752829, 0.8801548312250407, 0.877465140664391, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8708881600963793, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8574437230885931, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.6566030832385159, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.8801548312250407, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8786605091136247, 0.878959462775807, 0.878959462775807, 0.8801552774245364, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8819476607991433, 0.8810507998125964]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"precision_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8819476607991433, 0.8819476607991433]}], \"recall_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"recall_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.878959462775807, 0.878959462775807, 0.6560176695000335, 0.6279262878432947, 0.877764986725565, 0.8780626017892599, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.8780626017892599, 0.8813493072752829, 0.8801548312250407, 0.877465140664391, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8708881600963793, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8574437230885931, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.6566030832385159, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.8801548312250407, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8786605091136247, 0.878959462775807, 0.878959462775807, 0.8801552774245364, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8819476607991433, 0.8810507998125964]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"recall_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8819476607991433, 0.8819476607991433]}], \"weighted_accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"weighted_accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9665117770787506, 0.9786065299501534, 0.9813554620305336, 0.9813554620305336, 0.6717494820250518, 0.6409579406286754, 0.9728159468189691, 0.9768964161365838, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9805321057926966, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9814028520129764, 0.9813554620305336, 0.9781886802185374, 0.9768964161365838, 0.9782569698570445, 0.9803789380574457, 0.9727800069013416, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9656951416624107, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9458047701386455, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.6744121723625218, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9811151228169257, 0.9815330388127952, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9810212680670899, 0.9813554620305336, 0.9813554620305336, 0.979809920140187, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9812271957797485, 0.9796523841744232]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"weighted_accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9665117770787506, 0.9786065299501534, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9813554620305336, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9814028520129764, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952, 0.9815330388127952]}], \"f1_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"f1_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8317049505588496, 0.8298557104668042, 0.8223432579203689, 0.8223432579203689, 0.7133373166759465, 0.6912818838855739, 0.8343733245693283, 0.8283320254742186, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8247607352817615, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8230645104453013, 0.8223432579203689, 0.8283159011577496, 0.8283320254742186, 0.8340038964486345, 0.8274057341812663, 0.8336738701929575, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8293809130013485, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8279544612318214, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.7136846875693615, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8236291863135168, 0.8252147639474552, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8221942996134691, 0.8223432579203689, 0.8223432579203689, 0.8284670145127391, 0.8223432579203689, 0.8223432579203689, 0.8223432579203689, 0.8304641276084948, 0.8310004708511786]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"f1_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8317049505588496, 0.8317049505588496, 0.8317049505588496, 0.8317049505588496, 0.8317049505588496, 0.8317049505588496, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283, 0.8343733245693283]}], \"average_precision_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"average_precision_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.818001567591072, 0.8309574381689986, 0.829644257217514, 0.8253645019996536, 0.8317022646213232, 0.8287671433778694, 0.8242615544336784, 0.8239178579909586, 0.8337728870693388, 0.8261533758022201, 0.830304347693148, 0.8312066152578408, 0.8319501448771425, 0.8298441496964554, 0.8324747676961477, 0.8313028834626411, 0.820781339907332, 0.8322890649236852, 0.8247487801351466, 0.8238306097456387, 0.8201944153223895, 0.825159045725511, 0.8207088439199935, 0.8331251979061556, 0.819990738153842, 0.7872565217938792, 0.825781787332027, 0.8317515489821423, 0.8263751362824895, 0.8314871680542144, 0.7872565217938792, 0.829443419184735, 0.8312592742595631, 0.8249846635084598, 0.829994371630726, 0.7872565217938792, 0.8238619595992885, 0.8166458542301147, 0.8320405212219457, 0.8266145066038255, 0.7872565217938792, 0.8308547037160696, 0.8271421589007371, 0.8273272482076296, 0.8313422566050527, 0.7872565217938792, 0.830737380102495, 0.8322208272004314, 0.8299452038559203, 0.8289548262324564, 0.7872565217938792, 0.8291444876475982, 0.8327105306639877, 0.8261570887264884, 0.832058156537746, 0.7872565217938792, 0.8289522829845598, 0.8237898767513651, 0.8320409498137531, 0.8304114070009538, 0.7872565217938792, 0.8262735988335352, 0.8129835223544912, 0.8231566700073836, 0.8309345950175823, 0.7872565217938792, 0.8308127852164013, 0.8284627426687834, 0.8327719153198891, 0.8315445279671563, 0.7872565217938792, 0.8289250941264488, 0.8183025226087406, 0.8329058513742573, 0.8305512924761211, 0.7872565217938792, 0.8246653488697289, 0.8282055629287294, 0.8251607417754376, 0.8301511769932677, 0.8275860830972966, 0.8318286433350568]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"average_precision_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.818001567591072, 0.8309574381689986, 0.8309574381689986, 0.8309574381689986, 0.8317022646213232, 0.8317022646213232, 0.8317022646213232, 0.8317022646213232, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388, 0.8337728870693388]}], \"AUC_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"AUC_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.5777155368356112, 0.6305110509347944, 0.6335583604100532, 0.6258116849245801, 0.6356173671204847, 0.6211979632544319, 0.6063063240080938, 0.6181742923711174, 0.6371822492378356, 0.6164143534010468, 0.6342933428487124, 0.6319119220327073, 0.6346125141175203, 0.6251343833428058, 0.6365909697726307, 0.631471251386176, 0.6082539473680375, 0.6323414251284074, 0.6221544194372628, 0.6181780786913152, 0.605521075474749, 0.6041384331531718, 0.5990316604750665, 0.6334692192242535, 0.6022801747930784, 0.5, 0.6126982694370844, 0.6346861683437994, 0.6142918610053384, 0.6333515277736165, 0.5, 0.6323577336234807, 0.6333514427665932, 0.6148739602964552, 0.6315939148547655, 0.5, 0.6117686850723691, 0.5770577868888347, 0.635412444322883, 0.6140898476780768, 0.5, 0.6217766394697619, 0.6160524365480751, 0.6161951128211223, 0.6297145951174972, 0.5, 0.6251513298504217, 0.6334976276501612, 0.6261489475264866, 0.6262829843301321, 0.5, 0.6232867569160379, 0.633576075053597, 0.6227573704938333, 0.6323514175982934, 0.5, 0.6237645598121215, 0.6176379881363026, 0.6403289324259205, 0.6293083772589999, 0.5, 0.6190213810423357, 0.5832462705547183, 0.6130036700540654, 0.6310580926780616, 0.5, 0.6280001285756633, 0.6284817410898801, 0.6361893408398975, 0.6356708712532727, 0.5, 0.6243506896655318, 0.596958741563032, 0.6327004082247988, 0.632017318100363, 0.5, 0.6093574508360327, 0.627083662685884, 0.6139126555507943, 0.6274198054021178, 0.6187331807428572, 0.629385361320245]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"AUC_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.5777155368356112, 0.6305110509347944, 0.6335583604100532, 0.6335583604100532, 0.6356173671204847, 0.6356173671204847, 0.6356173671204847, 0.6356173671204847, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205]}], \"average_precision_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"average_precision_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.5502394688010714, 0.5666383423613146, 0.5626104881352652, 0.5507825894272143, 0.5643494903822921, 0.5550195069905071, 0.5615249612158257, 0.5585473667124592, 0.56136384093279, 0.5518178545597646, 0.5616572410391347, 0.5552349005149471, 0.566100914108215, 0.5610207364636155, 0.5605845862409862, 0.5560855492360005, 0.5471134079216563, 0.5578259966176884, 0.5597143660614136, 0.5584436255196921, 0.5592609281214621, 0.5566460121732828, 0.5551178602422677, 0.5601926998067175, 0.5410080003247095, 0.5, 0.5560809322581727, 0.5606743718805891, 0.5529738563448522, 0.5552326746635725, 0.5, 0.5608904463868234, 0.5586580104720976, 0.5609306645576684, 0.5538956305902502, 0.5, 0.551637219092681, 0.5510981286804961, 0.5667561907005529, 0.5560695077921674, 0.5, 0.5568892690418143, 0.5557113277233251, 0.5561504138511995, 0.5581248241708541, 0.5, 0.5608817426397403, 0.556215131928907, 0.5550278300280613, 0.5524004528740142, 0.5, 0.55566513407174, 0.5594092594006004, 0.5625089371443541, 0.5600037530624082, 0.5, 0.5563576463262727, 0.5499912539981936, 0.5646591655993716, 0.5553673158339785, 0.5, 0.5532278697203598, 0.5408473477461034, 0.5579319265305825, 0.5600378872980173, 0.5, 0.5594958112887934, 0.5552535243747494, 0.5678678561788167, 0.5572715067676148, 0.5, 0.5561392729798611, 0.5470230436783634, 0.5674098517003145, 0.5533552442706734, 0.5, 0.55863908993774, 0.5516109876677222, 0.5480106409935888, 0.5578350306724775, 0.567234045526078, 0.57164643052494]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"average_precision_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.5502394688010714, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5666383423613146, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5667561907005529, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.5678678561788167, 0.57164643052494]}], \"AUC_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"AUC_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8944236958602897, 0.90665702245026, 0.9073139298112525, 0.9056060955061962, 0.7427548684160967, 0.6839162044484418, 0.9008720905216224, 0.9033065180317742, 0.9081001545263524, 0.9035598543252968, 0.9074334261007302, 0.9069030076073382, 0.9074881780738696, 0.9054996799705964, 0.9079095074980164, 0.9068447273572033, 0.9018377689431114, 0.9070006890847067, 0.9048351282296127, 0.903301190290537, 0.9012542359898067, 0.9009974840507409, 0.899347628750685, 0.9072952240688105, 0.9007142396947765, 0.878959462775807, 0.9027973965080186, 0.9074308741965897, 0.9032030166279383, 0.9072208356296398, 0.878959462775807, 0.9070407417833131, 0.9072847725252785, 0.9019100430160535, 0.9068677361479163, 0.878959462775807, 0.9026723615402078, 0.8916948726601536, 0.9076266023733242, 0.9030966814048632, 0.878959462775807, 0.9047888510594879, 0.9035508230237201, 0.9035821443309592, 0.9064786800775643, 0.878959462775807, 0.9055830450590479, 0.9072448031335705, 0.9056674301756932, 0.9057083622764355, 0.878959462775807, 0.9050859222850078, 0.9073026255222791, 0.6832899935929513, 0.9070031833391914, 0.878959462775807, 0.9051625037315378, 0.9039083988341735, 0.9087147928505501, 0.9064174960286435, 0.878959462775807, 0.9042169542928015, 0.8966076502177257, 0.9028440602290797, 0.9067354500137309, 0.878959462775807, 0.906135729896777, 0.90616075808489, 0.9078645346045662, 0.9077522921463359, 0.878959462775807, 0.9053280642561358, 0.8995033128943792, 0.9070764677723231, 0.9069823422330732, 0.878959462775807, 0.9020589795060555, 0.9059000533257823, 0.9030961385661401, 0.9059607258614282, 0.9041245675687868, 0.9064210229746953]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"AUC_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8944236958602897, 0.90665702245026, 0.9073139298112525, 0.9073139298112525, 0.9073139298112525, 0.9073139298112525, 0.9073139298112525, 0.9073139298112525, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9081001545263524, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501, 0.9087147928505501]}], \"matthews_correlation\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"matthews_correlation\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.11018258412797255, 0.11002904638722885, 0.0, 0.0, 0.13467346783304887, 0.1077369811678492, 0.13680033454832868, 0.08515876016947388, 0.0, 0.0, 0.0, 0.0, 0.05805305128620335, 0.0, 0.0, 0.0, 0.020561345286516215, 0.0, 0.09620704741709699, 0.08515876016947388, 0.14272716168903815, 0.09085317691010443, 0.12774780595379792, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.09381727915799687, 0.0, 0.0, 0.0, 0.09656931731252813, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1245808492491973, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.03340610378826443, 0.07215194380456343, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.0028922087787057777, 0.0, 0.0, 0.11151209549648784, 0.0, 0.0, 0.0, 0.1310981942998046, 0.13120366386801727]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"matthews_correlation_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.11018258412797255, 0.11018258412797255, 0.11018258412797255, 0.11018258412797255, 0.13467346783304887, 0.13467346783304887, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.13680033454832868, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815, 0.14272716168903815]}], \"precision_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"precision_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.6257814464359828, 0.7114984973388535, 0.4394797313879035, 0.4394797313879035, 0.545953159887907, 0.5360952111595852, 0.6834639415474276, 0.6491341770017877, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.6899534514180035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.5396108592666751, 0.4394797313879035, 0.724071134336341, 0.6491341770017877, 0.737164063889119, 0.7071311746461839, 0.6691178137265068, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.6135207538018648, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.5767307898610723, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.542429776252389, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.5897248104687486, 0.740008274507251, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.439461382435584, 0.4394797313879035, 0.4394797313879035, 0.7990280398447204, 0.4394797313879035, 0.4394797313879035, 0.4394797313879035, 0.7876990337092691, 0.792637304723637]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"precision_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.6257814464359828, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.7114984973388535, 0.724071134336341, 0.724071134336341, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.737164063889119, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.740008274507251, 0.7990280398447204, 0.7990280398447204, 0.7990280398447204, 0.7990280398447204, 0.7990280398447204, 0.7990280398447204]}], \"norm_macro_recall\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"norm_macro_recall\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.04860981058229541, 0.029206021078487777, 0.0, 0.0, 0.19754614826948708, 0.1608824636359342, 0.0523731748961274, 0.024920851395867017, 0.0, 0.0, 0.0, 0.0, 0.009805127224841791, 0.0, 0.0, 0.0, 0.002409638554216853, 0.0, 0.023649268510098808, 0.024920851395867017, 0.04603713262128846, 0.018791327356497068, 0.04981467574796259, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.04188632500567002, 0.0, 0.0, 0.0, 0.06116679684041722, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.18301871002522682, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.004570655503369414, 0.010172796448953702, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.02296965626265579, 0.0, 0.0, 0.0, 0.029543943124992377, 0.032725334967724784]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"norm_macro_recall_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.04860981058229541, 0.04860981058229541, 0.04860981058229541, 0.04860981058229541, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708, 0.19754614826948708]}], \"f1_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"f1_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.520574928174174, 0.4985077022912276, 0.4677878338156226, 0.4677878338156226, 0.5212383250117166, 0.5012235804246237, 0.5216799755142278, 0.4952293850060273, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4777516417800933, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4702432570531805, 0.4677878338156226, 0.49337094555862376, 0.4952293850060273, 0.5136159214787135, 0.4874630725966583, 0.5191427071595363, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.5132731368094254, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.5331140569641261, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.5183343375682423, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4726721873156833, 0.4780641541780641, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4677029504192717, 0.4677878338156226, 0.4677878338156226, 0.49188716438430785, 0.4677878338156226, 0.4677878338156226, 0.4677878338156226, 0.4973371657510099, 0.5012115565130668]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"f1_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.520574928174174, 0.520574928174174, 0.520574928174174, 0.520574928174174, 0.5212383250117166, 0.5212383250117166, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5216799755142278, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261, 0.5331140569641261]}], \"log_loss\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"log_loss\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.4128544260681367, 0.3581560383870387, 0.3574246973076634, 0.3618457527250725, 0.6211846208235114, 0.6525580728995276, 0.37488262842592235, 0.36193625584201755, 0.36414226708265807, 0.3606709140064427, 0.35706404397804026, 0.3585776514599269, 0.35678834373681967, 0.3588809154558716, 0.3578208402379143, 0.3579074754430679, 0.4203946116874772, 0.35788906874247084, 0.3593504602422005, 0.362058077254065, 0.36388161631768423, 0.3627612497562333, 0.3781231269627226, 0.3585614629680019, 0.3640266386094582, 0.36907150272375905, 0.36103587708958595, 0.3574576856500783, 0.36038873713981034, 0.3587766168487521, 0.36904393745247965, 0.3597008722268059, 0.35794496109101565, 0.386356770881035, 0.3592637537975661, 0.36906350534645876, 0.3627027277858854, 0.4723513112241953, 0.3570326709610789, 0.36004227298182756, 0.4244464811070713, 0.36196301417925963, 0.3603693905172588, 0.3595398933646522, 0.36023403834242906, 0.3690440791387606, 0.3592026713986829, 0.35782260539614386, 0.3601531663358723, 0.3598923430217199, 0.3690386289310361, 0.35929860872706765, 0.35965051420661615, 0.6586932544718724, 0.35741980713908933, 0.3690408283876396, 0.3603906680522099, 0.3634298338246024, 0.357025188851106, 0.3600002617145662, 0.36904154368182585, 0.35991915701364324, 0.4211341175239113, 0.4184875758320465, 0.35877162859009243, 0.3690433692354728, 0.3629367180412242, 0.3586493911870906, 0.35841503939565106, 0.3620367187411236, 0.36904111982297866, 0.3602989811242744, 0.3676338186762587, 0.3618762195279066, 0.3582792497317019, 0.36904964622667114, 0.36456131899637373, 0.36100970873789273, 0.3610420363663427, 0.35895357265912253, 0.3631963089285146, 0.3561342356431786]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"log_loss_min\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.4128544260681367, 0.3581560383870387, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.3574246973076634, 0.35706404397804026, 0.35706404397804026, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.35678834373681967, 0.3561342356431786]}], \"recall_score_macro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"recall_score_macro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.5243049052911477, 0.514603010539244, 0.5, 0.5, 0.5987730741347435, 0.5804412318179671, 0.5261865874480637, 0.5124604256979335, 0.5, 0.5, 0.5, 0.5, 0.5045615573969793, 0.5, 0.5, 0.5, 0.5012048192771085, 0.5, 0.5118246342550494, 0.5124604256979335, 0.5230185663106443, 0.5093956636782485, 0.5249073378739813, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5200762516359242, 0.5, 0.5, 0.5, 0.5305833984202086, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5915093550126134, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5022853277516848, 0.5050863982244769, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.4998296422487223, 0.5, 0.5, 0.5114848281313279, 0.5, 0.5, 0.5, 0.5147719715624961, 0.5163626674838624]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"recall_score_macro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.5243049052911477, 0.5243049052911477, 0.5243049052911477, 0.5243049052911477, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435]}], \"average_precision_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"average_precision_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8755859487731611, 0.8895267269038122, 0.8891202950860801, 0.8869408371187143, 0.7415432200644418, 0.680371689396445, 0.881684400376261, 0.8816830265233111, 0.8942647027544904, 0.8872011077791331, 0.8901923296080628, 0.8928757809687928, 0.8909337715082624, 0.8893925379408234, 0.8930397216561177, 0.8927039257483749, 0.878573428118526, 0.8933515810507865, 0.8838742552652545, 0.8816059000100751, 0.8779762814396609, 0.8841722134328279, 0.8789600920782069, 0.8937415796208938, 0.8817068382066312, 0.8331079922848431, 0.8854182892479662, 0.8922073637095753, 0.886921418818613, 0.8932025289503489, 0.8331079922848431, 0.8893123187248371, 0.8920762375470211, 0.8825316034157293, 0.8918059482689478, 0.8331079922848431, 0.8843620466794017, 0.8717042495414511, 0.8909224359366185, 0.8864171771919734, 0.8331079922848431, 0.8912825550506216, 0.887197069099102, 0.8873488776884202, 0.8920096447939244, 0.8331079922848431, 0.8904400038189195, 0.8938034607890021, 0.8911835839293738, 0.8906407014478169, 0.8331079922848431, 0.8897818064996118, 0.8934285850243345, 0.6357957909959607, 0.8926060594930687, 0.8331079922848431, 0.8896476299819389, 0.884488870779013, 0.8918497765255303, 0.8917067053730336, 0.8331079922848431, 0.8868942181136867, 0.8678458234929819, 0.8781321646011098, 0.8911254722863908, 0.8331079922848431, 0.8905129187046235, 0.8895994691929264, 0.8913780013481725, 0.8927667861772386, 0.8331079922848431, 0.8895512840282223, 0.8785451587856381, 0.8903571331548891, 0.8924600965151457, 0.8331079922848431, 0.8834827175523303, 0.8901865304299683, 0.8868242439517363, 0.8907569274186798, 0.8850169345226563, 0.889175968896095]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"average_precision_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8755859487731611, 0.8895267269038122, 0.8895267269038122, 0.8895267269038122, 0.8895267269038122, 0.8895267269038122, 0.8895267269038122, 0.8895267269038122, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904, 0.8942647027544904]}], \"balanced_accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"balanced_accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.5243049052911477, 0.514603010539244, 0.5, 0.5, 0.5987730741347435, 0.5804412318179671, 0.5261865874480637, 0.5124604256979335, 0.5, 0.5, 0.5, 0.5, 0.5045615573969793, 0.5, 0.5, 0.5, 0.5012048192771085, 0.5, 0.5118246342550494, 0.5124604256979335, 0.5230185663106443, 0.5093956636782485, 0.5249073378739813, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5200762516359242, 0.5, 0.5, 0.5, 0.5305833984202086, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5915093550126134, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5022853277516848, 0.5050863982244769, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.4998296422487223, 0.5, 0.5, 0.5114848281313279, 0.5, 0.5, 0.5, 0.5147719715624961, 0.5163626674838624]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"balanced_accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.5243049052911477, 0.5243049052911477, 0.5243049052911477, 0.5243049052911477, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435, 0.5987730741347435]}], \"f1_score_micro\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"f1_score_micro\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.878959462775807, 0.878959462775807, 0.6560176695000335, 0.6279262878432947, 0.877764986725565, 0.8780626017892599, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.8780626017892599, 0.8813493072752829, 0.8801548312250407, 0.877465140664391, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8708881600963793, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8574437230885931, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.6566030832385159, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.8801548312250407, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8786605091136247, 0.878959462775807, 0.878959462775807, 0.8801552774245364, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8819476607991433, 0.8810507998125964]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"f1_score_micro_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8819476607991433, 0.8819476607991433]}], \"recall_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"recall_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.878959462775807, 0.878959462775807, 0.6560176695000335, 0.6279262878432947, 0.877764986725565, 0.8780626017892599, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.8780626017892599, 0.8813493072752829, 0.8801548312250407, 0.877465140664391, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8708881600963793, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8574437230885931, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.6566030832385159, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.8801548312250407, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8786605091136247, 0.878959462775807, 0.878959462775807, 0.8801552774245364, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8819476607991433, 0.8810507998125964]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"recall_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8819476607991433, 0.8819476607991433]}], \"AUC_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"AUC_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.577715536835611, 0.6305110509347944, 0.6335583604100532, 0.6258116849245801, 0.6356173671204847, 0.6211979632544319, 0.6063063240080939, 0.6181742923711174, 0.6371822492378356, 0.6164143534010468, 0.6342933428487124, 0.6319119220327073, 0.6346125141175203, 0.6251343833428058, 0.6365909697726305, 0.631471251386176, 0.6082539473680374, 0.6323414251284074, 0.6221544194372628, 0.6181780786913151, 0.6055210754747491, 0.6041384331531718, 0.5990316604750665, 0.6334692192242535, 0.6022801747930784, 0.5, 0.6126982694370844, 0.6346861683437994, 0.6142918610053384, 0.6333515277736165, 0.5, 0.6323577336234808, 0.6333514427665932, 0.6148739602964552, 0.6315939148547652, 0.5, 0.6117686850723691, 0.5770577868888347, 0.635412444322883, 0.614089847678077, 0.5, 0.6217766394697619, 0.6160524365480751, 0.6161951128211223, 0.6297145951174972, 0.5, 0.6251513298504217, 0.6334976276501612, 0.6261489475264866, 0.6262829843301319, 0.5, 0.6232867569160379, 0.633576075053597, 0.6227573704938333, 0.6323514175982934, 0.5, 0.6237645598121215, 0.6176379881363026, 0.6403289324259205, 0.6293083772589999, 0.5, 0.6190213810423357, 0.5832462705547183, 0.6130036700540654, 0.6310580926780616, 0.5, 0.6280001285756633, 0.6284817410898801, 0.6361893408398975, 0.6356708712532728, 0.5, 0.6243506896655318, 0.596958741563032, 0.6327004082247988, 0.632017318100363, 0.5, 0.6093574508360327, 0.627083662685884, 0.6139126555507943, 0.6274198054021178, 0.6187331807428572, 0.629385361320245]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"AUC_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.577715536835611, 0.6305110509347944, 0.6335583604100532, 0.6335583604100532, 0.6356173671204847, 0.6356173671204847, 0.6356173671204847, 0.6356173671204847, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6371822492378356, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205, 0.6403289324259205]}], \"precision_score_weighted\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"precision_score_weighted\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8211084127437663, 0.8404274025740228, 0.7725877236727466, 0.7725877236727466, 0.8229296308474398, 0.816921936517325, 0.8355445244264965, 0.8251456830974799, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8331966818185361, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7976305964289804, 0.7725877236727466, 0.8434434437775428, 0.8251456830974799, 0.8480169484715905, 0.8376782366663222, 0.8322422263644794, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8173585094922367, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8107386874950198, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8202485731455192, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8097715851132291, 0.8449346805564147, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.7725555238670319, 0.7725877236727466, 0.7725877236727466, 0.861804237023114, 0.7725877236727466, 0.7725877236727466, 0.7725877236727466, 0.8583053096012531, 0.8606434051869168]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"precision_score_weighted_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8211084127437663, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8404274025740228, 0.8434434437775428, 0.8434434437775428, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.8480169484715905, 0.861804237023114, 0.861804237023114, 0.861804237023114, 0.861804237023114, 0.861804237023114, 0.861804237023114]}], \"accuracy\": [{\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"markers\", \"name\": \"accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.878959462775807, 0.878959462775807, 0.6560176695000335, 0.6279262878432947, 0.877764986725565, 0.8780626017892599, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.878959462775807, 0.878959462775807, 0.8780626017892599, 0.8813493072752829, 0.8801548312250407, 0.877465140664391, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8708881600963793, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8574437230885931, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.6566030832385159, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8792584164379894, 0.8801548312250407, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8786605091136247, 0.878959462775807, 0.878959462775807, 0.8801552774245364, 0.878959462775807, 0.878959462775807, 0.878959462775807, 0.8819476607991433, 0.8810507998125964]}, {\"categories\": [\"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\", \"10\", \"11\", \"12\", \"13\", \"14\", \"15\", \"16\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"28\", \"29\", \"30\", \"31\", \"32\", \"33\", \"34\", \"35\", \"36\", \"37\", \"38\", \"39\", \"40\", \"41\", \"42\", \"43\", \"44\", \"45\", \"46\", \"47\", \"48\", \"49\", \"50\", \"51\", \"52\", \"53\", \"54\", \"55\", \"56\", \"57\", \"58\", \"59\", \"60\", \"61\", \"62\", \"63\", \"64\", \"65\", \"66\", \"67\", \"68\", \"69\", \"70\", \"71\", \"72\", \"73\", \"74\", \"75\", \"76\", \"77\", \"78\", \"79\", \"80\", \"81\"], \"mode\": \"lines\", \"name\": \"accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8723860518037615, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8798549851638668, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8813493072752829, 0.8819476607991433, 0.8819476607991433]}]}, \"metricName\": null, \"primaryMetricName\": \"accuracy\", \"showLegend\": false}, \"run_metrics\": [{\"name\": \"experiment_status\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], \"series\": [{\"data\": [\"DatasetEvaluation\", \"FeaturesGeneration\", \"DatasetFeaturization\", \"DatasetFeaturizationCompleted\", \"DatasetBalancing\", \"DatasetCrossValidationSplit\", \"ModelSelection\", \"BestRunExplainModel\", \"ModelExplanationDataSetSetup\", \"PickSurrogateModel\", \"EngineeredFeatureExplanations\", \"EngineeredFeatureExplanations\", \"RawFeaturesExplanations\", \"RawFeaturesExplanations\", \"BestRunExplainModel\"]}]}, {\"name\": \"experiment_status_description\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], \"series\": [{\"data\": [\"Gathering dataset statistics.\", \"Generating features for the dataset.\", \"Beginning to fit featurizers and featurize the dataset.\", \"Completed fit featurizers and featurizing the dataset.\", \"Performing class balancing sweeping\", \"Generating individually featurized CV splits.\", \"Beginning model selection.\", \"Best run model explanations started\", \"Model explanations data setup completed\", \"Choosing LightGBM as the surrogate model for explanations\", \"Computation of engineered features started\", \"Computation of engineered features completed\", \"Computation of raw features started\", \"Computation of raw features completed\", \"Best run model explanations completed\"]}]}, {\"name\": \"recall_score_macro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.5147719715624961]}]}, {\"name\": \"recall_score_weighted\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8819476607991433]}]}, {\"name\": \"AUC_micro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.9041245675687868]}]}, {\"name\": \"f1_score_micro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8819476607991433]}]}, {\"name\": \"precision_score_weighted\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8583053096012531]}]}, {\"name\": \"average_precision_score_macro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.567234045526078]}]}, {\"name\": \"recall_score_micro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8819476607991433]}]}, {\"name\": \"f1_score_macro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.4973371657510099]}]}, {\"name\": \"precision_score_macro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.7876990337092691]}]}, {\"name\": \"average_precision_score_micro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8850169345226563]}]}, {\"name\": \"norm_macro_recall\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.029543943124992377]}]}, {\"name\": \"AUC_macro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.6187331807428572]}]}, {\"name\": \"average_precision_score_weighted\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8275860830972966]}]}, {\"name\": \"AUC_weighted\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.6187331807428572]}]}, {\"name\": \"balanced_accuracy\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.5147719715624961]}]}, {\"name\": \"f1_score_weighted\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8304641276084948]}]}, {\"name\": \"matthews_correlation\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.1310981942998046]}]}, {\"name\": \"weighted_accuracy\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.9812271957797485]}]}, {\"name\": \"accuracy\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8819476607991433]}]}, {\"name\": \"log_loss\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.3631963089285146]}]}, {\"name\": \"precision_score_micro\", \"run_id\": \"AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5\", \"categories\": [0], \"series\": [{\"data\": [0.8819476607991433]}]}], \"run_logs\": \"\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.20.0\"}, \"loading\": false}"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Current status: FeaturesGeneration. Generating features for the dataset.\n",
      "Current status: DatasetBalancing. Performing class balancing sweeping\n",
      "Current status: DatasetCrossValidationSplit. Generating individually featurized CV splits.\n",
      "Current status: ModelSelection. Beginning model selection.\n",
      "\n",
      "****************************************************************************************************\n",
      "DATA GUARDRAILS: \n",
      "\n",
      "TYPE:         Class balancing detection\n",
      "STATUS:       ALERTED\n",
      "DESCRIPTION:  To decrease model bias, please cancel the current run and fix balancing problem.\n",
      "              Learn more about imbalanced data: https://aka.ms/AutomatedMLImbalancedData\n",
      "DETAILS:      Imbalanced data can lead to a falsely perceived positive effect of a model's accuracy because the input data has bias towards one class.\n",
      "+---------------------------------+---------------------------------+--------------------------------------+\n",
      "|Size of the smallest class       |Name/Label of the smallest class |Number of samples in the training data|\n",
      "+=================================+=================================+======================================+\n",
      "|405                              |1                                |3346                                  |\n",
      "+---------------------------------+---------------------------------+--------------------------------------+\n",
      "\n",
      "****************************************************************************************************\n",
      "\n",
      "TYPE:         Missing feature values imputation\n",
      "STATUS:       PASSED\n",
      "DESCRIPTION:  No feature missing values were detected in the training data.\n",
      "              Learn more about missing value imputation: https://aka.ms/AutomatedMLFeaturization\n",
      "\n",
      "****************************************************************************************************\n",
      "\n",
      "TYPE:         High cardinality feature detection\n",
      "STATUS:       PASSED\n",
      "DESCRIPTION:  Your inputs were analyzed, and no high cardinality features were detected.\n",
      "              Learn more about high cardinality feature handling: https://aka.ms/AutomatedMLFeaturization\n",
      "\n",
      "****************************************************************************************************\n",
      "\n",
      "****************************************************************************************************\n",
      "ITERATION: The iteration being evaluated.\n",
      "PIPELINE: A summary description of the pipeline being evaluated.\n",
      "DURATION: Time taken for the current iteration.\n",
      "METRIC: The result of computing score on the fitted pipeline.\n",
      "BEST: The best observed score thus far.\n",
      "****************************************************************************************************\n",
      "\n",
      " ITERATION   PIPELINE                                       DURATION      METRIC      BEST\n",
      "         2   MaxAbsScaler RandomForest                      0:00:50       0.8790    0.8790\n",
      "         0   MaxAbsScaler LightGBM                          0:00:58       0.8724    0.8790\n",
      "         6   SparseNormalizer XGBoostClassifier             0:00:59       0.8778    0.8790\n",
      "         7   MaxAbsScaler LogisticRegression                0:00:55       0.8781    0.8790\n",
      "         1   MaxAbsScaler XGBoostClassifier                 0:03:18       0.8799    0.8799\n",
      "         3   MaxAbsScaler RandomForest                      0:03:26       0.8790    0.8799\n",
      "         4   MaxAbsScaler RandomForest                      0:03:33       0.6560    0.8799\n",
      "         5   MaxAbsScaler RandomForest                      0:03:23       0.6279    0.8799\n",
      "         8   MaxAbsScaler RandomForest                      0:00:53       0.8790    0.8799\n",
      "         9   SparseNormalizer LightGBM                      0:00:53       0.8790    0.8799\n",
      "        10   StandardScalerWrapper RandomForest             0:00:51       0.8790    0.8799\n",
      "        11   StandardScalerWrapper LightGBM                 0:00:53       0.8790    0.8799\n",
      "        12   StandardScalerWrapper XGBoostClassifier        0:00:56       0.8793    0.8799\n",
      "        13   SparseNormalizer RandomForest                  0:00:56       0.8790    0.8799\n",
      "        14   MaxAbsScaler RandomForest                      0:01:01       0.8790    0.8799\n",
      "        15   MaxAbsScaler LightGBM                          0:00:52       0.8790    0.8799\n",
      "        16   SparseNormalizer XGBoostClassifier             0:00:50       0.8793    0.8799\n",
      "        17   MaxAbsScaler GradientBoosting                  0:00:50       0.8790    0.8799\n",
      "        18   MaxAbsScaler LogisticRegression                0:00:53       0.8790    0.8799\n",
      "        19   MaxAbsScaler LogisticRegression                0:00:51       0.8781    0.8799\n",
      "        21   SparseNormalizer XGBoostClassifier             0:00:56       0.8802    0.8802\n",
      "        22   SparseNormalizer XGBoostClassifier             0:00:51       0.8775    0.8802\n",
      "        20   SparseNormalizer XGBoostClassifier             0:00:57       0.8813    0.8813\n",
      "        23   StandardScalerWrapper ExtremeRandomTrees       0:00:56       0.8790    0.8813\n",
      "        24   SparseNormalizer GradientBoosting              0:00:55       0.8790    0.8813\n",
      "        25   SparseNormalizer XGBoostClassifier             0:00:46       0.8790    0.8813\n",
      "        27   StandardScalerWrapper LightGBM                 0:00:46       0.8790    0.8813\n",
      "        26   SparseNormalizer LightGBM                      0:00:56       0.8790    0.8813\n",
      "        28   SparseNormalizer LightGBM                      0:00:46       0.8790    0.8813\n",
      "        29   MaxAbsScaler LightGBM                          0:00:47       0.8790    0.8813\n",
      "        30   SparseNormalizer XGBoostClassifier             0:00:48       0.8790    0.8813\n",
      "        31   MaxAbsScaler RandomForest                      0:00:47       0.8790    0.8813\n",
      "        32   MaxAbsScaler ExtremeRandomTrees                0:00:48       0.8790    0.8813\n",
      "        33   StandardScalerWrapper XGBoostClassifier        0:00:54       0.8709    0.8813\n",
      "        34   StandardScalerWrapper LightGBM                 0:00:48       0.8790    0.8813\n",
      "        35   SparseNormalizer XGBoostClassifier             0:00:47       0.8790    0.8813\n",
      "        36   SparseNormalizer LightGBM                      0:00:50       0.8790    0.8813\n",
      "        37   StandardScalerWrapper XGBoostClassifier        0:00:58       0.8574    0.8813\n",
      "        38   StandardScalerWrapper LightGBM                 0:00:47       0.8790    0.8813\n",
      "        39   SparseNormalizer LightGBM                      0:01:02       0.8790    0.8813\n",
      "        40   StandardScalerWrapper XGBoostClassifier        0:00:49       0.8790    0.8813\n",
      "        41   StandardScalerWrapper LightGBM                 0:00:54       0.8790    0.8813\n",
      "        43   MaxAbsScaler GradientBoosting                  0:00:48       0.8790    0.8813\n",
      "        42   SparseNormalizer LightGBM                      0:00:55       0.8790    0.8813\n",
      "        44   StandardScalerWrapper LightGBM                 0:00:47       0.8790    0.8813\n",
      "        45   TruncatedSVDWrapper XGBoostClassifier          0:00:57       0.8790    0.8813\n",
      "        46   SparseNormalizer GradientBoosting              0:01:01       0.8790    0.8813\n",
      "        47   StandardScalerWrapper LightGBM                 0:00:48       0.8790    0.8813\n",
      "        48   StandardScalerWrapper LightGBM                 0:00:53       0.8790    0.8813\n",
      "        49   StandardScalerWrapper LightGBM                 0:00:53       0.8790    0.8813\n",
      "        50   SparseNormalizer XGBoostClassifier             0:00:46       0.8790    0.8813\n",
      "        51   SparseNormalizer LightGBM                      0:00:50       0.8790    0.8813\n",
      "        53   MaxAbsScaler LogisticRegression                0:00:47       0.6566    0.8813\n",
      "        52   StandardScalerWrapper GradientBoosting         0:00:57       0.8790    0.8813\n",
      "        54   MaxAbsScaler LightGBM                          0:00:47       0.8790    0.8813\n",
      "        55   SparseNormalizer XGBoostClassifier             0:00:55       0.8790    0.8813\n",
      "        57   MaxAbsScaler LightGBM                          0:00:48       0.8790    0.8813\n",
      "        56   MaxAbsScaler LightGBM                          0:00:59       0.8790    0.8813\n",
      "        58   MaxAbsScaler LightGBM                          0:00:50       0.8790    0.8813\n",
      "        59   MaxAbsScaler LightGBM                          0:00:46       0.8790    0.8813\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        60   StandardScalerWrapper XGBoostClassifier        0:00:56       0.8790    0.8813\n",
      "        61   SparseNormalizer LightGBM                      0:00:54       0.8790    0.8813\n",
      "        62   SparseNormalizer XGBoostClassifier             0:00:53       0.8793    0.8813\n",
      "        63   StandardScalerWrapper XGBoostClassifier        0:00:48       0.8802    0.8813\n",
      "        64   MaxAbsScaler LightGBM                          0:00:49       0.8790    0.8813\n",
      "        65   StandardScalerWrapper XGBoostClassifier        0:00:53       0.8790    0.8813\n",
      "        66   SparseNormalizer XGBoostClassifier             0:00:49       0.8790    0.8813\n",
      "        67   StandardScalerWrapper LightGBM                 0:00:59       0.8790    0.8813\n",
      "        68   StandardScalerWrapper RandomForest             0:00:52       0.8790    0.8813\n",
      "        69   MaxAbsScaler LightGBM                          0:00:45       0.8790    0.8813\n",
      "        71   SparseNormalizer LightGBM                      0:00:45       0.8790    0.8813\n",
      "        70   StandardScalerWrapper XGBoostClassifier        0:00:59       0.8790    0.8813\n",
      "        72   MaxAbsScaler GradientBoosting                  0:00:54       0.8790    0.8813\n",
      "        73   StandardScalerWrapper XGBoostClassifier        0:00:51       0.8787    0.8813\n",
      "        74   StandardScalerWrapper LightGBM                 0:00:49       0.8790    0.8813\n",
      "        75   TruncatedSVDWrapper XGBoostClassifier          0:00:52       0.8790    0.8813\n",
      "        76   SparseNormalizer XGBoostClassifier             0:00:48       0.8802    0.8813\n",
      "ERROR: {\n",
      "    \"additional_properties\": {},\n",
      "    \"error\": {\n",
      "        \"additional_properties\": {\n",
      "            \"debugInfo\": null\n",
      "        },\n",
      "        \"code\": \"UserError\",\n",
      "        \"severity\": null,\n",
      "        \"message\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_format\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_parameters\": {},\n",
      "        \"reference_code\": null,\n",
      "        \"details_uri\": null,\n",
      "        \"target\": null,\n",
      "        \"details\": [],\n",
      "        \"inner_error\": {\n",
      "            \"additional_properties\": {},\n",
      "            \"code\": \"ResourceExhausted\",\n",
      "            \"inner_error\": {\n",
      "                \"additional_properties\": {},\n",
      "                \"code\": \"Timeout\",\n",
      "                \"inner_error\": {\n",
      "                    \"additional_properties\": {},\n",
      "                    \"code\": \"ExperimentTimeoutForIterations\",\n",
      "                    \"inner_error\": null\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"correlation\": null,\n",
      "    \"environment\": null,\n",
      "    \"location\": null,\n",
      "    \"time\": {},\n",
      "    \"component_name\": null\n",
      "}\n",
      "        77   MaxAbsScaler LightGBM                          0:00:46       0.8790    0.8813\n",
      "ERROR: {\n",
      "    \"additional_properties\": {},\n",
      "    \"error\": {\n",
      "        \"additional_properties\": {\n",
      "            \"debugInfo\": null\n",
      "        },\n",
      "        \"code\": \"UserError\",\n",
      "        \"severity\": null,\n",
      "        \"message\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_format\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_parameters\": {},\n",
      "        \"reference_code\": null,\n",
      "        \"details_uri\": null,\n",
      "        \"target\": null,\n",
      "        \"details\": [],\n",
      "        \"inner_error\": {\n",
      "            \"additional_properties\": {},\n",
      "            \"code\": \"ResourceExhausted\",\n",
      "            \"inner_error\": {\n",
      "                \"additional_properties\": {},\n",
      "                \"code\": \"Timeout\",\n",
      "                \"inner_error\": {\n",
      "                    \"additional_properties\": {},\n",
      "                    \"code\": \"ExperimentTimeoutForIterations\",\n",
      "                    \"inner_error\": null\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"correlation\": null,\n",
      "    \"environment\": null,\n",
      "    \"location\": null,\n",
      "    \"time\": {},\n",
      "    \"component_name\": null\n",
      "}\n",
      "        78   SparseNormalizer LightGBM                      0:00:44       0.8790    0.8813\n",
      "ERROR: {\n",
      "    \"additional_properties\": {},\n",
      "    \"error\": {\n",
      "        \"additional_properties\": {\n",
      "            \"debugInfo\": null\n",
      "        },\n",
      "        \"code\": \"UserError\",\n",
      "        \"severity\": null,\n",
      "        \"message\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_format\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_parameters\": {},\n",
      "        \"reference_code\": null,\n",
      "        \"details_uri\": null,\n",
      "        \"target\": null,\n",
      "        \"details\": [],\n",
      "        \"inner_error\": {\n",
      "            \"additional_properties\": {},\n",
      "            \"code\": \"ResourceExhausted\",\n",
      "            \"inner_error\": {\n",
      "                \"additional_properties\": {},\n",
      "                \"code\": \"Timeout\",\n",
      "                \"inner_error\": {\n",
      "                    \"additional_properties\": {},\n",
      "                    \"code\": \"ExperimentTimeoutForIterations\",\n",
      "                    \"inner_error\": null\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"correlation\": null,\n",
      "    \"environment\": null,\n",
      "    \"location\": null,\n",
      "    \"time\": {},\n",
      "    \"component_name\": null\n",
      "}\n",
      "        79   MaxAbsScaler LightGBM                          0:00:36       0.8790    0.8813\n",
      "ERROR: {\n",
      "    \"additional_properties\": {},\n",
      "    \"error\": {\n",
      "        \"additional_properties\": {\n",
      "            \"debugInfo\": null\n",
      "        },\n",
      "        \"code\": \"UserError\",\n",
      "        \"severity\": null,\n",
      "        \"message\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_format\": \"Experiment timeout reached, please consider increasing your experiment timeout.\",\n",
      "        \"message_parameters\": {},\n",
      "        \"reference_code\": null,\n",
      "        \"details_uri\": null,\n",
      "        \"target\": null,\n",
      "        \"details\": [],\n",
      "        \"inner_error\": {\n",
      "            \"additional_properties\": {},\n",
      "            \"code\": \"ResourceExhausted\",\n",
      "            \"inner_error\": {\n",
      "                \"additional_properties\": {},\n",
      "                \"code\": \"Timeout\",\n",
      "                \"inner_error\": {\n",
      "                    \"additional_properties\": {},\n",
      "                    \"code\": \"ExperimentTimeoutForIterations\",\n",
      "                    \"inner_error\": null\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"correlation\": null,\n",
      "    \"environment\": null,\n",
      "    \"location\": null,\n",
      "    \"time\": {},\n",
      "    \"component_name\": null\n",
      "}\n",
      "        80    VotingEnsemble                                0:01:47       0.8819    0.8819\n",
      "        81    StackEnsemble                                 0:01:55       0.8811    0.8819\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'runId': 'AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5',\n",
       " 'target': 'ccp03aml',\n",
       " 'status': 'Completed',\n",
       " 'startTimeUtc': '2021-01-29T12:15:59.253016Z',\n",
       " 'endTimeUtc': '2021-01-29T12:47:19.976252Z',\n",
       " 'properties': {'num_iterations': '1000',\n",
       "  'training_type': 'TrainFull',\n",
       "  'acquisition_function': 'EI',\n",
       "  'primary_metric': 'accuracy',\n",
       "  'train_split': '0',\n",
       "  'acquisition_parameter': '0',\n",
       "  'num_cross_validation': '5',\n",
       "  'target': 'ccp03aml',\n",
       "  'AMLSettingsJsonString': '{\"path\":null,\"name\":\"expp03ht\",\"subscription_id\":\"1b944a9b-fdae-4f97-aeb1-b7eea0beac53\",\"resource_group\":\"aml-quickstarts-136496\",\"workspace_name\":\"quick-starts-ws-136496\",\"region\":\"southcentralus\",\"compute_target\":\"ccp03aml\",\"spark_service\":null,\"azure_service\":\"remote\",\"many_models\":false,\"pipeline_fetch_max_batch_size\":1,\"iterations\":1000,\"primary_metric\":\"accuracy\",\"task_type\":\"classification\",\"data_script\":null,\"validation_size\":0.0,\"n_cross_validations\":5,\"y_min\":null,\"y_max\":null,\"num_classes\":null,\"featurization\":\"auto\",\"_ignore_package_version_incompatibilities\":false,\"is_timeseries\":false,\"max_cores_per_iteration\":1,\"max_concurrent_iterations\":5,\"iteration_timeout_minutes\":null,\"mem_in_mb\":null,\"enforce_time_on_windows\":false,\"experiment_timeout_minutes\":20,\"experiment_exit_score\":null,\"whitelist_models\":null,\"blacklist_algos\":[\"TensorFlowLinearClassifier\",\"TensorFlowDNN\"],\"supported_models\":[\"LightGBM\",\"AveragedPerceptronClassifier\",\"LogisticRegression\",\"SGD\",\"BernoulliNaiveBayes\",\"MultinomialNaiveBayes\",\"KNN\",\"RandomForest\",\"ExtremeRandomTrees\",\"SVM\",\"XGBoostClassifier\",\"GradientBoosting\",\"TensorFlowLinearClassifier\",\"TensorFlowDNN\",\"DecisionTree\",\"LinearSVM\"],\"auto_blacklist\":true,\"blacklist_samples_reached\":false,\"exclude_nan_labels\":true,\"verbosity\":20,\"_debug_log\":\"azureml_automl.log\",\"show_warnings\":false,\"model_explainability\":true,\"service_url\":null,\"sdk_url\":null,\"sdk_packages\":null,\"enable_onnx_compatible_models\":false,\"enable_split_onnx_featurizer_estimator_models\":false,\"vm_type\":\"STANDARD_D2_V2\",\"telemetry_verbosity\":20,\"send_telemetry\":true,\"enable_dnn\":false,\"scenario\":\"SDK-1.13.0\",\"environment_label\":null,\"force_text_dnn\":false,\"enable_feature_sweeping\":true,\"enable_early_stopping\":false,\"early_stopping_n_iters\":10,\"metrics\":null,\"enable_ensembling\":true,\"enable_stack_ensembling\":true,\"ensemble_iterations\":15,\"enable_tf\":false,\"enable_subsampling\":null,\"subsample_seed\":null,\"enable_nimbusml\":false,\"enable_streaming\":false,\"force_streaming\":false,\"track_child_runs\":true,\"allowed_private_models\":[],\"label_column_name\":\"sexo\",\"weight_column_name\":null,\"cv_split_column_names\":null,\"enable_local_managed\":false,\"_local_managed_run_id\":null,\"cost_mode\":1,\"lag_length\":0,\"metric_operation\":\"maximize\",\"preprocess\":true}',\n",
       "  'DataPrepJsonString': '{\\\\\"training_data\\\\\": \\\\\"{\\\\\\\\\\\\\"blocks\\\\\\\\\\\\\": [{\\\\\\\\\\\\\"id\\\\\\\\\\\\\": \\\\\\\\\\\\\"d781c78a-7bc1-4170-8771-ec781cb006f0\\\\\\\\\\\\\", \\\\\\\\\\\\\"type\\\\\\\\\\\\\": \\\\\\\\\\\\\"Microsoft.DPrep.GetFilesBlock\\\\\\\\\\\\\", \\\\\\\\\\\\\"arguments\\\\\\\\\\\\\": {\\\\\\\\\\\\\"isArchive\\\\\\\\\\\\\": false, \\\\\\\\\\\\\"path\\\\\\\\\\\\\": {\\\\\\\\\\\\\"target\\\\\\\\\\\\\": 4, \\\\\\\\\\\\\"resourceDetails\\\\\\\\\\\\\": [{\\\\\\\\\\\\\"path\\\\\\\\\\\\\": \\\\\\\\\\\\\"https://raw.githubusercontent.com/hftamayo/azuremlproj03/main/cad2018.csv\\\\\\\\\\\\\"}]}}, \\\\\\\\\\\\\"localData\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\": true, \\\\\\\\\\\\\"name\\\\\\\\\\\\\": null, \\\\\\\\\\\\\"annotation\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\"id\\\\\\\\\\\\\": \\\\\\\\\\\\\"e0ac160c-c9ef-40fb-9a8e-6bc8635d5908\\\\\\\\\\\\\", \\\\\\\\\\\\\"type\\\\\\\\\\\\\": \\\\\\\\\\\\\"Microsoft.DPrep.ParseDelimitedBlock\\\\\\\\\\\\\", \\\\\\\\\\\\\"arguments\\\\\\\\\\\\\": {\\\\\\\\\\\\\"columnHeadersMode\\\\\\\\\\\\\": 3, \\\\\\\\\\\\\"fileEncoding\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\"handleQuotedLineBreaks\\\\\\\\\\\\\": false, \\\\\\\\\\\\\"preview\\\\\\\\\\\\\": false, \\\\\\\\\\\\\"separator\\\\\\\\\\\\\": \\\\\\\\\\\\\",\\\\\\\\\\\\\", \\\\\\\\\\\\\"skipRows\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\"skipRowsMode\\\\\\\\\\\\\": 0}, \\\\\\\\\\\\\"localData\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\": true, \\\\\\\\\\\\\"name\\\\\\\\\\\\\": null, \\\\\\\\\\\\\"annotation\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\"id\\\\\\\\\\\\\": \\\\\\\\\\\\\"66a5e446-94f3-4f37-b0f0-4404b27c2559\\\\\\\\\\\\\", \\\\\\\\\\\\\"type\\\\\\\\\\\\\": \\\\\\\\\\\\\"Microsoft.DPrep.DropColumnsBlock\\\\\\\\\\\\\", \\\\\\\\\\\\\"arguments\\\\\\\\\\\\\": {\\\\\\\\\\\\\"columns\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 0, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumns\\\\\\\\\\\\\": [\\\\\\\\\\\\\"Path\\\\\\\\\\\\\"]}}}, \\\\\\\\\\\\\"localData\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\": true, \\\\\\\\\\\\\"name\\\\\\\\\\\\\": null, \\\\\\\\\\\\\"annotation\\\\\\\\\\\\\": null}, {\\\\\\\\\\\\\"id\\\\\\\\\\\\\": \\\\\\\\\\\\\"be0bb52d-5419-4c3a-880a-6086d37b8cfd\\\\\\\\\\\\\", \\\\\\\\\\\\\"type\\\\\\\\\\\\\": \\\\\\\\\\\\\"Microsoft.DPrep.SetColumnTypesBlock\\\\\\\\\\\\\", \\\\\\\\\\\\\"arguments\\\\\\\\\\\\\": {\\\\\\\\\\\\\"columnConversion\\\\\\\\\\\\\": [{\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"id\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"regfalle\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"edad\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"sexo\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"deptoocuhe\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"tipoarma\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}, {\\\\\\\\\\\\\"column\\\\\\\\\\\\\": {\\\\\\\\\\\\\"type\\\\\\\\\\\\\": 2, \\\\\\\\\\\\\"details\\\\\\\\\\\\\": {\\\\\\\\\\\\\"selectedColumn\\\\\\\\\\\\\": \\\\\\\\\\\\\"pracaut\\\\\\\\\\\\\"}}, \\\\\\\\\\\\\"typeProperty\\\\\\\\\\\\\": 2}]}, \\\\\\\\\\\\\"localData\\\\\\\\\\\\\": {}, \\\\\\\\\\\\\"isEnabled\\\\\\\\\\\\\": true, \\\\\\\\\\\\\"name\\\\\\\\\\\\\": null, \\\\\\\\\\\\\"annotation\\\\\\\\\\\\\": null}], \\\\\\\\\\\\\"inspectors\\\\\\\\\\\\\": [], \\\\\\\\\\\\\"meta\\\\\\\\\\\\\": {\\\\\\\\\\\\\"savedDatasetId\\\\\\\\\\\\\": \\\\\\\\\\\\\"83126fd9-f456-48a6-ae73-466351bfaa69\\\\\\\\\\\\\", \\\\\\\\\\\\\"datasetType\\\\\\\\\\\\\": \\\\\\\\\\\\\"tabular\\\\\\\\\\\\\", \\\\\\\\\\\\\"subscriptionId\\\\\\\\\\\\\": \\\\\\\\\\\\\"1b944a9b-fdae-4f97-aeb1-b7eea0beac53\\\\\\\\\\\\\", \\\\\\\\\\\\\"workspaceId\\\\\\\\\\\\\": \\\\\\\\\\\\\"2bc4379e-e9ea-4b1a-9fb9-b7297d2b4a58\\\\\\\\\\\\\", \\\\\\\\\\\\\"workspaceLocation\\\\\\\\\\\\\": \\\\\\\\\\\\\"southcentralus\\\\\\\\\\\\\"}}\\\\\", \\\\\"activities\\\\\": 0}',\n",
       "  'EnableSubsampling': None,\n",
       "  'runTemplate': 'AutoML',\n",
       "  'azureml.runsource': 'automl',\n",
       "  'display_task_type': 'classification',\n",
       "  'dependencies_versions': '{\"azureml-widgets\": \"1.20.0\", \"azureml-train\": \"1.20.0\", \"azureml-train-restclients-hyperdrive\": \"1.20.0\", \"azureml-train-core\": \"1.20.0\", \"azureml-train-automl\": \"1.20.0\", \"azureml-train-automl-runtime\": \"1.20.0\", \"azureml-train-automl-client\": \"1.20.0\", \"azureml-tensorboard\": \"1.20.0\", \"azureml-telemetry\": \"1.20.0\", \"azureml-sdk\": \"1.20.0\", \"azureml-samples\": \"0+unknown\", \"azureml-pipeline\": \"1.20.0\", \"azureml-pipeline-steps\": \"1.20.0\", \"azureml-pipeline-core\": \"1.20.0\", \"azureml-opendatasets\": \"1.20.0\", \"azureml-model-management-sdk\": \"1.0.1b6.post1\", \"azureml-mlflow\": \"1.20.0.post1\", \"azureml-interpret\": \"1.20.0\", \"azureml-explain-model\": \"1.20.0\", \"azureml-defaults\": \"1.20.0\", \"azureml-dataset-runtime\": \"1.20.0\", \"azureml-dataprep\": \"2.7.3\", \"azureml-dataprep-rslex\": \"1.5.0\", \"azureml-dataprep-native\": \"27.0.0\", \"azureml-datadrift\": \"1.20.0\", \"azureml-core\": \"1.20.0\", \"azureml-contrib-services\": \"1.20.0\", \"azureml-contrib-server\": \"1.20.0\", \"azureml-contrib-reinforcementlearning\": \"1.20.0\", \"azureml-contrib-pipeline-steps\": \"1.20.0\", \"azureml-contrib-notebook\": \"1.20.0\", \"azureml-contrib-interpret\": \"1.20.0\", \"azureml-contrib-gbdt\": \"1.20.0\", \"azureml-contrib-fairness\": \"1.20.0\", \"azureml-contrib-dataset\": \"1.20.0\", \"azureml-cli-common\": \"1.20.0\", \"azureml-automl-runtime\": \"1.20.0\", \"azureml-automl-core\": \"1.20.0\", \"azureml-accel-models\": \"1.20.0\"}',\n",
       "  '_aml_system_scenario_identification': 'Remote.Parent',\n",
       "  'ClientType': 'SDK',\n",
       "  'environment_cpu_name': 'AzureML-AutoML',\n",
       "  'environment_cpu_label': 'prod',\n",
       "  'environment_gpu_name': 'AzureML-AutoML-GPU',\n",
       "  'environment_gpu_label': 'prod',\n",
       "  'root_attribution': 'automl',\n",
       "  'attribution': 'AutoML',\n",
       "  'Orchestrator': 'AutoML',\n",
       "  'CancelUri': 'https://southcentralus.experiments.azureml.net/jasmine/v1.0/subscriptions/1b944a9b-fdae-4f97-aeb1-b7eea0beac53/resourceGroups/aml-quickstarts-136496/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-136496/experimentids/7229f264-e173-4e00-b190-95618964ceaf/cancel/AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5',\n",
       "  'ClientSdkVersion': '1.20.0.post1',\n",
       "  'snapshotId': '00000000-0000-0000-0000-000000000000',\n",
       "  'SetupRunId': 'AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_setup',\n",
       "  'SetupRunContainerId': 'dcid.AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_setup',\n",
       "  'FeaturizationRunJsonPath': 'featurizer_container.json',\n",
       "  'FeaturizationRunId': 'AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_featurize',\n",
       "  'ProblemInfoJsonString': '{\"dataset_num_categorical\": 0, \"is_sparse\": true, \"subsampling\": false, \"dataset_classes\": 2, \"dataset_features\": 34, \"dataset_samples\": 3346, \"single_frequency_class_detected\": false}',\n",
       "  'ModelExplainRunId': 'AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_ModelExplain'},\n",
       " 'inputDatasets': [{'dataset': {'id': '83126fd9-f456-48a6-ae73-466351bfaa69'}, 'consumptionDetails': {'type': 'RunInput', 'inputName': 'training_data', 'mechanism': 'Direct'}}],\n",
       " 'outputDatasets': [],\n",
       " 'logFiles': {},\n",
       " 'submittedBy': 'ODL_User 136496'}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RunDetails(exprun).show()\n",
    "exprun.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Best Model\n",
    "\n",
    "TODO: In the cell below, get the best model from the automl experiments and display all the properties of the model.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "gather": {
     "logged": 1598431425670
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Run(Experiment: expp03ht,\n",
      "Id: AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80,\n",
      "Type: azureml.scriptrun,\n",
      "Status: Completed)\n",
      "precision_score_micro 0.8819476607991433\n",
      "recall_score_macro 0.5147719715624961\n",
      "f1_score_weighted 0.8304641276084948\n",
      "accuracy 0.8819476607991433\n",
      "weighted_accuracy 0.9812271957797485\n",
      "AUC_weighted 0.6187331807428572\n",
      "log_loss 0.3631963089285146\n",
      "recall_score_weighted 0.8819476607991433\n",
      "balanced_accuracy 0.5147719715624961\n",
      "matthews_correlation 0.1310981942998046\n",
      "AUC_macro 0.6187331807428572\n",
      "precision_score_macro 0.7876990337092691\n",
      "average_precision_score_micro 0.8850169345226563\n",
      "f1_score_macro 0.4973371657510099\n",
      "f1_score_micro 0.8819476607991433\n",
      "average_precision_score_macro 0.567234045526078\n",
      "precision_score_weighted 0.8583053096012531\n",
      "recall_score_micro 0.8819476607991433\n",
      "norm_macro_recall 0.029543943124992377\n",
      "AUC_micro 0.9041245675687868\n",
      "average_precision_score_weighted 0.8275860830972966\n",
      "accuracy_table aml://artifactId/ExperimentRun/dcid.AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80/accuracy_table\n",
      "confusion_matrix aml://artifactId/ExperimentRun/dcid.AutoML_9038d9fb-0e41-4e62-95b0-a349f7ddbda5_80/confusion_matrix\n"
     ]
    }
   ],
   "source": [
    "#import azureml.train.automl\n",
    "# obtaining best run and fitted model\n",
    "brmodel, fmodel = exprun.get_output()\n",
    "\n",
    "# Print the best run\n",
    "print(brmodel)\n",
    "\n",
    "# Get all metrics of the best run model\n",
    "brmetrics = brmodel.get_metrics()\n",
    "\n",
    "# Print all metrics of the best run model\n",
    "for metric_name in brmetrics:\n",
    "    metric = brmetrics[metric_name]\n",
    "    print(metric_name, metric)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "datatransformer\n",
      "{'enable_dnn': None,\n",
      " 'enable_feature_sweeping': None,\n",
      " 'feature_sweeping_config': None,\n",
      " 'feature_sweeping_timeout': None,\n",
      " 'featurization_config': None,\n",
      " 'force_text_dnn': None,\n",
      " 'is_cross_validation': None,\n",
      " 'is_onnx_compatible': None,\n",
      " 'logger': None,\n",
      " 'observer': None,\n",
      " 'task': None,\n",
      " 'working_dir': None}\n",
      "\n",
      "prefittedsoftvotingclassifier\n",
      "{'estimators': ['20', '76', '63', '21', '1', '16'],\n",
      " 'weights': [0.4, 0.2, 0.1, 0.1, 0.1, 0.1]}\n",
      "\n",
      "20 - sparsenormalizer\n",
      "{'copy': True, 'norm': 'max'}\n",
      "\n",
      "20 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 1,\n",
      " 'eta': 0.2,\n",
      " 'gamma': 0,\n",
      " 'learning_rate': 0.1,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 6,\n",
      " 'max_leaves': 15,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 25,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'reg:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 0,\n",
      " 'reg_lambda': 0.4166666666666667,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 1,\n",
      " 'tree_method': 'auto',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n",
      "76 - sparsenormalizer\n",
      "{'copy': True, 'norm': 'l2'}\n",
      "\n",
      "76 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 1,\n",
      " 'eta': 0.001,\n",
      " 'gamma': 5,\n",
      " 'learning_rate': 0.1,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 6,\n",
      " 'max_leaves': 7,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 200,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'reg:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 1.4583333333333335,\n",
      " 'reg_lambda': 0.625,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 0.6,\n",
      " 'tree_method': 'auto',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n",
      "63 - standardscalerwrapper\n",
      "{'class_name': 'StandardScaler',\n",
      " 'copy': True,\n",
      " 'module_name': 'sklearn.preprocessing._data',\n",
      " 'with_mean': False,\n",
      " 'with_std': False}\n",
      "\n",
      "63 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 0.7,\n",
      " 'eta': 0.3,\n",
      " 'gamma': 0,\n",
      " 'grow_policy': 'lossguide',\n",
      " 'learning_rate': 0.1,\n",
      " 'max_bin': 1023,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 2,\n",
      " 'max_leaves': 0,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 10,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'reg:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 0.9375,\n",
      " 'reg_lambda': 1.0416666666666667,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 1,\n",
      " 'tree_method': 'hist',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n",
      "21 - sparsenormalizer\n",
      "{'copy': True, 'norm': 'max'}\n",
      "\n",
      "21 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 1,\n",
      " 'eta': 0.3,\n",
      " 'gamma': 10,\n",
      " 'learning_rate': 0.1,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 6,\n",
      " 'max_leaves': 7,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 100,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'reg:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 0,\n",
      " 'reg_lambda': 0.3125,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 0.5,\n",
      " 'tree_method': 'auto',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n",
      "1 - maxabsscaler\n",
      "{'copy': True}\n",
      "\n",
      "1 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 1,\n",
      " 'gamma': 0,\n",
      " 'learning_rate': 0.1,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 3,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 100,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'binary:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 0,\n",
      " 'reg_lambda': 1,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 1,\n",
      " 'tree_method': 'auto',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n",
      "16 - sparsenormalizer\n",
      "{'copy': True, 'norm': 'l1'}\n",
      "\n",
      "16 - xgboostclassifier\n",
      "{'base_score': 0.5,\n",
      " 'booster': 'gbtree',\n",
      " 'colsample_bylevel': 1,\n",
      " 'colsample_bynode': 1,\n",
      " 'colsample_bytree': 0.7,\n",
      " 'eta': 0.3,\n",
      " 'gamma': 5,\n",
      " 'grow_policy': 'lossguide',\n",
      " 'learning_rate': 0.1,\n",
      " 'max_bin': 255,\n",
      " 'max_delta_step': 0,\n",
      " 'max_depth': 0,\n",
      " 'max_leaves': 127,\n",
      " 'min_child_weight': 1,\n",
      " 'missing': nan,\n",
      " 'n_estimators': 10,\n",
      " 'n_jobs': 1,\n",
      " 'nthread': None,\n",
      " 'objective': 'reg:logistic',\n",
      " 'random_state': 0,\n",
      " 'reg_alpha': 1.9791666666666667,\n",
      " 'reg_lambda': 0.10416666666666667,\n",
      " 'scale_pos_weight': 1,\n",
      " 'seed': None,\n",
      " 'silent': None,\n",
      " 'subsample': 0.9,\n",
      " 'tree_method': 'hist',\n",
      " 'verbose': -10,\n",
      " 'verbosity': 0}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Display parameters of the fitted model\n",
    "def print_model(model, prefix=\"\"):\n",
    "    for step in model.steps:\n",
    "        print(prefix + step[0])\n",
    "        if hasattr(step[1], 'estimators') and hasattr(step[1], 'weights'):\n",
    "            pprint({'estimators': list(\n",
    "                e[0] for e in step[1].estimators), 'weights': step[1].weights})\n",
    "            print()\n",
    "            for estimator in step[1].estimators:\n",
    "                print_model(estimator[1], estimator[0] + ' - ')\n",
    "        else:\n",
    "            pprint(step[1].get_params())\n",
    "            print()\n",
    "\n",
    "print_model(fmodel)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['model.pkl']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import joblib\n",
    "joblib.dump(value=fmodel, filename='model.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model(workspace=Workspace.create(name='quick-starts-ws-136496', subscription_id='1b944a9b-fdae-4f97-aeb1-b7eea0beac53', resource_group='aml-quickstarts-136496'), name=p3automl_ht, id=p3automl_ht:2, version=2, tags={'Method of execution': 'Auto ML'}, properties={'Accuracy': '0.8819476607991433'})\n"
     ]
    }
   ],
   "source": [
    "#TODO: Save the best model\n",
    "autobmlp3ht = brmodel.register_model(model_path='outputs/model.pkl', model_name='p3automl_ht',\n",
    "                        tags={'Method of execution':'Auto ML'},\n",
    "                        properties={'Accuracy': brmetrics['accuracy']})\n",
    "\n",
    "print(autobmlp3ht)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Deployment\n",
    "\n",
    "Remember you have to deploy only one of the two models you trained.. Perform the steps in the rest of this notebook only if you wish to deploy this model.\n",
    "\n",
    "TODO: In the cell below, register the model, create an inference config and deploy the model as a web service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download scoring file \n",
    "brmodel.download_file('outputs/scoring_file_v_1_0_0.py', 'scoring.py')\n",
    "\n",
    "# Download environment file\n",
    "brmodel.download_file('outputs/conda_env_v_1_0_0.yml', 'enviro.yml')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tips: You can try get_logs(): https://aka.ms/debugimage#dockerlog or local deployment: https://aka.ms/debugimage#debug-locally to debug if deployment takes longer than 10 minutes.\n",
      "Running.............................................\n",
      "Succeeded\n",
      "ACI service creation operation finished, operation \"Succeeded\"\n",
      "Healthy\n",
      "http://94e18276-4983-4b4f-925f-845d23fa7c95.southcentralus.azurecontainer.io/score\n",
      "http://94e18276-4983-4b4f-925f-845d23fa7c95.southcentralus.azurecontainer.io/swagger.json\n"
     ]
    }
   ],
   "source": [
    "from azureml.core.environment import Environment\n",
    "from azureml.core.model import InferenceConfig\n",
    "\n",
    "infcfg = InferenceConfig(entry_script='scoring.py',\n",
    "                                    environment=Environment.from_conda_specification(name='myenv',file_path='enviro.yml'))\n",
    "\n",
    "# deploying the model via WebService\n",
    "from azureml.core.webservice import AciWebservice\n",
    "\n",
    "depcfg = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)\n",
    "webservice = Model.deploy(ws, \"wsp03ht01\", [autobmlp3ht], infcfg, depcfg)\n",
    "webservice.wait_for_deployment(show_output = True)\n",
    "print(webservice.state)\n",
    "\n",
    "print(webservice.scoring_uri)\n",
    "\n",
    "print(webservice.swagger_uri)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"data\": [{\"id\": 3053, \"regfalle\": 24117, \"edad\": 25, \"deptoocuhe\": 12, \"tipoarma\": 1, \"pracaut\": 1}, {\"id\": 3291, \"regfalle\": 23895, \"edad\": 25, \"deptoocuhe\": 11, \"tipoarma\": 1, \"pracaut\": 1}, {\"id\": 2855, \"regfalle\": 23586, \"edad\": 52, \"deptoocuhe\": 3, \"tipoarma\": 1, \"pracaut\": 1}, {\"id\": 2478, \"regfalle\": 21574, \"edad\": 56, \"deptoocuhe\": 6, \"tipoarma\": 1, \"pracaut\": 1}, {\"id\": 3170, \"regfalle\": 22237, \"edad\": 31, \"deptoocuhe\": 6, \"tipoarma\": 1, \"pracaut\": 0}, {\"id\": 3264, \"regfalle\": 22189, \"edad\": 29, \"deptoocuhe\": 8, \"tipoarma\": 1, \"pracaut\": 0}, {\"id\": 2144, \"regfalle\": 20845, \"edad\": 27, \"deptoocuhe\": 3, \"tipoarma\": 13, \"pracaut\": 1}, {\"id\": 1086, \"regfalle\": 21339, \"edad\": 25, \"deptoocuhe\": 10, \"tipoarma\": 1, \"pracaut\": 1}, {\"id\": 2613, \"regfalle\": 23573, \"edad\": 17, \"deptoocuhe\": 3, \"tipoarma\": 9, \"pracaut\": 1}, {\"id\": 1652, \"regfalle\": 23416, \"edad\": 34, \"deptoocuhe\": 7, \"tipoarma\": 13, \"pracaut\": 1}]}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "#Importing the data set for testing \n",
    "tsthomic2018 = pdhomic2018.sample(10) \n",
    "lblhomic2018 = tsthomic2018.pop('sexo')\n",
    "\n",
    "tstdatahomic2018 = json.dumps({'data': tsthomic2018.to_dict(orient='records')})\n",
    "\n",
    "print(tstdatahomic2018)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "gather": {
     "logged": 1598431657736
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "TODO: In the cell below, send a request to the web service you deployed to test it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "headers = {'Content-type': 'application/json'}\n",
    "\n",
    "response = requests.post(webservice.scoring_uri, tstdatahomic2018, headers=headers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"{\\\"result\\\": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}\"\n"
     ]
    }
   ],
   "source": [
    "# Print results \n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1465    0\n",
      "2849    0\n",
      "1726    0\n",
      "3190    0\n",
      "2201    1\n",
      "1940    0\n",
      "1142    0\n",
      "1557    0\n",
      "801     0\n",
      "3002    0\n",
      "Name: sexo, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Print original labels\n",
    "print(lblhomic2018)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "gather": {
     "logged": 1598432765711
    },
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "TODO: In the cell below, print the logs of the web service and delete the service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/sbin/nginx: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libcrypto.so.1.0.0: no version information available (required by /usr/sbin/nginx)\n",
      "/usr/sbin/nginx: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libcrypto.so.1.0.0: no version information available (required by /usr/sbin/nginx)\n",
      "/usr/sbin/nginx: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)\n",
      "/usr/sbin/nginx: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)\n",
      "/usr/sbin/nginx: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)\n",
      "2021-01-29T12:55:48,207323048+00:00 - iot-server/run \n",
      "2021-01-29T12:55:48,207505750+00:00 - nginx/run \n",
      "2021-01-29T12:55:48,209240467+00:00 - rsyslog/run \n",
      "rsyslogd: /azureml-envs/azureml_4f26b5b9300bb3a3b89415c34dafac63/lib/libuuid.so.1: no version information available (required by rsyslogd)\n",
      "2021-01-29T12:55:48,219837372+00:00 - gunicorn/run \n",
      "EdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting...\n",
      "2021-01-29T12:55:48,365620822+00:00 - iot-server/finish 1 0\n",
      "2021-01-29T12:55:48,367098036+00:00 - Exit code 1 is normal. Not restarting iot-server.\n",
      "Starting gunicorn 19.9.0\n",
      "Listening at: http://127.0.0.1:31311 (14)\n",
      "Using worker: sync\n",
      "worker timeout is set to 300\n",
      "Booting worker with pid: 42\n",
      "SPARK_HOME not set. Skipping PySpark Initialization.\n",
      "Generating new fontManager, this may take some time...\n",
      "Initializing logger\n",
      "2021-01-29 12:55:50,129 | root | INFO | Starting up app insights client\n",
      "2021-01-29 12:55:50,129 | root | INFO | Starting up request id generator\n",
      "2021-01-29 12:55:50,129 | root | INFO | Starting up app insight hooks\n",
      "2021-01-29 12:55:50,129 | root | INFO | Invoking user's init function\n",
      "2021-01-29 12:55:53,957 | azureml.core | WARNING | Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception cannot import name 'RunType'.\n",
      "Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception cannot import name 'RunType'.\n",
      "2021-01-29 12:55:53,987 | root | INFO | Users's init has completed successfully\n",
      "2021-01-29 12:55:53,989 | root | INFO | Skipping middleware: dbg_model_info as it's not enabled.\n",
      "2021-01-29 12:55:53,990 | root | INFO | Skipping middleware: dbg_resource_usage as it's not enabled.\n",
      "2021-01-29 12:55:53,990 | root | INFO | Scoring timeout is found from os.environ: 60000 ms\n",
      "2021-01-29 12:55:59,415 | root | INFO | 200\n",
      "127.0.0.1 - - [29/Jan/2021:12:55:59 +0000] \"GET /swagger.json HTTP/1.0\" 200 2338 \"-\" \"Go-http-client/1.1\"\n",
      "2021-01-29 12:56:01,528 | root | INFO | 200\n",
      "127.0.0.1 - - [29/Jan/2021:12:56:01 +0000] \"GET /swagger.json HTTP/1.0\" 200 2338 \"-\" \"Go-http-client/1.1\"\n",
      "2021-01-29 12:56:31,937 | root | INFO | Validation Request Content-Type\n",
      "2021-01-29 12:56:31,937 | root | INFO | Scoring Timer is set to 60.0 seconds\n",
      "2021-01-29 12:56:31,986 | root | INFO | 200\n",
      "127.0.0.1 - - [29/Jan/2021:12:56:31 +0000] \"POST /score HTTP/1.0\" 200 46 \"-\" \"python-requests/2.25.1\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(webservice.get_logs())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#best practice: deleting webservice that it won't be used anymore\n",
    "webservice.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#best practice: deleting CCluster that it won't be used anymore\n",
    "ccluster.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}