From 16ed30c6466a194d6c294c903741836be1c63041 Mon Sep 17 00:00:00 2001 From: Priti Date: Tue, 22 Nov 2022 10:35:28 -0500 Subject: [PATCH 01/14] [M2][#174] ElectricityMaps Data Source - Emissions --- docs/configuration.md | 63 +++-- .../mock/ElectricityMapDataSourceMocker.cs | 40 ++- .../src/Client/ElectricityMapsClient.cs | 103 ++++++-- .../src/Client/IElectricityMapsClient.cs | 17 ++ .../ElectricityMapsClientConfiguration.cs | 17 ++ .../ServiceCollectionExtensions.cs | 4 +- .../src/Constants/Paths.cs | 1 + .../src/Constants/QueryStrings.cs | 4 + .../src/ElectricityMapsDataSource.cs | 108 ++++++++- .../src/Model/EmissionsFactor.cs | 13 - .../src/Model/HistoryCarbonIntensityData.cs | 33 ++- .../test/Client/ElectricityMapsClientTests.cs | 8 +- .../test/Client/TestData.cs | 2 +- .../test/ElectricityMapsDataSourceTests.cs | 228 ++++++++++++++++++ .../ServiceCollectionExtensions.cs | 3 +- .../CarbonAwareControllerTests.cs | 12 - .../IntegrationTestingBase.cs | 1 + 17 files changed, 577 insertions(+), 80 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs diff --git a/docs/configuration.md b/docs/configuration.md index 4932749f9..2c120e8f3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,4 +1,3 @@ - - [Configuration](#configuration) - [Logging](#logging) - [DataSources](#datasources) @@ -10,9 +9,11 @@ - [WattTime Caching BalancingAuthority](#watttime-caching-balancingauthority) - [Json Configuration](#json-configuration) - [ElectricityMaps Configuration](#electricitymaps-configuration) - - [ApiTokenHeader](#api-token-header) - - [ApiToken](#api-token) - - [baseUrl](#baseurl) + - [API Token Header](#api-token-header) + - [API Token](#api-token) + - [BaseUrl](#baseurl-1) + - [Emission Factor Type](#emission-factor-type) + - [Disable Estimations](#disable-estimations) - [CarbonAwareVars](#carbonawarevars) - [Tracing and Monitoring Configuration](#tracing-and-monitoring-configuration) - [Verbosity](#verbosity) @@ -69,10 +70,10 @@ Logging__LogLevel__Default="Debug" dotnet run ## DataSources -The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file and [WattTime](https://www.watttime.org/) are supported. +The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and ElectricityMaps (https://www.electricitymaps.com/) are supported. Each data source interface is configured with a specific data source implementation. -If set to `WattTime`, WattTime configuration must also be supplied. +If set to `WattTime` or `ElectricityMaps`, the configuration specific to that data provider must also be supplied. `JSON` will result in the data being loaded from the file specified in the `DataFileLocation` property @@ -93,6 +94,11 @@ If set to `WattTime`, WattTime configuration must also be supplied. "username": "proxyUsername", "password": "proxyPassword" } + }, + "ElectricityMaps": { + "Type": "ElectricityMaps", + "APITokenHeader": "auth-token", + "APIToken": "myAwesomeToken" }, "Json": { "Type": "Json", @@ -115,7 +121,7 @@ If using the WattTime data source, WattTime configuration is required. } ``` -> **Sign up for a test account:** To create an account, follow these steps : https://www.watttime.org/api-documentation/#best-practices-for-api-usage +> **Sign up for a test account:** To create an account, follow these steps [from the WattTime documentation](https://www.watttime.org/api-documentation/#best-practices-for-api-usage) #### username @@ -170,7 +176,17 @@ info: CarbonAware.DataSources.Json.JsonDataSource[0] If using the ElectricityMaps data source, ElectricityMaps configuration is required. -__With an account token:__ +> **NOTE** +> The ElectricityMaps API does not currently support access to historical forecasts. +> This means that functionality such as the CLI `emissions-forecasts` `--requested-at` flag +> and the API `/forecasts/batch` `requestedAt` input will respond with a `NotImplemented` error. +> +> Depending on the goal, the historical measured `emissions` commands may be a reasonable workaround. +> This would treat the measured emissions as a "perfect historical forecast" effectively. +> Otherwise, use a data source that has support for historical forecasts, such as [WattTime](#watttime-configuration). + +**With an account token:** + ```json { "APITokenHeader": "auth-token", @@ -179,7 +195,8 @@ __With an account token:__ } ``` -__With a free trial token:__ +**With a free trial token:** + ```json { "APITokenHeader": "X-BLOBR-KEY", @@ -188,20 +205,32 @@ __With a free trial token:__ } ``` -> **Sign up for a free trial:** To get a free trial: https://api-portal.electricitymaps.com/ +> **Sign up for a free trial:** Select the free trial product from [the ElectricityMaps catalog](https://api-portal.electricitymaps.com/) #### API Token Header -The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOB-KEY" +The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOBR-KEY" #### API Token The ElectricityMaps token you receive with your account or free trial. -#### baseUrl +#### BaseUrl The url to use when connecting to ElectricityMaps. Defaults to "https://api.electricitymap.org/v3/" but can be overridden in the config if needed (such as for free-trial users or enable integration testing scenarios). +#### Emission Factor Type + +String value for the optional `emissionFactorType` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#emission-factors) for more details and valid values. + +#### Disable Estimations + +Boolean value for the optional `disableEstimations` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#estimations) for more details. + ## CarbonAwareVars This section contains the global settings for the SDK. The configuration looks like this: @@ -230,7 +259,7 @@ This application is integrated with Application Insights for monitoring purposes ApplicationInsights_Connection_String="AppInsightsConnectionString" ``` -You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net. +You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to [the documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net). ```bash AppInsights_InstrumentationKey="AppInsightsInstrumentationKey" @@ -335,6 +364,7 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" ``` ## Configuration for Forecast data Using ElectricityMaps + ```json { "DataSources": { @@ -350,11 +380,12 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" } ``` -## Configuration for Emissions data Using WattTime and Forecast data Using ElectricityMaps +## Configuration for Emissions data using ElectricityMaps and Forecast data using WattTime + ```json "DataSources": { - "EmissionsDataSource": "WattTime", - "ForecastDataSource": "ElectricityMaps", + "EmissionsDataSource": "ElectricityMaps", + "ForecastDataSource": "WattTime", "Configurations": { "WattTime": { "Type": "WattTime", diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs index cf86c02ab..4a2875133 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs @@ -26,19 +26,19 @@ public ElectricityMapsDataSourceMocker() public void SetupHistoryMock(decimal latitude, decimal longitude) { - var data = new List(); + var data = new List(); DateTimeOffset now = DateTimeOffset.UtcNow; DateTimeOffset past24 = now.AddHours(-24); while (past24 < now) { - var newDataPoint = new HistoryCarbonIntensity() + var newDataPoint = new CarbonIntensity() { - CarbonIntensity = 999, + Value = 999, DateTime = past24, UpdatedAt = now, CreatedAt = now, - EmissionFactorType = EmissionsFactor.Lifecycle, + EmissionFactorType = "lifecycle", IsEstimated = false, EstimationMethod = null }; @@ -101,15 +101,29 @@ private void SetupZonesMock() SetupResponseGivenGetRequest(Paths.Zones, result); } - public void Initialize() => SetupZonesMock(); + public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) + { + var data = new List(); + DateTimeOffset pointTime = start; + TimeSpan duration = TimeSpan.FromHours(1); - public void Reset() => _server.Reset(); + while (pointTime < end) + { + var newDataPoint = new CarbonIntensity() + { + Value = 100, + DateTime = pointTime, + }; - public void Dispose() => _server.Dispose(); + data.Add(newDataPoint); + pointTime = newDataPoint.DateTime + duration; + } - public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) - { - throw new NotImplementedException(); + HistoryCarbonIntensityData history = new() { HistoryData = data }; + PastRangeData pastRange = new() { HistoryData = data }; + + SetupResponseGivenGetRequest(Paths.History, history); + SetupResponseGivenGetRequest(Paths.PastRange, pastRange); } public void SetupBatchForecastMock() @@ -117,6 +131,12 @@ public void SetupBatchForecastMock() throw new NotImplementedException(); } + public void Initialize() => SetupZonesMock(); + + public void Reset() => _server.Reset(); + + public void Dispose() => _server.Dispose(); + private void SetupResponseGivenGetRequest(string path, object body) { var jsonBody = JsonSerializer.Serialize(body, _options); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs index 00c766e53..0cc295214 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs @@ -3,6 +3,7 @@ using CarbonAware.DataSources.ElectricityMaps.Model; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; +using System.Globalization; using System.Net.Http.Headers; using System.Net.Mime; using System.Text.Json; @@ -17,7 +18,7 @@ internal class ElectricityMapsClient : IElectricityMapsClient private readonly IOptionsMonitor _configurationMonitor; private ElectricityMapsClientConfiguration _configuration => this._configurationMonitor.CurrentValue; private readonly ILogger _log; - private Lazy>> _zonesAllowed; + private readonly Lazy>> _zonesAllowed; public ElectricityMapsClient(IHttpClientFactory factory, IOptionsMonitor monitor, ILogger log) { @@ -64,16 +65,6 @@ public async Task GetRecentCarbonIntensityHistoryAsy return await GetHistoryCarbonIntensityDataAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to History endpoint - private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) - { - await CheckZonesAllowedForPath(Paths.History, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.History, parameters)) - { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); - } - } - /// public async Task GetForecastedCarbonIntensityAsync (string zoneName) { @@ -103,14 +94,92 @@ public async Task GetForecastedCarbonIntensityAsy return await GetCurrentForecastAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to Forecast endpoint - private async Task GetCurrentForecastAsync(Dictionary parameters) + /// + public async Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using latitude {latitude} longitude {longitude}", + latitude, longitude); + + var parameters = new Dictionary() + { + { QueryStrings.Latitude, latitude }, + { QueryStrings.Longitude, longitude }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + /// + public async Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using zone {zone}", + zone); + + var parameters = new Dictionary() + { + { QueryStrings.ZoneName, zone }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + // The ElectricityMaps API has strict checks about datetime formatting. + // This helper method ensures that all DateTimeOffsets are properly formatted. + private static string DateTimeToString(DateTimeOffset dateTime) + { + return dateTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture); + } + + /// + /// Async method to check for allowed zones and then make GET request to History endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.History, parameters); + AddOptionalParameters(parameters); + + using Stream result = await this.MakeRequestGetStreamAsync(Paths.History, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); + } + + private void AddOptionalParameters(Dictionary parameters) { - await CheckZonesAllowedForPath(Paths.Forecast, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters)) + if (_configuration.EmissionFactorType != null) { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); + parameters.Add(QueryStrings.EmissionFactorType, _configuration.EmissionFactorType); } + if (_configuration.DisableEstimations != null) + { + parameters.Add(QueryStrings.DisableEstimations, _configuration.DisableEstimations.ToString()!.ToLowerInvariant()); + } + } + + private async Task GetPastRangeDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.PastRange, parameters); + AddOptionalParameters(parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.PastRange, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting emissions data"); + } + + /// + /// Async method to check for allowed zones and then make GET request to Forecast endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetCurrentForecastAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.Forecast, parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); } private async Task GetResponseAsync(string uriPath) @@ -157,7 +226,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q } // Checks the current supported client's endpoint paths. - private async Task CheckZonesAllowedForPath(string path, Dictionary parameters) + private async Task CheckZonesAllowedForPathAsync(string path, Dictionary parameters) { // Parameters don't contain a ZoneName to check, exit if (!parameters.ContainsKey(QueryStrings.ZoneName)) return; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs index fd7864ff8..d6e371e6d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs @@ -42,4 +42,21 @@ public interface IElectricityMapsClient /// A which contains all emissions data points in the 24 hour period. /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. public Task GetRecentCarbonIntensityHistoryAsync(string zoneName); + + /// + /// Async method to get the historical observed emission data for a given latitude and longitude over a given time period. + /// + /// Latitude for query + /// Longitude for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime); + + /// + /// Async method to get the historical observed emission data for a given zone over a given time period. + /// + /// Zone name for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs index 30661f5f9..2d050e60f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs @@ -1,5 +1,6 @@ using CarbonAware.Exceptions; using CarbonAware.DataSources.ElectricityMaps.Constants; +using CarbonAware.DataSources.ElectricityMaps.Model; namespace CarbonAware.DataSources.ElectricityMaps.Configuration; @@ -23,6 +24,22 @@ public class ElectricityMapsClientConfiguration /// public string BaseUrl { get; set; } = BaseUrls.TokenBaseUrl; + /// + /// Gets or sets the optional emissionFactorType parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#emission-factors for valid types + /// + public string? EmissionFactorType { get; set; } + + /// + /// Gets or sets the optional disableEstimations parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#estimations for details + /// + public bool? DisableEstimations { get; set; } + /// /// Validate that this object is properly configured. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs index 378146af1..1cdcf76c0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs @@ -20,7 +20,9 @@ public static IServiceCollection AddElectricityMapsForecastDataSource(this IServ public static IServiceCollection AddElectricityMapsEmissionsDataSource(this IServiceCollection services, DataSourcesConfiguration dataSourcesConfig) { - throw new NotImplementedException(); + AddElectricityMapsClient(services, dataSourcesConfig.EmissionsConfigurationSection()); + services.TryAddSingleton(); + return services; } private static void AddElectricityMapsClient(IServiceCollection services, IConfigurationSection configSection) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs index 3c4fc87de..c21bfbfb7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs @@ -4,5 +4,6 @@ internal class Paths { public const string History = "carbon-intensity/history"; public const string Forecast = "carbon-intensity/forecast"; + public const string PastRange = "carbon-intensity/past-range"; public const string Zones = "zones"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs index 27c33ecd9..d9bcc22c2 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs @@ -5,4 +5,8 @@ internal class QueryStrings public const string Latitude = "lat"; public const string Longitude = "lon"; public const string ZoneName = "zone"; + public const string StartTime = "start"; + public const string EndTime = "end"; + public const string DisableEstimations = "disableEstimations"; + public const string EmissionFactorType = "emissionFactorType"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs index fff633851..81ca4b5eb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs @@ -1,5 +1,6 @@ using CarbonAware.DataSources.ElectricityMaps.Client; using CarbonAware.DataSources.ElectricityMaps.Model; +using CarbonAware.Exceptions; using CarbonAware.Interfaces; using CarbonAware.Model; using Microsoft.Extensions.Logging; @@ -10,7 +11,7 @@ namespace CarbonAware.DataSources.ElectricityMaps; /// /// Represents a Electricity Maps data source. /// -public class ElectricityMapsDataSource : IForecastDataSource +public class ElectricityMapsDataSource : IForecastDataSource, IEmissionsDataSource { public string _name => "ElectricityMapsDataSource"; @@ -82,4 +83,109 @@ public async Task GetCarbonIntensityForecastAsync(Location lo await Task.Run(() => true); throw new NotImplementedException(); } + + /// + public async Task> GetCarbonIntensityAsync(IEnumerable locations, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + this._logger.LogDebug("Getting carbon intensity for locations {locations} for period {periodStartTime} to {periodEndTime}.", locations, periodStartTime, periodEndTime); + using (var activity = _activity.StartActivity()) + { + List result = new(); + foreach (var location in locations) + { + IEnumerable interimResult = await GetCarbonIntensityAsync(location, periodStartTime, periodEndTime); + result.AddRange(interimResult); + } + return result; + } + } + + /// + public async Task> GetCarbonIntensityAsync(Location location, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + using (var activity = _activity.StartActivity()) + { + var geolocation = await this._locationSource.ToGeopositionLocationAsync(location); + IEnumerable historyCarbonIntensity; + DateTime now = DateTime.UtcNow; + var isDateRangeWithin24Hours = (periodStartTime > now.AddHours(-24) && periodStartTime <= now) && (periodEndTime > now.AddHours(-24) && periodEndTime <= now); + if (isDateRangeWithin24Hours) + { + historyCarbonIntensity = await GetRecentCarbonInstensityData(geolocation); + } + else + { + historyCarbonIntensity = await GetPastCarbonIntensityData(periodStartTime, periodEndTime, geolocation); + } + + return HistoryCarbonIntensityToEmissionsData(location, historyCarbonIntensity, periodStartTime, periodEndTime); + } + } + + private async Task> GetPastCarbonIntensityData(DateTimeOffset periodStartTime, DateTimeOffset periodEndTime, Location geolocation) + { + PastRangeData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? "", periodStartTime, periodEndTime); + else + { + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Name ?? "", periodStartTime, periodEndTime); + } + + return data.HistoryData; + } + + private async Task> GetRecentCarbonInstensityData(Location geolocation) + { + HistoryCarbonIntensityData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? ""); + else + { + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Name ?? ""); + } + + return data.HistoryData; + } + + private IEnumerable HistoryCarbonIntensityToEmissionsData(Location location, IEnumerable data, DateTimeOffset startTime, DateTimeOffset endTime) + { + IEnumerable emissions; + var duration = GetDurationFromHistoryDataPointsOrDefault(data, TimeSpan.Zero); + emissions = data.Select(d => + { + var emission = (EmissionsData) d; + emission.Location = location.Name; + emission.Time = d.DateTime; + emission.Duration = duration; + return emission; + }); + + return emissions; + } + + private TimeSpan GetDurationFromHistoryDataPointsOrDefault(IEnumerable carbonIntensityDataPoints, TimeSpan defaultValue) + { + try + { + return GetDurationFromHistoryDataPoints(carbonIntensityDataPoints); + } + catch (CarbonAwareException) + { + return defaultValue; + } + } + + private TimeSpan GetDurationFromHistoryDataPoints(IEnumerable dataPoints) + { + var firstPoint = dataPoints.FirstOrDefault(); + var secondPoint = dataPoints.Skip(1)?.FirstOrDefault(); + + var first = firstPoint ?? throw new CarbonAwareException("Too few data points returned"); + var second = secondPoint ?? throw new CarbonAwareException("Too few data points returned"); + + // Handle chronological and reverse-chronological data by using `.Duration()` to get + // the absolute value of the TimeSpan between the two points. + return first.DateTime.Subtract(second.DateTime).Duration(); + } } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs deleted file mode 100644 index 27ef63379..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CarbonAware.DataSources.ElectricityMaps.Model; - -/// -/// Type of EmissionFactor to use for calculating carbon intensity as described in the Electricity Maps documentation - https://static.electricitymaps.com/api/docs/index.html#emission-factors -/// -[JsonConverter(typeof(JsonStringEnumConverter))] -public enum EmissionsFactor -{ - Lifecycle, - Direct -} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs index 5f0326d70..b23d658b0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs @@ -1,3 +1,5 @@ +using CarbonAware.Exceptions; +using CarbonAware.Model; using System.Text.Json.Serialization; namespace CarbonAware.DataSources.ElectricityMaps.Model; @@ -18,20 +20,20 @@ public record HistoryCarbonIntensityData /// List of History Carbon Intensity instances. /// [JsonPropertyName("history")] - public IEnumerable HistoryData { get; init; } = Array.Empty(); + public IEnumerable HistoryData { get; init; } = Array.Empty(); } /// /// A history carbon intensity. /// [Serializable] -public record HistoryCarbonIntensity +public record CarbonIntensity { /// /// Carbon Intensity value. /// [JsonPropertyName("carbonIntensity")] - public int CarbonIntensity { get; init; } + public int Value { get; init; } /// /// Indicates the datetime of the carbon intensity @@ -55,7 +57,7 @@ public record HistoryCarbonIntensity /// Indicated the emission factor type used for computing the carbon intensity. /// [JsonPropertyName("emissionFactorType")] - public EmissionsFactor EmissionFactorType { get; init; } + public string? EmissionFactorType { get; init; } /// /// Indicates whether the result is estimated or no @@ -69,4 +71,27 @@ public record HistoryCarbonIntensity [JsonPropertyName("estimationMethod")] public string? EstimationMethod { get; init; } + public static explicit operator EmissionsData(CarbonIntensity historyCarbonIntensity) + { + return new EmissionsData + { + Rating = historyCarbonIntensity.Value, + Time = historyCarbonIntensity.UpdatedAt, + }; + } } + +/// +/// Carbon intensity data for past date range. +/// +[Serializable] +public record PastRangeData +{ + /// + /// Carbon Intensity value. + /// + [JsonPropertyName("data")] + public IEnumerable HistoryData { get; init; } = Array.Empty(); + +} + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs index c8f47f8f5..6afd06ece 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs @@ -53,8 +53,8 @@ public void ClientInstantiation_FailsForInvalidConfig(string baseUrl) CreateBasicClient(TestData.GetZonesAllowedJsonString(), "{}"); this.Configuration = new ElectricityMapsClientConfiguration() { - APITokenHeader = null, - APIToken = null, + APITokenHeader = "", + APIToken = "", BaseUrl = baseUrl, }; @@ -254,8 +254,8 @@ public async Task GetRecentCarbonIntensityHistoryAsync_DeserializesExpectedRespo Assert.That(dataPoint?.DateTime, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.UpdatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.CreatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); - Assert.That(dataPoint?.CarbonIntensity, Is.EqualTo(999)); - Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo(EmissionsFactor.Lifecycle)); + Assert.That(dataPoint?.Value, Is.EqualTo(999)); + Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo("lifecycle")); Assert.That(dataPoint?.IsEstimated, Is.False); Assert.That(dataPoint?.EstimationMethod, Is.Null); }); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs index 9c028332d..48db3dab9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs @@ -63,7 +63,7 @@ public static string GetHistoryCarbonIntensityDataJsonString() ["updatedAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["createdAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["carbonIntensity"] = 999, - ["emissionFactorType"] = "Lifecycle", + ["emissionFactorType"] = "lifecycle", ["isEstimated"] = false, ["estimatedMethod"] = null, } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs index e39d37397..444868f5d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs @@ -96,4 +96,232 @@ public void GetCurrentCarbonIntensityForecastAsync_ThrowsWhenRegionNotFound() Assert.ThrowsAsync(async () => await _dataSource.GetCurrentCarbonIntensityForecastAsync(_defaultLocation)); } + + [TestCase(8, 1, 1, 0, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when date within 24 hours")] + [TestCase(36, 1, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when date outside of 24 hours")] + [TestCase(30, 12, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when start date outside of 24 hours but enddate within 24 hours")] + [TestCase(8, 12, 0, 1, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when start date within 24 hours but enddate outside of 24 hours")] + public async Task GetCarbonIntensity_CallsExpectedClientEndpoint(int startTimeOffset, int endTimeOffset, int expectedHistoryCalls, int expectedPastRangeCalls) + { + var now = DateTimeOffset.UtcNow; + var startDate = now.AddHours(-startTimeOffset); + var endDate = startDate.AddHours(endTimeOffset); + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new(); + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + PastRangeData pastRange = new(); + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => pastRange); + + await _dataSource.GetCarbonIntensityAsync(_defaultLocation, startDate, endDate); + + _electricityMapsClient.Verify(c => c.GetPastRangeDataAsync(_defaultLatitude, _defaultLongitude, startDate, endDate), Times.Exactly(expectedPastRangeCalls)); + + _electricityMapsClient.Verify(c => c.GetRecentCarbonIntensityHistoryAsync(_defaultLatitude, _defaultLongitude), Times.Exactly(expectedHistoryCalls)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeWithin24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-10); + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeMore24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = _defaultDataStartTime; + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + PastRangeData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_PastRange_ReturnsEmptyListWhenNoRecordsFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddHours(1); + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + + endDate) + ).ReturnsAsync(() => new PastRangeData()); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(0)); + } + + [Test] + public void GetCarbonIntensity_ThrowsWhenRegionNotFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddMinutes(1); + + this._locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Throws(); + + Assert.ThrowsAsync(async () => await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_ReturnsDefaultDuration_WhenOneDatapointReturned() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(1)); + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(TimeSpan.Zero)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_WhenMultipleDataPoints_ReturnsExpectedDuration() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + var expectedDuration = TimeSpan.FromHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + DateTime = startDate, + + }, + new CarbonIntensity() + { + DateTime= startDate + expectedDuration, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(expectedDuration)); + + var second = result.Skip(1)?.First(); + Assert.IsNotNull(second); + Assert.That(second.Duration, Is.EqualTo(expectedDuration)); + } } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs index 551891515..cb635adf4 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs @@ -32,7 +32,8 @@ public static IServiceCollection AddDataSourceService(this IServiceCollection se } case DataSourceType.ElectricityMaps: { - throw new ArgumentException("ElectricityMaps data source is not supported for emissions data"); + services.AddElectricityMapsEmissionsDataSource(dataSources); + break; } case DataSourceType.None: { diff --git a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs index ea6620d89..8bba52f7a 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs @@ -48,8 +48,6 @@ public async Task FakeEndPoint_ReturnsNotFound() [TestCase("2021-12-25", "2021-12-26", "westus")] public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Sets up any data endpoints needed for mocking purposes _dataSourceMocker?.SetupDataMock(start, end, location); @@ -72,8 +70,6 @@ public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset e [TestCase("non-location-param", "", TestName = "location param not present")] public async Task BestLocations_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Call the private method to construct with parameters var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -213,8 +209,6 @@ public async Task EmissionsForecastsBatch_SupportedDataSources_ReturnsOk(string [TestCase("2021-12-25", "2021-12-26", "westus", TestName = "EmissionsMarginalCarbonIntensity expects OK date only, no time")] public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, string end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/forecasts/current'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); @@ -246,8 +240,6 @@ public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, strin [TestCase("non-location-param", "", TestName = "EmissionsMarginalCarbonIntensity returns BadRequest for location not present")] public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -265,8 +257,6 @@ public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_Retu [TestCase("westus", "2022-3-1T15:30:00Z", "2022-3-1T18:00:00Z", TestName = "EmissionsMarginalCarbonIntensityBatch returns BadRequest for wrong date format")] public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_ReturnsBadRequest(string location, string startTime, string endTime) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var intesityData = Enumerable.Range(0, 1).Select(x => new { location = location, @@ -283,8 +273,6 @@ public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_Re [TestCase("2021-12-25", "2021-12-26", "westus", 3, TestName = "EmissionsMarginalCarbonIntensityBatch expects OK for multiple element batch")] public async Task EmissionsMarginalCarbonIntensityBatch_SupportedDataSources_ReturnsOk(string start, string end, string location, int nelems) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); diff --git a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs index 7fdf38ce9..3c7eb05b1 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs @@ -101,6 +101,7 @@ public void Setup() } case DataSourceType.ElectricityMaps: { + Environment.SetEnvironmentVariable("DataSources__EmissionsDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__ForecastDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__Type", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__APITokenHeader", "token"); From 1ccb2472e2684f4ab4b9ff736dadc6cb900097bd Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Wed, 7 Dec 2022 16:31:10 -0500 Subject: [PATCH 02/14] Add Azure Functions Example --- .../build-azure-functions-packages.yaml | 65 +++++++++ .gitignore | 1 + samples/azure-function/README.md | 125 ++++++++++++++++++ .../.devcontainer/devcontainer.json | 5 + .../emissions-azure-function/Dockerfile | 23 ++++ .../GetCarbonIntensity.cs | 66 +++++++++ .../emissions-azure-function/Startup.cs | 30 +++++ .../emissions-azure-function/appsettings.json | 14 ++ .../emissions-azure-function/function.csproj | 29 ++++ .../emissions-azure-function/host.json | 11 ++ .../.devcontainer/devcontainer.json | 5 + .../forecast-azure-function/Dockerfile | 23 ++++ .../forecast-azure-function/GetForecast.cs | 65 +++++++++ .../forecast-azure-function/Startup.cs | 30 +++++ .../forecast-azure-function/appsettings.json | 15 +++ .../forecast-azure-function/function.csproj | 29 ++++ .../forecast-azure-function/host.json | 11 ++ 17 files changed, 547 insertions(+) create mode 100644 .github/workflows/build-azure-functions-packages.yaml create mode 100644 samples/azure-function/README.md create mode 100644 samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json create mode 100644 samples/azure-function/emissions-azure-function/Dockerfile create mode 100644 samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs create mode 100644 samples/azure-function/emissions-azure-function/Startup.cs create mode 100644 samples/azure-function/emissions-azure-function/appsettings.json create mode 100644 samples/azure-function/emissions-azure-function/function.csproj create mode 100644 samples/azure-function/emissions-azure-function/host.json create mode 100644 samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json create mode 100644 samples/azure-function/forecast-azure-function/Dockerfile create mode 100644 samples/azure-function/forecast-azure-function/GetForecast.cs create mode 100644 samples/azure-function/forecast-azure-function/Startup.cs create mode 100644 samples/azure-function/forecast-azure-function/appsettings.json create mode 100644 samples/azure-function/forecast-azure-function/function.csproj create mode 100644 samples/azure-function/forecast-azure-function/host.json diff --git a/.github/workflows/build-azure-functions-packages.yaml b/.github/workflows/build-azure-functions-packages.yaml new file mode 100644 index 000000000..e52e60b8f --- /dev/null +++ b/.github/workflows/build-azure-functions-packages.yaml @@ -0,0 +1,65 @@ +name: Build and Install GSF Packages on Sample Azure Functions + +on: + push: + branches: [ dev, main ] + pull_request: + branches: [ dev, main ] + paths: + - 'src/**' + - '.github/workflows/**' + - 'samples/azure-function/**' + +env: + EMISSIONS_DOCKERFILE_PATH: samples/azure-function/emissions-azure-function/Dockerfile + FORECAST_DOCKERFILE_PATH: samples/azure-function/forecast-azure-function/Dockerfile + EM_CONTAINER_IMAGE_NAME: em-runnable-container + EM_CONTAINER_RUNTIME_NAME: em-az-func + FC_CONTAINER_IMAGE_NAME: fc-runnable-container + FC_CONTAINER_RUNTIME_NAME: fc-az-func + +jobs: + container_emissions_azure_function: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Docker RM container name + continue-on-error: true + run: docker rm -f ${{ env.EM_CONTAINER_RUNTIME_NAME }} + + - name: Docker Target Final + run: | + docker build --no-cache . -f ${{ env.EMISSIONS_DOCKERFILE_PATH }} -t ${{ env.EM_CONTAINER_IMAGE_NAME }} + docker rmi -f $(docker images -f "dangling=true" -q) + + - name: Docker Run Container + run: docker run -d --name ${{ env.EM_CONTAINER_RUNTIME_NAME }} -p 8080:80 ${{ env.EM_CONTAINER_IMAGE_NAME }} + + # Request fails with authentication error. Expected + - name: Get Emissions + run: wget -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus" || exit 0 + + container_forecast_azure_function: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Docker RM container name + continue-on-error: true + run: docker rm -f ${{ env.FC_CONTAINER_RUNTIME_NAME }} + + - name: Docker Target Final + run: | + docker build --no-cache . -f ${{ env.FORECAST_DOCKERFILE_PATH }} -t ${{ env.FC_CONTAINER_IMAGE_NAME }} + docker rmi -f $(docker images -f "dangling=true" -q) + + - name: Docker Run Container + run: docker run -d --name ${{ env.FC_CONTAINER_RUNTIME_NAME }} -p 8081:80 ${{ env.FC_CONTAINER_IMAGE_NAME }} + + # Request fails with authentication error. Expected + - name: Get Current Forecast + run: | + wget -t 5 --waitretry=5 "http://0.0.0.0:8081/api/GetCurrentForecast" --header "Content-Type: application/json" --post-data '{"startDate":"2022-11-02T15:30:00Z","endDate":"2022-11-02T18:30:00Z","location":"eastus","duration":"15"}' || exit 0 diff --git a/.gitignore b/.gitignore index 2d30e8e9d..db29bcb5b 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ build TestResults /src/clients/generated +/packages # Build-gen'd OpenAPI Spec src/CarbonAware.WebApi/src/CarbonAware.WebApi.xml diff --git a/samples/azure-function/README.md b/samples/azure-function/README.md new file mode 100644 index 000000000..dc74a2117 --- /dev/null +++ b/samples/azure-function/README.md @@ -0,0 +1,125 @@ +# Use Carbon Aware SDK with an Azure Function + +## Overview +The two samples included showcase the Azure Functions tooling for the Carbon Aware SDK [C# Class Library](../../docs/architecture/c%23-client-library.md). The emissions function app implements GetAverageCarbonIntensity and the forecast function app implements GetCurrentForecast. GetAverageCarbonIntensity uses the EmissionsHandler to return the Carbon Intensity rate of a location for a specific timespan. GetCurrentForecast uses the ForecastHandler to yield the optimal time of a specified location and duration. The functions can run locally for debugging or be deployed to Azure. + +## Azure Function Dependency Injection +The Carbon Aware SDK is included in the function .csproj file by [creating and adding the SDK as a package](../../docs/packaging.md#included-scripts). The [Startup.cs](Startup.cs) file uses dependency injection to access the handlers in the library. The following code initializes the C# Library: + +_For the emissions function app:_ + +```C# + public override void Configure(IFunctionsHostBuilder builder) + { + var configuration = builder.GetContext().Configuration; + builder.Services + .AddEmissionsServices(configuration) + } +``` + +_For the forecast function app:_ +```C# + public override void Configure(IFunctionsHostBuilder builder) + { + var configuration = builder.GetContext().Configuration; + builder.Services + .AddForecastServices(configuration); + } +``` + + +## Run Function Locally +Both azure function apps can be run locally without needing an azure subscription. The process for running both is the same, as they use the same configuration, just call different paths within the SDK. + +### Prerequisites +[.NET Core SDK](https://dotnet.microsoft.com/download) + +[Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) + +### Start Function +To run and debug locally, update the [appsettings.json](appsettings.json) file to include the desired [configuration](../../docs/configuration.md). + +In the app folder (`samples/azure-function/emissions-azure-function` or `samples/azure-function/forecast-azure-function`), run the command: ```func start``` + +After the function has compiled and is running, the URLs to the functions will be presented. + +_Example call for Emissions azure function_ + +The following example will retrieve the Average Carbon Intensity. For this example, query parameters were used, but the values could also be sent in the body of the request. + +``` +curl --location --request GET 'http://localhost:7071/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus' +``` + +_Example call for Forecast azure function_ + +The following example will call the Current Forecast route. If an error is returned, update the start and end dates. The request can use either the request body or query parameters. + +``` +curl --location --request GET 'http://localhost:7071/api/GetCurrentForecast' \ +--header 'Content-Type: application/json' \ +--data-raw '{ + "startDate": "2022-11-02T15:30:00Z", + "endDate": "2022-11-02T18:30:00Z", + "location" : "eastus", + "duration": 15 +}' +``` + +## Deploy to Azure +If you have an azure subscription, you can also deploy these functions to Azure. + +### Prerequisites + +You must have the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) or [Azure PowerShell](https://learn.microsoft.com/en-us/powershell/azure/install-az-ps) installed locally to be able to publish to Azure. + +[Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) + +### Create Function App +Log in to Azure: ```az login``` + +Once the correct subscription is selected run the following script to create a new function app: + +``` +# Function app and storage account names must be unique. + +# Variable block +let "randomIdentifier=$RANDOM*$RANDOM" +location="eastus" +resourceGroup="carbon-aware-$randomIdentifier" +storage="casaccount$randomIdentifier" +functionApp="carbon-aware-functionapp-$randomIdentifier" +skuStorage="Standard_LRS" +functionsVersion="3" + +# Create a resource group +echo "Creating $resourceGroup in "$location"..." +az group create --name $resourceGroup --location "$location" + +# Create an Azure storage account in the resource group. +echo "Creating $storage" +az storage account create --name $storage --location "$location" --resource-group $resourceGroup --sku $skuStorage + +# Create a serverless function app in the resource group. +echo "Creating $functionApp" +az functionapp create --name $functionApp --storage-account $storage --consumption-plan-location "$location" --resource-group $resourceGroup --functions-version $functionsVersion +``` + +### Publish Functions +Update the [appsettings.json](appsettings.json) file to include the desired [configuration](../../docs/configuration.md). + +To publish the function code to a function app in Azure, use the publish command in the samples/azure-function folder: + +``` +func azure functionapp publish $functionApp +``` + +## References + +[Azure Functions developer guide](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference?tabs=blob) + +[Use dependency injection in .NET Azure Functions](https://learn.microsoft.com/en-us/azure/azure-functions/functions-dotnet-dependency-injection) + +[Run functions locally](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=v4%2Cwindows%2Ccsharp%2Cportal%2Cbash#start) + +[Create a function app for serverless code execution](https://learn.microsoft.com/en-us/azure/azure-functions/scripts/functions-cli-create-serverless?source=recommendations) \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json b/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json new file mode 100644 index 000000000..f8aea8b28 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json @@ -0,0 +1,5 @@ +{ + "name": "Azure Functions & C# - .NET 6 (In-Process)", + "dockerFile": "../Dockerfile", + "forwardPorts": [ 7071 ], +} \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/Dockerfile b/samples/azure-function/emissions-azure-function/Dockerfile new file mode 100644 index 000000000..f16ee79b9 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/Dockerfile @@ -0,0 +1,23 @@ +# Find the Dockerfile at this URL +# https://github.com/Azure/azure-functions-docker/blob/dev/host/4/bullseye/amd64/dotnet/dotnet-inproc/dotnet.Dockerfile + +FROM mcr.microsoft.com/azure-functions/dotnet:4.0 AS base +WORKDIR /home/site/wwwroot + +FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build +COPY ["src/", "data/src/"] +COPY ["scripts/", "data/scripts/"] +COPY ["samples/", "data/samples/"] + +WORKDIR /data +RUN scripts/package/create_packages.sh src/CarbonAwareSDK.sln /packages && \ + dotnet restore "samples/azure-function/emissions-azure-function/function.csproj" && \ + scripts/package/add_packages.sh samples/azure-function/emissions-azure-function/function.csproj /packages && \ + dotnet build "samples/azure-function/emissions-azure-function/function.csproj" -c Release -o /app/build + +FROM build AS publish +RUN dotnet publish "samples/azure-function/emissions-azure-function/function.csproj" -c Release -o /app/publish + +FROM base AS final +WORKDIR /home/site/wwwroot +COPY --from=publish /app/publish . diff --git a/samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs b/samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs new file mode 100644 index 000000000..9b1ae7f0d --- /dev/null +++ b/samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs @@ -0,0 +1,66 @@ +using System; +using System.IO; +using System.Threading.Tasks; +using Microsoft.AspNetCore.Mvc; +using Microsoft.Azure.WebJobs; +using Microsoft.Azure.WebJobs.Extensions.Http; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.Logging; +using Newtonsoft.Json; +using GSF.CarbonAware.Handlers; + +namespace function +{ + public class GetCarbonIntensity + { + private readonly IEmissionsHandler _handler; + + public GetCarbonIntensity(IEmissionsHandler handler) + { + this._handler = handler; + } + + + [FunctionName("GetAverageCarbonIntensity")] + public async Task Run( + [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, + ILogger log) + { + //Get the startDate, endDate, and location from the request query if the values are present in the query + string startDate = req.Query["startdate"]; + string endDate = req.Query["enddate"]; + string location = req.Query["location"]; + + //If the parameters are includes in the body of the request, read the values from the request body + string requestBody = String.Empty; + using (StreamReader streamReader = new(req.Body)) + { + requestBody = await streamReader.ReadToEndAsync(); + } + dynamic data = JsonConvert.DeserializeObject(requestBody); + + startDate ??= data?.startDate; + endDate ??= data?.endDate; + location ??= data?.location; + + + try + { + var result = await _handler.GetAverageCarbonIntensityAsync(location, DateTimeOffset.Parse(startDate), DateTimeOffset.Parse(endDate)); + log.LogInformation($"For location {location} Starting at: {startDate} Ending at: {endDate} the Average Emissions Rating is: {result}."); + + return new OkObjectResult(result); + } + catch (Exception e) + { + //Messages related to incorrect parameter values (ie dates outside of range) are returned in the data section + //Otherwise send the returned error messages + if (e.Data.Count>0) + return new BadRequestObjectResult(e.Data); + else + return new BadRequestObjectResult(e.Message); + } + + } + } +} diff --git a/samples/azure-function/emissions-azure-function/Startup.cs b/samples/azure-function/emissions-azure-function/Startup.cs new file mode 100644 index 000000000..5c35f0584 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/Startup.cs @@ -0,0 +1,30 @@ +using System.IO; +using GSF.CarbonAware.Configuration; +using Microsoft.Azure.Functions.Extensions.DependencyInjection; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; + +[assembly: FunctionsStartup(typeof(CarbonAwareFunctions.Startup))] + +namespace CarbonAwareFunctions +{ + public class Startup : FunctionsStartup + { + public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder) + { + FunctionsHostBuilderContext context = builder.GetContext(); + + builder.ConfigurationBuilder + .AddJsonFile(Path.Combine(context.ApplicationRootPath, "appsettings.json"), optional: true, reloadOnChange: false) + .AddJsonFile(Path.Combine(context.ApplicationRootPath, $"appsettings.{context.EnvironmentName}.json"), optional: true, reloadOnChange: false) + .AddEnvironmentVariables(); + } + + public override void Configure(IFunctionsHostBuilder builder) + { + var configuration = builder.GetContext().Configuration; + builder.Services + .AddEmissionsServices(configuration); + } + } +} \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/appsettings.json b/samples/azure-function/emissions-azure-function/appsettings.json new file mode 100644 index 000000000..34b0294f7 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/appsettings.json @@ -0,0 +1,14 @@ +{ + "DataSources": { + "EmissionsDataSource": "WattTime", + "ForecastDataSource": "", + "Configurations": { + "WattTime": { + "Type": "WattTime", + "Username": "username", + "Password": "password", + "BaseURL": "https://api2.watttime.org/v2/" + } + } + } +} \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/function.csproj b/samples/azure-function/emissions-azure-function/function.csproj new file mode 100644 index 000000000..d2d6991e2 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/function.csproj @@ -0,0 +1,29 @@ + + + net6.0 + v3 + + + + + + + + + + + + + + + PreserveNewest + + + PreserveNewest + Never + + + PreserveNewest + + + diff --git a/samples/azure-function/emissions-azure-function/host.json b/samples/azure-function/emissions-azure-function/host.json new file mode 100644 index 000000000..beb2e4020 --- /dev/null +++ b/samples/azure-function/emissions-azure-function/host.json @@ -0,0 +1,11 @@ +{ + "version": "2.0", + "logging": { + "applicationInsights": { + "samplingSettings": { + "isEnabled": true, + "excludedTypes": "Request" + } + } + } +} \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json b/samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json new file mode 100644 index 000000000..c4bf7bcfa --- /dev/null +++ b/samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json @@ -0,0 +1,5 @@ +{ + "name": "Azure Functions & C# - .NET 6 (In-Process)", + "dockerFile": "../Dockerfile", + "forwardPorts": [ 7071 ] +} \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/Dockerfile b/samples/azure-function/forecast-azure-function/Dockerfile new file mode 100644 index 000000000..57403d1c2 --- /dev/null +++ b/samples/azure-function/forecast-azure-function/Dockerfile @@ -0,0 +1,23 @@ +# Find the Dockerfile at this URL +# https://github.com/Azure/azure-functions-docker/blob/dev/host/4/bullseye/amd64/dotnet/dotnet-inproc/dotnet.Dockerfile + +FROM mcr.microsoft.com/azure-functions/dotnet:4.0 AS base +WORKDIR /home/site/wwwroot + +FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build +COPY ["src/", "data/src/"] +COPY ["scripts/", "data/scripts/"] +COPY ["samples/", "data/samples/"] + +WORKDIR /data +RUN scripts/package/create_packages.sh src/CarbonAwareSDK.sln /packages && \ + dotnet restore "samples/azure-function/forecast-azure-function/function.csproj" && \ + scripts/package/add_packages.sh samples/azure-function/forecast-azure-function/function.csproj /packages && \ + dotnet build "samples/azure-function/forecast-azure-function/function.csproj" -c Release -o /app/build + +FROM build AS publish +RUN dotnet publish "samples/azure-function/forecast-azure-function/function.csproj" -c Release -o /app/publish + +FROM base AS final +WORKDIR /home/site/wwwroot +COPY --from=publish /app/publish . diff --git a/samples/azure-function/forecast-azure-function/GetForecast.cs b/samples/azure-function/forecast-azure-function/GetForecast.cs new file mode 100644 index 000000000..b06074f22 --- /dev/null +++ b/samples/azure-function/forecast-azure-function/GetForecast.cs @@ -0,0 +1,65 @@ +using System; +using System.IO; +using System.Threading.Tasks; +using Microsoft.AspNetCore.Mvc; +using Microsoft.Azure.WebJobs; +using Microsoft.Azure.WebJobs.Extensions.Http; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.Logging; +using Newtonsoft.Json; +using GSF.CarbonAware.Handlers; + +namespace CarbonAwareFunctions +{ + public class GetForecast + { + private readonly IForecastHandler _handler; + + public GetForecast(IForecastHandler handler) + { + this._handler = handler; + } + + [FunctionName("GetCurrentForecast")] + public async Task Run( + [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, + ILogger log) + { + //Get the startDate, endDate, location, and duration from the request query if the values are present in the query + string startDate = req.Query["startdate"]; + string endDate = req.Query["enddate"]; + string location = req.Query["location"]; + string duration = req.Query["duration"]; + + //If the parameters are includes in the body of the request, read the values from the request body + string requestBody = String.Empty; + using (StreamReader streamReader = new(req.Body)) + { + requestBody = await streamReader.ReadToEndAsync(); + } + dynamic data = JsonConvert.DeserializeObject(requestBody); + + startDate ??= data?.startDate; + endDate ??= data?.endDate; + location ??= data?.location; + duration ??= data?.duration; + + try + { + var result = await _handler.GetCurrentForecastAsync(new string[] { location }, DateTimeOffset.Parse(startDate), DateTimeOffset.Parse(endDate), int.Parse(duration)); + + return new OkObjectResult(result); + } + catch (Exception e) + { + //Messages related to incorrect parameter values (ie dates outside of range) are returned in the data section + //Otherwise send the returned error messages + if (e.Data.Count > 0) + return new BadRequestObjectResult(e.Data); + else + return new BadRequestObjectResult(e.Message); + } + + } + } +} diff --git a/samples/azure-function/forecast-azure-function/Startup.cs b/samples/azure-function/forecast-azure-function/Startup.cs new file mode 100644 index 000000000..473bf0188 --- /dev/null +++ b/samples/azure-function/forecast-azure-function/Startup.cs @@ -0,0 +1,30 @@ +using System.IO; +using GSF.CarbonAware.Configuration; +using Microsoft.Azure.Functions.Extensions.DependencyInjection; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; + +[assembly: FunctionsStartup(typeof(CarbonAwareFunctions.Startup))] + +namespace CarbonAwareFunctions +{ + public class Startup : FunctionsStartup + { + public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder) + { + FunctionsHostBuilderContext context = builder.GetContext(); + + builder.ConfigurationBuilder + .AddJsonFile(Path.Combine(context.ApplicationRootPath, "appsettings.json"), optional: true, reloadOnChange: false) + .AddJsonFile(Path.Combine(context.ApplicationRootPath, $"appsettings.{context.EnvironmentName}.json"), optional: true, reloadOnChange: false) + .AddEnvironmentVariables(); + } + + public override void Configure(IFunctionsHostBuilder builder) + { + var configuration = builder.GetContext().Configuration; + builder.Services + .AddForecastServices(configuration); + } + } +} \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/appsettings.json b/samples/azure-function/forecast-azure-function/appsettings.json new file mode 100644 index 000000000..bdf5c068a --- /dev/null +++ b/samples/azure-function/forecast-azure-function/appsettings.json @@ -0,0 +1,15 @@ +{ + "DataSources": { + "EmissionsDataSource": "", + "ForecastDataSource": "WattTime", + "Configurations": { + "WattTime": { + "Type": "WattTime", + "Username": "username", + "Password": "password", + "BaseURL": "https://api2.watttime.org/v2/" + } + } + } +} + \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/function.csproj b/samples/azure-function/forecast-azure-function/function.csproj new file mode 100644 index 000000000..d2d6991e2 --- /dev/null +++ b/samples/azure-function/forecast-azure-function/function.csproj @@ -0,0 +1,29 @@ + + + net6.0 + v3 + + + + + + + + + + + + + + + PreserveNewest + + + PreserveNewest + Never + + + PreserveNewest + + + diff --git a/samples/azure-function/forecast-azure-function/host.json b/samples/azure-function/forecast-azure-function/host.json new file mode 100644 index 000000000..beb2e4020 --- /dev/null +++ b/samples/azure-function/forecast-azure-function/host.json @@ -0,0 +1,11 @@ +{ + "version": "2.0", + "logging": { + "applicationInsights": { + "samplingSettings": { + "isEnabled": true, + "excludedTypes": "Request" + } + } + } +} \ No newline at end of file From 8a43a463e4f1a26ef3dcdf1a77659b4f49373001 Mon Sep 17 00:00:00 2001 From: Juan Zuluaga Date: Thu, 8 Dec 2022 16:07:10 -0500 Subject: [PATCH 03/14] Fail in case the error is not known --- .../workflows/build-azure-functions-packages.yaml | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/.github/workflows/build-azure-functions-packages.yaml b/.github/workflows/build-azure-functions-packages.yaml index e52e60b8f..05e269457 100644 --- a/.github/workflows/build-azure-functions-packages.yaml +++ b/.github/workflows/build-azure-functions-packages.yaml @@ -39,7 +39,12 @@ jobs: # Request fails with authentication error. Expected - name: Get Emissions - run: wget -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus" || exit 0 + run: | + set +e + ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus" 2>&1 | grep "HTTP/" | awk '{print $2}') + set -e + [ "401" == $ret_code ] + container_forecast_azure_function: runs-on: ubuntu-latest @@ -62,4 +67,7 @@ jobs: # Request fails with authentication error. Expected - name: Get Current Forecast run: | - wget -t 5 --waitretry=5 "http://0.0.0.0:8081/api/GetCurrentForecast" --header "Content-Type: application/json" --post-data '{"startDate":"2022-11-02T15:30:00Z","endDate":"2022-11-02T18:30:00Z","location":"eastus","duration":"15"}' || exit 0 + set +e + ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8081/api/GetCurrentForecast" --header "Content-Type: application/json" --post-data '{"startDate":"2022-11-02T15:30:00Z","endDate":"2022-11-02T18:30:00Z","location":"eastus","duration":"15"}' 2>&1 | grep "HTTP/" | awk '{print $2}') + set -e + [ "401" == $ret_code ] From 5f4e3060747a890cda8e39a051666eca34cefc2e Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Mon, 12 Dec 2022 12:29:30 -0500 Subject: [PATCH 04/14] PR comments --- .../build-azure-functions-packages.yaml | 73 ------------------- .../verify-azure-function-with-packages.yaml | 51 +++++++++++++ .gitignore | 1 - ...t-library.md => c-sharp-client-library.md} | 0 .../.devcontainer/devcontainer.json | 0 .../{forecast-azure-function => }/Dockerfile | 8 +- .../GetCarbonIntensity.cs | 12 +-- .../GetForecast.cs | 13 ++-- samples/azure-function/README.md | 50 ++++++------- .../{forecast-azure-function => }/Startup.cs | 3 +- .../appsettings.json | 0 .../.devcontainer/devcontainer.json | 5 -- .../emissions-azure-function/Dockerfile | 23 ------ .../emissions-azure-function/Startup.cs | 30 -------- .../forecast-azure-function/appsettings.json | 15 ---- .../forecast-azure-function/function.csproj | 29 -------- .../forecast-azure-function/host.json | 11 --- .../function.csproj | 2 +- .../{emissions-azure-function => }/host.json | 0 19 files changed, 94 insertions(+), 232 deletions(-) delete mode 100644 .github/workflows/build-azure-functions-packages.yaml create mode 100644 .github/workflows/verify-azure-function-with-packages.yaml rename docs/architecture/{c#-client-library.md => c-sharp-client-library.md} (100%) rename samples/azure-function/{forecast-azure-function => }/.devcontainer/devcontainer.json (100%) rename samples/azure-function/{forecast-azure-function => }/Dockerfile (58%) rename samples/azure-function/{emissions-azure-function => }/GetCarbonIntensity.cs (95%) rename samples/azure-function/{forecast-azure-function => }/GetForecast.cs (95%) rename samples/azure-function/{forecast-azure-function => }/Startup.cs (95%) rename samples/azure-function/{emissions-azure-function => }/appsettings.json (100%) delete mode 100644 samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json delete mode 100644 samples/azure-function/emissions-azure-function/Dockerfile delete mode 100644 samples/azure-function/emissions-azure-function/Startup.cs delete mode 100644 samples/azure-function/forecast-azure-function/appsettings.json delete mode 100644 samples/azure-function/forecast-azure-function/function.csproj delete mode 100644 samples/azure-function/forecast-azure-function/host.json rename samples/azure-function/{emissions-azure-function => }/function.csproj (96%) rename samples/azure-function/{emissions-azure-function => }/host.json (100%) diff --git a/.github/workflows/build-azure-functions-packages.yaml b/.github/workflows/build-azure-functions-packages.yaml deleted file mode 100644 index 05e269457..000000000 --- a/.github/workflows/build-azure-functions-packages.yaml +++ /dev/null @@ -1,73 +0,0 @@ -name: Build and Install GSF Packages on Sample Azure Functions - -on: - push: - branches: [ dev, main ] - pull_request: - branches: [ dev, main ] - paths: - - 'src/**' - - '.github/workflows/**' - - 'samples/azure-function/**' - -env: - EMISSIONS_DOCKERFILE_PATH: samples/azure-function/emissions-azure-function/Dockerfile - FORECAST_DOCKERFILE_PATH: samples/azure-function/forecast-azure-function/Dockerfile - EM_CONTAINER_IMAGE_NAME: em-runnable-container - EM_CONTAINER_RUNTIME_NAME: em-az-func - FC_CONTAINER_IMAGE_NAME: fc-runnable-container - FC_CONTAINER_RUNTIME_NAME: fc-az-func - -jobs: - container_emissions_azure_function: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v3 - - - name: Docker RM container name - continue-on-error: true - run: docker rm -f ${{ env.EM_CONTAINER_RUNTIME_NAME }} - - - name: Docker Target Final - run: | - docker build --no-cache . -f ${{ env.EMISSIONS_DOCKERFILE_PATH }} -t ${{ env.EM_CONTAINER_IMAGE_NAME }} - docker rmi -f $(docker images -f "dangling=true" -q) - - - name: Docker Run Container - run: docker run -d --name ${{ env.EM_CONTAINER_RUNTIME_NAME }} -p 8080:80 ${{ env.EM_CONTAINER_IMAGE_NAME }} - - # Request fails with authentication error. Expected - - name: Get Emissions - run: | - set +e - ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus" 2>&1 | grep "HTTP/" | awk '{print $2}') - set -e - [ "401" == $ret_code ] - - - container_forecast_azure_function: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v3 - - - name: Docker RM container name - continue-on-error: true - run: docker rm -f ${{ env.FC_CONTAINER_RUNTIME_NAME }} - - - name: Docker Target Final - run: | - docker build --no-cache . -f ${{ env.FORECAST_DOCKERFILE_PATH }} -t ${{ env.FC_CONTAINER_IMAGE_NAME }} - docker rmi -f $(docker images -f "dangling=true" -q) - - - name: Docker Run Container - run: docker run -d --name ${{ env.FC_CONTAINER_RUNTIME_NAME }} -p 8081:80 ${{ env.FC_CONTAINER_IMAGE_NAME }} - - # Request fails with authentication error. Expected - - name: Get Current Forecast - run: | - set +e - ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8081/api/GetCurrentForecast" --header "Content-Type: application/json" --post-data '{"startDate":"2022-11-02T15:30:00Z","endDate":"2022-11-02T18:30:00Z","location":"eastus","duration":"15"}' 2>&1 | grep "HTTP/" | awk '{print $2}') - set -e - [ "401" == $ret_code ] diff --git a/.github/workflows/verify-azure-function-with-packages.yaml b/.github/workflows/verify-azure-function-with-packages.yaml new file mode 100644 index 000000000..afab08c33 --- /dev/null +++ b/.github/workflows/verify-azure-function-with-packages.yaml @@ -0,0 +1,51 @@ +name: Verify Sample Azure Functions with GSF Packages + +on: + push: + branches: [ dev, main ] + pull_request: + branches: [ dev, main ] + paths: + - 'src/**' + - '.github/workflows/**' + - 'samples/azure-function/**' + +env: + DOCKERFILE_PATH: samples/azure-function/Dockerfile + CONTAINER_IMAGE_NAME: runnable-container + CONTAINER_RUNTIME_NAME: az-func + +jobs: + container_azure_function: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Docker RM container name + continue-on-error: true + run: docker rm -f ${{ env.CONTAINER_RUNTIME_NAME }} + + - name: Docker Target Final + run: | + docker build --no-cache . -f ${{ env.DOCKERFILE_PATH }} -t ${{ env.CONTAINER_IMAGE_NAME }} + docker rmi -f $(docker images -f "dangling=true" -q) + + - name: Docker Run Container + run: docker run -d --name ${{ env.CONTAINER_RUNTIME_NAME }} -p 8080:80 ${{ env.CONTAINER_IMAGE_NAME }} + + # Request fails with authentication error. Expected + - name: Get Average Carbon Intensity + run: | + set +e + ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus" 2>&1 | grep "HTTP/" | awk '{print $2}') + set -e + [ "401" == $ret_code ] + + # Request fails with authentication error. Expected + - name: Get Current Forecast + run: | + set +e + ret_code=$(wget -S -t 5 --waitretry=5 "http://0.0.0.0:8080/api/GetCurrentForecast" --header "Content-Type: application/json" --post-data '{"startDate":"2022-11-02T15:30:00Z","endDate":"2022-11-02T18:30:00Z","location":"eastus","duration":"15"}' 2>&1 | grep "HTTP/" | awk '{print $2}') + set -e + [ "401" == $ret_code ] diff --git a/.gitignore b/.gitignore index db29bcb5b..2d30e8e9d 100644 --- a/.gitignore +++ b/.gitignore @@ -13,7 +13,6 @@ build TestResults /src/clients/generated -/packages # Build-gen'd OpenAPI Spec src/CarbonAware.WebApi/src/CarbonAware.WebApi.xml diff --git a/docs/architecture/c#-client-library.md b/docs/architecture/c-sharp-client-library.md similarity index 100% rename from docs/architecture/c#-client-library.md rename to docs/architecture/c-sharp-client-library.md diff --git a/samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json b/samples/azure-function/.devcontainer/devcontainer.json similarity index 100% rename from samples/azure-function/forecast-azure-function/.devcontainer/devcontainer.json rename to samples/azure-function/.devcontainer/devcontainer.json diff --git a/samples/azure-function/forecast-azure-function/Dockerfile b/samples/azure-function/Dockerfile similarity index 58% rename from samples/azure-function/forecast-azure-function/Dockerfile rename to samples/azure-function/Dockerfile index 57403d1c2..8211fc34f 100644 --- a/samples/azure-function/forecast-azure-function/Dockerfile +++ b/samples/azure-function/Dockerfile @@ -11,12 +11,12 @@ COPY ["samples/", "data/samples/"] WORKDIR /data RUN scripts/package/create_packages.sh src/CarbonAwareSDK.sln /packages && \ - dotnet restore "samples/azure-function/forecast-azure-function/function.csproj" && \ - scripts/package/add_packages.sh samples/azure-function/forecast-azure-function/function.csproj /packages && \ - dotnet build "samples/azure-function/forecast-azure-function/function.csproj" -c Release -o /app/build + dotnet restore "samples/azure-function/function.csproj" && \ + scripts/package/add_packages.sh samples/azure-function/function.csproj /packages && \ + dotnet build "samples/azure-function/function.csproj" -c Release -o /app/build FROM build AS publish -RUN dotnet publish "samples/azure-function/forecast-azure-function/function.csproj" -c Release -o /app/publish +RUN dotnet publish "samples/azure-function/function.csproj" -c Release -o /app/publish FROM base AS final WORKDIR /home/site/wwwroot diff --git a/samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs b/samples/azure-function/GetCarbonIntensity.cs similarity index 95% rename from samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs rename to samples/azure-function/GetCarbonIntensity.cs index 9b1ae7f0d..c1e2a441e 100644 --- a/samples/azure-function/emissions-azure-function/GetCarbonIntensity.cs +++ b/samples/azure-function/GetCarbonIntensity.cs @@ -1,13 +1,13 @@ -using System; -using System.IO; -using System.Threading.Tasks; +using GSF.CarbonAware.Handlers; using Microsoft.AspNetCore.Mvc; using Microsoft.Azure.WebJobs; using Microsoft.Azure.WebJobs.Extensions.Http; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Logging; -using Newtonsoft.Json; -using GSF.CarbonAware.Handlers; +using System; +using System.IO; +using System.Threading.Tasks; +using System.Text.Json; namespace function { @@ -37,7 +37,7 @@ public async Task Run( { requestBody = await streamReader.ReadToEndAsync(); } - dynamic data = JsonConvert.DeserializeObject(requestBody); + dynamic data = JsonSerializer.Deserialize(requestBody); startDate ??= data?.startDate; endDate ??= data?.endDate; diff --git a/samples/azure-function/forecast-azure-function/GetForecast.cs b/samples/azure-function/GetForecast.cs similarity index 95% rename from samples/azure-function/forecast-azure-function/GetForecast.cs rename to samples/azure-function/GetForecast.cs index b06074f22..564a0138e 100644 --- a/samples/azure-function/forecast-azure-function/GetForecast.cs +++ b/samples/azure-function/GetForecast.cs @@ -1,13 +1,14 @@ -using System; -using System.IO; -using System.Threading.Tasks; +using GSF.CarbonAware.Handlers; using Microsoft.AspNetCore.Mvc; using Microsoft.Azure.WebJobs; using Microsoft.Azure.WebJobs.Extensions.Http; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Logging; -using Newtonsoft.Json; -using GSF.CarbonAware.Handlers; +using System; +using System.IO; +using System.Threading.Tasks; +using System.Text.Json; + namespace CarbonAwareFunctions { @@ -37,7 +38,7 @@ public async Task Run( { requestBody = await streamReader.ReadToEndAsync(); } - dynamic data = JsonConvert.DeserializeObject(requestBody); + dynamic data = JsonSerializer.Deserialize(requestBody); startDate ??= data?.startDate; endDate ??= data?.endDate; diff --git a/samples/azure-function/README.md b/samples/azure-function/README.md index dc74a2117..c20fcc2cd 100644 --- a/samples/azure-function/README.md +++ b/samples/azure-function/README.md @@ -1,12 +1,12 @@ # Use Carbon Aware SDK with an Azure Function ## Overview -The two samples included showcase the Azure Functions tooling for the Carbon Aware SDK [C# Class Library](../../docs/architecture/c%23-client-library.md). The emissions function app implements GetAverageCarbonIntensity and the forecast function app implements GetCurrentForecast. GetAverageCarbonIntensity uses the EmissionsHandler to return the Carbon Intensity rate of a location for a specific timespan. GetCurrentForecast uses the ForecastHandler to yield the optimal time of a specified location and duration. The functions can run locally for debugging or be deployed to Azure. + +The sample included showcase the Azure Functions tooling for the Carbon Aware SDK [C# Class Library](../../docs/architecture/c-sharp-client-library.md). It includes an implementation for `GetAverageCarbonIntensity` and for `GetCurrentForecast`. `GetAverageCarbonIntensity` uses the `EmissionsHandler` to return the carbon intensity rate of a location for a specific timespan. `GetCurrentForecast` uses the `ForecastHandler` to yield the optimal time of a specified location and duration. The functions can run locally for debugging or be deployed to Azure. ## Azure Function Dependency Injection -The Carbon Aware SDK is included in the function .csproj file by [creating and adding the SDK as a package](../../docs/packaging.md#included-scripts). The [Startup.cs](Startup.cs) file uses dependency injection to access the handlers in the library. The following code initializes the C# Library: -_For the emissions function app:_ +The Carbon Aware SDK is included in the function .csproj file by [creating and adding the SDK as a package](../../docs/packaging.md#included-scripts). The [Startup.cs](./Startup.cs) file uses dependency injection to access the handlers in the library. The following code initializes the C# Library: ```C# public override void Configure(IFunctionsHostBuilder builder) @@ -14,48 +14,41 @@ _For the emissions function app:_ var configuration = builder.GetContext().Configuration; builder.Services .AddEmissionsServices(configuration) - } -``` - -_For the forecast function app:_ -```C# - public override void Configure(IFunctionsHostBuilder builder) - { - var configuration = builder.GetContext().Configuration; - builder.Services .AddForecastServices(configuration); } ``` - ## Run Function Locally -Both azure function apps can be run locally without needing an azure subscription. The process for running both is the same, as they use the same configuration, just call different paths within the SDK. -### Prerequisites +Both Azure Function apps can be run locally without needing an azure subscription. The process for running both is the same, as they use the same configuration, just call different paths within the SDK. + +### Prerequisites to Run + [.NET Core SDK](https://dotnet.microsoft.com/download) [Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) ### Start Function + To run and debug locally, update the [appsettings.json](appsettings.json) file to include the desired [configuration](../../docs/configuration.md). -In the app folder (`samples/azure-function/emissions-azure-function` or `samples/azure-function/forecast-azure-function`), run the command: ```func start``` +In the app folder (`samples/azure-function`), run the command: ```func start``` After the function has compiled and is running, the URLs to the functions will be presented. -_Example call for Emissions azure function_ +#### _Example call for Get Average Carbon Intensity function_ The following example will retrieve the Average Carbon Intensity. For this example, query parameters were used, but the values could also be sent in the body of the request. -``` +```bash curl --location --request GET 'http://localhost:7071/api/GetAverageCarbonIntensity?startDate=2022-03-01T15:30:00Z&endDate=2022-03-01T18:30:00Z&location=eastus' ``` -_Example call for Forecast azure function_ +#### _Example call for Get Current Forecast function_ The following example will call the Current Forecast route. If an error is returned, update the start and end dates. The request can use either the request body or query parameters. -``` +```bash curl --location --request GET 'http://localhost:7071/api/GetCurrentForecast' \ --header 'Content-Type: application/json' \ --data-raw '{ @@ -67,20 +60,22 @@ curl --location --request GET 'http://localhost:7071/api/GetCurrentForecast' \ ``` ## Deploy to Azure + If you have an azure subscription, you can also deploy these functions to Azure. -### Prerequisites +### Prerequisites to Deploy You must have the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) or [Azure PowerShell](https://learn.microsoft.com/en-us/powershell/azure/install-az-ps) installed locally to be able to publish to Azure. [Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local) ### Create Function App + Log in to Azure: ```az login``` Once the correct subscription is selected run the following script to create a new function app: -``` +```bash # Function app and storage account names must be unique. # Variable block @@ -105,13 +100,14 @@ echo "Creating $functionApp" az functionapp create --name $functionApp --storage-account $storage --consumption-plan-location "$location" --resource-group $resourceGroup --functions-version $functionsVersion ``` -### Publish Functions -Update the [appsettings.json](appsettings.json) file to include the desired [configuration](../../docs/configuration.md). +### Publish Functions + +Update the [appsettings.json](./appsettings.json) file to include the desired [configuration](../../docs/configuration.md). To publish the function code to a function app in Azure, use the publish command in the samples/azure-function folder: -``` -func azure functionapp publish $functionApp +```bash +func Azure Functionapp publish $functionApp ``` ## References @@ -122,4 +118,4 @@ func azure functionapp publish $functionApp [Run functions locally](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=v4%2Cwindows%2Ccsharp%2Cportal%2Cbash#start) -[Create a function app for serverless code execution](https://learn.microsoft.com/en-us/azure/azure-functions/scripts/functions-cli-create-serverless?source=recommendations) \ No newline at end of file +[Create a function app for serverless code execution](https://learn.microsoft.com/en-us/azure/azure-functions/scripts/functions-cli-create-serverless?source=recommendations) diff --git a/samples/azure-function/forecast-azure-function/Startup.cs b/samples/azure-function/Startup.cs similarity index 95% rename from samples/azure-function/forecast-azure-function/Startup.cs rename to samples/azure-function/Startup.cs index 473bf0188..1660065ba 100644 --- a/samples/azure-function/forecast-azure-function/Startup.cs +++ b/samples/azure-function/Startup.cs @@ -1,8 +1,8 @@ -using System.IO; using GSF.CarbonAware.Configuration; using Microsoft.Azure.Functions.Extensions.DependencyInjection; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; +using System.IO; [assembly: FunctionsStartup(typeof(CarbonAwareFunctions.Startup))] @@ -24,6 +24,7 @@ public override void Configure(IFunctionsHostBuilder builder) { var configuration = builder.GetContext().Configuration; builder.Services + .AddEmissionsServices(configuration) .AddForecastServices(configuration); } } diff --git a/samples/azure-function/emissions-azure-function/appsettings.json b/samples/azure-function/appsettings.json similarity index 100% rename from samples/azure-function/emissions-azure-function/appsettings.json rename to samples/azure-function/appsettings.json diff --git a/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json b/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json deleted file mode 100644 index f8aea8b28..000000000 --- a/samples/azure-function/emissions-azure-function/.devcontainer/devcontainer.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "name": "Azure Functions & C# - .NET 6 (In-Process)", - "dockerFile": "../Dockerfile", - "forwardPorts": [ 7071 ], -} \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/Dockerfile b/samples/azure-function/emissions-azure-function/Dockerfile deleted file mode 100644 index f16ee79b9..000000000 --- a/samples/azure-function/emissions-azure-function/Dockerfile +++ /dev/null @@ -1,23 +0,0 @@ -# Find the Dockerfile at this URL -# https://github.com/Azure/azure-functions-docker/blob/dev/host/4/bullseye/amd64/dotnet/dotnet-inproc/dotnet.Dockerfile - -FROM mcr.microsoft.com/azure-functions/dotnet:4.0 AS base -WORKDIR /home/site/wwwroot - -FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build -COPY ["src/", "data/src/"] -COPY ["scripts/", "data/scripts/"] -COPY ["samples/", "data/samples/"] - -WORKDIR /data -RUN scripts/package/create_packages.sh src/CarbonAwareSDK.sln /packages && \ - dotnet restore "samples/azure-function/emissions-azure-function/function.csproj" && \ - scripts/package/add_packages.sh samples/azure-function/emissions-azure-function/function.csproj /packages && \ - dotnet build "samples/azure-function/emissions-azure-function/function.csproj" -c Release -o /app/build - -FROM build AS publish -RUN dotnet publish "samples/azure-function/emissions-azure-function/function.csproj" -c Release -o /app/publish - -FROM base AS final -WORKDIR /home/site/wwwroot -COPY --from=publish /app/publish . diff --git a/samples/azure-function/emissions-azure-function/Startup.cs b/samples/azure-function/emissions-azure-function/Startup.cs deleted file mode 100644 index 5c35f0584..000000000 --- a/samples/azure-function/emissions-azure-function/Startup.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.IO; -using GSF.CarbonAware.Configuration; -using Microsoft.Azure.Functions.Extensions.DependencyInjection; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; - -[assembly: FunctionsStartup(typeof(CarbonAwareFunctions.Startup))] - -namespace CarbonAwareFunctions -{ - public class Startup : FunctionsStartup - { - public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder) - { - FunctionsHostBuilderContext context = builder.GetContext(); - - builder.ConfigurationBuilder - .AddJsonFile(Path.Combine(context.ApplicationRootPath, "appsettings.json"), optional: true, reloadOnChange: false) - .AddJsonFile(Path.Combine(context.ApplicationRootPath, $"appsettings.{context.EnvironmentName}.json"), optional: true, reloadOnChange: false) - .AddEnvironmentVariables(); - } - - public override void Configure(IFunctionsHostBuilder builder) - { - var configuration = builder.GetContext().Configuration; - builder.Services - .AddEmissionsServices(configuration); - } - } -} \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/appsettings.json b/samples/azure-function/forecast-azure-function/appsettings.json deleted file mode 100644 index bdf5c068a..000000000 --- a/samples/azure-function/forecast-azure-function/appsettings.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "DataSources": { - "EmissionsDataSource": "", - "ForecastDataSource": "WattTime", - "Configurations": { - "WattTime": { - "Type": "WattTime", - "Username": "username", - "Password": "password", - "BaseURL": "https://api2.watttime.org/v2/" - } - } - } -} - \ No newline at end of file diff --git a/samples/azure-function/forecast-azure-function/function.csproj b/samples/azure-function/forecast-azure-function/function.csproj deleted file mode 100644 index d2d6991e2..000000000 --- a/samples/azure-function/forecast-azure-function/function.csproj +++ /dev/null @@ -1,29 +0,0 @@ - - - net6.0 - v3 - - - - - - - - - - - - - - - PreserveNewest - - - PreserveNewest - Never - - - PreserveNewest - - - diff --git a/samples/azure-function/forecast-azure-function/host.json b/samples/azure-function/forecast-azure-function/host.json deleted file mode 100644 index beb2e4020..000000000 --- a/samples/azure-function/forecast-azure-function/host.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "version": "2.0", - "logging": { - "applicationInsights": { - "samplingSettings": { - "isEnabled": true, - "excludedTypes": "Request" - } - } - } -} \ No newline at end of file diff --git a/samples/azure-function/emissions-azure-function/function.csproj b/samples/azure-function/function.csproj similarity index 96% rename from samples/azure-function/emissions-azure-function/function.csproj rename to samples/azure-function/function.csproj index d2d6991e2..e48592f82 100644 --- a/samples/azure-function/emissions-azure-function/function.csproj +++ b/samples/azure-function/function.csproj @@ -1,7 +1,7 @@ net6.0 - v3 + v4 diff --git a/samples/azure-function/emissions-azure-function/host.json b/samples/azure-function/host.json similarity index 100% rename from samples/azure-function/emissions-azure-function/host.json rename to samples/azure-function/host.json From 09ce9e10fd4c80f0fa7d8273fc47413a8cb0b76b Mon Sep 17 00:00:00 2001 From: Priti Date: Tue, 22 Nov 2022 10:35:28 -0500 Subject: [PATCH 05/14] [M2][#174] ElectricityMaps Data Source - Emissions --- docs/configuration.md | 63 +++-- .../mock/ElectricityMapDataSourceMocker.cs | 40 ++- .../src/Client/ElectricityMapsClient.cs | 103 ++++++-- .../src/Client/IElectricityMapsClient.cs | 17 ++ .../ElectricityMapsClientConfiguration.cs | 17 ++ .../ServiceCollectionExtensions.cs | 4 +- .../src/Constants/Paths.cs | 1 + .../src/Constants/QueryStrings.cs | 4 + .../src/ElectricityMapsDataSource.cs | 108 ++++++++- .../src/Model/EmissionsFactor.cs | 13 - .../src/Model/HistoryCarbonIntensityData.cs | 33 ++- .../test/Client/ElectricityMapsClientTests.cs | 8 +- .../test/Client/TestData.cs | 2 +- .../test/ElectricityMapsDataSourceTests.cs | 228 ++++++++++++++++++ .../ServiceCollectionExtensions.cs | 3 +- .../CarbonAwareControllerTests.cs | 12 - .../IntegrationTestingBase.cs | 1 + 17 files changed, 577 insertions(+), 80 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs diff --git a/docs/configuration.md b/docs/configuration.md index 4932749f9..2c120e8f3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,4 +1,3 @@ - - [Configuration](#configuration) - [Logging](#logging) - [DataSources](#datasources) @@ -10,9 +9,11 @@ - [WattTime Caching BalancingAuthority](#watttime-caching-balancingauthority) - [Json Configuration](#json-configuration) - [ElectricityMaps Configuration](#electricitymaps-configuration) - - [ApiTokenHeader](#api-token-header) - - [ApiToken](#api-token) - - [baseUrl](#baseurl) + - [API Token Header](#api-token-header) + - [API Token](#api-token) + - [BaseUrl](#baseurl-1) + - [Emission Factor Type](#emission-factor-type) + - [Disable Estimations](#disable-estimations) - [CarbonAwareVars](#carbonawarevars) - [Tracing and Monitoring Configuration](#tracing-and-monitoring-configuration) - [Verbosity](#verbosity) @@ -69,10 +70,10 @@ Logging__LogLevel__Default="Debug" dotnet run ## DataSources -The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file and [WattTime](https://www.watttime.org/) are supported. +The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and ElectricityMaps (https://www.electricitymaps.com/) are supported. Each data source interface is configured with a specific data source implementation. -If set to `WattTime`, WattTime configuration must also be supplied. +If set to `WattTime` or `ElectricityMaps`, the configuration specific to that data provider must also be supplied. `JSON` will result in the data being loaded from the file specified in the `DataFileLocation` property @@ -93,6 +94,11 @@ If set to `WattTime`, WattTime configuration must also be supplied. "username": "proxyUsername", "password": "proxyPassword" } + }, + "ElectricityMaps": { + "Type": "ElectricityMaps", + "APITokenHeader": "auth-token", + "APIToken": "myAwesomeToken" }, "Json": { "Type": "Json", @@ -115,7 +121,7 @@ If using the WattTime data source, WattTime configuration is required. } ``` -> **Sign up for a test account:** To create an account, follow these steps : https://www.watttime.org/api-documentation/#best-practices-for-api-usage +> **Sign up for a test account:** To create an account, follow these steps [from the WattTime documentation](https://www.watttime.org/api-documentation/#best-practices-for-api-usage) #### username @@ -170,7 +176,17 @@ info: CarbonAware.DataSources.Json.JsonDataSource[0] If using the ElectricityMaps data source, ElectricityMaps configuration is required. -__With an account token:__ +> **NOTE** +> The ElectricityMaps API does not currently support access to historical forecasts. +> This means that functionality such as the CLI `emissions-forecasts` `--requested-at` flag +> and the API `/forecasts/batch` `requestedAt` input will respond with a `NotImplemented` error. +> +> Depending on the goal, the historical measured `emissions` commands may be a reasonable workaround. +> This would treat the measured emissions as a "perfect historical forecast" effectively. +> Otherwise, use a data source that has support for historical forecasts, such as [WattTime](#watttime-configuration). + +**With an account token:** + ```json { "APITokenHeader": "auth-token", @@ -179,7 +195,8 @@ __With an account token:__ } ``` -__With a free trial token:__ +**With a free trial token:** + ```json { "APITokenHeader": "X-BLOBR-KEY", @@ -188,20 +205,32 @@ __With a free trial token:__ } ``` -> **Sign up for a free trial:** To get a free trial: https://api-portal.electricitymaps.com/ +> **Sign up for a free trial:** Select the free trial product from [the ElectricityMaps catalog](https://api-portal.electricitymaps.com/) #### API Token Header -The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOB-KEY" +The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOBR-KEY" #### API Token The ElectricityMaps token you receive with your account or free trial. -#### baseUrl +#### BaseUrl The url to use when connecting to ElectricityMaps. Defaults to "https://api.electricitymap.org/v3/" but can be overridden in the config if needed (such as for free-trial users or enable integration testing scenarios). +#### Emission Factor Type + +String value for the optional `emissionFactorType` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#emission-factors) for more details and valid values. + +#### Disable Estimations + +Boolean value for the optional `disableEstimations` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#estimations) for more details. + ## CarbonAwareVars This section contains the global settings for the SDK. The configuration looks like this: @@ -230,7 +259,7 @@ This application is integrated with Application Insights for monitoring purposes ApplicationInsights_Connection_String="AppInsightsConnectionString" ``` -You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net. +You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to [the documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net). ```bash AppInsights_InstrumentationKey="AppInsightsInstrumentationKey" @@ -335,6 +364,7 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" ``` ## Configuration for Forecast data Using ElectricityMaps + ```json { "DataSources": { @@ -350,11 +380,12 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" } ``` -## Configuration for Emissions data Using WattTime and Forecast data Using ElectricityMaps +## Configuration for Emissions data using ElectricityMaps and Forecast data using WattTime + ```json "DataSources": { - "EmissionsDataSource": "WattTime", - "ForecastDataSource": "ElectricityMaps", + "EmissionsDataSource": "ElectricityMaps", + "ForecastDataSource": "WattTime", "Configurations": { "WattTime": { "Type": "WattTime", diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs index cf86c02ab..4a2875133 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs @@ -26,19 +26,19 @@ public ElectricityMapsDataSourceMocker() public void SetupHistoryMock(decimal latitude, decimal longitude) { - var data = new List(); + var data = new List(); DateTimeOffset now = DateTimeOffset.UtcNow; DateTimeOffset past24 = now.AddHours(-24); while (past24 < now) { - var newDataPoint = new HistoryCarbonIntensity() + var newDataPoint = new CarbonIntensity() { - CarbonIntensity = 999, + Value = 999, DateTime = past24, UpdatedAt = now, CreatedAt = now, - EmissionFactorType = EmissionsFactor.Lifecycle, + EmissionFactorType = "lifecycle", IsEstimated = false, EstimationMethod = null }; @@ -101,15 +101,29 @@ private void SetupZonesMock() SetupResponseGivenGetRequest(Paths.Zones, result); } - public void Initialize() => SetupZonesMock(); + public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) + { + var data = new List(); + DateTimeOffset pointTime = start; + TimeSpan duration = TimeSpan.FromHours(1); - public void Reset() => _server.Reset(); + while (pointTime < end) + { + var newDataPoint = new CarbonIntensity() + { + Value = 100, + DateTime = pointTime, + }; - public void Dispose() => _server.Dispose(); + data.Add(newDataPoint); + pointTime = newDataPoint.DateTime + duration; + } - public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) - { - throw new NotImplementedException(); + HistoryCarbonIntensityData history = new() { HistoryData = data }; + PastRangeData pastRange = new() { HistoryData = data }; + + SetupResponseGivenGetRequest(Paths.History, history); + SetupResponseGivenGetRequest(Paths.PastRange, pastRange); } public void SetupBatchForecastMock() @@ -117,6 +131,12 @@ public void SetupBatchForecastMock() throw new NotImplementedException(); } + public void Initialize() => SetupZonesMock(); + + public void Reset() => _server.Reset(); + + public void Dispose() => _server.Dispose(); + private void SetupResponseGivenGetRequest(string path, object body) { var jsonBody = JsonSerializer.Serialize(body, _options); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs index 00c766e53..0cc295214 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs @@ -3,6 +3,7 @@ using CarbonAware.DataSources.ElectricityMaps.Model; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; +using System.Globalization; using System.Net.Http.Headers; using System.Net.Mime; using System.Text.Json; @@ -17,7 +18,7 @@ internal class ElectricityMapsClient : IElectricityMapsClient private readonly IOptionsMonitor _configurationMonitor; private ElectricityMapsClientConfiguration _configuration => this._configurationMonitor.CurrentValue; private readonly ILogger _log; - private Lazy>> _zonesAllowed; + private readonly Lazy>> _zonesAllowed; public ElectricityMapsClient(IHttpClientFactory factory, IOptionsMonitor monitor, ILogger log) { @@ -64,16 +65,6 @@ public async Task GetRecentCarbonIntensityHistoryAsy return await GetHistoryCarbonIntensityDataAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to History endpoint - private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) - { - await CheckZonesAllowedForPath(Paths.History, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.History, parameters)) - { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); - } - } - /// public async Task GetForecastedCarbonIntensityAsync (string zoneName) { @@ -103,14 +94,92 @@ public async Task GetForecastedCarbonIntensityAsy return await GetCurrentForecastAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to Forecast endpoint - private async Task GetCurrentForecastAsync(Dictionary parameters) + /// + public async Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using latitude {latitude} longitude {longitude}", + latitude, longitude); + + var parameters = new Dictionary() + { + { QueryStrings.Latitude, latitude }, + { QueryStrings.Longitude, longitude }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + /// + public async Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using zone {zone}", + zone); + + var parameters = new Dictionary() + { + { QueryStrings.ZoneName, zone }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + // The ElectricityMaps API has strict checks about datetime formatting. + // This helper method ensures that all DateTimeOffsets are properly formatted. + private static string DateTimeToString(DateTimeOffset dateTime) + { + return dateTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture); + } + + /// + /// Async method to check for allowed zones and then make GET request to History endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.History, parameters); + AddOptionalParameters(parameters); + + using Stream result = await this.MakeRequestGetStreamAsync(Paths.History, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); + } + + private void AddOptionalParameters(Dictionary parameters) { - await CheckZonesAllowedForPath(Paths.Forecast, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters)) + if (_configuration.EmissionFactorType != null) { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); + parameters.Add(QueryStrings.EmissionFactorType, _configuration.EmissionFactorType); } + if (_configuration.DisableEstimations != null) + { + parameters.Add(QueryStrings.DisableEstimations, _configuration.DisableEstimations.ToString()!.ToLowerInvariant()); + } + } + + private async Task GetPastRangeDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.PastRange, parameters); + AddOptionalParameters(parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.PastRange, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting emissions data"); + } + + /// + /// Async method to check for allowed zones and then make GET request to Forecast endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetCurrentForecastAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.Forecast, parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); } private async Task GetResponseAsync(string uriPath) @@ -157,7 +226,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q } // Checks the current supported client's endpoint paths. - private async Task CheckZonesAllowedForPath(string path, Dictionary parameters) + private async Task CheckZonesAllowedForPathAsync(string path, Dictionary parameters) { // Parameters don't contain a ZoneName to check, exit if (!parameters.ContainsKey(QueryStrings.ZoneName)) return; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs index fd7864ff8..d6e371e6d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs @@ -42,4 +42,21 @@ public interface IElectricityMapsClient /// A which contains all emissions data points in the 24 hour period. /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. public Task GetRecentCarbonIntensityHistoryAsync(string zoneName); + + /// + /// Async method to get the historical observed emission data for a given latitude and longitude over a given time period. + /// + /// Latitude for query + /// Longitude for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime); + + /// + /// Async method to get the historical observed emission data for a given zone over a given time period. + /// + /// Zone name for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs index 30661f5f9..2d050e60f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs @@ -1,5 +1,6 @@ using CarbonAware.Exceptions; using CarbonAware.DataSources.ElectricityMaps.Constants; +using CarbonAware.DataSources.ElectricityMaps.Model; namespace CarbonAware.DataSources.ElectricityMaps.Configuration; @@ -23,6 +24,22 @@ public class ElectricityMapsClientConfiguration /// public string BaseUrl { get; set; } = BaseUrls.TokenBaseUrl; + /// + /// Gets or sets the optional emissionFactorType parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#emission-factors for valid types + /// + public string? EmissionFactorType { get; set; } + + /// + /// Gets or sets the optional disableEstimations parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#estimations for details + /// + public bool? DisableEstimations { get; set; } + /// /// Validate that this object is properly configured. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs index 378146af1..1cdcf76c0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs @@ -20,7 +20,9 @@ public static IServiceCollection AddElectricityMapsForecastDataSource(this IServ public static IServiceCollection AddElectricityMapsEmissionsDataSource(this IServiceCollection services, DataSourcesConfiguration dataSourcesConfig) { - throw new NotImplementedException(); + AddElectricityMapsClient(services, dataSourcesConfig.EmissionsConfigurationSection()); + services.TryAddSingleton(); + return services; } private static void AddElectricityMapsClient(IServiceCollection services, IConfigurationSection configSection) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs index 3c4fc87de..c21bfbfb7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs @@ -4,5 +4,6 @@ internal class Paths { public const string History = "carbon-intensity/history"; public const string Forecast = "carbon-intensity/forecast"; + public const string PastRange = "carbon-intensity/past-range"; public const string Zones = "zones"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs index 27c33ecd9..d9bcc22c2 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs @@ -5,4 +5,8 @@ internal class QueryStrings public const string Latitude = "lat"; public const string Longitude = "lon"; public const string ZoneName = "zone"; + public const string StartTime = "start"; + public const string EndTime = "end"; + public const string DisableEstimations = "disableEstimations"; + public const string EmissionFactorType = "emissionFactorType"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs index fff633851..81ca4b5eb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs @@ -1,5 +1,6 @@ using CarbonAware.DataSources.ElectricityMaps.Client; using CarbonAware.DataSources.ElectricityMaps.Model; +using CarbonAware.Exceptions; using CarbonAware.Interfaces; using CarbonAware.Model; using Microsoft.Extensions.Logging; @@ -10,7 +11,7 @@ namespace CarbonAware.DataSources.ElectricityMaps; /// /// Represents a Electricity Maps data source. /// -public class ElectricityMapsDataSource : IForecastDataSource +public class ElectricityMapsDataSource : IForecastDataSource, IEmissionsDataSource { public string _name => "ElectricityMapsDataSource"; @@ -82,4 +83,109 @@ public async Task GetCarbonIntensityForecastAsync(Location lo await Task.Run(() => true); throw new NotImplementedException(); } + + /// + public async Task> GetCarbonIntensityAsync(IEnumerable locations, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + this._logger.LogDebug("Getting carbon intensity for locations {locations} for period {periodStartTime} to {periodEndTime}.", locations, periodStartTime, periodEndTime); + using (var activity = _activity.StartActivity()) + { + List result = new(); + foreach (var location in locations) + { + IEnumerable interimResult = await GetCarbonIntensityAsync(location, periodStartTime, periodEndTime); + result.AddRange(interimResult); + } + return result; + } + } + + /// + public async Task> GetCarbonIntensityAsync(Location location, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + using (var activity = _activity.StartActivity()) + { + var geolocation = await this._locationSource.ToGeopositionLocationAsync(location); + IEnumerable historyCarbonIntensity; + DateTime now = DateTime.UtcNow; + var isDateRangeWithin24Hours = (periodStartTime > now.AddHours(-24) && periodStartTime <= now) && (periodEndTime > now.AddHours(-24) && periodEndTime <= now); + if (isDateRangeWithin24Hours) + { + historyCarbonIntensity = await GetRecentCarbonInstensityData(geolocation); + } + else + { + historyCarbonIntensity = await GetPastCarbonIntensityData(periodStartTime, periodEndTime, geolocation); + } + + return HistoryCarbonIntensityToEmissionsData(location, historyCarbonIntensity, periodStartTime, periodEndTime); + } + } + + private async Task> GetPastCarbonIntensityData(DateTimeOffset periodStartTime, DateTimeOffset periodEndTime, Location geolocation) + { + PastRangeData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? "", periodStartTime, periodEndTime); + else + { + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Name ?? "", periodStartTime, periodEndTime); + } + + return data.HistoryData; + } + + private async Task> GetRecentCarbonInstensityData(Location geolocation) + { + HistoryCarbonIntensityData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? ""); + else + { + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Name ?? ""); + } + + return data.HistoryData; + } + + private IEnumerable HistoryCarbonIntensityToEmissionsData(Location location, IEnumerable data, DateTimeOffset startTime, DateTimeOffset endTime) + { + IEnumerable emissions; + var duration = GetDurationFromHistoryDataPointsOrDefault(data, TimeSpan.Zero); + emissions = data.Select(d => + { + var emission = (EmissionsData) d; + emission.Location = location.Name; + emission.Time = d.DateTime; + emission.Duration = duration; + return emission; + }); + + return emissions; + } + + private TimeSpan GetDurationFromHistoryDataPointsOrDefault(IEnumerable carbonIntensityDataPoints, TimeSpan defaultValue) + { + try + { + return GetDurationFromHistoryDataPoints(carbonIntensityDataPoints); + } + catch (CarbonAwareException) + { + return defaultValue; + } + } + + private TimeSpan GetDurationFromHistoryDataPoints(IEnumerable dataPoints) + { + var firstPoint = dataPoints.FirstOrDefault(); + var secondPoint = dataPoints.Skip(1)?.FirstOrDefault(); + + var first = firstPoint ?? throw new CarbonAwareException("Too few data points returned"); + var second = secondPoint ?? throw new CarbonAwareException("Too few data points returned"); + + // Handle chronological and reverse-chronological data by using `.Duration()` to get + // the absolute value of the TimeSpan between the two points. + return first.DateTime.Subtract(second.DateTime).Duration(); + } } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs deleted file mode 100644 index 27ef63379..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CarbonAware.DataSources.ElectricityMaps.Model; - -/// -/// Type of EmissionFactor to use for calculating carbon intensity as described in the Electricity Maps documentation - https://static.electricitymaps.com/api/docs/index.html#emission-factors -/// -[JsonConverter(typeof(JsonStringEnumConverter))] -public enum EmissionsFactor -{ - Lifecycle, - Direct -} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs index 5f0326d70..b23d658b0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs @@ -1,3 +1,5 @@ +using CarbonAware.Exceptions; +using CarbonAware.Model; using System.Text.Json.Serialization; namespace CarbonAware.DataSources.ElectricityMaps.Model; @@ -18,20 +20,20 @@ public record HistoryCarbonIntensityData /// List of History Carbon Intensity instances. /// [JsonPropertyName("history")] - public IEnumerable HistoryData { get; init; } = Array.Empty(); + public IEnumerable HistoryData { get; init; } = Array.Empty(); } /// /// A history carbon intensity. /// [Serializable] -public record HistoryCarbonIntensity +public record CarbonIntensity { /// /// Carbon Intensity value. /// [JsonPropertyName("carbonIntensity")] - public int CarbonIntensity { get; init; } + public int Value { get; init; } /// /// Indicates the datetime of the carbon intensity @@ -55,7 +57,7 @@ public record HistoryCarbonIntensity /// Indicated the emission factor type used for computing the carbon intensity. /// [JsonPropertyName("emissionFactorType")] - public EmissionsFactor EmissionFactorType { get; init; } + public string? EmissionFactorType { get; init; } /// /// Indicates whether the result is estimated or no @@ -69,4 +71,27 @@ public record HistoryCarbonIntensity [JsonPropertyName("estimationMethod")] public string? EstimationMethod { get; init; } + public static explicit operator EmissionsData(CarbonIntensity historyCarbonIntensity) + { + return new EmissionsData + { + Rating = historyCarbonIntensity.Value, + Time = historyCarbonIntensity.UpdatedAt, + }; + } } + +/// +/// Carbon intensity data for past date range. +/// +[Serializable] +public record PastRangeData +{ + /// + /// Carbon Intensity value. + /// + [JsonPropertyName("data")] + public IEnumerable HistoryData { get; init; } = Array.Empty(); + +} + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs index c8f47f8f5..6afd06ece 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs @@ -53,8 +53,8 @@ public void ClientInstantiation_FailsForInvalidConfig(string baseUrl) CreateBasicClient(TestData.GetZonesAllowedJsonString(), "{}"); this.Configuration = new ElectricityMapsClientConfiguration() { - APITokenHeader = null, - APIToken = null, + APITokenHeader = "", + APIToken = "", BaseUrl = baseUrl, }; @@ -254,8 +254,8 @@ public async Task GetRecentCarbonIntensityHistoryAsync_DeserializesExpectedRespo Assert.That(dataPoint?.DateTime, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.UpdatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.CreatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); - Assert.That(dataPoint?.CarbonIntensity, Is.EqualTo(999)); - Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo(EmissionsFactor.Lifecycle)); + Assert.That(dataPoint?.Value, Is.EqualTo(999)); + Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo("lifecycle")); Assert.That(dataPoint?.IsEstimated, Is.False); Assert.That(dataPoint?.EstimationMethod, Is.Null); }); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs index 9c028332d..48db3dab9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs @@ -63,7 +63,7 @@ public static string GetHistoryCarbonIntensityDataJsonString() ["updatedAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["createdAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["carbonIntensity"] = 999, - ["emissionFactorType"] = "Lifecycle", + ["emissionFactorType"] = "lifecycle", ["isEstimated"] = false, ["estimatedMethod"] = null, } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs index e39d37397..444868f5d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs @@ -96,4 +96,232 @@ public void GetCurrentCarbonIntensityForecastAsync_ThrowsWhenRegionNotFound() Assert.ThrowsAsync(async () => await _dataSource.GetCurrentCarbonIntensityForecastAsync(_defaultLocation)); } + + [TestCase(8, 1, 1, 0, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when date within 24 hours")] + [TestCase(36, 1, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when date outside of 24 hours")] + [TestCase(30, 12, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when start date outside of 24 hours but enddate within 24 hours")] + [TestCase(8, 12, 0, 1, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when start date within 24 hours but enddate outside of 24 hours")] + public async Task GetCarbonIntensity_CallsExpectedClientEndpoint(int startTimeOffset, int endTimeOffset, int expectedHistoryCalls, int expectedPastRangeCalls) + { + var now = DateTimeOffset.UtcNow; + var startDate = now.AddHours(-startTimeOffset); + var endDate = startDate.AddHours(endTimeOffset); + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new(); + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + PastRangeData pastRange = new(); + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => pastRange); + + await _dataSource.GetCarbonIntensityAsync(_defaultLocation, startDate, endDate); + + _electricityMapsClient.Verify(c => c.GetPastRangeDataAsync(_defaultLatitude, _defaultLongitude, startDate, endDate), Times.Exactly(expectedPastRangeCalls)); + + _electricityMapsClient.Verify(c => c.GetRecentCarbonIntensityHistoryAsync(_defaultLatitude, _defaultLongitude), Times.Exactly(expectedHistoryCalls)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeWithin24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-10); + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeMore24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = _defaultDataStartTime; + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + PastRangeData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_PastRange_ReturnsEmptyListWhenNoRecordsFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddHours(1); + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + + endDate) + ).ReturnsAsync(() => new PastRangeData()); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(0)); + } + + [Test] + public void GetCarbonIntensity_ThrowsWhenRegionNotFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddMinutes(1); + + this._locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Throws(); + + Assert.ThrowsAsync(async () => await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_ReturnsDefaultDuration_WhenOneDatapointReturned() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(1)); + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(TimeSpan.Zero)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_WhenMultipleDataPoints_ReturnsExpectedDuration() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + var expectedDuration = TimeSpan.FromHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + DateTime = startDate, + + }, + new CarbonIntensity() + { + DateTime= startDate + expectedDuration, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(expectedDuration)); + + var second = result.Skip(1)?.First(); + Assert.IsNotNull(second); + Assert.That(second.Duration, Is.EqualTo(expectedDuration)); + } } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs index 551891515..cb635adf4 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs @@ -32,7 +32,8 @@ public static IServiceCollection AddDataSourceService(this IServiceCollection se } case DataSourceType.ElectricityMaps: { - throw new ArgumentException("ElectricityMaps data source is not supported for emissions data"); + services.AddElectricityMapsEmissionsDataSource(dataSources); + break; } case DataSourceType.None: { diff --git a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs index ea6620d89..8bba52f7a 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs @@ -48,8 +48,6 @@ public async Task FakeEndPoint_ReturnsNotFound() [TestCase("2021-12-25", "2021-12-26", "westus")] public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Sets up any data endpoints needed for mocking purposes _dataSourceMocker?.SetupDataMock(start, end, location); @@ -72,8 +70,6 @@ public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset e [TestCase("non-location-param", "", TestName = "location param not present")] public async Task BestLocations_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Call the private method to construct with parameters var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -213,8 +209,6 @@ public async Task EmissionsForecastsBatch_SupportedDataSources_ReturnsOk(string [TestCase("2021-12-25", "2021-12-26", "westus", TestName = "EmissionsMarginalCarbonIntensity expects OK date only, no time")] public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, string end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/forecasts/current'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); @@ -246,8 +240,6 @@ public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, strin [TestCase("non-location-param", "", TestName = "EmissionsMarginalCarbonIntensity returns BadRequest for location not present")] public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -265,8 +257,6 @@ public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_Retu [TestCase("westus", "2022-3-1T15:30:00Z", "2022-3-1T18:00:00Z", TestName = "EmissionsMarginalCarbonIntensityBatch returns BadRequest for wrong date format")] public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_ReturnsBadRequest(string location, string startTime, string endTime) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var intesityData = Enumerable.Range(0, 1).Select(x => new { location = location, @@ -283,8 +273,6 @@ public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_Re [TestCase("2021-12-25", "2021-12-26", "westus", 3, TestName = "EmissionsMarginalCarbonIntensityBatch expects OK for multiple element batch")] public async Task EmissionsMarginalCarbonIntensityBatch_SupportedDataSources_ReturnsOk(string start, string end, string location, int nelems) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); diff --git a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs index 7fdf38ce9..3c7eb05b1 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs @@ -101,6 +101,7 @@ public void Setup() } case DataSourceType.ElectricityMaps: { + Environment.SetEnvironmentVariable("DataSources__EmissionsDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__ForecastDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__Type", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__APITokenHeader", "token"); From f45a946ef301cd75027fd12e8b0c3c3e42ab23b1 Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Mon, 12 Dec 2022 11:40:44 -0500 Subject: [PATCH 06/14] Add data-source-matrix docs --- docs/configuration.md | 9 +++++--- docs/data-source-matrix.md | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 3 deletions(-) create mode 100644 docs/data-source-matrix.md diff --git a/docs/configuration.md b/docs/configuration.md index 2c120e8f3..3af58b79f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -98,7 +98,8 @@ If set to `WattTime` or `ElectricityMaps`, the configuration specific to that da "ElectricityMaps": { "Type": "ElectricityMaps", "APITokenHeader": "auth-token", - "APIToken": "myAwesomeToken" + "APIToken": "myAwesomeToken", + "BaseURL": "https://api.electricitymap.org/v3/" }, "Json": { "Type": "Json", @@ -373,7 +374,8 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" "ElectricityMaps": { "Type": "ElectricityMaps", "APITokenHeader": "auth-token", - "APIToken": "token" + "APIToken": "token", + "BaseURL": "https://api.electricitymap.org/v3/" } } } @@ -396,7 +398,8 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" "ElectricityMaps": { "Type": "ElectricityMaps", "APITokenHeader": "auth-token", - "APIToken": "token" + "APIToken": "token", + "BaseURL": "https://api.electricitymap.org/v3/" } } } diff --git a/docs/data-source-matrix.md b/docs/data-source-matrix.md new file mode 100644 index 000000000..c35171f00 --- /dev/null +++ b/docs/data-source-matrix.md @@ -0,0 +1,45 @@ +# Data Source Matrix +The Carbon Aware SDK includes access to various data sources of carbon aware data, including WattTime, ElectricityMaps, and a custom JSON source. This matrix is an attempt to track what features of the Carbon Aware SDK are enabled for which data sources. + +## Type of DataSource +In the CarbonAware SDK configuration, you can set what data source to use as the `EmissionsDataSource` and the `ForecastDataSource`. +| Type | WattTime | ElectricityMaps | JSON | +|--------------|:-----------:|:-----------------:|:------:| +| Emissions DataSource | Yes | Yes | Yes || +| Forecast DataSource | Yes | Yes | No | + +## Data Source Routes Available +Not all data sources support all the routes provided in the interfaces (`IEmissionsDataSource`/`IForecastDataSource`). The list below maps the interface route to the relevant consumer call, while the table lists only the interface route. + +- GetCarbonIntensityAsync + - CLI: `emissions` + - API: `emissions/bylocation` / `emissions/bylocations` / `emissions/bylocations/best` / `emissions/average-carbon-intensity` / `average-carbon-intensity/batch` + - Library: `GetEmissionsDataAsync(...)` / `GetBestEmissionsDataAsync(...)` / `GetAverageCarbonIntensityDataAsync(...)` +- GetCurrentForecastAsync + - CLI: `emissions-forecasts` + - API: `forecasts/current` + - Library: `GetCurrentForecastAsync(...)` +- GetForecastByDateAsync + - CLI: `emissions-forecasts --requested-at` + - API: `forecasts/batch` with `requestedAt` field + - Library: `GetForecastByDateAsync(...)` + +| Route | WattTime | ElectricityMaps | JSON | +|--------------|:-----------:|:-----------------:|:------:| +| GetCarbonIntensityAsync | Yes | Yes | Yes | +| GetCurrentForecastAsync | Yes | Yes | No | +| GetForecastByDateAsync | Yes | No | No | + +## Data Source Configuration +| Configuration Type | WattTime | ElectricityMaps | JSON | +|--------------|:-----------:|:-----------------:|:------:| +| Authentication Required | Yes - username/password | Yes - token/header | N/A | +| BaseUrl Override | Only for testing | Switch between trial + full version, testing | N/A | +| Support trial + full account | Yes | Yes* (*different URL and token header required) | N/A | + + +## Miscellaneous +| Note | WattTime | ElectricityMaps | JSON | +|--------------|:-----------:|:-----------------:|:------:| +| Makes HTTP(s) call | Yes | Yes | No | +| Can use custom data | No | No | Yes | \ No newline at end of file From 7699ee17ba2bc6810646de7d7f7805d183a5afa0 Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Tue, 13 Dec 2022 10:58:27 -0500 Subject: [PATCH 07/14] updating tables --- docs/data-source-matrix.md | 63 ++++++++++++++++++++++---------------- 1 file changed, 36 insertions(+), 27 deletions(-) diff --git a/docs/data-source-matrix.md b/docs/data-source-matrix.md index c35171f00..559cc3bba 100644 --- a/docs/data-source-matrix.md +++ b/docs/data-source-matrix.md @@ -1,28 +1,42 @@ -# Data Source Matrix -The Carbon Aware SDK includes access to various data sources of carbon aware data, including WattTime, ElectricityMaps, and a custom JSON source. This matrix is an attempt to track what features of the Carbon Aware SDK are enabled for which data sources. +# Data Source Matrices -## Type of DataSource -In the CarbonAware SDK configuration, you can set what data source to use as the `EmissionsDataSource` and the `ForecastDataSource`. +The Carbon Aware SDK includes access to various data sources of carbon aware data, including WattTime, ElectricityMaps, and a custom JSON source. These matrices are an attempt to track what features of the Carbon Aware SDK are enabled for which data sources. + +## Contents + +- [Type of Data Sources and Configuration](#type-of-data-sources-and-configuration) +- [Data Source Routes Available](#data-source-routes-available) +- [Location Source Availability](#location-source-availability) + +## Type of Data Sources and Configuration + +In the CarbonAware SDK configuration, you can set what data source to use as the `EmissionsDataSource` and the `ForecastDataSource`. There are also certain configuration fields that must be set in order to access the raw data. | Type | WattTime | ElectricityMaps | JSON | -|--------------|:-----------:|:-----------------:|:------:| -| Emissions DataSource | Yes | Yes | Yes || -| Forecast DataSource | Yes | Yes | No | +|--------------------------|-----------|-----------------|------| +| Is Emissions DataSource | Yes | Yes | Yes | +| Is Forecast DataSource | Yes | Yes | No | +| Makes HTTP(s) call | Yes | Yes | No | +| Can Use Custom Data | No | No | Yes | +| Needs Authentication Config | Yes - username/password | Yes - token/header | No | +| BaseUrl Override Config | Only for testing | Switch between trial + full version, testing | No | +| Supports Trial + Full Account | Yes | Yes (*different URL and token header required) | N/A | ## Data Source Routes Available + Not all data sources support all the routes provided in the interfaces (`IEmissionsDataSource`/`IForecastDataSource`). The list below maps the interface route to the relevant consumer call, while the table lists only the interface route. - GetCarbonIntensityAsync - - CLI: `emissions` - - API: `emissions/bylocation` / `emissions/bylocations` / `emissions/bylocations/best` / `emissions/average-carbon-intensity` / `average-carbon-intensity/batch` - - Library: `GetEmissionsDataAsync(...)` / `GetBestEmissionsDataAsync(...)` / `GetAverageCarbonIntensityDataAsync(...)` + - CLI: `emissions` + - API: `emissions/bylocation` / `emissions/bylocations` / `emissions/bylocations/best` / `emissions/average-carbon-intensity` / `average-carbon-intensity/batch` + - Library: `GetEmissionsDataAsync(...)` / `GetBestEmissionsDataAsync(...)` / `GetAverageCarbonIntensityDataAsync(...)` - GetCurrentForecastAsync - - CLI: `emissions-forecasts` - - API: `forecasts/current` - - Library: `GetCurrentForecastAsync(...)` + - CLI: `emissions-forecasts` + - API: `forecasts/current` + - Library: `GetCurrentForecastAsync(...)` - GetForecastByDateAsync - - CLI: `emissions-forecasts --requested-at` - - API: `forecasts/batch` with `requestedAt` field - - Library: `GetForecastByDateAsync(...)` + - CLI: `emissions-forecasts --requested-at` + - API: `forecasts/batch` with `requestedAt` field + - Library: `GetForecastByDateAsync(...)` | Route | WattTime | ElectricityMaps | JSON | |--------------|:-----------:|:-----------------:|:------:| @@ -30,16 +44,11 @@ Not all data sources support all the routes provided in the interfaces (`IEmissi | GetCurrentForecastAsync | Yes | Yes | No | | GetForecastByDateAsync | Yes | No | No | -## Data Source Configuration -| Configuration Type | WattTime | ElectricityMaps | JSON | -|--------------|:-----------:|:-----------------:|:------:| -| Authentication Required | Yes - username/password | Yes - token/header | N/A | -| BaseUrl Override | Only for testing | Switch between trial + full version, testing | N/A | -| Support trial + full account | Yes | Yes* (*different URL and token header required) | N/A | +## Location Source Availability +*I wasn't sure exactly what to fill here but wanted something along the lines of: -## Miscellaneous -| Note | WattTime | ElectricityMaps | JSON | -|--------------|:-----------:|:-----------------:|:------:| -| Makes HTTP(s) call | Yes | Yes | No | -| Can use custom data | No | No | Yes | \ No newline at end of file +- WattTime is better to capture data for XX locations +- Electricity Maps is better to capture data for YY locations +- Electricity Maps can take a zone name or lat/long +- WattTime takes a region name (azure region?) or lat long From 0fc025541e4eac341615ed2f6a4767d5aef588bd Mon Sep 17 00:00:00 2001 From: Priti Date: Tue, 22 Nov 2022 10:35:28 -0500 Subject: [PATCH 08/14] [M2][#174] ElectricityMaps Data Source - Emissions --- docs/configuration.md | 63 +++-- .../mock/ElectricityMapDataSourceMocker.cs | 40 ++- .../src/Client/ElectricityMapsClient.cs | 103 ++++++-- .../src/Client/IElectricityMapsClient.cs | 17 ++ .../ElectricityMapsClientConfiguration.cs | 17 ++ .../ServiceCollectionExtensions.cs | 4 +- .../src/Constants/Paths.cs | 1 + .../src/Constants/QueryStrings.cs | 4 + .../src/ElectricityMapsDataSource.cs | 108 ++++++++- .../src/Model/EmissionsFactor.cs | 13 - .../src/Model/HistoryCarbonIntensityData.cs | 33 ++- .../test/Client/ElectricityMapsClientTests.cs | 8 +- .../test/Client/TestData.cs | 2 +- .../test/ElectricityMapsDataSourceTests.cs | 228 ++++++++++++++++++ .../ServiceCollectionExtensions.cs | 3 +- .../CarbonAwareControllerTests.cs | 12 - .../IntegrationTestingBase.cs | 1 + 17 files changed, 577 insertions(+), 80 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs diff --git a/docs/configuration.md b/docs/configuration.md index 4932749f9..2c120e8f3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,4 +1,3 @@ - - [Configuration](#configuration) - [Logging](#logging) - [DataSources](#datasources) @@ -10,9 +9,11 @@ - [WattTime Caching BalancingAuthority](#watttime-caching-balancingauthority) - [Json Configuration](#json-configuration) - [ElectricityMaps Configuration](#electricitymaps-configuration) - - [ApiTokenHeader](#api-token-header) - - [ApiToken](#api-token) - - [baseUrl](#baseurl) + - [API Token Header](#api-token-header) + - [API Token](#api-token) + - [BaseUrl](#baseurl-1) + - [Emission Factor Type](#emission-factor-type) + - [Disable Estimations](#disable-estimations) - [CarbonAwareVars](#carbonawarevars) - [Tracing and Monitoring Configuration](#tracing-and-monitoring-configuration) - [Verbosity](#verbosity) @@ -69,10 +70,10 @@ Logging__LogLevel__Default="Debug" dotnet run ## DataSources -The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file and [WattTime](https://www.watttime.org/) are supported. +The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and ElectricityMaps (https://www.electricitymaps.com/) are supported. Each data source interface is configured with a specific data source implementation. -If set to `WattTime`, WattTime configuration must also be supplied. +If set to `WattTime` or `ElectricityMaps`, the configuration specific to that data provider must also be supplied. `JSON` will result in the data being loaded from the file specified in the `DataFileLocation` property @@ -93,6 +94,11 @@ If set to `WattTime`, WattTime configuration must also be supplied. "username": "proxyUsername", "password": "proxyPassword" } + }, + "ElectricityMaps": { + "Type": "ElectricityMaps", + "APITokenHeader": "auth-token", + "APIToken": "myAwesomeToken" }, "Json": { "Type": "Json", @@ -115,7 +121,7 @@ If using the WattTime data source, WattTime configuration is required. } ``` -> **Sign up for a test account:** To create an account, follow these steps : https://www.watttime.org/api-documentation/#best-practices-for-api-usage +> **Sign up for a test account:** To create an account, follow these steps [from the WattTime documentation](https://www.watttime.org/api-documentation/#best-practices-for-api-usage) #### username @@ -170,7 +176,17 @@ info: CarbonAware.DataSources.Json.JsonDataSource[0] If using the ElectricityMaps data source, ElectricityMaps configuration is required. -__With an account token:__ +> **NOTE** +> The ElectricityMaps API does not currently support access to historical forecasts. +> This means that functionality such as the CLI `emissions-forecasts` `--requested-at` flag +> and the API `/forecasts/batch` `requestedAt` input will respond with a `NotImplemented` error. +> +> Depending on the goal, the historical measured `emissions` commands may be a reasonable workaround. +> This would treat the measured emissions as a "perfect historical forecast" effectively. +> Otherwise, use a data source that has support for historical forecasts, such as [WattTime](#watttime-configuration). + +**With an account token:** + ```json { "APITokenHeader": "auth-token", @@ -179,7 +195,8 @@ __With an account token:__ } ``` -__With a free trial token:__ +**With a free trial token:** + ```json { "APITokenHeader": "X-BLOBR-KEY", @@ -188,20 +205,32 @@ __With a free trial token:__ } ``` -> **Sign up for a free trial:** To get a free trial: https://api-portal.electricitymaps.com/ +> **Sign up for a free trial:** Select the free trial product from [the ElectricityMaps catalog](https://api-portal.electricitymaps.com/) #### API Token Header -The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOB-KEY" +The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOBR-KEY" #### API Token The ElectricityMaps token you receive with your account or free trial. -#### baseUrl +#### BaseUrl The url to use when connecting to ElectricityMaps. Defaults to "https://api.electricitymap.org/v3/" but can be overridden in the config if needed (such as for free-trial users or enable integration testing scenarios). +#### Emission Factor Type + +String value for the optional `emissionFactorType` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#emission-factors) for more details and valid values. + +#### Disable Estimations + +Boolean value for the optional `disableEstimations` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#estimations) for more details. + ## CarbonAwareVars This section contains the global settings for the SDK. The configuration looks like this: @@ -230,7 +259,7 @@ This application is integrated with Application Insights for monitoring purposes ApplicationInsights_Connection_String="AppInsightsConnectionString" ``` -You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net. +You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to [the documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net). ```bash AppInsights_InstrumentationKey="AppInsightsInstrumentationKey" @@ -335,6 +364,7 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" ``` ## Configuration for Forecast data Using ElectricityMaps + ```json { "DataSources": { @@ -350,11 +380,12 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" } ``` -## Configuration for Emissions data Using WattTime and Forecast data Using ElectricityMaps +## Configuration for Emissions data using ElectricityMaps and Forecast data using WattTime + ```json "DataSources": { - "EmissionsDataSource": "WattTime", - "ForecastDataSource": "ElectricityMaps", + "EmissionsDataSource": "ElectricityMaps", + "ForecastDataSource": "WattTime", "Configurations": { "WattTime": { "Type": "WattTime", diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs index cf86c02ab..4a2875133 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs @@ -26,19 +26,19 @@ public ElectricityMapsDataSourceMocker() public void SetupHistoryMock(decimal latitude, decimal longitude) { - var data = new List(); + var data = new List(); DateTimeOffset now = DateTimeOffset.UtcNow; DateTimeOffset past24 = now.AddHours(-24); while (past24 < now) { - var newDataPoint = new HistoryCarbonIntensity() + var newDataPoint = new CarbonIntensity() { - CarbonIntensity = 999, + Value = 999, DateTime = past24, UpdatedAt = now, CreatedAt = now, - EmissionFactorType = EmissionsFactor.Lifecycle, + EmissionFactorType = "lifecycle", IsEstimated = false, EstimationMethod = null }; @@ -101,15 +101,29 @@ private void SetupZonesMock() SetupResponseGivenGetRequest(Paths.Zones, result); } - public void Initialize() => SetupZonesMock(); + public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) + { + var data = new List(); + DateTimeOffset pointTime = start; + TimeSpan duration = TimeSpan.FromHours(1); - public void Reset() => _server.Reset(); + while (pointTime < end) + { + var newDataPoint = new CarbonIntensity() + { + Value = 100, + DateTime = pointTime, + }; - public void Dispose() => _server.Dispose(); + data.Add(newDataPoint); + pointTime = newDataPoint.DateTime + duration; + } - public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) - { - throw new NotImplementedException(); + HistoryCarbonIntensityData history = new() { HistoryData = data }; + PastRangeData pastRange = new() { HistoryData = data }; + + SetupResponseGivenGetRequest(Paths.History, history); + SetupResponseGivenGetRequest(Paths.PastRange, pastRange); } public void SetupBatchForecastMock() @@ -117,6 +131,12 @@ public void SetupBatchForecastMock() throw new NotImplementedException(); } + public void Initialize() => SetupZonesMock(); + + public void Reset() => _server.Reset(); + + public void Dispose() => _server.Dispose(); + private void SetupResponseGivenGetRequest(string path, object body) { var jsonBody = JsonSerializer.Serialize(body, _options); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs index 00c766e53..0cc295214 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs @@ -3,6 +3,7 @@ using CarbonAware.DataSources.ElectricityMaps.Model; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; +using System.Globalization; using System.Net.Http.Headers; using System.Net.Mime; using System.Text.Json; @@ -17,7 +18,7 @@ internal class ElectricityMapsClient : IElectricityMapsClient private readonly IOptionsMonitor _configurationMonitor; private ElectricityMapsClientConfiguration _configuration => this._configurationMonitor.CurrentValue; private readonly ILogger _log; - private Lazy>> _zonesAllowed; + private readonly Lazy>> _zonesAllowed; public ElectricityMapsClient(IHttpClientFactory factory, IOptionsMonitor monitor, ILogger log) { @@ -64,16 +65,6 @@ public async Task GetRecentCarbonIntensityHistoryAsy return await GetHistoryCarbonIntensityDataAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to History endpoint - private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) - { - await CheckZonesAllowedForPath(Paths.History, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.History, parameters)) - { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); - } - } - /// public async Task GetForecastedCarbonIntensityAsync (string zoneName) { @@ -103,14 +94,92 @@ public async Task GetForecastedCarbonIntensityAsy return await GetCurrentForecastAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to Forecast endpoint - private async Task GetCurrentForecastAsync(Dictionary parameters) + /// + public async Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using latitude {latitude} longitude {longitude}", + latitude, longitude); + + var parameters = new Dictionary() + { + { QueryStrings.Latitude, latitude }, + { QueryStrings.Longitude, longitude }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + /// + public async Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using zone {zone}", + zone); + + var parameters = new Dictionary() + { + { QueryStrings.ZoneName, zone }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + // The ElectricityMaps API has strict checks about datetime formatting. + // This helper method ensures that all DateTimeOffsets are properly formatted. + private static string DateTimeToString(DateTimeOffset dateTime) + { + return dateTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture); + } + + /// + /// Async method to check for allowed zones and then make GET request to History endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.History, parameters); + AddOptionalParameters(parameters); + + using Stream result = await this.MakeRequestGetStreamAsync(Paths.History, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); + } + + private void AddOptionalParameters(Dictionary parameters) { - await CheckZonesAllowedForPath(Paths.Forecast, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters)) + if (_configuration.EmissionFactorType != null) { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); + parameters.Add(QueryStrings.EmissionFactorType, _configuration.EmissionFactorType); } + if (_configuration.DisableEstimations != null) + { + parameters.Add(QueryStrings.DisableEstimations, _configuration.DisableEstimations.ToString()!.ToLowerInvariant()); + } + } + + private async Task GetPastRangeDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.PastRange, parameters); + AddOptionalParameters(parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.PastRange, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting emissions data"); + } + + /// + /// Async method to check for allowed zones and then make GET request to Forecast endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetCurrentForecastAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.Forecast, parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); } private async Task GetResponseAsync(string uriPath) @@ -157,7 +226,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q } // Checks the current supported client's endpoint paths. - private async Task CheckZonesAllowedForPath(string path, Dictionary parameters) + private async Task CheckZonesAllowedForPathAsync(string path, Dictionary parameters) { // Parameters don't contain a ZoneName to check, exit if (!parameters.ContainsKey(QueryStrings.ZoneName)) return; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs index fd7864ff8..d6e371e6d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs @@ -42,4 +42,21 @@ public interface IElectricityMapsClient /// A which contains all emissions data points in the 24 hour period. /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. public Task GetRecentCarbonIntensityHistoryAsync(string zoneName); + + /// + /// Async method to get the historical observed emission data for a given latitude and longitude over a given time period. + /// + /// Latitude for query + /// Longitude for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime); + + /// + /// Async method to get the historical observed emission data for a given zone over a given time period. + /// + /// Zone name for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs index 30661f5f9..2d050e60f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs @@ -1,5 +1,6 @@ using CarbonAware.Exceptions; using CarbonAware.DataSources.ElectricityMaps.Constants; +using CarbonAware.DataSources.ElectricityMaps.Model; namespace CarbonAware.DataSources.ElectricityMaps.Configuration; @@ -23,6 +24,22 @@ public class ElectricityMapsClientConfiguration /// public string BaseUrl { get; set; } = BaseUrls.TokenBaseUrl; + /// + /// Gets or sets the optional emissionFactorType parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#emission-factors for valid types + /// + public string? EmissionFactorType { get; set; } + + /// + /// Gets or sets the optional disableEstimations parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#estimations for details + /// + public bool? DisableEstimations { get; set; } + /// /// Validate that this object is properly configured. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs index 378146af1..1cdcf76c0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs @@ -20,7 +20,9 @@ public static IServiceCollection AddElectricityMapsForecastDataSource(this IServ public static IServiceCollection AddElectricityMapsEmissionsDataSource(this IServiceCollection services, DataSourcesConfiguration dataSourcesConfig) { - throw new NotImplementedException(); + AddElectricityMapsClient(services, dataSourcesConfig.EmissionsConfigurationSection()); + services.TryAddSingleton(); + return services; } private static void AddElectricityMapsClient(IServiceCollection services, IConfigurationSection configSection) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs index 3c4fc87de..c21bfbfb7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs @@ -4,5 +4,6 @@ internal class Paths { public const string History = "carbon-intensity/history"; public const string Forecast = "carbon-intensity/forecast"; + public const string PastRange = "carbon-intensity/past-range"; public const string Zones = "zones"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs index 27c33ecd9..d9bcc22c2 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs @@ -5,4 +5,8 @@ internal class QueryStrings public const string Latitude = "lat"; public const string Longitude = "lon"; public const string ZoneName = "zone"; + public const string StartTime = "start"; + public const string EndTime = "end"; + public const string DisableEstimations = "disableEstimations"; + public const string EmissionFactorType = "emissionFactorType"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs index fff633851..81ca4b5eb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs @@ -1,5 +1,6 @@ using CarbonAware.DataSources.ElectricityMaps.Client; using CarbonAware.DataSources.ElectricityMaps.Model; +using CarbonAware.Exceptions; using CarbonAware.Interfaces; using CarbonAware.Model; using Microsoft.Extensions.Logging; @@ -10,7 +11,7 @@ namespace CarbonAware.DataSources.ElectricityMaps; /// /// Represents a Electricity Maps data source. /// -public class ElectricityMapsDataSource : IForecastDataSource +public class ElectricityMapsDataSource : IForecastDataSource, IEmissionsDataSource { public string _name => "ElectricityMapsDataSource"; @@ -82,4 +83,109 @@ public async Task GetCarbonIntensityForecastAsync(Location lo await Task.Run(() => true); throw new NotImplementedException(); } + + /// + public async Task> GetCarbonIntensityAsync(IEnumerable locations, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + this._logger.LogDebug("Getting carbon intensity for locations {locations} for period {periodStartTime} to {periodEndTime}.", locations, periodStartTime, periodEndTime); + using (var activity = _activity.StartActivity()) + { + List result = new(); + foreach (var location in locations) + { + IEnumerable interimResult = await GetCarbonIntensityAsync(location, periodStartTime, periodEndTime); + result.AddRange(interimResult); + } + return result; + } + } + + /// + public async Task> GetCarbonIntensityAsync(Location location, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + using (var activity = _activity.StartActivity()) + { + var geolocation = await this._locationSource.ToGeopositionLocationAsync(location); + IEnumerable historyCarbonIntensity; + DateTime now = DateTime.UtcNow; + var isDateRangeWithin24Hours = (periodStartTime > now.AddHours(-24) && periodStartTime <= now) && (periodEndTime > now.AddHours(-24) && periodEndTime <= now); + if (isDateRangeWithin24Hours) + { + historyCarbonIntensity = await GetRecentCarbonInstensityData(geolocation); + } + else + { + historyCarbonIntensity = await GetPastCarbonIntensityData(periodStartTime, periodEndTime, geolocation); + } + + return HistoryCarbonIntensityToEmissionsData(location, historyCarbonIntensity, periodStartTime, periodEndTime); + } + } + + private async Task> GetPastCarbonIntensityData(DateTimeOffset periodStartTime, DateTimeOffset periodEndTime, Location geolocation) + { + PastRangeData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? "", periodStartTime, periodEndTime); + else + { + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Name ?? "", periodStartTime, periodEndTime); + } + + return data.HistoryData; + } + + private async Task> GetRecentCarbonInstensityData(Location geolocation) + { + HistoryCarbonIntensityData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? ""); + else + { + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Name ?? ""); + } + + return data.HistoryData; + } + + private IEnumerable HistoryCarbonIntensityToEmissionsData(Location location, IEnumerable data, DateTimeOffset startTime, DateTimeOffset endTime) + { + IEnumerable emissions; + var duration = GetDurationFromHistoryDataPointsOrDefault(data, TimeSpan.Zero); + emissions = data.Select(d => + { + var emission = (EmissionsData) d; + emission.Location = location.Name; + emission.Time = d.DateTime; + emission.Duration = duration; + return emission; + }); + + return emissions; + } + + private TimeSpan GetDurationFromHistoryDataPointsOrDefault(IEnumerable carbonIntensityDataPoints, TimeSpan defaultValue) + { + try + { + return GetDurationFromHistoryDataPoints(carbonIntensityDataPoints); + } + catch (CarbonAwareException) + { + return defaultValue; + } + } + + private TimeSpan GetDurationFromHistoryDataPoints(IEnumerable dataPoints) + { + var firstPoint = dataPoints.FirstOrDefault(); + var secondPoint = dataPoints.Skip(1)?.FirstOrDefault(); + + var first = firstPoint ?? throw new CarbonAwareException("Too few data points returned"); + var second = secondPoint ?? throw new CarbonAwareException("Too few data points returned"); + + // Handle chronological and reverse-chronological data by using `.Duration()` to get + // the absolute value of the TimeSpan between the two points. + return first.DateTime.Subtract(second.DateTime).Duration(); + } } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs deleted file mode 100644 index 27ef63379..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CarbonAware.DataSources.ElectricityMaps.Model; - -/// -/// Type of EmissionFactor to use for calculating carbon intensity as described in the Electricity Maps documentation - https://static.electricitymaps.com/api/docs/index.html#emission-factors -/// -[JsonConverter(typeof(JsonStringEnumConverter))] -public enum EmissionsFactor -{ - Lifecycle, - Direct -} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs index 5f0326d70..b23d658b0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs @@ -1,3 +1,5 @@ +using CarbonAware.Exceptions; +using CarbonAware.Model; using System.Text.Json.Serialization; namespace CarbonAware.DataSources.ElectricityMaps.Model; @@ -18,20 +20,20 @@ public record HistoryCarbonIntensityData /// List of History Carbon Intensity instances. /// [JsonPropertyName("history")] - public IEnumerable HistoryData { get; init; } = Array.Empty(); + public IEnumerable HistoryData { get; init; } = Array.Empty(); } /// /// A history carbon intensity. /// [Serializable] -public record HistoryCarbonIntensity +public record CarbonIntensity { /// /// Carbon Intensity value. /// [JsonPropertyName("carbonIntensity")] - public int CarbonIntensity { get; init; } + public int Value { get; init; } /// /// Indicates the datetime of the carbon intensity @@ -55,7 +57,7 @@ public record HistoryCarbonIntensity /// Indicated the emission factor type used for computing the carbon intensity. /// [JsonPropertyName("emissionFactorType")] - public EmissionsFactor EmissionFactorType { get; init; } + public string? EmissionFactorType { get; init; } /// /// Indicates whether the result is estimated or no @@ -69,4 +71,27 @@ public record HistoryCarbonIntensity [JsonPropertyName("estimationMethod")] public string? EstimationMethod { get; init; } + public static explicit operator EmissionsData(CarbonIntensity historyCarbonIntensity) + { + return new EmissionsData + { + Rating = historyCarbonIntensity.Value, + Time = historyCarbonIntensity.UpdatedAt, + }; + } } + +/// +/// Carbon intensity data for past date range. +/// +[Serializable] +public record PastRangeData +{ + /// + /// Carbon Intensity value. + /// + [JsonPropertyName("data")] + public IEnumerable HistoryData { get; init; } = Array.Empty(); + +} + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs index c8f47f8f5..6afd06ece 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs @@ -53,8 +53,8 @@ public void ClientInstantiation_FailsForInvalidConfig(string baseUrl) CreateBasicClient(TestData.GetZonesAllowedJsonString(), "{}"); this.Configuration = new ElectricityMapsClientConfiguration() { - APITokenHeader = null, - APIToken = null, + APITokenHeader = "", + APIToken = "", BaseUrl = baseUrl, }; @@ -254,8 +254,8 @@ public async Task GetRecentCarbonIntensityHistoryAsync_DeserializesExpectedRespo Assert.That(dataPoint?.DateTime, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.UpdatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.CreatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); - Assert.That(dataPoint?.CarbonIntensity, Is.EqualTo(999)); - Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo(EmissionsFactor.Lifecycle)); + Assert.That(dataPoint?.Value, Is.EqualTo(999)); + Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo("lifecycle")); Assert.That(dataPoint?.IsEstimated, Is.False); Assert.That(dataPoint?.EstimationMethod, Is.Null); }); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs index 9c028332d..48db3dab9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs @@ -63,7 +63,7 @@ public static string GetHistoryCarbonIntensityDataJsonString() ["updatedAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["createdAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["carbonIntensity"] = 999, - ["emissionFactorType"] = "Lifecycle", + ["emissionFactorType"] = "lifecycle", ["isEstimated"] = false, ["estimatedMethod"] = null, } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs index e39d37397..444868f5d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs @@ -96,4 +96,232 @@ public void GetCurrentCarbonIntensityForecastAsync_ThrowsWhenRegionNotFound() Assert.ThrowsAsync(async () => await _dataSource.GetCurrentCarbonIntensityForecastAsync(_defaultLocation)); } + + [TestCase(8, 1, 1, 0, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when date within 24 hours")] + [TestCase(36, 1, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when date outside of 24 hours")] + [TestCase(30, 12, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when start date outside of 24 hours but enddate within 24 hours")] + [TestCase(8, 12, 0, 1, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when start date within 24 hours but enddate outside of 24 hours")] + public async Task GetCarbonIntensity_CallsExpectedClientEndpoint(int startTimeOffset, int endTimeOffset, int expectedHistoryCalls, int expectedPastRangeCalls) + { + var now = DateTimeOffset.UtcNow; + var startDate = now.AddHours(-startTimeOffset); + var endDate = startDate.AddHours(endTimeOffset); + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new(); + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + PastRangeData pastRange = new(); + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => pastRange); + + await _dataSource.GetCarbonIntensityAsync(_defaultLocation, startDate, endDate); + + _electricityMapsClient.Verify(c => c.GetPastRangeDataAsync(_defaultLatitude, _defaultLongitude, startDate, endDate), Times.Exactly(expectedPastRangeCalls)); + + _electricityMapsClient.Verify(c => c.GetRecentCarbonIntensityHistoryAsync(_defaultLatitude, _defaultLongitude), Times.Exactly(expectedHistoryCalls)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeWithin24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-10); + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeMore24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = _defaultDataStartTime; + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + PastRangeData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_PastRange_ReturnsEmptyListWhenNoRecordsFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddHours(1); + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + + endDate) + ).ReturnsAsync(() => new PastRangeData()); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(0)); + } + + [Test] + public void GetCarbonIntensity_ThrowsWhenRegionNotFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddMinutes(1); + + this._locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Throws(); + + Assert.ThrowsAsync(async () => await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_ReturnsDefaultDuration_WhenOneDatapointReturned() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(1)); + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(TimeSpan.Zero)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_WhenMultipleDataPoints_ReturnsExpectedDuration() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + var expectedDuration = TimeSpan.FromHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + DateTime = startDate, + + }, + new CarbonIntensity() + { + DateTime= startDate + expectedDuration, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(expectedDuration)); + + var second = result.Skip(1)?.First(); + Assert.IsNotNull(second); + Assert.That(second.Duration, Is.EqualTo(expectedDuration)); + } } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs index 551891515..cb635adf4 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs @@ -32,7 +32,8 @@ public static IServiceCollection AddDataSourceService(this IServiceCollection se } case DataSourceType.ElectricityMaps: { - throw new ArgumentException("ElectricityMaps data source is not supported for emissions data"); + services.AddElectricityMapsEmissionsDataSource(dataSources); + break; } case DataSourceType.None: { diff --git a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs index ea6620d89..8bba52f7a 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs @@ -48,8 +48,6 @@ public async Task FakeEndPoint_ReturnsNotFound() [TestCase("2021-12-25", "2021-12-26", "westus")] public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Sets up any data endpoints needed for mocking purposes _dataSourceMocker?.SetupDataMock(start, end, location); @@ -72,8 +70,6 @@ public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset e [TestCase("non-location-param", "", TestName = "location param not present")] public async Task BestLocations_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Call the private method to construct with parameters var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -213,8 +209,6 @@ public async Task EmissionsForecastsBatch_SupportedDataSources_ReturnsOk(string [TestCase("2021-12-25", "2021-12-26", "westus", TestName = "EmissionsMarginalCarbonIntensity expects OK date only, no time")] public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, string end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/forecasts/current'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); @@ -246,8 +240,6 @@ public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, strin [TestCase("non-location-param", "", TestName = "EmissionsMarginalCarbonIntensity returns BadRequest for location not present")] public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -265,8 +257,6 @@ public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_Retu [TestCase("westus", "2022-3-1T15:30:00Z", "2022-3-1T18:00:00Z", TestName = "EmissionsMarginalCarbonIntensityBatch returns BadRequest for wrong date format")] public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_ReturnsBadRequest(string location, string startTime, string endTime) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var intesityData = Enumerable.Range(0, 1).Select(x => new { location = location, @@ -283,8 +273,6 @@ public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_Re [TestCase("2021-12-25", "2021-12-26", "westus", 3, TestName = "EmissionsMarginalCarbonIntensityBatch expects OK for multiple element batch")] public async Task EmissionsMarginalCarbonIntensityBatch_SupportedDataSources_ReturnsOk(string start, string end, string location, int nelems) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); diff --git a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs index 7fdf38ce9..3c7eb05b1 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs @@ -101,6 +101,7 @@ public void Setup() } case DataSourceType.ElectricityMaps: { + Environment.SetEnvironmentVariable("DataSources__EmissionsDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__ForecastDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__Type", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__APITokenHeader", "token"); From 88f0d1de682a42bcb1b9663c2b4b405fd667950b Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Tue, 13 Dec 2022 14:29:02 -0500 Subject: [PATCH 09/14] Move WattTimeClient to live within datasource --- docs/packaging.md | 1 - ...bonAware.DataSources.WattTime.Mocks.csproj | 2 +- .../mock/WattTimeDataSourceMocker.cs | 4 +- .../CarbonAware.DataSources.WattTime.csproj | 12 +- .../src/Client}/IWattTimeClient.cs | 6 +- .../src/Client/InternalsVisibleTo.cs | 4 + .../src/Client}/WattTimeClient.cs | 28 ++-- .../src/Client}/WattTimeClientException.cs | 2 +- .../Client}/WattTimeClientHttpException.cs | 2 +- .../ServiceCollectionExtensions.cs | 3 +- .../WattTimeClientConfiguration.cs | 4 +- .../Constants/AuthenticationHeaderTypes.cs | 2 +- .../src/Constants/Paths.cs | 2 +- .../src/Constants/QueryStrings.cs | 2 +- .../src/InternalsVisibleTo.cs | 2 +- .../src/Model/BalancingAuthority.cs | 2 +- .../src/Model/Forecast.cs | 2 +- .../src/Model/GridEmissionDataPoint.cs | 2 +- .../src/Model/LoginResult.cs | 2 +- .../src/WattTimeDataSource.cs | 8 +- .../test/Client}/TestData.cs | 2 +- .../test/Client}/WattTimeClientTests.cs | 123 ++++++------------ .../ServiceCollectionExtensionTests.cs | 70 ++++++++++ .../WattTimeClientConfigurationTests.cs | 44 +++++++ .../test/WattTimeDataSourceTests.cs | 4 +- .../CarbonAware.Tools.WattTimeClient.csproj | 23 ---- .../Configuration/ConfigurationException.cs | 11 -- .../ServiceCollectionExtensions.cs | 62 --------- .../src/InternalsVisibleTo.cs | 4 - .../test/AssemblyInfo.cs | 3 - ...bonAware.Tools.WattTimeClient.Tests.csproj | 29 ----- src/CarbonAwareSDK.sln | 12 -- .../test/EmissionsHandlerTests.cs | 7 +- 33 files changed, 211 insertions(+), 275 deletions(-) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client}/IWattTimeClient.cs (98%) create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client}/WattTimeClient.cs (97%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client}/WattTimeClientException.cs (80%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client}/WattTimeClientHttpException.cs (97%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Configuration/WattTimeClientConfiguration.cs (94%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Constants/AuthenticationHeaderTypes.cs (69%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Constants/Paths.cs (82%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Constants/QueryStrings.cs (85%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Model/BalancingAuthority.cs (93%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Model/Forecast.cs (92%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Model/GridEmissionDataPoint.cs (96%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient => CarbonAware.DataSources/CarbonAware.DataSources.WattTime}/src/Model/LoginResult.cs (88%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client}/TestData.cs (97%) rename src/{CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test => CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client}/WattTimeClientTests.cs (84%) create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/WattTimeClientConfigurationTests.cs delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/CarbonAware.Tools.WattTimeClient.csproj delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ConfigurationException.cs delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ServiceCollectionExtensions.cs delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/InternalsVisibleTo.cs delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/AssemblyInfo.cs delete mode 100644 src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/CarbonAware.Tools.WattTimeClient.Tests.csproj diff --git a/docs/packaging.md b/docs/packaging.md index e6d3637b3..65d336a94 100644 --- a/docs/packaging.md +++ b/docs/packaging.md @@ -14,7 +14,6 @@ The current package include 8 projects from the SDK: 6. "CarbonAware.DataSources.Registration" 7. "CarbonAware.DataSources.WattTime" 8. "CarbonAware.LocationSources.Azure" -9. "CarbonAware.Tools.WattTimeClient" These 8 projects enable users of the library to consume the current endpoints exposed by the library. The package that needs to be added to a new C# project is `GSF.CarbonAware`. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj index c8f3fa6a1..5fd613739 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj @@ -13,7 +13,7 @@ - + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs index 7468a11db..44252d4eb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs @@ -1,6 +1,6 @@ using CarbonAware.DataSources.Mocks; -using CarbonAware.Tools.WattTimeClient.Constants; -using CarbonAware.Tools.WattTimeClient.Model; +using CarbonAware.DataSources.WattTime.Constants; +using CarbonAware.DataSources.WattTime.Model; using System.Net; using System.Net.Mime; using System.Text.Json; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj index 18c784ceb..213fca4a1 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj @@ -1,4 +1,4 @@ - + net6.0 @@ -9,7 +9,15 @@ - + + + + + + + + + diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs similarity index 98% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/IWattTimeClient.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index 1661eae4a..8830f57a9 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -1,6 +1,6 @@ -using CarbonAware.Tools.WattTimeClient.Model; +using CarbonAware.DataSources.WattTime.Model; -namespace CarbonAware.Tools.WattTimeClient; +namespace CarbonAware.DataSources.WattTime.Client; /// /// An interface for interacting with the WattTime API. @@ -8,7 +8,7 @@ namespace CarbonAware.Tools.WattTimeClient; public interface IWattTimeClient { public const string NamedClient = "WattTimeClient"; - + /// /// Async method to get observed emission data for a given balancing authority and time period. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs new file mode 100644 index 000000000..65ae2dda4 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs @@ -0,0 +1,4 @@ +using System.Runtime.CompilerServices; + +[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Client.Tests")] +[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Mocks")] diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs similarity index 97% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClient.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 4bf13495e..fbfcd1d4b 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -1,19 +1,19 @@ -using Microsoft.Extensions.Options; +using CarbonAware.DataSources.WattTime.Configuration; +using CarbonAware.DataSources.WattTime.Constants; +using CarbonAware.DataSources.WattTime.Model; +using Microsoft.Extensions.Caching.Memory; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using System.Diagnostics; +using System.Globalization; +using System.Net; using System.Net.Http.Headers; +using System.Net.Mime; using System.Text; using System.Text.Json; -using CarbonAware.Tools.WattTimeClient.Model; using System.Web; -using System.Diagnostics; -using Microsoft.Extensions.Logging; -using System.Net.Mime; -using System.Net; -using CarbonAware.Tools.WattTimeClient.Configuration; -using CarbonAware.Tools.WattTimeClient.Constants; -using System.Globalization; -using Microsoft.Extensions.Caching.Memory; -namespace CarbonAware.Tools.WattTimeClient; +namespace CarbonAware.DataSources.WattTime.Client; public class WattTimeClient : IWattTimeClient { @@ -21,7 +21,7 @@ public class WattTimeClient : IWattTimeClient private static readonly HttpStatusCode[] RetriableStatusCodes = new HttpStatusCode[] { - HttpStatusCode.Unauthorized, + HttpStatusCode.Unauthorized, HttpStatusCode.Forbidden }; @@ -291,7 +291,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q // this will get a specialized namevalue collection for formatting query strings. var query = HttpUtility.ParseQueryString(string.Empty); - foreach(var kvp in queryStringParams) + foreach (var kvp in queryStringParams) { query[kvp.Key] = kvp.Value; } @@ -308,7 +308,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q private async Task GetBalancingAuthorityFromCacheAsync(string latitude, string longitude) { - var key = new Tuple( latitude, longitude ); + var key = new Tuple(latitude, longitude); var balancingAuthority = await this.memoryCache.GetOrCreateAsync(key, async entry => { var parameters = new Dictionary() diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientException.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientException.cs similarity index 80% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientException.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientException.cs index d6c4413cb..a2bb7714f 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientException.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientException.cs @@ -1,6 +1,6 @@ using CarbonAware.Exceptions; -namespace CarbonAware.Tools.WattTimeClient; +namespace CarbonAware.DataSources.WattTime; public class WattTimeClientException : CarbonAwareException { diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientHttpException.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs similarity index 97% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientHttpException.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs index 4a88358df..8743bec1b 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/WattTimeClientHttpException.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs @@ -1,7 +1,7 @@ using CarbonAware.Exceptions; using CarbonAware.Interfaces; -namespace CarbonAware.Tools.WattTimeClient; +namespace CarbonAware.DataSources.WattTime.Client; public class WattTimeClientHttpException : CarbonAwareException, IHttpResponseException { diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs index 4f30e941b..437922d6c 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs @@ -1,8 +1,7 @@ using CarbonAware.Configuration; +using CarbonAware.DataSources.WattTime.Client; using CarbonAware.Exceptions; using CarbonAware.Interfaces; -using CarbonAware.Tools.WattTimeClient; -using CarbonAware.Tools.WattTimeClient.Configuration; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection.Extensions; diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/WattTimeClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs similarity index 94% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/WattTimeClientConfiguration.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs index 612fd8a20..db2cc8a88 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/WattTimeClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs @@ -1,5 +1,7 @@ -namespace CarbonAware.Tools.WattTimeClient.Configuration; +using CarbonAware.Exceptions; + +namespace CarbonAware.DataSources.WattTime.Configuration; /// /// A configuration class for holding WattTime client config values. diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/AuthenticationHeaderTypes.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/AuthenticationHeaderTypes.cs similarity index 69% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/AuthenticationHeaderTypes.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/AuthenticationHeaderTypes.cs index c0373868c..99d160530 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/AuthenticationHeaderTypes.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/AuthenticationHeaderTypes.cs @@ -1,4 +1,4 @@ -namespace CarbonAware.Tools.WattTimeClient.Constants; +namespace CarbonAware.DataSources.WattTime.Constants; internal class AuthenticationHeaderTypes { diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs similarity index 82% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/Paths.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs index 40dba9cfb..d564a9c26 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs @@ -1,4 +1,4 @@ -namespace CarbonAware.Tools.WattTimeClient.Constants; +namespace CarbonAware.DataSources.WattTime.Constants; internal class Paths { diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs similarity index 85% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/QueryStrings.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs index 8749940ac..4aad4aa6f 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs @@ -1,4 +1,4 @@ -namespace CarbonAware.Tools.WattTimeClient.Constants; +namespace CarbonAware.DataSources.WattTime.Constants; internal class QueryStrings { diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs index d9984da0e..8ffa1dafa 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs @@ -1,3 +1,3 @@ using System.Runtime.CompilerServices; -[assembly:InternalsVisibleTo("CarbonAware.DataSources.WattTime.Tests")] \ No newline at end of file +[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Tests")] \ No newline at end of file diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/BalancingAuthority.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs similarity index 93% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/BalancingAuthority.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs index aa0107563..65b6ffd66 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/BalancingAuthority.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace CarbonAware.Tools.WattTimeClient.Model; +namespace CarbonAware.DataSources.WattTime.Model; /// /// The details of the balancing authority (BA) serving a particular location. diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/Forecast.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs similarity index 92% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/Forecast.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs index e41aaa282..0ab507fff 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/Forecast.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace CarbonAware.Tools.WattTimeClient.Model; +namespace CarbonAware.DataSources.WattTime.Model; /// /// An emissions forecast for a given time period. diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/GridEmissionDataPoint.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs similarity index 96% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/GridEmissionDataPoint.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs index cea28b366..538e6caf6 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/GridEmissionDataPoint.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace CarbonAware.Tools.WattTimeClient.Model; +namespace CarbonAware.DataSources.WattTime.Model; /// /// An object describing the emissions for a given time period and balancing authority. diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/LoginResult.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/LoginResult.cs similarity index 88% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/LoginResult.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/LoginResult.cs index 741c0208f..9ce89151e 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Model/LoginResult.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/LoginResult.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace CarbonAware.Tools.WattTimeClient.Model; +namespace CarbonAware.DataSources.WattTime.Model; /// /// Serializable object describing the WattTime login response object. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index 2972ea56e..25efda411 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -1,9 +1,8 @@ -using CarbonAware.Exceptions; +using CarbonAware.DataSources.WattTime.Client; +using CarbonAware.DataSources.WattTime.Model; using CarbonAware.Interfaces; using CarbonAware.LocationSources.Exceptions; using CarbonAware.Model; -using CarbonAware.Tools.WattTimeClient; -using CarbonAware.Tools.WattTimeClient.Model; using Microsoft.Extensions.Logging; using System.Diagnostics; using System.Globalization; @@ -11,7 +10,7 @@ namespace CarbonAware.DataSources.WattTime; /// -/// Reprsents a wattime data source. +/// Represents a WattTime data source. /// public class WattTimeDataSource : IEmissionsDataSource, IForecastDataSource { @@ -35,7 +34,6 @@ public class WattTimeDataSource : IEmissionsDataSource, IForecastDataSource const double LBS_TO_GRAMS_CONVERSION_FACTOR = 453.59237; public double MinSamplingWindow => 120; // 2hrs of data - /// /// Creates a new instance of the class. /// diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs similarity index 97% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/TestData.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs index 43006cd7b..0dcf42b7b 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs @@ -2,7 +2,7 @@ using System.Text.Json; using System.Text.Json.Nodes; -namespace CarbonAware.Tools.WattTimeClient.Tests; +namespace CarbonAware.DataSources.WattTime.Client.Tests; public static class TestData { diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs similarity index 84% rename from src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/WattTimeClientTests.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 26db74ddf..68cfb9ec5 100644 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -1,5 +1,5 @@ -using CarbonAware.Tools.WattTimeClient.Configuration; -using CarbonAware.Tools.WattTimeClient.Model; +using CarbonAware.DataSources.WattTime.Configuration; +using CarbonAware.DataSources.WattTime.Model; using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; @@ -18,7 +18,7 @@ using System.Threading; using System.Threading.Tasks; -namespace CarbonAware.Tools.WattTimeClient.Tests; +namespace CarbonAware.DataSources.WattTime.Client.Tests; #pragma warning disable CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider declaring as nullable. public class WattTimeClientTests @@ -106,7 +106,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() var data = await client.GetDataAsync("balauth", new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); - Assert.IsTrue(data.Count() > 0); + Assert.IsTrue(data.Any()); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); Assert.AreEqual("dt", gridDataPoint.Datatype); @@ -132,7 +132,7 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Count() > 0); + Assert.IsTrue(data.Any()); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); } @@ -151,7 +151,7 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Count() > 0); + Assert.IsTrue(data.Any()); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); } @@ -458,101 +458,61 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenNoneSet() [Test] public async Task GetHistoricalDataAsync_StreamsExpectedContent() { - using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) + using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); + this.CreateHttpClient(m => { - this.CreateHttpClient(m => - { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream)); - return Task.FromResult(response); - }); + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream)); + return Task.FromResult(response); + }); - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - Assert.AreEqual("myStreamResults", streamResult); - } + Assert.AreEqual("myStreamResults", streamResult); } [Test] public async Task GetHistoricalDataAsync_RefreshesTokenWhenExpired() { - using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) + using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); + this.CreateHttpClient(m => { - this.CreateHttpClient(m => - { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); - return Task.FromResult(response); - }); + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); + return Task.FromResult(response); + }); - this.HttpClient.DefaultRequestHeaders.Authorization = null; - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + this.HttpClient.DefaultRequestHeaders.Authorization = null; + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - Assert.AreEqual("myStreamResults", streamResult); - } + Assert.AreEqual("myStreamResults", streamResult); } [Test] public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() { - using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) + using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); + this.CreateHttpClient(m => { - this.CreateHttpClient(m => - { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); - return Task.FromResult(response); - }); - - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); + return Task.FromResult(response); + }); - Assert.AreEqual("myStreamResults", streamResult); - } - } + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - [Test] - public void TestClient_With_Proxy_Failure() - { - var key1 = $"{CarbonAwareVariablesConfiguration.Key}:Proxy:UseProxy"; - var key2 = $"{CarbonAwareVariablesConfiguration.Key}:Proxy:Url"; - var settings = new Dictionary { - {key1, "true"}, - {key2, "http://fakeproxy:8080"}, - }; - var configuration = new ConfigurationBuilder() - .AddInMemoryCollection(settings) - .Build(); - var serviceCollection = new ServiceCollection(); - serviceCollection.ConfigureWattTimeClient(configuration); - serviceCollection.AddMemoryCache(); - var serviceProvider = serviceCollection.BuildServiceProvider(); - var client = serviceProvider.GetRequiredService(); - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); - } + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - [Test] - public void TestClient_With_Missing_Proxy_URL() - { - var key1 = $"{CarbonAwareVariablesConfiguration.Key}:Proxy:UseProxy"; - var settings = new Dictionary { - {key1, "true"}, - }; - var configuration = new ConfigurationBuilder() - .AddInMemoryCollection(settings) - .Build(); - var serviceCollection = new ServiceCollection(); - Assert.Throws(() => serviceCollection.ConfigureWattTimeClient(configuration)); + Assert.AreEqual("myStreamResults", streamResult); } private void CreateHttpClient(Func> requestDelegate) @@ -567,10 +527,7 @@ private void CreateHttpClient(Func private HttpResponseMessage MockWattTimeAuthResponse(HttpRequestMessage request, HttpContent reponseContent, string? validToken = null) { - if (validToken == null) - { - validToken = this.DefaultTokenValue; - } + validToken ??= this.DefaultTokenValue; var auth = this.HttpClient.DefaultRequestHeaders.Authorization; if (auth == null) { diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs new file mode 100644 index 000000000..6086edecc --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs @@ -0,0 +1,70 @@ +using CarbonAware.Configuration; +using CarbonAware.DataSources.WattTime.Client; +using CarbonAware.DataSources.WattTime.Configuration; +using CarbonAware.Exceptions; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using NUnit.Framework; +using System.Collections.Generic; +using System.Net.Http; + +namespace CarbonAware.DataSources.WattTime.Tests; + +[TestFixture] +public class ServiceCollectionExtensionTests +{ + private readonly string ForecastDataSourceKey = $"DataSources:ForecastDataSource"; + private readonly string ForecastDataSourceValue = $"WattTimeTest"; + private readonly string UsernameKey = $"DataSources:Configurations:WattTimeTest:Username"; + private readonly string Username = "devuser"; + private readonly string PasswordKey = $"DataSources:Configurations:WattTimeTest:Password"; + private readonly string Password = "12345"; + private readonly string ProxyUrl = $"DataSources:Configurations:WattTimeTest:Proxy:Url"; + private readonly string UseProxyKey = $"DataSources:Configurations:WattTimeTest:Proxy:UseProxy"; + + [Test] + public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() + { + // Arrange + var settings = new Dictionary { + { ForecastDataSourceKey, ForecastDataSourceValue }, + { UsernameKey, Username }, + { PasswordKey, Password }, + { ProxyUrl, "http://fakeproxy:8080" }, + { UseProxyKey, "true" }, + }; + + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection(settings) + .AddEnvironmentVariables() + .Build(); + var serviceCollection = new ServiceCollection(); + serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources()); + var serviceProvider = serviceCollection.BuildServiceProvider(); + var client = serviceProvider.GetRequiredService(); + + // Act & Assert + Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); + } + + [Test] + public void ClientProxyTest_With_Missing_ProxyURL_ThrowsException() + { + // Arrange + var settings = new Dictionary { + { ForecastDataSourceKey, ForecastDataSourceValue }, + { UsernameKey, Username }, + { PasswordKey, Password }, + { UseProxyKey, "true" }, + }; + + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection(settings) + .AddEnvironmentVariables() + .Build(); + var serviceCollection = new ServiceCollection(); + + // Act & Assert + Assert.Throws(() => serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources())); + } +} \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/WattTimeClientConfigurationTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/WattTimeClientConfigurationTests.cs new file mode 100644 index 000000000..90e8e0420 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/WattTimeClientConfigurationTests.cs @@ -0,0 +1,44 @@ +using CarbonAware.DataSources.WattTime.Configuration; +using CarbonAware.Exceptions; +using NUnit.Framework; + +namespace CarbonAware.DataSources.WattTime.Tests; + +[TestFixture] +public class WattTimeClientConfigurationTests +{ + [TestCase("testuser", "12345", "http://example.com", TestName = "Validate does not throw: username; password; url")] + public void Validate_DoesNotThrow(string? username, string? password, string? url) + { + // Arrange + var config = new WattTimeClientConfiguration(); + if (username != null) + config.Username = username; + if (password != null) + config.Password = password; + if (url != null) + config.BaseUrl = url; + + // Act & Assert + Assert.DoesNotThrow(() => config.Validate()); + } + + [TestCase("testuser", "12345", "not a url", TestName = "Validate throws: username; password; bad url")] + [TestCase(null, "12345", "http://example.com", TestName = "Validate throws: no username; password; url")] + [TestCase("testuser", null, "http://example.com", TestName = "Validate throws: no username; password; url")] + [TestCase(null, null, "http://example.com", TestName = "Validate throws: no username; no password; url")] + public void Validate_Throws(string? username, string? password, string? url) + { + // Arrange + var config = new WattTimeClientConfiguration(); + if (username != null) + config.Username = username; + if (password != null) + config.Password = password; + if (url != null) + config.BaseUrl = url; + + // Act & Assert + Assert.Throws(() => config.Validate()); + } +} \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index 7023b1ade..4a5d2d6a1 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -2,8 +2,8 @@ using CarbonAware.Interfaces; using CarbonAware.LocationSources.Exceptions; using CarbonAware.Model; -using CarbonAware.Tools.WattTimeClient; -using CarbonAware.Tools.WattTimeClient.Model; +using CarbonAware.DataSources.WattTime.Client; +using CarbonAware.DataSources.WattTime.Model; using Microsoft.Extensions.Logging; using Moq; using NUnit.Framework; diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/CarbonAware.Tools.WattTimeClient.csproj b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/CarbonAware.Tools.WattTimeClient.csproj deleted file mode 100644 index fe62177c1..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/CarbonAware.Tools.WattTimeClient.csproj +++ /dev/null @@ -1,23 +0,0 @@ - - - - net6.0 - enable - enable - true - false - - - - - - - - - - - - - - - diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ConfigurationException.cs b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ConfigurationException.cs deleted file mode 100644 index 8c57638dd..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ConfigurationException.cs +++ /dev/null @@ -1,11 +0,0 @@ -namespace CarbonAware.Tools.WattTimeClient.Configuration; - -/// -/// An exception class thrown when the WattTime client is misconfigured. -/// -public class ConfigurationException : Exception -{ - public ConfigurationException(string message) : base(message) - { - } -} diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ServiceCollectionExtensions.cs deleted file mode 100644 index 02498b260..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/Configuration/ServiceCollectionExtensions.cs +++ /dev/null @@ -1,62 +0,0 @@ -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.DependencyInjection.Extensions; -using Microsoft.Extensions.Logging; -using System.Net; - -namespace CarbonAware.Tools.WattTimeClient.Configuration; - -public static class ServiceCollectionExtensions -{ - /// - /// Method to configure and add the WattTime client to the service collection. - /// The service collection to add the client to. - /// The configuration to use to configure the client. - /// The service collection with the configured client added. - /// - public static IServiceCollection ConfigureWattTimeClient(this IServiceCollection services, IConfiguration configuration) - { - - // configuring dependency injection to have config. - services.Configure(c => - { - configuration.GetSection(WattTimeClientConfiguration.Key).Bind(c); - }); - var configVars = configuration.GetSection(CarbonAwareVariablesConfiguration.Key).Get(); - if (configVars?.Proxy?.UseProxy == true) - { - if (String.IsNullOrEmpty(configVars.Proxy.Url)) - { - throw new ConfigurationException("Url is missing."); - } - LogProxyConfiguration(configuration, configVars); - services.AddHttpClient(IWattTimeClient.NamedClient) - .ConfigurePrimaryHttpMessageHandler(() => - new HttpClientHandler() { - Proxy = new WebProxy { - Address = new Uri(configVars.Proxy.Url), - Credentials = new NetworkCredential(configVars.Proxy.Username, configVars.Proxy.Password), - BypassProxyOnLocal = true - } - }); - } - else - { - services.AddHttpClient(IWattTimeClient.NamedClient); - } - services.AddMemoryCache(); - services.TryAddSingleton(); - - return services; - } - - private static void LogProxyConfiguration(IConfiguration config, CarbonAwareVariablesConfiguration caVars) - { - ILoggerFactory factory = LoggerFactory.Create(b => { - b.AddConfiguration(config.GetSection("Logging")); - b.AddConsole(); - }); - var logger = factory.CreateLogger(); - logger.LogInformation($"Proxy configured to Url {caVars?.Proxy?.Url} with username {caVars?.Proxy?.Username}"); - } -} diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/InternalsVisibleTo.cs b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/InternalsVisibleTo.cs deleted file mode 100644 index c6da31c22..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/src/InternalsVisibleTo.cs +++ /dev/null @@ -1,4 +0,0 @@ -using System.Runtime.CompilerServices; - -[assembly:InternalsVisibleTo("CarbonAware.Tools.WattTimeClient.Tests")] -[assembly:InternalsVisibleTo("CarbonAware.DataSources.WattTime.Mocks")] diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/AssemblyInfo.cs b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/AssemblyInfo.cs deleted file mode 100644 index 913f4ec3d..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/AssemblyInfo.cs +++ /dev/null @@ -1,3 +0,0 @@ -using NUnit.Framework; - -[assembly: Category("Unit")] \ No newline at end of file diff --git a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/CarbonAware.Tools.WattTimeClient.Tests.csproj b/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/CarbonAware.Tools.WattTimeClient.Tests.csproj deleted file mode 100644 index b9e6347b1..000000000 --- a/src/CarbonAware.Tools/CarbonAware.Tools.WattTimeClient/test/CarbonAware.Tools.WattTimeClient.Tests.csproj +++ /dev/null @@ -1,29 +0,0 @@ - - - - net6.0 - enable - - false - - - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - runtime; build; native; contentfiles; analyzers; buildtransitive - all - - - - - - - - - - - - diff --git a/src/CarbonAwareSDK.sln b/src/CarbonAwareSDK.sln index 26d2c7fd5..b07c97e0f 100644 --- a/src/CarbonAwareSDK.sln +++ b/src/CarbonAwareSDK.sln @@ -19,10 +19,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.DataSources.Jso EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.DataSources.Json.Tests", "CarbonAware.DataSources\CarbonAware.DataSources.Json\test\CarbonAware.DataSources.Json.Tests.csproj", "{7261E3D9-A27F-4D55-AED0-9315E081E3B2}" EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.Tools.WattTimeClient", "CarbonAware.Tools\CarbonAware.Tools.WattTimeClient\src\CarbonAware.Tools.WattTimeClient.csproj", "{BDA9EDD2-291A-4554-A617-27923FEDCB9B}" -EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.Tools.WattTimeClient.Tests", "CarbonAware.Tools\CarbonAware.Tools.WattTimeClient\test\CarbonAware.Tools.WattTimeClient.Tests.csproj", "{9122F502-56BB-4B44-9E9D-F481A043963A}" -EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.WebApi.UnitTests", "CarbonAware.WebApi\test\unitTests\CarbonAware.WebApi.UnitTests.csproj", "{7E9E3B13-8C74-4E51-9224-E209D2B13E7A}" EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CarbonAware.DataSources.WattTime.Tests", "CarbonAware.DataSources\CarbonAware.DataSources.WattTime\test\CarbonAware.DataSources.WattTime.Tests.csproj", "{74CEA7CE-FD6E-498C-9552-161D6EDD4B69}" @@ -107,14 +103,6 @@ Global {7261E3D9-A27F-4D55-AED0-9315E081E3B2}.Debug|Any CPU.Build.0 = Debug|Any CPU {7261E3D9-A27F-4D55-AED0-9315E081E3B2}.Release|Any CPU.ActiveCfg = Release|Any CPU {7261E3D9-A27F-4D55-AED0-9315E081E3B2}.Release|Any CPU.Build.0 = Release|Any CPU - {BDA9EDD2-291A-4554-A617-27923FEDCB9B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {BDA9EDD2-291A-4554-A617-27923FEDCB9B}.Debug|Any CPU.Build.0 = Debug|Any CPU - {BDA9EDD2-291A-4554-A617-27923FEDCB9B}.Release|Any CPU.ActiveCfg = Release|Any CPU - {BDA9EDD2-291A-4554-A617-27923FEDCB9B}.Release|Any CPU.Build.0 = Release|Any CPU - {9122F502-56BB-4B44-9E9D-F481A043963A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {9122F502-56BB-4B44-9E9D-F481A043963A}.Debug|Any CPU.Build.0 = Debug|Any CPU - {9122F502-56BB-4B44-9E9D-F481A043963A}.Release|Any CPU.ActiveCfg = Release|Any CPU - {9122F502-56BB-4B44-9E9D-F481A043963A}.Release|Any CPU.Build.0 = Release|Any CPU {7E9E3B13-8C74-4E51-9224-E209D2B13E7A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {7E9E3B13-8C74-4E51-9224-E209D2B13E7A}.Debug|Any CPU.Build.0 = Debug|Any CPU {7E9E3B13-8C74-4E51-9224-E209D2B13E7A}.Release|Any CPU.ActiveCfg = Release|Any CPU diff --git a/src/GSF.CarbonAware/test/EmissionsHandlerTests.cs b/src/GSF.CarbonAware/test/EmissionsHandlerTests.cs index c34624de6..04cdbe2bc 100644 --- a/src/GSF.CarbonAware/test/EmissionsHandlerTests.cs +++ b/src/GSF.CarbonAware/test/EmissionsHandlerTests.cs @@ -1,9 +1,8 @@ using CarbonAware.Aggregators.CarbonAware; -using CarbonAware.Tools.WattTimeClient; +using CarbonAware.Aggregators.Emissions; using EmissionsData = CarbonAware.Model.EmissionsData; using GSF.CarbonAware.Exceptions; using GSF.CarbonAware.Handlers; -using GSF.CarbonAware.Models; using Microsoft.Extensions.Logging; using Moq; using NUnit.Framework; @@ -11,7 +10,7 @@ using System.Threading.Tasks; using System; using static CarbonAware.Aggregators.CarbonAware.CarbonAwareParameters; -using CarbonAware.Aggregators.Emissions; + namespace GSF.CarbonAware.Tests; @@ -142,7 +141,7 @@ public void GetAverageCarbonIntensity_ErrorThrowsCustomException() { // Arrange var aggregator = new Mock(); - aggregator.Setup(x => x.CalculateAverageCarbonIntensityAsync(It.IsAny())).ThrowsAsync(new WattTimeClientException("")); + aggregator.Setup(x => x.CalculateAverageCarbonIntensityAsync(It.IsAny())).ThrowsAsync(new CarbonAwareException("")); var emissionsHandler = new EmissionsHandler(Logger!.Object, aggregator.Object); // Act/Assert From 8670eb7a88795a90da35e031dc4b3e4f6c35dad1 Mon Sep 17 00:00:00 2001 From: Priti Date: Tue, 22 Nov 2022 10:35:28 -0500 Subject: [PATCH 10/14] [M2][#174] ElectricityMaps Data Source - Emissions --- docs/configuration.md | 63 +++-- .../mock/ElectricityMapDataSourceMocker.cs | 40 ++- .../src/Client/ElectricityMapsClient.cs | 103 ++++++-- .../src/Client/IElectricityMapsClient.cs | 17 ++ .../ElectricityMapsClientConfiguration.cs | 17 ++ .../ServiceCollectionExtensions.cs | 4 +- .../src/Constants/Paths.cs | 1 + .../src/Constants/QueryStrings.cs | 4 + .../src/ElectricityMapsDataSource.cs | 108 ++++++++- .../src/Model/EmissionsFactor.cs | 13 - .../src/Model/HistoryCarbonIntensityData.cs | 33 ++- .../test/Client/ElectricityMapsClientTests.cs | 8 +- .../test/Client/TestData.cs | 2 +- .../test/ElectricityMapsDataSourceTests.cs | 228 ++++++++++++++++++ .../ServiceCollectionExtensions.cs | 3 +- .../CarbonAwareControllerTests.cs | 12 - .../IntegrationTestingBase.cs | 1 + 17 files changed, 577 insertions(+), 80 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs diff --git a/docs/configuration.md b/docs/configuration.md index 4932749f9..2c120e8f3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,4 +1,3 @@ - - [Configuration](#configuration) - [Logging](#logging) - [DataSources](#datasources) @@ -10,9 +9,11 @@ - [WattTime Caching BalancingAuthority](#watttime-caching-balancingauthority) - [Json Configuration](#json-configuration) - [ElectricityMaps Configuration](#electricitymaps-configuration) - - [ApiTokenHeader](#api-token-header) - - [ApiToken](#api-token) - - [baseUrl](#baseurl) + - [API Token Header](#api-token-header) + - [API Token](#api-token) + - [BaseUrl](#baseurl-1) + - [Emission Factor Type](#emission-factor-type) + - [Disable Estimations](#disable-estimations) - [CarbonAwareVars](#carbonawarevars) - [Tracing and Monitoring Configuration](#tracing-and-monitoring-configuration) - [Verbosity](#verbosity) @@ -69,10 +70,10 @@ Logging__LogLevel__Default="Debug" dotnet run ## DataSources -The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file and [WattTime](https://www.watttime.org/) are supported. +The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and ElectricityMaps (https://www.electricitymaps.com/) are supported. Each data source interface is configured with a specific data source implementation. -If set to `WattTime`, WattTime configuration must also be supplied. +If set to `WattTime` or `ElectricityMaps`, the configuration specific to that data provider must also be supplied. `JSON` will result in the data being loaded from the file specified in the `DataFileLocation` property @@ -93,6 +94,11 @@ If set to `WattTime`, WattTime configuration must also be supplied. "username": "proxyUsername", "password": "proxyPassword" } + }, + "ElectricityMaps": { + "Type": "ElectricityMaps", + "APITokenHeader": "auth-token", + "APIToken": "myAwesomeToken" }, "Json": { "Type": "Json", @@ -115,7 +121,7 @@ If using the WattTime data source, WattTime configuration is required. } ``` -> **Sign up for a test account:** To create an account, follow these steps : https://www.watttime.org/api-documentation/#best-practices-for-api-usage +> **Sign up for a test account:** To create an account, follow these steps [from the WattTime documentation](https://www.watttime.org/api-documentation/#best-practices-for-api-usage) #### username @@ -170,7 +176,17 @@ info: CarbonAware.DataSources.Json.JsonDataSource[0] If using the ElectricityMaps data source, ElectricityMaps configuration is required. -__With an account token:__ +> **NOTE** +> The ElectricityMaps API does not currently support access to historical forecasts. +> This means that functionality such as the CLI `emissions-forecasts` `--requested-at` flag +> and the API `/forecasts/batch` `requestedAt` input will respond with a `NotImplemented` error. +> +> Depending on the goal, the historical measured `emissions` commands may be a reasonable workaround. +> This would treat the measured emissions as a "perfect historical forecast" effectively. +> Otherwise, use a data source that has support for historical forecasts, such as [WattTime](#watttime-configuration). + +**With an account token:** + ```json { "APITokenHeader": "auth-token", @@ -179,7 +195,8 @@ __With an account token:__ } ``` -__With a free trial token:__ +**With a free trial token:** + ```json { "APITokenHeader": "X-BLOBR-KEY", @@ -188,20 +205,32 @@ __With a free trial token:__ } ``` -> **Sign up for a free trial:** To get a free trial: https://api-portal.electricitymaps.com/ +> **Sign up for a free trial:** Select the free trial product from [the ElectricityMaps catalog](https://api-portal.electricitymaps.com/) #### API Token Header -The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOB-KEY" +The API Token Header for ElectricityMaps. If you have a paid account, the header is "auth-token". If you're using the free trial, the header is "X-BLOBR-KEY" #### API Token The ElectricityMaps token you receive with your account or free trial. -#### baseUrl +#### BaseUrl The url to use when connecting to ElectricityMaps. Defaults to "https://api.electricitymap.org/v3/" but can be overridden in the config if needed (such as for free-trial users or enable integration testing scenarios). +#### Emission Factor Type + +String value for the optional `emissionFactorType` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#emission-factors) for more details and valid values. + +#### Disable Estimations + +Boolean value for the optional `disableEstimations` parameter to be sent on every ElectricityMaps API request that accepts this parameter. + +See the [ElectricityMaps API Documentation](https://static.electricitymaps.com/api/docs/index.html#estimations) for more details. + ## CarbonAwareVars This section contains the global settings for the SDK. The configuration looks like this: @@ -230,7 +259,7 @@ This application is integrated with Application Insights for monitoring purposes ApplicationInsights_Connection_String="AppInsightsConnectionString" ``` -You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net. +You can alternatively configure using Instrumentation Key by setting the `AppInsights_InstrumentationKey` variable. However, Microsoft is ending technical support for instrumentation key�based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to [the documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net). ```bash AppInsights_InstrumentationKey="AppInsightsInstrumentationKey" @@ -335,6 +364,7 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" ``` ## Configuration for Forecast data Using ElectricityMaps + ```json { "DataSources": { @@ -350,11 +380,12 @@ DataSources__Configurations__WattTime__Password="wattTimePassword" } ``` -## Configuration for Emissions data Using WattTime and Forecast data Using ElectricityMaps +## Configuration for Emissions data using ElectricityMaps and Forecast data using WattTime + ```json "DataSources": { - "EmissionsDataSource": "WattTime", - "ForecastDataSource": "ElectricityMaps", + "EmissionsDataSource": "ElectricityMaps", + "ForecastDataSource": "WattTime", "Configurations": { "WattTime": { "Type": "WattTime", diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs index cf86c02ab..4a2875133 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs @@ -26,19 +26,19 @@ public ElectricityMapsDataSourceMocker() public void SetupHistoryMock(decimal latitude, decimal longitude) { - var data = new List(); + var data = new List(); DateTimeOffset now = DateTimeOffset.UtcNow; DateTimeOffset past24 = now.AddHours(-24); while (past24 < now) { - var newDataPoint = new HistoryCarbonIntensity() + var newDataPoint = new CarbonIntensity() { - CarbonIntensity = 999, + Value = 999, DateTime = past24, UpdatedAt = now, CreatedAt = now, - EmissionFactorType = EmissionsFactor.Lifecycle, + EmissionFactorType = "lifecycle", IsEstimated = false, EstimationMethod = null }; @@ -101,15 +101,29 @@ private void SetupZonesMock() SetupResponseGivenGetRequest(Paths.Zones, result); } - public void Initialize() => SetupZonesMock(); + public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) + { + var data = new List(); + DateTimeOffset pointTime = start; + TimeSpan duration = TimeSpan.FromHours(1); - public void Reset() => _server.Reset(); + while (pointTime < end) + { + var newDataPoint = new CarbonIntensity() + { + Value = 100, + DateTime = pointTime, + }; - public void Dispose() => _server.Dispose(); + data.Add(newDataPoint); + pointTime = newDataPoint.DateTime + duration; + } - public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location) - { - throw new NotImplementedException(); + HistoryCarbonIntensityData history = new() { HistoryData = data }; + PastRangeData pastRange = new() { HistoryData = data }; + + SetupResponseGivenGetRequest(Paths.History, history); + SetupResponseGivenGetRequest(Paths.PastRange, pastRange); } public void SetupBatchForecastMock() @@ -117,6 +131,12 @@ public void SetupBatchForecastMock() throw new NotImplementedException(); } + public void Initialize() => SetupZonesMock(); + + public void Reset() => _server.Reset(); + + public void Dispose() => _server.Dispose(); + private void SetupResponseGivenGetRequest(string path, object body) { var jsonBody = JsonSerializer.Serialize(body, _options); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs index 00c766e53..0cc295214 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/ElectricityMapsClient.cs @@ -3,6 +3,7 @@ using CarbonAware.DataSources.ElectricityMaps.Model; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; +using System.Globalization; using System.Net.Http.Headers; using System.Net.Mime; using System.Text.Json; @@ -17,7 +18,7 @@ internal class ElectricityMapsClient : IElectricityMapsClient private readonly IOptionsMonitor _configurationMonitor; private ElectricityMapsClientConfiguration _configuration => this._configurationMonitor.CurrentValue; private readonly ILogger _log; - private Lazy>> _zonesAllowed; + private readonly Lazy>> _zonesAllowed; public ElectricityMapsClient(IHttpClientFactory factory, IOptionsMonitor monitor, ILogger log) { @@ -64,16 +65,6 @@ public async Task GetRecentCarbonIntensityHistoryAsy return await GetHistoryCarbonIntensityDataAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to History endpoint - private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) - { - await CheckZonesAllowedForPath(Paths.History, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.History, parameters)) - { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); - } - } - /// public async Task GetForecastedCarbonIntensityAsync (string zoneName) { @@ -103,14 +94,92 @@ public async Task GetForecastedCarbonIntensityAsy return await GetCurrentForecastAsync(parameters); } - // Internal call to check for allowed zones and then make GET request to Forecast endpoint - private async Task GetCurrentForecastAsync(Dictionary parameters) + /// + public async Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using latitude {latitude} longitude {longitude}", + latitude, longitude); + + var parameters = new Dictionary() + { + { QueryStrings.Latitude, latitude }, + { QueryStrings.Longitude, longitude }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + /// + public async Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime) + { + _log.LogDebug("Requesting carbon intensity using zone {zone}", + zone); + + var parameters = new Dictionary() + { + { QueryStrings.ZoneName, zone }, + { QueryStrings.StartTime, DateTimeToString(startTime) }, + { QueryStrings.EndTime, DateTimeToString(endTime) }, + }; + + return await GetPastRangeDataAsync(parameters); + } + + // The ElectricityMaps API has strict checks about datetime formatting. + // This helper method ensures that all DateTimeOffsets are properly formatted. + private static string DateTimeToString(DateTimeOffset dateTime) + { + return dateTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture); + } + + /// + /// Async method to check for allowed zones and then make GET request to History endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetHistoryCarbonIntensityDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.History, parameters); + AddOptionalParameters(parameters); + + using Stream result = await this.MakeRequestGetStreamAsync(Paths.History, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting history carbon intensity data"); + } + + private void AddOptionalParameters(Dictionary parameters) { - await CheckZonesAllowedForPath(Paths.Forecast, parameters); - using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters)) + if (_configuration.EmissionFactorType != null) { - return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); + parameters.Add(QueryStrings.EmissionFactorType, _configuration.EmissionFactorType); } + if (_configuration.DisableEstimations != null) + { + parameters.Add(QueryStrings.DisableEstimations, _configuration.DisableEstimations.ToString()!.ToLowerInvariant()); + } + } + + private async Task GetPastRangeDataAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.PastRange, parameters); + AddOptionalParameters(parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.PastRange, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting emissions data"); + } + + /// + /// Async method to check for allowed zones and then make GET request to Forecast endpoint + /// + /// List of query params + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + private async Task GetCurrentForecastAsync(Dictionary parameters) + { + await CheckZonesAllowedForPathAsync(Paths.Forecast, parameters); + using Stream result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new ElectricityMapsClientException($"Error getting forecasted data"); } private async Task GetResponseAsync(string uriPath) @@ -157,7 +226,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q } // Checks the current supported client's endpoint paths. - private async Task CheckZonesAllowedForPath(string path, Dictionary parameters) + private async Task CheckZonesAllowedForPathAsync(string path, Dictionary parameters) { // Parameters don't contain a ZoneName to check, exit if (!parameters.ContainsKey(QueryStrings.ZoneName)) return; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs index fd7864ff8..d6e371e6d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Client/IElectricityMapsClient.cs @@ -42,4 +42,21 @@ public interface IElectricityMapsClient /// A which contains all emissions data points in the 24 hour period. /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. public Task GetRecentCarbonIntensityHistoryAsync(string zoneName); + + /// + /// Async method to get the historical observed emission data for a given latitude and longitude over a given time period. + /// + /// Latitude for query + /// Longitude for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string latitude, string longitude, DateTimeOffset startTime, DateTimeOffset endTime); + + /// + /// Async method to get the historical observed emission data for a given zone over a given time period. + /// + /// Zone name for query + /// A which contains all emissions data points in the 24 hour period. + /// Can be thrown when errors occur connecting to ElectricityMaps client. See the ElectricityMapsClientException class for documentation of expected status codes. + public Task GetPastRangeDataAsync(string zone, DateTimeOffset startTime, DateTimeOffset endTime); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs index 30661f5f9..2d050e60f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ElectricityMapsClientConfiguration.cs @@ -1,5 +1,6 @@ using CarbonAware.Exceptions; using CarbonAware.DataSources.ElectricityMaps.Constants; +using CarbonAware.DataSources.ElectricityMaps.Model; namespace CarbonAware.DataSources.ElectricityMaps.Configuration; @@ -23,6 +24,22 @@ public class ElectricityMapsClientConfiguration /// public string BaseUrl { get; set; } = BaseUrls.TokenBaseUrl; + /// + /// Gets or sets the optional emissionFactorType parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#emission-factors for valid types + /// + public string? EmissionFactorType { get; set; } + + /// + /// Gets or sets the optional disableEstimations parameter used in API requests + /// + /// + /// See https://static.electricitymaps.com/api/docs/index.html#estimations for details + /// + public bool? DisableEstimations { get; set; } + /// /// Validate that this object is properly configured. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs index 378146af1..1cdcf76c0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Configuration/ServiceCollectionExtensions.cs @@ -20,7 +20,9 @@ public static IServiceCollection AddElectricityMapsForecastDataSource(this IServ public static IServiceCollection AddElectricityMapsEmissionsDataSource(this IServiceCollection services, DataSourcesConfiguration dataSourcesConfig) { - throw new NotImplementedException(); + AddElectricityMapsClient(services, dataSourcesConfig.EmissionsConfigurationSection()); + services.TryAddSingleton(); + return services; } private static void AddElectricityMapsClient(IServiceCollection services, IConfigurationSection configSection) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs index 3c4fc87de..c21bfbfb7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/Paths.cs @@ -4,5 +4,6 @@ internal class Paths { public const string History = "carbon-intensity/history"; public const string Forecast = "carbon-intensity/forecast"; + public const string PastRange = "carbon-intensity/past-range"; public const string Zones = "zones"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs index 27c33ecd9..d9bcc22c2 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Constants/QueryStrings.cs @@ -5,4 +5,8 @@ internal class QueryStrings public const string Latitude = "lat"; public const string Longitude = "lon"; public const string ZoneName = "zone"; + public const string StartTime = "start"; + public const string EndTime = "end"; + public const string DisableEstimations = "disableEstimations"; + public const string EmissionFactorType = "emissionFactorType"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs index fff633851..81ca4b5eb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs @@ -1,5 +1,6 @@ using CarbonAware.DataSources.ElectricityMaps.Client; using CarbonAware.DataSources.ElectricityMaps.Model; +using CarbonAware.Exceptions; using CarbonAware.Interfaces; using CarbonAware.Model; using Microsoft.Extensions.Logging; @@ -10,7 +11,7 @@ namespace CarbonAware.DataSources.ElectricityMaps; /// /// Represents a Electricity Maps data source. /// -public class ElectricityMapsDataSource : IForecastDataSource +public class ElectricityMapsDataSource : IForecastDataSource, IEmissionsDataSource { public string _name => "ElectricityMapsDataSource"; @@ -82,4 +83,109 @@ public async Task GetCarbonIntensityForecastAsync(Location lo await Task.Run(() => true); throw new NotImplementedException(); } + + /// + public async Task> GetCarbonIntensityAsync(IEnumerable locations, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + this._logger.LogDebug("Getting carbon intensity for locations {locations} for period {periodStartTime} to {periodEndTime}.", locations, periodStartTime, periodEndTime); + using (var activity = _activity.StartActivity()) + { + List result = new(); + foreach (var location in locations) + { + IEnumerable interimResult = await GetCarbonIntensityAsync(location, periodStartTime, periodEndTime); + result.AddRange(interimResult); + } + return result; + } + } + + /// + public async Task> GetCarbonIntensityAsync(Location location, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) + { + using (var activity = _activity.StartActivity()) + { + var geolocation = await this._locationSource.ToGeopositionLocationAsync(location); + IEnumerable historyCarbonIntensity; + DateTime now = DateTime.UtcNow; + var isDateRangeWithin24Hours = (periodStartTime > now.AddHours(-24) && periodStartTime <= now) && (periodEndTime > now.AddHours(-24) && periodEndTime <= now); + if (isDateRangeWithin24Hours) + { + historyCarbonIntensity = await GetRecentCarbonInstensityData(geolocation); + } + else + { + historyCarbonIntensity = await GetPastCarbonIntensityData(periodStartTime, periodEndTime, geolocation); + } + + return HistoryCarbonIntensityToEmissionsData(location, historyCarbonIntensity, periodStartTime, periodEndTime); + } + } + + private async Task> GetPastCarbonIntensityData(DateTimeOffset periodStartTime, DateTimeOffset periodEndTime, Location geolocation) + { + PastRangeData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? "", periodStartTime, periodEndTime); + else + { + data = await this._electricityMapsClient.GetPastRangeDataAsync(geolocation.Name ?? "", periodStartTime, periodEndTime); + } + + return data.HistoryData; + } + + private async Task> GetRecentCarbonInstensityData(Location geolocation) + { + HistoryCarbonIntensityData data; + if (geolocation.Latitude != null && geolocation.Latitude != null) + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Latitude.ToString() ?? "", geolocation.Longitude.ToString() ?? ""); + else + { + data = await this._electricityMapsClient.GetRecentCarbonIntensityHistoryAsync(geolocation.Name ?? ""); + } + + return data.HistoryData; + } + + private IEnumerable HistoryCarbonIntensityToEmissionsData(Location location, IEnumerable data, DateTimeOffset startTime, DateTimeOffset endTime) + { + IEnumerable emissions; + var duration = GetDurationFromHistoryDataPointsOrDefault(data, TimeSpan.Zero); + emissions = data.Select(d => + { + var emission = (EmissionsData) d; + emission.Location = location.Name; + emission.Time = d.DateTime; + emission.Duration = duration; + return emission; + }); + + return emissions; + } + + private TimeSpan GetDurationFromHistoryDataPointsOrDefault(IEnumerable carbonIntensityDataPoints, TimeSpan defaultValue) + { + try + { + return GetDurationFromHistoryDataPoints(carbonIntensityDataPoints); + } + catch (CarbonAwareException) + { + return defaultValue; + } + } + + private TimeSpan GetDurationFromHistoryDataPoints(IEnumerable dataPoints) + { + var firstPoint = dataPoints.FirstOrDefault(); + var secondPoint = dataPoints.Skip(1)?.FirstOrDefault(); + + var first = firstPoint ?? throw new CarbonAwareException("Too few data points returned"); + var second = secondPoint ?? throw new CarbonAwareException("Too few data points returned"); + + // Handle chronological and reverse-chronological data by using `.Duration()` to get + // the absolute value of the TimeSpan between the two points. + return first.DateTime.Subtract(second.DateTime).Duration(); + } } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs deleted file mode 100644 index 27ef63379..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/EmissionsFactor.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CarbonAware.DataSources.ElectricityMaps.Model; - -/// -/// Type of EmissionFactor to use for calculating carbon intensity as described in the Electricity Maps documentation - https://static.electricitymaps.com/api/docs/index.html#emission-factors -/// -[JsonConverter(typeof(JsonStringEnumConverter))] -public enum EmissionsFactor -{ - Lifecycle, - Direct -} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs index 5f0326d70..b23d658b0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/Model/HistoryCarbonIntensityData.cs @@ -1,3 +1,5 @@ +using CarbonAware.Exceptions; +using CarbonAware.Model; using System.Text.Json.Serialization; namespace CarbonAware.DataSources.ElectricityMaps.Model; @@ -18,20 +20,20 @@ public record HistoryCarbonIntensityData /// List of History Carbon Intensity instances. /// [JsonPropertyName("history")] - public IEnumerable HistoryData { get; init; } = Array.Empty(); + public IEnumerable HistoryData { get; init; } = Array.Empty(); } /// /// A history carbon intensity. /// [Serializable] -public record HistoryCarbonIntensity +public record CarbonIntensity { /// /// Carbon Intensity value. /// [JsonPropertyName("carbonIntensity")] - public int CarbonIntensity { get; init; } + public int Value { get; init; } /// /// Indicates the datetime of the carbon intensity @@ -55,7 +57,7 @@ public record HistoryCarbonIntensity /// Indicated the emission factor type used for computing the carbon intensity. /// [JsonPropertyName("emissionFactorType")] - public EmissionsFactor EmissionFactorType { get; init; } + public string? EmissionFactorType { get; init; } /// /// Indicates whether the result is estimated or no @@ -69,4 +71,27 @@ public record HistoryCarbonIntensity [JsonPropertyName("estimationMethod")] public string? EstimationMethod { get; init; } + public static explicit operator EmissionsData(CarbonIntensity historyCarbonIntensity) + { + return new EmissionsData + { + Rating = historyCarbonIntensity.Value, + Time = historyCarbonIntensity.UpdatedAt, + }; + } } + +/// +/// Carbon intensity data for past date range. +/// +[Serializable] +public record PastRangeData +{ + /// + /// Carbon Intensity value. + /// + [JsonPropertyName("data")] + public IEnumerable HistoryData { get; init; } = Array.Empty(); + +} + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs index c8f47f8f5..6afd06ece 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/ElectricityMapsClientTests.cs @@ -53,8 +53,8 @@ public void ClientInstantiation_FailsForInvalidConfig(string baseUrl) CreateBasicClient(TestData.GetZonesAllowedJsonString(), "{}"); this.Configuration = new ElectricityMapsClientConfiguration() { - APITokenHeader = null, - APIToken = null, + APITokenHeader = "", + APIToken = "", BaseUrl = baseUrl, }; @@ -254,8 +254,8 @@ public async Task GetRecentCarbonIntensityHistoryAsync_DeserializesExpectedRespo Assert.That(dataPoint?.DateTime, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.UpdatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); Assert.That(dataPoint?.CreatedAt, Is.EqualTo(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero))); - Assert.That(dataPoint?.CarbonIntensity, Is.EqualTo(999)); - Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo(EmissionsFactor.Lifecycle)); + Assert.That(dataPoint?.Value, Is.EqualTo(999)); + Assert.That(dataPoint?.EmissionFactorType, Is.EqualTo("lifecycle")); Assert.That(dataPoint?.IsEstimated, Is.False); Assert.That(dataPoint?.EstimationMethod, Is.Null); }); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs index 9c028332d..48db3dab9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/Client/TestData.cs @@ -63,7 +63,7 @@ public static string GetHistoryCarbonIntensityDataJsonString() ["updatedAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["createdAt"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), ["carbonIntensity"] = 999, - ["emissionFactorType"] = "Lifecycle", + ["emissionFactorType"] = "lifecycle", ["isEstimated"] = false, ["estimatedMethod"] = null, } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs index e39d37397..444868f5d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/test/ElectricityMapsDataSourceTests.cs @@ -96,4 +96,232 @@ public void GetCurrentCarbonIntensityForecastAsync_ThrowsWhenRegionNotFound() Assert.ThrowsAsync(async () => await _dataSource.GetCurrentCarbonIntensityForecastAsync(_defaultLocation)); } + + [TestCase(8, 1, 1, 0, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when date within 24 hours")] + [TestCase(36, 1, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when date outside of 24 hours")] + [TestCase(30, 12, 0, 1, TestName = "GetCarbonIntensity calls GetPastRangeDataAsync method when start date outside of 24 hours but enddate within 24 hours")] + [TestCase(8, 12, 0, 1, TestName = "GetCarbonIntensity calls GetRecentCarbonIntensityDataAsync method when start date within 24 hours but enddate outside of 24 hours")] + public async Task GetCarbonIntensity_CallsExpectedClientEndpoint(int startTimeOffset, int endTimeOffset, int expectedHistoryCalls, int expectedPastRangeCalls) + { + var now = DateTimeOffset.UtcNow; + var startDate = now.AddHours(-startTimeOffset); + var endDate = startDate.AddHours(endTimeOffset); + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new(); + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + PastRangeData pastRange = new(); + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => pastRange); + + await _dataSource.GetCarbonIntensityAsync(_defaultLocation, startDate, endDate); + + _electricityMapsClient.Verify(c => c.GetPastRangeDataAsync(_defaultLatitude, _defaultLongitude, startDate, endDate), Times.Exactly(expectedPastRangeCalls)); + + _electricityMapsClient.Verify(c => c.GetRecentCarbonIntensityHistoryAsync(_defaultLatitude, _defaultLongitude), Times.Exactly(expectedHistoryCalls)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeWithin24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-10); + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_DateRangeMore24Hours_ReturnsResultsWhenRecordsFound() + { + var startDate = _defaultDataStartTime; + var endDate = startDate.AddHours(1); + var expectedCarbonIntensity = 100; + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + PastRangeData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + }, + new CarbonIntensity() + { + Value = expectedCarbonIntensity, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + endDate) + ).ReturnsAsync(() => emissionData); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Rating, Is.EqualTo(expectedCarbonIntensity)); + Assert.That(first.Location, Is.EqualTo(_defaultLocation.Name)); + + this._locationSource.Verify(l => l.ToGeopositionLocationAsync(_defaultLocation)); + } + + [Test] + public async Task GetCarbonIntensity_PastRange_ReturnsEmptyListWhenNoRecordsFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddHours(1); + + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + this._electricityMapsClient.Setup(c => c.GetPastRangeDataAsync( + _defaultLatitude, + _defaultLongitude, + startDate, + + endDate) + ).ReturnsAsync(() => new PastRangeData()); + + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.IsNotNull(result); + Assert.That(result.Count(), Is.EqualTo(0)); + } + + [Test] + public void GetCarbonIntensity_ThrowsWhenRegionNotFound() + { + var startDate = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); + var endDate = startDate.AddMinutes(1); + + this._locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Throws(); + + Assert.ThrowsAsync(async () => await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_ReturnsDefaultDuration_WhenOneDatapointReturned() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(1)); + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(TimeSpan.Zero)); + } + + [Test] + public async Task GetDurationBetweenHistoryDataPoints_WhenMultipleDataPoints_ReturnsExpectedDuration() + { + var startDate = DateTimeOffset.UtcNow.AddHours(-8); + var endDate = startDate.AddHours(1); + var expectedDuration = TimeSpan.FromHours(1); + // Arrange + _locationSource.Setup(l => l.ToGeopositionLocationAsync(_defaultLocation)).Returns(Task.FromResult(_defaultLocation)); + + HistoryCarbonIntensityData emissionData = new() + { + HistoryData = new List() + { + new CarbonIntensity() + { + DateTime = startDate, + + }, + new CarbonIntensity() + { + DateTime= startDate + expectedDuration, + } + } + }; + + this._electricityMapsClient.Setup(c => c.GetRecentCarbonIntensityHistoryAsync( + _defaultLatitude, + _defaultLongitude) + ).ReturnsAsync(() => emissionData); + + + // Act & Assert + var result = await this._dataSource.GetCarbonIntensityAsync(new List() { _defaultLocation }, startDate, endDate); + + Assert.That(result.Count(), Is.EqualTo(2)); + + var first = result.First(); + Assert.IsNotNull(first); + Assert.That(first.Duration, Is.EqualTo(expectedDuration)); + + var second = result.Skip(1)?.First(); + Assert.IsNotNull(second); + Assert.That(second.Duration, Is.EqualTo(expectedDuration)); + } } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs index 551891515..cb635adf4 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.Registration/Configuration/ServiceCollectionExtensions.cs @@ -32,7 +32,8 @@ public static IServiceCollection AddDataSourceService(this IServiceCollection se } case DataSourceType.ElectricityMaps: { - throw new ArgumentException("ElectricityMaps data source is not supported for emissions data"); + services.AddElectricityMapsEmissionsDataSource(dataSources); + break; } case DataSourceType.None: { diff --git a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs index ea6620d89..8bba52f7a 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs @@ -48,8 +48,6 @@ public async Task FakeEndPoint_ReturnsNotFound() [TestCase("2021-12-25", "2021-12-26", "westus")] public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Sets up any data endpoints needed for mocking purposes _dataSourceMocker?.SetupDataMock(start, end, location); @@ -72,8 +70,6 @@ public async Task BestLocations_ReturnsOK(DateTimeOffset start, DateTimeOffset e [TestCase("non-location-param", "", TestName = "location param not present")] public async Task BestLocations_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/bylocations/best'", DataSourceType.ElectricityMaps); - //Call the private method to construct with parameters var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -213,8 +209,6 @@ public async Task EmissionsForecastsBatch_SupportedDataSources_ReturnsOk(string [TestCase("2021-12-25", "2021-12-26", "westus", TestName = "EmissionsMarginalCarbonIntensity expects OK date only, no time")] public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, string end, string location) { - IgnoreTestForDataSource("data source does not implement '/emissions/forecasts/current'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); @@ -246,8 +240,6 @@ public async Task EmissionsMarginalCarbonIntensity_ReturnsOk(string start, strin [TestCase("non-location-param", "", TestName = "EmissionsMarginalCarbonIntensity returns BadRequest for location not present")] public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_ReturnsBadRequest(string queryString, string value) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var queryStrings = new Dictionary(); queryStrings[queryString] = value; @@ -265,8 +257,6 @@ public async Task EmissionsMarginalCarbonIntensity_EmptyLocationQueryString_Retu [TestCase("westus", "2022-3-1T15:30:00Z", "2022-3-1T18:00:00Z", TestName = "EmissionsMarginalCarbonIntensityBatch returns BadRequest for wrong date format")] public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_ReturnsBadRequest(string location, string startTime, string endTime) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var intesityData = Enumerable.Range(0, 1).Select(x => new { location = location, @@ -283,8 +273,6 @@ public async Task EmissionsMarginalCarbonIntensityBatch_MissingRequiredParams_Re [TestCase("2021-12-25", "2021-12-26", "westus", 3, TestName = "EmissionsMarginalCarbonIntensityBatch expects OK for multiple element batch")] public async Task EmissionsMarginalCarbonIntensityBatch_SupportedDataSources_ReturnsOk(string start, string end, string location, int nelems) { - IgnoreTestForDataSource("data source does not implement '/emissions/average-carbon-intensity'", DataSourceType.ElectricityMaps); - var startDate = DateTimeOffset.Parse(start); var endDate = DateTimeOffset.Parse(end); _dataSourceMocker?.SetupDataMock(startDate, endDate, location); diff --git a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs index 7fdf38ce9..3c7eb05b1 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/IntegrationTestingBase.cs @@ -101,6 +101,7 @@ public void Setup() } case DataSourceType.ElectricityMaps: { + Environment.SetEnvironmentVariable("DataSources__EmissionsDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__ForecastDataSource", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__Type", "ElectricityMaps"); Environment.SetEnvironmentVariable("DataSources__Configurations__ElectricityMaps__APITokenHeader", "token"); From 82cb16e3802537d861d37889149e32bf227c8566 Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Thu, 15 Dec 2022 14:24:07 -0500 Subject: [PATCH 11/14] PR comments --- .../CarbonAware.DataSources.WattTime.csproj | 12 +++ .../src/Client/InternalsVisibleTo.cs | 4 - .../src/InternalsVisibleTo.cs | 3 - .../test/Client/WattTimeClientTests.cs | 89 ++++++++++--------- 4 files changed, 59 insertions(+), 49 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj index 213fca4a1..90b7b8dc3 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/CarbonAware.DataSources.WattTime.csproj @@ -8,6 +8,18 @@ false + + + <_Parameter1>CarbonAware.DataSources.WattTime.Client.Tests + + + <_Parameter1>CarbonAware.DataSources.WattTime.Mocks + + + <_Parameter1>CarbonAware.DataSources.WattTime.Tests + + + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs deleted file mode 100644 index 65ae2dda4..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/InternalsVisibleTo.cs +++ /dev/null @@ -1,4 +0,0 @@ -using System.Runtime.CompilerServices; - -[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Client.Tests")] -[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Mocks")] diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs deleted file mode 100644 index 8ffa1dafa..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/InternalsVisibleTo.cs +++ /dev/null @@ -1,3 +0,0 @@ -using System.Runtime.CompilerServices; - -[assembly: InternalsVisibleTo("CarbonAware.DataSources.WattTime.Tests")] \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 68cfb9ec5..8725bd76c 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -1,14 +1,11 @@ using CarbonAware.DataSources.WattTime.Configuration; using CarbonAware.DataSources.WattTime.Model; using Microsoft.Extensions.Caching.Memory; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; using Moq; using NUnit.Framework; using System; -using System.Collections.Generic; using System.IO; using System.Linq; using System.Net.Http; @@ -106,7 +103,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() var data = await client.GetDataAsync("balauth", new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); - Assert.IsTrue(data.Any()); + Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); Assert.AreEqual("dt", gridDataPoint.Datatype); @@ -132,7 +129,7 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Any()); + Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); } @@ -151,7 +148,7 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Any()); + Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); } @@ -458,61 +455,67 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenNoneSet() [Test] public async Task GetHistoricalDataAsync_StreamsExpectedContent() { - using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); - this.CreateHttpClient(m => + using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream)); - return Task.FromResult(response); - }); + this.CreateHttpClient(m => + { + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream)); + return Task.FromResult(response); + }); - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - Assert.AreEqual("myStreamResults", streamResult); + Assert.AreEqual("myStreamResults", streamResult); + } } [Test] public async Task GetHistoricalDataAsync_RefreshesTokenWhenExpired() { - using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); - this.CreateHttpClient(m => + using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); - return Task.FromResult(response); - }); + this.CreateHttpClient(m => + { + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); + return Task.FromResult(response); + }); - this.HttpClient.DefaultRequestHeaders.Authorization = null; - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + this.HttpClient.DefaultRequestHeaders.Authorization = null; + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - Assert.AreEqual("myStreamResults", streamResult); + Assert.AreEqual("myStreamResults", streamResult); + } } [Test] public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() { - using var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults")); - this.CreateHttpClient(m => + using (var testStream = new MemoryStream(Encoding.UTF8.GetBytes("myStreamResults"))) { - var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); - return Task.FromResult(response); - }); + this.CreateHttpClient(m => + { + var response = this.MockWattTimeAuthResponse(m, new StreamContent(testStream), "REFRESHTOKEN"); + return Task.FromResult(response); + }); - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); + client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync("ba"); - var sr = new StreamReader(result); - string streamResult = sr.ReadToEnd(); + var result = await client.GetHistoricalDataAsync("ba"); + var sr = new StreamReader(result); + string streamResult = sr.ReadToEnd(); - Assert.AreEqual("myStreamResults", streamResult); + Assert.AreEqual("myStreamResults", streamResult); + } } private void CreateHttpClient(Func> requestDelegate) @@ -527,7 +530,10 @@ private void CreateHttpClient(Func private HttpResponseMessage MockWattTimeAuthResponse(HttpRequestMessage request, HttpContent reponseContent, string? validToken = null) { - validToken ??= this.DefaultTokenValue; + if (validToken == null) + { + validToken = this.DefaultTokenValue; + } var auth = this.HttpClient.DefaultRequestHeaders.Authorization; if (auth == null) { @@ -566,5 +572,4 @@ protected override async Task SendAsync(HttpRequestMessage } } } -#pragma warning restore CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider declaring as nullable. - +#pragma warning restore CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider declaring as nullable. \ No newline at end of file From e6572b4a9fe60aeb3e5e92ed9456f0c9d4b23bfa Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Mon, 19 Dec 2022 14:25:46 -0500 Subject: [PATCH 12/14] PR comments --- docs/data-source-matrix.md | 54 --------------------------------- docs/selecting-a-data-source.md | 37 ++++++++++++++++++++++ 2 files changed, 37 insertions(+), 54 deletions(-) delete mode 100644 docs/data-source-matrix.md create mode 100644 docs/selecting-a-data-source.md diff --git a/docs/data-source-matrix.md b/docs/data-source-matrix.md deleted file mode 100644 index 559cc3bba..000000000 --- a/docs/data-source-matrix.md +++ /dev/null @@ -1,54 +0,0 @@ -# Data Source Matrices - -The Carbon Aware SDK includes access to various data sources of carbon aware data, including WattTime, ElectricityMaps, and a custom JSON source. These matrices are an attempt to track what features of the Carbon Aware SDK are enabled for which data sources. - -## Contents - -- [Type of Data Sources and Configuration](#type-of-data-sources-and-configuration) -- [Data Source Routes Available](#data-source-routes-available) -- [Location Source Availability](#location-source-availability) - -## Type of Data Sources and Configuration - -In the CarbonAware SDK configuration, you can set what data source to use as the `EmissionsDataSource` and the `ForecastDataSource`. There are also certain configuration fields that must be set in order to access the raw data. -| Type | WattTime | ElectricityMaps | JSON | -|--------------------------|-----------|-----------------|------| -| Is Emissions DataSource | Yes | Yes | Yes | -| Is Forecast DataSource | Yes | Yes | No | -| Makes HTTP(s) call | Yes | Yes | No | -| Can Use Custom Data | No | No | Yes | -| Needs Authentication Config | Yes - username/password | Yes - token/header | No | -| BaseUrl Override Config | Only for testing | Switch between trial + full version, testing | No | -| Supports Trial + Full Account | Yes | Yes (*different URL and token header required) | N/A | - -## Data Source Routes Available - -Not all data sources support all the routes provided in the interfaces (`IEmissionsDataSource`/`IForecastDataSource`). The list below maps the interface route to the relevant consumer call, while the table lists only the interface route. - -- GetCarbonIntensityAsync - - CLI: `emissions` - - API: `emissions/bylocation` / `emissions/bylocations` / `emissions/bylocations/best` / `emissions/average-carbon-intensity` / `average-carbon-intensity/batch` - - Library: `GetEmissionsDataAsync(...)` / `GetBestEmissionsDataAsync(...)` / `GetAverageCarbonIntensityDataAsync(...)` -- GetCurrentForecastAsync - - CLI: `emissions-forecasts` - - API: `forecasts/current` - - Library: `GetCurrentForecastAsync(...)` -- GetForecastByDateAsync - - CLI: `emissions-forecasts --requested-at` - - API: `forecasts/batch` with `requestedAt` field - - Library: `GetForecastByDateAsync(...)` - -| Route | WattTime | ElectricityMaps | JSON | -|--------------|:-----------:|:-----------------:|:------:| -| GetCarbonIntensityAsync | Yes | Yes | Yes | -| GetCurrentForecastAsync | Yes | Yes | No | -| GetForecastByDateAsync | Yes | No | No | - -## Location Source Availability - -*I wasn't sure exactly what to fill here but wanted something along the lines of: - -- WattTime is better to capture data for XX locations -- Electricity Maps is better to capture data for YY locations -- Electricity Maps can take a zone name or lat/long -- WattTime takes a region name (azure region?) or lat long diff --git a/docs/selecting-a-data-source.md b/docs/selecting-a-data-source.md new file mode 100644 index 000000000..9ec9e7f41 --- /dev/null +++ b/docs/selecting-a-data-source.md @@ -0,0 +1,37 @@ +# Selecting a Data Source + +The Carbon Aware SDK includes access to various data sources of carbon aware data, including WattTime, ElectricityMaps, and a custom JSON source. These matrices are an attempt to track what features of the Carbon Aware SDK are enabled for which data sources. + +## Contents + +- [Type of Data Sources and Configuration](#type-of-data-sources-and-configuration) +- [Data Source Methods Available](#data-source-methods-available) +- [Location Coverage](#location-coverage) + +## Type of Data Sources and Configuration + +In the CarbonAware SDK configuration, you can set what data source to use as the `EmissionsDataSource` and the `ForecastDataSource`. There are also certain configuration fields that must be set in order to access the raw data. +| Type | WattTime | ElectricityMaps | JSON | +|--------------------------|-----------|-----------------|------| +| Is Emissions DataSource | ✅ | ✅ | ✅ | +| Is Forecast DataSource | ✅ | ✅ | ❌ | +| Makes HTTP(s) call | ✅ | ✅ | ❌ | +| Can Use Custom Data | ❌ | ❌ | ✅ | +| Supports Trial + Full Account | ✅ | ✅ (*[different config required](./configuration.md#electricitymaps-configuration)) | N/A | + +## Data Source Methods Available + +Not all data sources support all the routes provided in the interfaces (`IEmissionsDataSource`/`IForecastDataSource`). + +| Methods | WattTime | ElectricityMaps | JSON | CLI Usage | Web Api Usage | SDK Usage +|--------------|:-----------:|:-----------------:|:------:|:-:|:-:|:-:| +| GetCarbonIntensityAsync | ✅ | ✅ | ✅ |`emissions`|`emissions/bylocation` or `emissions/bylocations` or `emissions/bylocations/best` or `emissions/average-carbon-intensity` or `average-carbon-intensity/batch`|`GetEmissionsDataAsync(...)` or `GetBestEmissionsDataAsync(...)` or `GetAverageCarbonIntensityDataAsync(...)`| +| GetCurrentForecastAsync | ✅ | ✅ | ❌ |`emissions-forecasts`|`forecasts/current`|`GetCurrentForecastAsync(...)`| +| GetForecastByDateAsync | ✅ | ❌ | ❌ |`emissions-forecasts --requested-at`|`forecasts/batch` with `requestedAt` field|`GetForecastByDateAsync(...)`| + +## Location Coverage + +Different data sources provide both different features (as outlined above) but also coverage of different geographic areas. It's important to note that each data source may have different region names, which are handled through the location config. + +- For `WattTime`, see their [interactive coverage map](https://www.watttime.org/explorer) to find the relevant zone. +- For `ElectricityMaps`, see their [live map app](https://app.electricitymaps.com/map?utm_source=electricitymaps.com&utm_medium=website&utm_campaign=banner) to find the relevant zone and see current data coming in. From 8deedf18da56a3dc487459b2c005223893a17921 Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Tue, 3 Jan 2023 13:25:10 -0500 Subject: [PATCH 13/14] PR comments --- docs/configuration.md | 2 +- samples/azure-function/README.md | 16 +++++++++++----- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 2c120e8f3..366c26950 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -70,7 +70,7 @@ Logging__LogLevel__Default="Debug" dotnet run ## DataSources -The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and ElectricityMaps (https://www.electricitymaps.com/) are supported. +The SDK supports multiple data sources for getting carbon data. At this time, only a JSON file, [WattTime](https://www.watttime.org/) and [ElectricityMaps](https://www.electricitymaps.com/) are supported. Each data source interface is configured with a specific data source implementation. If set to `WattTime` or `ElectricityMaps`, the configuration specific to that data provider must also be supplied. diff --git a/samples/azure-function/README.md b/samples/azure-function/README.md index c20fcc2cd..ec932c485 100644 --- a/samples/azure-function/README.md +++ b/samples/azure-function/README.md @@ -2,7 +2,9 @@ ## Overview -The sample included showcase the Azure Functions tooling for the Carbon Aware SDK [C# Class Library](../../docs/architecture/c-sharp-client-library.md). It includes an implementation for `GetAverageCarbonIntensity` and for `GetCurrentForecast`. `GetAverageCarbonIntensity` uses the `EmissionsHandler` to return the carbon intensity rate of a location for a specific timespan. `GetCurrentForecast` uses the `ForecastHandler` to yield the optimal time of a specified location and duration. The functions can run locally for debugging or be deployed to Azure. +The sample included showcase the Azure Functions tooling for the Carbon Aware SDK [C# Class Library](../../docs/architecture/c-sharp-client-library.md). It includes an implementation for `GetAverageCarbonIntensity` and for `GetCurrentForecast`. `GetAverageCarbonIntensity` uses the `EmissionsHandler` to return the carbon intensity rate of a location for a specific timespan. `GetCurrentForecast` uses the `ForecastHandler` to yield the optimal time of a specified location and duration. + +The functions can run locally for debugging or be deployed to Azure. See the [data source configuration docs](../../docs/configuration.md#datasources) for detailed information about configuring the data source(s) your Azure Function will use. ## Azure Function Dependency Injection @@ -18,9 +20,11 @@ The Carbon Aware SDK is included in the function .csproj file by [creating and a } ``` +> Note as the in-process [Azure Function uses dependency injection](https://learn.microsoft.com/en-us/azure/azure-functions/functions-dotnet-dependency-injection) though via [Microsoft.Azure.Functions.Extensions](https://www.nuget.org/packages/Microsoft.Azure.Functions.Extensions/) there is a version conflict of [Microsoft.Extensions.Configuration](https://www.nuget.org/packages/Microsoft.Extensions.Configuration). It is fixed adding a version specific project dependency (in .csproj) to the same version as the Carbon Aware SDK. Microsoft.Extensions.Configuration is backwards compatible. + ## Run Function Locally -Both Azure Function apps can be run locally without needing an azure subscription. The process for running both is the same, as they use the same configuration, just call different paths within the SDK. +Both Azure Function apps can be [run locally](https://learn.microsoft.com/azure/azure-functions/functions-develop-local) without needing an Azure subscription. The process for running both is the same, as they use the same configuration, just call different paths within the SDK. ### Prerequisites to Run @@ -46,7 +50,7 @@ curl --location --request GET 'http://localhost:7071/api/GetAverageCarbonIntensi #### _Example call for Get Current Forecast function_ -The following example will call the Current Forecast route. If an error is returned, update the start and end dates. The request can use either the request body or query parameters. +The following example will call the Current Forecast route. The request can use either the request body or query parameters. ```bash curl --location --request GET 'http://localhost:7071/api/GetCurrentForecast' \ @@ -59,9 +63,11 @@ curl --location --request GET 'http://localhost:7071/api/GetCurrentForecast' \ }' ``` +> Note: startDate and endDate must be a valid interval within the future 24 hours from the time of calling. + ## Deploy to Azure -If you have an azure subscription, you can also deploy these functions to Azure. +If you have an Azure subscription, you can also deploy these functions to Azure. ### Prerequisites to Deploy @@ -107,7 +113,7 @@ Update the [appsettings.json](./appsettings.json) file to include the desired [c To publish the function code to a function app in Azure, use the publish command in the samples/azure-function folder: ```bash -func Azure Functionapp publish $functionApp +func azure functionapp publish $functionApp ``` ## References From 4938978ae5d4f10ee9e370384d21671ec3807b2e Mon Sep 17 00:00:00 2001 From: Jennifer Madiedo Date: Thu, 5 Jan 2023 13:11:53 -0500 Subject: [PATCH 14/14] Adding html for dashes and spaces --- docs/selecting-a-data-source.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/selecting-a-data-source.md b/docs/selecting-a-data-source.md index 9ec9e7f41..0dcaba02c 100644 --- a/docs/selecting-a-data-source.md +++ b/docs/selecting-a-data-source.md @@ -25,9 +25,9 @@ Not all data sources support all the routes provided in the interfaces (`IEmissi | Methods | WattTime | ElectricityMaps | JSON | CLI Usage | Web Api Usage | SDK Usage |--------------|:-----------:|:-----------------:|:------:|:-:|:-:|:-:| -| GetCarbonIntensityAsync | ✅ | ✅ | ✅ |`emissions`|`emissions/bylocation` or `emissions/bylocations` or `emissions/bylocations/best` or `emissions/average-carbon-intensity` or `average-carbon-intensity/batch`|`GetEmissionsDataAsync(...)` or `GetBestEmissionsDataAsync(...)` or `GetAverageCarbonIntensityDataAsync(...)`| -| GetCurrentForecastAsync | ✅ | ✅ | ❌ |`emissions-forecasts`|`forecasts/current`|`GetCurrentForecastAsync(...)`| -| GetForecastByDateAsync | ✅ | ❌ | ❌ |`emissions-forecasts --requested-at`|`forecasts/batch` with `requestedAt` field|`GetForecastByDateAsync(...)`| +| GetCarbonIntensityAsync | ✅ | ✅ | ✅ |`emissions`|`emissions/bylocation` or `emissions/bylocations` or `emissions/bylocations/best` or `emissions/average`‑`carbon`‑`intensity` or `emissions/average`‑`carbon`‑`intensity/batch`|`GetEmissionsDataAsync(...)` or `GetBestEmissionsDataAsync(...)` or `GetAverageCarbonIntensityDataAsync(...)`| +| GetCurrentForecastAsync | ✅ | ✅ | ❌ |`emissions`‑`forecasts`|`forecasts/current`|`GetCurrentForecastAsync(...)`| +| GetForecastByDateAsync | ✅ | ❌ | ❌ |`emissions`‑`forecasts` ‑‑`requested`‑`at`|`forecasts/batch` with `requestedAt` field|`GetForecastByDateAsync(...)`| ## Location Coverage