Audit.Net as a dedicated API?

[cbdg-mitch]

Hi there @thepirat000 - I have 4 .net Core applications that currently are 100% microservice driven - they do not have direct access to our back end database. I've created an "Audit API" that would take an audit event as a parameter (seen below), Unfortunately so far I have not been able to derive how I can make this work without basically copy/pasting all of the classes from the Audit.Net source code to parse the audit event natively like the library does. I am able to call the Audit API from my application using a custom provider and the insert event, it's just the receiving end I can't figure out.

Controller (Audit API):

[HttpPost] [Route("AddAuditEvent")] public ActionResult<AuditEvent> AddAuditEvent([FromBody] AuditEvent entity) { if (ModelState.IsValid) { // Do something here to process the Audit event object //_AuditRepository.AddAuditEvent(entity); //_sqlDataProvider.InsertEvent(entity); return Ok(); } else { var errors = ModelState .Where(x => x.Value.Errors.Count > 0) .Select(x => new { x.Key, x.Value.Errors }) .ToArray(); return BadRequest(); } }

Any suggestions here? I have Audit.Net working perfectly if I use the SQL provider in the actual application, but I really don't want to have to do that. I want the Audit.Net SQL provider in the Audit API and just to be able to call the endpoint with the Audit Event param.

Thanks for any advice.

[thepirat000]

What's the problem on the receiving part?
Are both the sending and receiving part targeting the same framework version?

[cbdg-mitch]

@thepirat000 thanks for the reply. So the API endpoint receives the audit event object, but I do not see anything in your documentation about a function that simply takes an audit event as a parameter, so I do not know what to do with the object to then get it into the SQL data provider.

[cbdg-mitch]

I guess to clarify, is that the API itself is not auditing anything, it's just storing data. The SQL data provider that you provide us (great work BTW) does that work all for you as part of the audit itself, so the API in this situation is "dumb" and just takes the event object and from there, I don't know how to just... store it, with the proper structure that the SQL provider would normally handle.

[cbdg-mitch]

Sort of just talking this out to myself, but like, I see how your source code in the SQL provider is generating columns and values based on the audit event all as part of the library, so without just trying to copy/paste all those functions, I hoped that I could somehow just pass the AuditEvent object from the API call parameter into some function and let the SQL data provider library do what it normally would do.

[thepirat000]

I guess you can instantiate a SqlDataProvider from your controller.

public class MyController : ControllerBase
{
    private AuditDataProvider _dataProvider;

    public MyController(AuditDataProvider dataProvider)
    {
        _dataProvider = dataProvider;
    }        

    [HttpPost]
    [Route("AddAuditEvent")]
    public ActionResult<AuditEvent> AddAuditEvent([FromBody] AuditEvent entity)
    {
        _dataProvider.InsertEvent(entity);
        // ...
    }
}

And configure the dependency on your startup, for example:

    services.AddSingleton<AuditDataProvider>(new Audit.SqlServer.Providers.SqlDataProvider(config => config
        .ConnectionString("data source=localhost;initial catalog=Audit;integrated security=true;")
        .TableName("Event")
        .IdColumnName("EventId")
        .JsonColumnName("JsonData")));

[cbdg-mitch]

Thanks! I'm trying it now and will report back.

[cbdg-mitch]

@thepirat000 so just a little more advice here.

On startup of the application (not the API), I'm registering the custom data provider that will call the API.

Audit.Core.Configuration.DataProvider = new AuditEventProvider();

Here is the Custom Provider:

private readonly JsonSerializerOptions _jsonSerializerOptions = new JsonSerializerOptions() {
            PropertyNamingPolicy = JsonNamingPolicy.CamelCase
        };

        private readonly HttpClient _httpClient;
        private readonly ITokenService _tokenService;
        private readonly ConfigOptions _configOptions;

        public AuditEventProvider(HttpClient httpClient, IConfiguration configuration,
            ITokenService tokenService, IOptions<ConfigOptions> configOptions) {
            _httpClient = httpClient;
            _tokenService = tokenService;
            _configOptions = configOptions.Value;
        }
        
        public override object InsertEvent(AuditEvent entity) {
            var token = _tokenService.GetToken(ApiScopes.ApiAuditScope);
            _httpClient.SetBearerToken(token.Result.AccessToken);

            var request = new HttpRequestMessage(HttpMethod.Post, ApiPaths.ApiAudit.Audit.AddAuditEvent) {
                Content = JsonContent.Create(entity)
            };

            var response = _httpClient.SendAsync(request);
            response.Result.EnsureSuccessStatusCode();

            return entity;
        }
        ```
        
        The issue is that the startup code is giving an error that there's no matching constructor, so I'm not sure how to alter the startup code to register the custom provider but still be able to inject all of my required stuff into the Custom provider to be able to call the API.
        
        Can I somehow set the custom default provider another way than new AuditEventProvider() ? Or will adding the empty constructor still allow the rest of the code to work fine?

[cbdg-mitch]

Wondering now if I need to simply make the custom data provider call the registered AuditEventService as a separate call so the service can be registered without the custom data provider being included.. trying that..

[cbdg-mitch]

I think this is still a valid Q - Even if I take the DI stuff out of the constructor for the custom data provider, I still need to DI the actual service for the custom data provider to call, so the "setup" line in startup.cs will still need an empty constructor, so then I'm not sure the service call will work.

Let me know what you think!

[cbdg-mitch]

Just to be super clear, here's what I have for the custom provider now

namespace AppFrdb.Extensions {
    public class AuditEventProvider : AuditDataProvider {
        private readonly IAuditEventService _auditEventService;

        public AuditEventProvider(IAuditEventService auditEventService) {
            _auditEventService = auditEventService;
        }

        public override object InsertEvent(AuditEvent auditEvent) {
            _auditEventService.AddAuditEvent(auditEvent);
            return auditEvent;
        }
    }
}

So obviously with that, the "setup" in startup doesn't work due to no default constructor. Thanks for any help you can give when you have a chance.

[thepirat000]

I think one option is to resolve the dependencies in the Data Provider Factory.

For example, on your Startup Configure method:

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddHttpClient();
        services.AddTransient<ITokenService, TokenService>();
        services.AddTransient<AuditDataProvider, AuditEventProvider>();
        // ...
    }

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        // Factory method to create the DataProvider to use on each audit scope
        Audit.Core.Configuration.Setup().UseFactory(() => app.ApplicationServices.GetService<AuditDataProvider>());
        // ...
    }

[cbdg-mitch]

In this case above, I do successfully get an API token and all of the request information looks fine, so I'm pretty sure it's how I'm injecting the AuditEventService into the AuditEventProvider. Thoughts?

[thepirat000]

You are not awaiting the PostAsync call, is there a reason for that?

[cbdg-mitch]

I've been staring at this too long lol. I got the actual error that's occurring now. Should be much easier to debug. I'll report back!

System.InvalidOperationException: Multiple constructors accepting all given argument types have been found in type 'ApiAudit.Controllers.AuditEventController'. There should only be one applicable constructor.

[cbdg-mitch]

That was because I forgot to remove the extra constructor on the Audit API controller 🤦

Alright, so we're now hitting the endpoint @thepirat000 - here's what we're getting on the API endpoint Insert Event call.

Microsoft.Data.SqlClient.SqlException: 'Cannot insert the value NULL into column 'EventType', table 'bscaudit.dbo.Event'; column does not allow nulls. INSERT fails.'

Debugging the object that was passed to the API controller, EventType is NOT null (it's AddressType:Create)

[thepirat000]

Please share a complete sample that reproduces the issue on the API side

[cbdg-mitch]

Here is the AuditAPI app - I'm not sure it'll work since the database will be unavailable, but for context, I'll add a screenshot of the debugging information separately.
ApiAuditDemo.zip

[cbdg-mitch]

Here's a screenshot of the debugger at the time the error occurs. You can see the incoming entity has all the correct fields populated.

[cbdg-mitch]

I have to wonder if the AuditEvent object that is sent into the endpoint is not formatted quite right

Here is the call to the endpoint:

public async Task AddAuditEvent(AuditEvent entity) {
            var token = await _tokenService.GetToken(ApiScopes.ApiAuditScope);
            _httpClient.SetBearerToken(token.AccessToken);

            var request = new HttpRequestMessage(HttpMethod.Post, ApiPaths.ApiAudit.Audit.AddAuditEvent) {
                Content = JsonContent.Create(entity)
            };

            var response = await _httpClient.SendAsync(request);
            response.EnsureSuccessStatusCode();
        }

Is it something to do with doing a JsonContent.Create() on the already Json formatted AuditEvent? Some sort of "Double Json-ing"? So the key/values don't line up right for your library to parse them?

[thepirat000]

Looks like your SQL table is expecting an EventType column, but you are not configuring the SqlDataProvider to insert that column.

 services.AddSingleton<AuditDataProvider>(new Audit.SqlServer.Providers.SqlDataProvider(config => config
                .ConnectionString(Environment.GetEnvironmentVariable("DBCREDENTIAL"))
                .TableName("Event")
                .IdColumnName("EventId")
                .CustomColumn("EventType", ev => ev.EventType) // <--- Add the missing custom column
                .JsonColumnName("JsonData")));

[cbdg-mitch]

Ah! ok. I took the default configuration from your documentation.

that did it!

{"EventType":"AddressType:Create","Environment":{"UserName":"Mitch Russell","MachineName":"MDR-DESKTOP","DomainName":"MDR-DESKTOP","CallingMethodName":"AppFrdb.Shared.Admin.AddressTypes\u002B\u003COnAddressTypeRowInserting\u003Ed__15.MoveNext()","AssemblyName":"AppFrdb, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null","Culture":"en-US"},"Target":{"Type":"AddressTypeAdd","Old":{"OrganizationId":"00000000-0000-0000-0000-000000000000","AddedByUserId":"00000000-0000-0000-0000-000000000000"},"New":{"OrganizationId":"5003a074-088e-ea11-86e9-000d3a11483c","AddressTypeName":"test","AddedByUserId":"6b14edd9-e399-ea11-86e9-000d3a11483c"}},"Comments":["test Address Type Created"],"StartDate":"2022-06-26T17:00:56.2045873-04:00","EndDate":"2022-06-26T17:00:56.3228151-04:00","Duration":118,"ReferenceId":"5003a074-088e-ea11-86e9-000d3a11483c","OrganizationId":"5003a074-088e-ea11-86e9-000d3a11483c","UserId":"6b14edd9-e399-ea11-86e9-000d3a11483c"}

@thepirat000 THANK YOU. I will for sure be sending over a donation this week for all of the amazing support you provided in getting this approach working. If you need anything from me in terms of adding a dedicated API section to your documentation, let me know!

Last Q - is there any plan to upgrade the library to .NET 6+?

[thepirat000]

The library is already compatible with .NET 6