AIS.NET Projects

Package	Status
Ais.Net
Ais.Net.Models
Ais.Net.Receiver
Ais.Net.Receiver.Storage.Azure.Blob

The AIS.NET project contains a series of layers, from a low-level high performance NMEA AIS sentence decoder, to a rich high-level C# 9.0 models of AIS message types, a receiver component that can listen to TCP streams of NMEA sentences and expose them as an IObservable<string> of raw sentences or an decoded IObservable<IAisMessage>, and finally a Storage Client implementation to persisting the raw NMEA sentence stream to Azure Blob storage for future processing.

Ais.Net.Receiver

A simple .NET AIS Receiver for capturing the Norwegian Coastal Administration's marine Automatic Identification System (AIS) AIVDM/AIVDO NMEA message network data (available under Norwegian license for public data (NLOD)) and persisting in Microsoft Azure Blob Storage.

The Norwegian Coastal Administration provide a TCP endpoint (153.44.253.27:5631) for broadcasting their raw AIS AIVDM/AIVDO sentences, captured by over 50 base stations, and covers the area 40-60 nautical miles from the Norwegian coastline.

This project contains a NmeaReceiver which consumes the raw NetworkStream, a NmeaToAisMessageTypeProcessor, which can decode the raw sentences into IAisMessage, and ReceiverHost which manages the process and exposes an IObservable<string> for raw sentences and an IObservable<IAisMessage> for decoded messages. ReceiverHost can be hosted in a console application or other runtime environments like .NET Interactive.

The project also includes a demo console which shows how the various pieces can fit together, including subscribing to the IObservable<string> and IObservable<IAisMessage> streams and displaying the results or batch the AIVDM/AIVDO sentences and write them to Azure Blob Storage using the Append Blob feature, to create timestamped hour-long rolling logs.

The purpose of this application is to provide sample data for Ais.Net - the .NET Standard, high performance, zero allocation AIS decoder. The majority of raw AIS data is only available via commercial sources, and thus creating AIS datasets large enough to test / benchmark Ais.Net is almost impossible.

The Norwegian Coastal Administration TCP endpoint produces:

• ~2.9 KB per second
• ~10.3 MB per hour
• ~248 MB per day
• ~1.7 GB per week
• ~7 GB per month
• ~81.4 GB per year

Azure Blob Storage Taxonomy

The AIS data is stored using the following taxonomy

<USER DEFINED CONTAINER NAME>/raw/yyyy/MM/dd/yyyyMMddTHH.nm4

An example directory listing, with a user defined container name of nmea-ais would look as follows:

\---nmea-ais
    \---raw
        \---2021
            \---07
                +---12
                | 20210712T00.nm4 |
                | 20210712T01.mm4 |
                | 20210712T02.nm4 |
                | 20210712T03.nm4 |
                | 20210712T04.nm4 |
                | 20210712T05.nm4 |
                | 20210712T06.nm4 |
                | 20210712T07.nm4 |
                | 20210712T08.nm4 |
                | 20210712T09.nm4 |
                | 20210712T10.nm4 |
                | 20210712T11.nm4 |
                | 20210712T12.nm4 |
                | 20210712T13.nm4 |
                | 20210712T14.nm4 |
                | 20210712T15.nm4 |
                | 20210712T16.nm4 |
                | 20210712T17.nm4 |
                | 20210712T18.nm4 |
                | 20210712T19.nm4 |
                | 20210712T20.nm4 |
                | 20210712T21.nm4 |
                | 20210712T22.nm4 |
                | 20210712T23.nm4 |
                +---20210713
                | 20210713T00.nm4 |

To Run

Update the values in the settings.json file:

{
  "Ais": {
    "host": "153.44.253.27",
    "port": "5631",
    "retryAttempts": 100,
    "retryPeriodicity": "00:00:00:00.500"
  },
  "Storage": {
    "connectionString": "<YOUR AZURE STORAGE CONNECTION STRING>",
    "containerName": "nmea-ais",
    "writeBatchSize": 500
  }
}

From the command line: dotnet Ais.Net.Receiver.Host.Console.exe

Raspberry Pi

As the AIS.NET stack is written in .NET 6.0 and .NET Standard you can publish the Ais.Net.Receiver.Host.Console application with a target runtime of Portable. This will allow you to run the receiver on a Raspberry Pi if you want to capture your own AIS data.

For reliability you can run Ais.Net.Receiver.Host.Console.dll as daemon.

Installation

The combination of Windows Terminal, .NET and PowerShell make a Raspberry Pi a very productive environment for .NET Devs.

Install Windows Terminal. You can download Windows Terminal from the Microsoft Store or from the GitHub releases page.

Open Windows Terminal and use ssh pi@<Raspberry PI IP Address> to connect to your Pi.

Install .NET 6.0

Use the following commands to install .NET 6.0 on your Pi.

1. curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel Current
2. echo 'export DOTNET_ROOT=$HOME/.dotnet' >> ~/.bashrc
3. echo 'export PATH=$PATH:$HOME/.dotnet' >> ~/.bashrc
4. source ~/.bashrc
5. dotnet --version

Install PowerShell 7.x

Use the following commands to install PowerShell on your Pi.

1. Download the latest package wget https://github.com/PowerShell/PowerShell/releases/download/v7.2.7/powershell-7.2.7-linux-arm32.tar.gz
2. Create a directory for it to be unpacked into mkdir ~/powershell
3. Unpack tar -xvf ./powershell-7.2.7-linux-arm32.tar.gz -C ~/powershell
4. Give it executable rights sudo chmod +x /opt/microsoft/powershell/7/pwsh
5. Create a symbolic link sudo ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh

use the command pwsh to enter the PowerShell session.

Install Ais.Net.Receiver.Host.Console

2. From the solution root, open a command prompt and type dotnet publish -c Release .\Solutions\Ais.Net.Receiver.sln
3. Add your Azure Blob Storage Account connection string to settings.json
4. Transfer (I use Beyond Compare as it has native SSH support) the contents of .\Solutions\Ais.Net.Receiver.Host.Console\bin\Release\net5.0\publish to a folder called aisr in the home/pi directory on your Raspberry Pi (assuming you still have the default set up.)
5. Copy Solutions\Ais.Net.Receiver.Host.Console.RaspberryPi\aisr.service to /lib/systemd/system/aisr.service
6. run sudo chmod 644 /lib/systemd/system/aisr.service
7. run sudo systemctl enable aisr.service
8. run sudo reboot

You can use journalctl -u "aisr" to view the console output of Ais.Net.Receiver.Host.Console.dll

You can use sudo systemctl restart aisr to restart the service.

If you need to look at / edit the deployed aisr.service use sudo nano /lib/systemd/system/aisr.service make your edits then use Ctrl+O and Ctrl+X to save the file and exit.

Use Azure Storage Explorer to browse to where files are captured.

Configuration

Configuration is read from settings.json and can also be overridden for local development by using a settings.local.json file.

{
  "Ais": {
    "host": "153.44.253.27",
    "port": "5631",
    "loggerVerbosity": "Minimal", 
    "statisticsPeriodicity": "00:01:00",
    "retryAttempts": 100,
    "retryPeriodicity": "00:00:00:00.500"
  },
  "Storage": {
    "enableCapture": true,
    "connectionString": "DefaultEndpointsProtocol=https;AccountName=<ACCOUNT_NAME>;AccountKey=<ACCOUNT_KEY>",
    "containerName": "nmea-ais-dev",
    "writeBatchSize": 500
  }
}

AIS

These settings control the ReceiverHost and its behaviour.

• host: IP Address or FQDN of the AIS Source
• port: Port number for the AIS Source
• loggerVerbosity: Controls the output to the console.
  • Quiet = Essential only,
  • Minimal = Statistics only. Sample rate of statistics controlled by statisticsPeriodicity,
  • Normal = Vessel Names and Positions,
  • Detailed = NMEA Sentences,
  • Diagnostic = Messages and Errors
• statisticsPeriodicity: TimeSpan defining the sample rate of statistics to display
• retryAttempts: Number of retry attempts when a connection error occurs
• retryPeriodicity: How long to wait before a retry attempt.

Storage

These settings control the capturing NMEA sentences to Azure Blob Storage.

• enableCapture: Whether you want to capture the NMEA sentences and write them to Azure Blob Storage
• connectionString: Azure Storage Account Connection String
• containerName: Name of the container to capture the NMEA sentences. You can use this to separate a local dev storage container from your production storage container, within the same storage account.
• writeBatchSize: How many NMEA sentences to batch before writing to Azure Blob Storage.

Licenses

AIS.Net.Receiver is also available under the Apache 2.0 open source license.

The Data ingested by the AIS.Net.Receiver is licensed under the Norwegian license for public data (NLOD).

Project Sponsor

This project is sponsored by endjin, a UK based Technology Consultancy which specializes in Data & Analytics, AI & Cloud Native App Dev, and is a .NET Foundation Corporate Sponsor.

We help small teams achieve big things.

We produce two free weekly newsletters:

• Azure Weekly for all things about the Microsoft Azure Platform
• Power BI Weekly for all things Power BI, Microsoft Fabric, and Azure Synapse Analytics

Keep up with everything that's going on at endjin via our blog, follow us on Twitter, YouTube or LinkedIn.

We have become the maintainers of a number of popular .NET Open Source Projects:

• Reactive Extensions for .NET
• Reaqtor
• Argotic Syndication Framework

And we have over 50 Open Source projects of our own, spread across the following GitHub Orgs:

• endjin
• Corvus
• Menes
• Marain
• AIS.NET

And the DevOps tooling we have created for managing all these projects is available on the PowerShell Gallery.

For more information about our products and services, or for commercial support of this project, please contact us.

Code of conduct

This project has adopted a code of conduct adapted from the Contributor Covenant to clarify expected behaviour in our community. This code of conduct has been adopted by many other projects. For more information see the Code of Conduct FAQ or contact hello@endjin.com with any additional questions or comments.

IP Maturity Model (IMM)