Orion-ld Crash after loading custom @context

[yalamovm]

Hello,

I am running Orion-LD version 1.12.0 with the core @context
https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld
in a Kubernetes cluster.

On the very first run, when only the core @context is present, Orion-LD starts and runs as expected:

I: 000000.000: orionld.cpp[1171]: main: Orion Context Broker is running
I: 000000.000: orionld.cpp[812]: versionInfo: Version Info:
I: 000000.000: orionld.cpp[813]: versionInfo: -----------------------------------------
I: 000000.000: orionld.cpp[814]: versionInfo: orionld version:    1.12.0
I: 000000.000: orionld.cpp[815]: versionInfo: based on orion:     1.15.0-next
I: 000000.000: orionld.cpp[816]: versionInfo: core @context:      https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld
I: 000000.000: orionld.cpp[817]: versionInfo: git hash:           nogitversion
I: 000000.000: orionld.cpp[818]: versionInfo: build branch:       
I: 000000.000: orionld.cpp[819]: versionInfo: compiled by:        root
I: 000000.000: orionld.cpp[820]: versionInfo: compiled in:        
I: 000000.000: orionld.cpp[821]: versionInfo: -----------------------------------------
I: 000000.015: mongocInit.cpp[307]: mongocInit: Connecting to mongo for the C driver (URI: mongodb://mongo-db/)
I: 000000.067: orionld.cpp[1334]: main: Auto-configured reqPoolSize to 8 (CPU cores: 4)
I: 000000.068: orionld.cpp[1388]: main: Startup completed
I: 000000.068: orionld.cpp[1403]: main: Initialization is Done
I: 000000.068: orionld.cpp[1404]: main:   Accepting REST requests on port 1026 (experimental API endpoints are enabled)
I: 000000.068: orionld.cpp[1405]: main:   TRoE:                      Enabled
I: 000000.068: orionld.cpp[1406]: main:   Distributed Operation:     Disabled
I: 000000.068: orionld.cpp[1407]: main:   Health Check:              Disabled
I: 000000.068: orionld.cpp[1408]: main:   Entity Maps:               Disabled
I: 000000.068: orionld.cpp[1409]: main:   Distributed Subscriptions: Disabled
I: 000000.068: orionld.cpp[1414]: main:   Postgres Server Version:   12.0.7
I: 000000.068: orionld.cpp[1420]: main:   Postgres Client Version:   18.0.1
I: 000000.068: orionld.cpp[1423]: main:   Mongo Server Version:      8.2.4
I: 000000.068: orionld.cpp[1427]: main:   Mongo Driver:              mongoc driver- ONLY (MongoDB C++ Legacy Driver is DISABLED)
I: 000000.068: orionld.cpp[1428]: main:   MongoC Driver Version:     2.2.0
I: 000000.784: mhdConnectionInit.cpp[1290]: mhdConnectionInit: ------------------------- Servicing NGSI-LD request 001: POST /ngsi-ld/v1/entityOperations/upsert/ --------------------------

After loading the following custom @context, Orion-LD continues to work correctly:

[root@orion /]# curl http://ld-context:3004/context.jsonld
{
  "@context": {
    "Building": "https://smartdatamodels.org/dataModel.Building/Building",
    "address": "https://smartdatamodels.org/address",
    "areaServed": "https://smartdatamodels.org/areaServed",
    "category": "https://smartdatamodels.org/dataModel.Building/category",
    "description": "http://purl.org/dc/terms/description",
    "peopleOccupancy": "https://smartdatamodels.org/dataModel.Building/peopleOccupancy",
    "location": "ngsi-ld:location",
    "floorsAboveGround": "https://smartdatamodels.org/dataModel.Building/floorsAboveGround",
    "floorsBelowGround": "https://smartdatamodels.org/dataModel.Building/floorsBelowGround",
    "BuildingSpace": "https://smartdatamodels.org/dataModel.S4BLDG/BuildingSpace",
    "airVolume": "https://smartdatamodels.org/dataModel.S4BLDG/airVolume",
    "controlledAsset": "https://smartdatamodels.org/dataModel.Device/controlledAsset",
    "Device": "https://smartdatamodels.org/dataModel.Device/Device",
    "deviceCategory": "https://smartdatamodels.org/dataModel.Device/deviceCategory",
    "isContainedInBuildingSpace": "https://smartdatamodels.org/dataModel.S4BLDG/isContainedInBuildingSpace",
    "controlledProperty": "https://smartdatamodels.org/dataModel.Device/controlledProperty",
    "serialNumber": "https://smartdatamodels.org/dataModel.Device/serialNumber",
    "value": "ngsi-ld:hasValue",
    "WeatherForecast": "https://smartdatamodels.org/dataModel.Weather/WeatherForecast",
    "dataProvider": "https://smartdatamodels.org/dataProvider",
    "dayMaximum": "https://smartdatamodels.org/dataModel.Weather/dayMaximum",
    "dayMinimum": "https://smartdatamodels.org/dataModel.Weather/dayMinimum",
    "precipitationProbability": "https://smartdatamodels.org/dataModel.Weather/precipitationProbability",
    "source": "https://smartdatamodels.org/source",
    "validFrom": "https://smartdatamodels.org/validFrom",
    "validTo": "https://smartdatamodels.org/dataModel.Weather/validTo",
    "WeatherObserved": "https://smartdatamodels.org/dataModel.Weather/WeatherObserved",
    "dateObserved": "https://smartdatamodels.org/dateObserved",
    "temperature": "https://smartdatamodels.org/dataModel.Weather/temperature",
    "feelLikesTemperature": "https://smartdatamodels.org/dataModel.Weather/feelLikesTemperature",
    "relativeHumidity": "https://smartdatamodels.org/dataModel.Weather/relativeHumidity",
    "precipitation": "https://smartdatamodels.org/dataModel.Weather/precipitation"
  }
}

However, after restarting Orion-LD, it no longer comes up correctly. There is no increase in memory usage, no CPU usage, and port 1026 is no longer open. The only service that remains reachable is Prometheus.

[root@orion /]# netstat  -lptn 
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      1/orionld         

There is also nothing meaningful in the logs, even with tracing enabled:

I: 000000.000: orionld.cpp[1171]: main: Orion Context Broker is running
I: 000000.000: orionld.cpp[812]: versionInfo: Version Info:
I: 000000.000: orionld.cpp[813]: versionInfo: -----------------------------------------
I: 000000.000: orionld.cpp[814]: versionInfo: orionld version:    1.12.0
I: 000000.000: orionld.cpp[815]: versionInfo: based on orion:     1.15.0-next
I: 000000.000: orionld.cpp[816]: versionInfo: core @context:      https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld
I: 000000.000: orionld.cpp[817]: versionInfo: git hash:           nogitversion
I: 000000.000: orionld.cpp[818]: versionInfo: build branch:       
I: 000000.000: orionld.cpp[819]: versionInfo: compiled by:        root
I: 000000.000: orionld.cpp[820]: versionInfo: compiled in:        
I: 000000.000: orionld.cpp[821]: versionInfo: -----------------------------------------
W: 000000.013: mongocInit.cpp[63]: mongocLog: MONGOC[mongoc]:warning: Falling back to malloc for counters.
I: 000000.014: mongocInit.cpp[307]: mongocInit: Connecting to mongo for the C driver (URI: mongodb://mongo-db/)

The only way I have found to make Orion-LD work again is to remove the static page that provides the custom @context:

[root@orion /]# curl http://ld-context:3004/context.jsonld

My assumption is that this is caused either by an invalid custom @context or by an invalid configuration related to context handling.

Any advice or guidance would be greatly appreciated.

Thanks,

[kzangeli]

That's weird. Very weird, very simple user context, nothing strange there.
You can remove "value" and "location" - those are handled by the core context anyway and you cannot override what's in the core context.

I will create a functest using your exact user context (inject it, kill the broker and restart it and then send it a ping) and with some luck the broker doesn't respond to that "ping" and I have the issue reproduced.
Once reproduced, to fix it is typically trivial.
A bit busy right now but as soon as I have a moment I'll have a look.