Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extend NGSI-LD support for NGSIToPostgreSQL processor (and other global upgrades) #56

Conversation

bobeal
Copy link

@bobeal bobeal commented Nov 2, 2021

Note: submitting the PR in draft status since there are still some things to finish (see TODOs below), but new features are already in good shape and tested on a real deployment.

Proposed changes

Globally extend the NGSI-LD support of the NGSIToPostgreSQL processor and also do some global upgrades (NiFi version, Java version):

  • chore: upgrade NiFi version to 1.14.0 (only tested for NGSIToPostgreSQL processor)
  • chore: set base Java version to 11 for nifi-ngsi-bundle
  • chore: replace part of System.out statements by log statements
  • feat: extract keys to ignore in payloads into a constant list for improved maintainability
  • feat: add support for multi-attributes (section 4.5.5 of the NGSI-LD spec) to avoid too long (and truncated) names
    • a configurable prefix can be removed from the dataset ids
  • feat: add support for sub-attributes of all types of attributes (was previously only supported for properties)
  • feat: use types as much as possible for columns (e.g. TIMESTAMPTZ for dates or NUMERIC for numeric values)
    • currently using TEXT (default), TIMESTAMPTZ and NUMERIC (more could be added in the future)
    • it won't break existing installations because columns already exist, but they could benefit from such a new feature

TODOs before removing the draft status

  • align NGSIv2 code where needed (e.g., in getValuesForInsert, listOfFields)
  • add some specific tests for the new features
  • upgrade bundle version (to which version?)

Types of changes

What types of changes does your code introduce to the project: Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in downstream modules

@bobeal
Copy link
Author

bobeal commented Nov 2, 2021

I have read the CLA Document and I hereby sign the CLA

@bobeal bobeal changed the title feat: extend NGSI-LD support for NGSIToPostgreSQL process (and other global upgrades) feat: extend NGSI-LD support for NGSIToPostgreSQL processor (and other global upgrades) Nov 2, 2021
@bobeal bobeal closed this Oct 31, 2024
@bobeal bobeal deleted the feature/improve-ngsild-support-global-upgrades branch October 31, 2024 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant