All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, markdownlint, and this project adheres to Semantic Versioning.
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.10.3
- In
requirements.txt
, updated:- boto3==1.34.131
- orjson==3.10.5
- psutil==6.0.0
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.10.1
- In
requirements.txt
, updated:- azure-servicebus==7.12.2
- boto3==1.34.110
- confluent-kafka==2.4.0
- orjson==3.10.3
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.9.0
- In
requirements.txt
, updated:- azure-servicebus==7.12.0
- boto3==1.34.64
- orjson==3.9.15
- psutil==5.9.8
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.8.0
- In
requirements.txt
, updated:- azure-servicebus==7.11.4
- boto3==1.29.0
- confluent-kafka==2.3.0
- orjson==3.9.10
- psutil==5.9.6
- psycopg2-binary==2.9.9
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.7.1
- In
requirements.txt
, updated:- azure-servicebus==7.11.2
- boto3==1.28.57
- confluent-kafka==2.2.0
- orjson==3.9.7
- psycopg2-binary==2.9.8
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.6.0
- In
requirements.txt
, updated:- boto3==1.26.163
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.5.3
- In
requirements.txt
, updated:- azure-servicebus==7.11.0
- boto3==1.26.153
- orjson==3.9.1
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.5.2
- In
requirements.txt
, updated:- azure-servicebus==7.9.0
- boto3==1.26.130
- confluent-kafka==2.1.1
- orjson==3.8.12
- pika==1.3.2
- psutil==5.9.5
- psycopg2-binary==2.9.6
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.5.0
- In
requirements.txt
, updated:- azure-servicebus==7.8.3
- boto3==1.26.104
- confluent-kafka==2.0.2
- orjson==3.8.9
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.4.0
- In
requirements.txt
, updated:- boto3==1.26.48
- orjson==3.8.5
- Updated SQS visibility timeout from 30s to 30m
- Fix for
rabbitmq-custom
logging.
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.3.2
- In
requirements.txt
, updated:- boto3==1.25.4
- orjson==3.8.1
- pika==1.3.1
- psutil==5.9.3
- psycopg2-binary==2.9.5
- Added
rabbitmq-custom
subcommand
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-runtime:3.3.1
- In
requirements.txt
, updated:- boto3==1.24.89
- psycopg2-binary==2.9.4
- In
Dockerfile
, updated FROM instruction tosenzing/senzingapi-tools:3.3.0
- In
requirements.txt
, updated:- boto3==1.24.82
- psutil==5.9.2
- In
Dockerfile
, bump fromsenzing/senzingapi-runtime:3.1.1
tosenzing/senzingapi-runtime:3.2.0
- Updated python dependencies
- Removed support for
SENZING_DEFAULT_ENTITY_TYPE
- Deleted
Dockerfile-with-data
- Changed from
SENZING_AZURE_CONNECTION_STRING
toSENZING_AZURE_QUEUE_CONNECTION_STRING
for clarity
- In
Dockerfile
, bump fromsenzing/senzingapi-runtime:3.1.0
tosenzing/senzingapi-runtime:3.1.1
- Migrated to
senzing/senzingapi-runtime
as Docker base image
- Add Support for
SENZING_LICENSE_BASE64_ENCODED
- Remove support for
SENZING_ENTITY_TYPE
- Remove support for
SENZING_DATA_SOURCE
parameter - Upgrade
Dockerfile
todebian:11.3-slim@sha256:f6957458017ec31c4e325a76f39d6323c4c21b0e31572efa006baa927a160891
- Increase monitoring check frequency
- Fixed issue with records not having
RECORD_ID
- Upgrade
Dockerfile
toFROM debian:11.3-slim@sha256:06a93cbdd49a265795ef7b24fe374fee670148a7973190fb798e43b3cf7c5d0f
- Added
libodbc1
to Dockerfile
- Added
poll()
calls for Kafka's output and failure queues - Terminate on failure to deliver to info or failure queues.
- Added additional logging around calls to Senzing
- Migrate from
senzingdata-v2
tosenzingdata-v3
- In
Dockerfile-with-data
, added docker build args to Dockerfile for more flexibility.- SENZING_APT_REPOSITORY_URL
- SENZING_DATA_PACKAGE_NAME
- SENZING_DATA_SUBDIRECTORY
- Remove entire messages from log as they may contain Personally Identifiable Information (PII)
- Support for
libcrypto
andlibssl
- Support for enhanced v3 python package styles
- Improved support for Senzing v2 and v3 python package styles
- Added label to runner image
- updated documentation
- Added label to runner image
- updated documentation
- Support for Senzing v2 and v3 python package styles
- Updated base image to address vulnerabilities
- Added support for Kafka configuration
- trimmed image size
- Updated to senzing/senzing-base:1.6.2
- Added subcommands for Azure Queue and Azure SQL Database:
azure-queue
azure-queue-withinfo
- Create a
Dockerfile-with-data
which embeds/opt/senzing/data
- Added performace timing for G2 load
- Updated to senzing/senzing-base:1.6.1
- Updated to senzing/senzing-base:1.6.0
- SQS based loads now only remove the record from the queue if it was successfully loaded to Senzing, permitting failed records to go to the dead letter queue, if configured.
- RabbitMQ virtual host is now a settable parameter.
- Properly handle long running records that take a while to load (> 1 minute) to prevent RabbitMQ heartbeat timeouts.
- Add logging messages around
g2_engine.initV2
andg2_engine.primeEngine()
- Additional debug logging for performance issues
- Support for
SENZING_SKIP_GOVERNOR
andSENZING_SKIP_INFO_FILTER
- Updated Dockerfile base to senzing/senzing-base:1.5.5
- Support for
SENZING_SQS_INFO_QUEUE_DELAY_SECONDS
- Added
endpoint_url
in AWS SQS configuration.
- RabbitMQ, Kafka, and SQS loading subcommands (including withinfo variations) now support multiple records per message when formated as a json array.
- Support for random delay on SQS empty queue
- Support for
SENZING_DELAY_RANDOMIZED
- Fixes in Governor support
- Support for stack traces using
gdb
. - Support for
SENZING_RABBITMQ_HEARTBEAT_IN_SECONDS
- Add
SENZING_PRIME_ENGINE
.
- Improved failure handling
- Support for Senzing Governor
- Support for RabbitMQ exchanges
- Deprecated support for multiple Kafka and RabbitMQ processes (i.e. not threads)
- Fix issue with empty
RECORD_ID
- Add support for
SENZING_SKIP_DATABASE_PERFORMANCE_TEST
- Upgrade to senzing/senzing-base:1.5.2
- Updated Dockerfile base to senzing/senzing-base:1.5.0
- Added support for AWS SQS
- Improve logging
- Fixed parameter to RabbitMQ
- Added assertions
- Improve logging
- Fix parameter to RabbitMQ
- Add assertions
- "future-proofed" Governor and InfoFilter
- Refactored: Renamed "with_info" to "withinfo
- Fixed messages
- Added "info" and "failure" queues.
- Added "info" and "failure" queues.
- Update to senzing/senzing-base:1.4.0
- Governor: Improve filename to avoid import conflict.
- Support for
SENZING_RABBITMQ_PREFETCH_COUNT
- Governor: A "hook" for an external class called before record is consumed
- Support for
SENZING_ENGINE_CONFIGURATION_JSON
- Added MSSQL support
- Update to senzing/senzing-base:1.3.0
- Improved license expiry warning
- Remove errant exception catching
- RPM based installation
- Added the ability to delay processing via
SENZING_DELAY_IN_SECONDS
- Based on
senzing/senzing-base:1.1.0
, a non-root, immutable container
- Migrate to python3
- Python 2.7 implementation.
- Support Kafka consumer.
- Support RabbitMQ consumer.