Error in getting ES parameters when using certificates in 8.0.5

[arrabito]

Dear all,

I've just updated to 8.0.5 but I get an error in the Monitoring System:

2022-12-08 11:27:15 UTC Monitoring/Monitoring INFO: Can not connect to DB will use Monitoring/Monitoring
2022-12-08 11:27:15 UTC Monitoring/Monitoring ERROR: Exception while calling initialization function
Traceback (most recent call last):
  File "/opt/dirac/versions/v2.2.0-1670311527/Linux-x86_64/lib/python3.9/site-packages/DIRAC/MonitoringSystem/DB/MonitoringDB.py", line 54, in __init__
    super().__init__(dbname=name.split("/")[1], fullName=name, indexPrefix=indexPrefix)
  File "/opt/dirac/versions/v2.2.0-1670311527/Linux-x86_64/lib/python3.9/site-packages/DIRAC/Core/Base/ElasticDB.py", line 25, in __init__
    raise RuntimeError("Cannot get database parameters: %s" % result["Message"])
RuntimeError: Cannot get database parameters: Failed to get the configuration parameter: Password.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/dirac/versions/v2.2.0-1670311527/Linux-x86_64/lib/python3.9/site-packages/DIRAC/Core/DISET/private/Service.py", line 124, in initialize
    result = initFunc(dict(self._serviceInfoDict))
  File "/opt/dirac/versions/v2.2.0-1670311527/Linux-x86_64/lib/python3.9/site-packages/DIRAC/MonitoringSystem/Service/MonitoringHandler.py", line 56, in initializeHandler
    cls.__db = result["Value"]()
  File "/opt/dirac/versions/v2.2.0-1670311527/Linux-x86_64/lib/python3.9/site-packages/DIRAC/MonitoringSystem/DB/MonitoringDB.py", line 56, in __init__
    self.log.error("Can't connect to MonitoringDB", repr(ex))
AttributeError: 'MonitoringDB' object has no attribute 'log'
2022-12-08 11:27:15 UTC Monitoring/Monitoring ERROR: Exception while calling initialization function: 'MonitoringDB' object has no attribute 'log'

while it was working with the previous 7.3 version.

I think this is due to the fact that we use certificates for authentication to ES, so no User/Password is provided. This use case was correctly handled in 7.3, but I have the impression that it's not anymore correctly handled now in the

ConfigurationSystem/Client/Utilities.py

Can you please confirm and eventually suggest a fix?

Thank you.

[arrabito]

I found a simple fix:

        result = gConfig.getOption("/Systems/NoSQLDatabases/Password")
	if not result["OK"]:
            gLogger.warn("No configuration parameter set for Password, assuming no Password")
            dbPass = None
        else:                                                                   
            dbPass = result["Value"]

and the same for the User parameter.

Let me know if you want me to create a PR.

[fstagni]

Yes I think you can are right. Would you create the PR yourself?

[arrabito]

yes no problem.
Should I target 8.0 ?

[fstagni]

yes, rel-v8r0 branch

[chaen]

No wait !
This was done by design: #6078
So the test needs to be more complex, for example testing that username & password are defined or certificate authentication is chosen.

[fstagni]

Good point. @arrabito would you still take care of this?

[arrabito]

Sorry, I didn't understand that you were suggesting that I take care of this.
I can do it, but not before the end of the next week. Is it fine for you?

[fstagni]

Sure, no rush.

[arrabito]

@chaen @fstagni I'm back to this.
Before I create a PR, could you please tell me if the parameter:
CRT=True/False
determines if we use auth by certificate or not?

If yes, maybe we could get this paramater first (before getting User and Password) and only if it's False, we retrieve User and Password.

What do you think?

[fstagni]

This was introduced in #4924 for CTA, I think.

I am not aware of all the possibilities for connecting to ES, maybe there are cases where both cert and username/password are needed?

[arrabito]

Yes it was introduced for CTA. I'm not aware either about all the possibilities for connecting to ElasticSearch but at least in our case, we just need certificates.
My proposal is to determine which kind of connection is used at the beginning.
Any other suggestion?

[fstagni]

What you are proposing seems OK. Go ahead.

[arrabito]

I've just created PR #6871 but 2 checks failed. I'm not sure how to pass them.