-
Notifications
You must be signed in to change notification settings - Fork 176
DIRAC v7r1
[TECH PREVIEW] MySQL version 8 is now in pre-supported mode: the tests that we have done didn't show any issue, but are not conclusive.
[TECH PREVIEW] ThreadPool - replaced by ThreadPoolExecutor(native python). To activate please set the DIRAC_USE_NEWTHREADPOOL=yes environment variable in bashrc.
FileCatalog is now instantiating plugins using ObjectLoader mechanism, which allows to provide plugins in extensions. One side effect is that SecurityPolicy plugins are dropped and the only one existing VOMSPolicy became VOMSSecurityManager. For those installations that used PolicyBasedSecurityManager, the configuration should be changed in order to use VOMSSecurityManager instead. The option SecurityPolicy can then be dropped
In order to acomodate for CTA storage, the Files table of the FTS3DB needs an update
alter table Files CHANGE status status enum('New','Submitted','Ready','Active','Finished','Canceled','Staging','Failed','Defunct','Started', 'Not_used') DEFAULT 'New';
For more details, see https://github.com/DIRACGrid/DIRAC/pull/4891
The DIRAC ProxyManager services keeps uploaded long-living proxies in order to provide them for asynchronous operations performed on the user's behalf. Proxies with different user's groups were uploaded independently. Starting from this release only one proxy is uploaded per user without an embedded DIRAC group extension. That is, to load a proxy in ProxyManager, the client prepares a proxy without a group. When a user proxy with a particular group will be requested by some DIRAC service, the proxy will be dressed with the DIRAC group extension on the fly and then delegated to the target component.
The already uploaded proxies with the DIRAC group extension embedded are still valid and will be used if no more recent proxy is available. If a user deletes the proxy from the ProxyManager, then proxies for all the user groups will be no more available, since the proxy is stored in one instance (excluding the VOMS proxy, everything is the same there) and is associated with all groups at once. This differs from the current behavior where the user can delete only the proxy with a particular user group and keep other proxies uploaded.
To protect against the possibility of downloading a proxy certificate with a specific group, the group option DownloadableProxy is implemented. If no value is set, then a proxy with this group is allowed to load by default.
PR https://github.com/DIRACGrid/DIRAC/pull/4491 makes order in the CS helpers for the /Resources CS section. The helpers DIRAC.Core.Utilities.SiteCEMapping and DIRAC.Core.Utilities.SitesDIRACGOCDBMapping are now deprecated in favor of DIRAC.ConfigurationSystem.Client.Helpers.Resources. While backward compatibility will be maintained, you are invited to adjust your DIRAC extension accordingly.
The home made wrapper of openssl (pyGSI) is deprecated in favour of the standard M2Crypto library.
It was first made available in v7r0, is the default as of v7r1 and will be removed in v7r2.
M2Crypto (or any standard tool that respects TLS..) will be stricter than PyGSI. So you may need to adapt your environment a bit. Here are a few hints:
- SAN in your certificates: if you are contacting a machine using its aliases, make sure that all the aliases are in the SubjectAlternativeName (SAN) field of the certificates
- FQDN in the configuration: SAN normally contains only FQDN, so make sure you use the FQDN in the CS as well (e.g.
mymachine.cern.chand notmymachine) - ComponentInstaller screwed: like any change you do on your hosts, the ComponentInstaller will duplicate the entry. So if you change the CS to put FQDN, the machine will appear twice.
If needed, it can be temporarily disabled by setting the environment variable DIRAC_USE_M2CRYPTO to No.
If already installed the following changes should be applied to the ProductionDB:
alter table ProductionSteps modify InputQuery longblob;
alter table ProductionSteps modify OutputQuery longblob; In this patch the RSS databases are updated in order to add VO specific behavior. It requires modification of the databases' schema. This can be achieved with the following SQL commands:::
USE ResourceManagementDB;
ALTER TABLE PilotCache ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER CE;
ALTER TABLE PilotCache DROP PRIMARY KEY, ADD PRIMARY KEY (`Site`,`CE`,`VO`);
ALTER TABLE PolicyResult ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER Name;
ALTER TABLE PolicyResult DROP PRIMARY KEY, ADD PRIMARY KEY (`Element`,`Name`,`StatusType`,`PolicyName`,`VO`);
USE ResourceStatusDB;
ALTER TABLE ResourceStatus ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE ResourceStatus DROP PRIMARY KEY, ADD PRIMARY KEY (`Name`,`StatusType`,`VO`);
ALTER TABLE ResourceLog ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE ResourceHistory ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE NodeStatus ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE NodeStatus DROP PRIMARY KEY, ADD PRIMARY KEY (`Name`,`StatusType`,`VO`);
ALTER TABLE NodeLog ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE NodeHistory ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE SiteStatus ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE SiteStatus DROP PRIMARY KEY, ADD PRIMARY KEY (`Name`,`StatusType`,`VO`);
ALTER TABLE SiteLog ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;
ALTER TABLE SiteHistory ADD VO VARCHAR(64) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'all' AFTER StatusType;Previously, any files ending with .cfg was interpreted as a DIRAC configuration file by DIRAC scripts. Now, this requires the explicit --cfg option.
In order to update all the run files for services and agents, you can run the following
find -L /opt/dirac/startup/ -name run -exec sed -i.bak -E '/--cfg/! s| ((/[a-zA-Z_0-9]+)+\.cfg)| --cfg \1|g' {} \;
The VOMS2CSAgent can now use plugins to validate and append to the CS when adding new users. Currently the only available plugin is one for validating users against CERN's LDAP servers. This can be enabled by adding SyncPluginName = CERNLDAP to the VOMS2CSAgent plugin and existing users can have this information added by running this script.
The stored procedure ps_get_all_info_of_replicas_bulk needs to be created. See https://github.com/DIRACGrid/DIRAC/pull/5144/files