forked from DIRACGrid/DIRAC
-
Notifications
You must be signed in to change notification settings - Fork 0
/
release.notes
11224 lines (8008 loc) · 385 KB
/
release.notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
[v8.1.0a1]
*WorkloadManagementSystem
FIX: (#6388) replace opPath by opChain in JobPath
*DataManagementSystem
FIX: (#6385) Monitoring does not update the dict using by the accounting, avoiding a crash
NEW: (#6385) FailedDataOperation monitoring now keeps track of interactive/job failures
NEW: (#6383) cancel the FTS transfers when an RMS request is found to be canceled
*Core
FIX: (#6379) Explicitly depend on db12
CHANGE: (#6337) Use Python's default SSL ciphers by default
*Resources
CHANGE: (#6378) negative free space value is transformed to 0
[v8.0.0]
*Resources
FIX: (#6359) new directory creation specifies the permission in octal mode
NEW: (#6357) Add a setGfalSetting decorator
CHANGE: (#6357) add a 30s timeout for removal at RAL
*WorkloadManagementSystem
NEW: (#6355) The default InputDataDirectory used by DownloadInputData can be overridden by setting /LocalSite/InputDataDirectory
FIX: (#6354) Stop instance from VirtualMachineMonitor
[v8.0.0a29]
*Core
NEW: (#6347) Introduce ``DIRAC_MYSQL_OPTIMIZER_TRACES_PATH`` environment variable for advance optimization of MySQL calls
CHANGE: (#6347) calls to the MySQL class with keyword parameters MUST now be named
FIX: (#6345) Setting codeProperties["version"] in AgentModule
[v8.0.0a28]
*Core
FIX: (#6341) createClient now creates subclasses of the original class
NEW: (#6335) DIRAC.initialize now provides keyword arguments to customise initialisation
NEW: (#6335) SecurityProperty enum for expressing security properties which can be assigned to groups/hosts
NEW: (#6335) LogLevel enum for choosing the logging level
NEW: (#6309) Introduce DReturnType/DOKReturnType/DErrorReturnType for DIRAC return structures
CHANGE: (#6299) Set OpenSSL ciphers to "DEFAULT:@SECLEVEL=1"
NEW: (#6299) Experimental support for Python 3.10
NEW: (#6299) Suppress InsecureRequestWarning when using dirac-configure with HTTPS servers
NEW: (#6299) Support overriding ciphers and methods with environment variables when using HTTPS
CHANGE: (#6298) Replace suds-jurko with suds
CHANGE: (#6297) Use psutil.net_if_addrs for Network.discoverInterfaces
NEW: (#6291) adding general flag for disabling Monitoring for Agents (used by JobAgent)
CHANGE: (#6286) Default proxy length is 2048 bits
CHANGE: (#6286) remove RFC argument for proxy generation
*WorkloadManagement
FIX: (#6338) Log error if CE Platform is missing.
CHANGE: (#6336) Remove CPU(MHz) from CacheSize(kB) job parameters
*Resources
FIX: (#6320) Forward jobDesc to innerCE in PoolCE
FIX: (#6318) Set HOME in SingularityCE
FIX: (#6317) Return proper error if provider value missing in CloudCE
CHANGE: (#6283) GFA2_SRM2 does not use anymore StorageElements/DefaultProtocols option in the CS (relies on specific OutputProtocols config)
CHANGE: (#6283) explicit distinction between PluginName and ProtocolSection name in all the StorageElement components
CHANGE: (#6283) API change for the StorageElement. See https://github.com/DIRACGrid/DIRAC/pull/6283 for details
CHANGE: (#6283) StorageElement favors local plugins
*WorkloadManagementSystem
FIX: (#6288) ElasticJobParametersDB: do not return parameters that are not needed
*tests
FIX: (#6287) replication transformation test: avoid target==source
FEAT: (#6287) transformation tests: allow -disk SEs, avoid RAL, RALPP, CESNET (don't work for me, YMMV)
*MonitoringSystem
FIX: (#6286) ElasticSearchDB generates full index names based on UTC
FIX: (#6286) DataOperationSender works on a copy of the documents
CHANGE: (#6286) DataOperationSender fails if the master accounting system fails
[v8.0.0a27]
*ConfigurationSystem
CHANGE: (#6285) Make GOCDB2CSAgent functionMap less private
*Core
CHANGE: (#6281) Removed Core.Utilities.MemStat module (use psutil module instead)
CHANGE: (#6281) Removed possibility to Register a DISET service with a secondary name
*Resources
FIX: (#6275) Conditional FC ProxyPlugin checks the ThreadConfig
*DataManagementSystem
FIX: (#6268) Fix FC change{group,owner,mode} for multiple LFNs
*WorkloadManagement
NEW: (#6267) ContainerBind option can be set on the per-site and per-CE level
*WorkloadManagementSystem
FIX: (#6256) Making getJobParameters handle specific parameters
CHANGE: (#6178) removing filtering user sites by platform during the CHECKING state
*FrameworkSystem
NEW: (#6248) Add StdoutJsonBackend
FIX: (#6246) dirac_login HTML error hiding the actual error
[v7.3.25]
*Resources
FIX: (#6275) Conditional FC ProxyPlugin checks the ThreadConfig
*DataManagementSystem
FIX: (#6268) Fix FC change{group,owner,mode} for multiple LFNs
*WorkloadManagement
NEW: (#6267) ContainerBind option can be set on the per-site and per-CE level
*FrameworkSystem
NEW: (#6248) Add StdoutJsonBackend
FIX: (#6246) dirac_login HTML error hiding the actual error
[v8.0.0a26]
*Resources
FIX: (#6242) remove REST information from the ARC6 pilot job references to bind them with jobs
FIX: (#6222) dirac_resource_info - restored the SE lookup logic
FIX: (#6218) ARCCE.getARCJob() becomes protected
FIX: (#6215) ARC6 typo issues
FIX: (#6215) ARC writeXRSL becomes protected
*WorkloadManagement
FIX: (#6241) Fix Platform key error in SD when CheckPlatform=True
*DataManagementSystem
FIX: (#6240) DatasetManager: bug fix in getDatasetsInDirectory() method
FIX: (#6209) Built-in metakeys were incorrectly modified in a MultiVO environment. This fix keeps them intact while modifying user-supplied metadata names by append a VO suffix, as before.
*tests
NEW: (#6239) added tests for the dirac-resource-info command
*Interfaces
CHANGE: (#6228) Make tmpdir for local cache instead of using CWD directly when replicating
*Core
FIX: (#6227) Increase VOMS command timeout
FIX: (#6204) Fixing errors, adding tests for TimeUtilities module
FIX: (#6128) Discovery of HTTPS methods from parent classes
*MonitoringSystem
FIX: (#6214) Fixing errors blocking commission of data operations to Elasticsearch
*ResourceStatusSystem
FIX: (#6201) convert ComputingElement to CE before searching in the configuration
*WorkloadManagementSystem
FIX: (#6196) Fixing errors, adding mapping for ESJobParameters
CHANGE: (#6193) new method to submit pilots from ARC
CHANGE: (#6161) removing the possibility to get the optimizers to run with a vo plugin
[v8.0.0a25]
*CI
NEW: (#6189) max open files for dirac CI container to 8192
*Core
CHANGE: (#6188) Propagate original proxy strength when generating new proxies
*WorkloadManagementSystem
CHANGE: (#6154) Addded timestamp to elasticjobparameters and changed parameter values into strings
[v8.0.0a24]
*Core
FIX: (#6179) better report of error when importing Module
FIX: (#6167) Replaced old vomses/vomsdir code to be able to handle cvmfs vomses. See https://github.com/DIRACGrid/DIRAC/issues/6150
FIX: (#6151) Replace deprecated threading.activeCount with active_count
FIX: (#6149) use AnyThreadEventLoopPolicy asyncio policy
FIX: (#6144) Removing time dependence of TimeUtilities Module
NEW: (#6137) add https to issuer address automatically
FIX: (#6132) fix error reporting in TornadoService
CHANGE: (#6091) changed name and simplified repetitive methods of Utilities.TIme
CHANGE: (#6084) BaseRequestHandler encode/decode static methods
*MonitoringSystem
FIX: (#6175) MonitoringReporter - late binding to MonitoringDB
FIX: (#6076) Removing multiplication in timestamp when committing to monitoring
*WorkloadManagementSystem
FIX: (#6173) restore /LocalSite/InstancePath for the case of JobAgents (see issue #6143)
FIX: (#6162) use the JobMonitoringClient to check for job parameters (that can be in the ES backend)
NEW: (#6134) adding a test to show the modifications made to the JDL before being inserted in the JobDB
*RequestManagementSystem
NEW: (#6172) Add option to automatically cancel Scheduled requests after a defined time period. Default behaviour unchanged. Relates to #6136
CHANGE: (#6114) Replace BLOBs with TEXTs for ReqDB
*docs
NEW: (#6147) add info on installing non-released code from within the pilot
*Resources/Cloud
FIX: (#6146) userData need to be in "ascii" string format to meet the requirement of "create_node" when creating VMs from cloud resources
*FrameworkSystem
FIX: (#6138) dirac_login: avoid misleading error message, fixes issue #6096
FIX: (#6135) dirac_login: Pass proxy as string to writeToProxyFile()
NEW: (#6130) add the possibility to add a prefix before the actual log message for a given subLogger
[v8.0.0a23]
*Tests
FIX: (#6122) restart Tornado after updating the configuration
*Core
FIX: (#6120) don't truncate first character in cli doc strings
FIX: (#6103) Race condition which causes multiple connectioons to be opened when the connection to an executor drops
CHANGE: (#6103) Improve logging split between message/varmessage in execuctors
FIX: (#6083) Grid: BDII make ldapsearch call forward compatible with newer ldapsearch versions
*DataManagementSystem
NEW: (#6115) Introduces an option 'ForceIndexedMetadata' to disable the creation of unindexed metadata. Default: False, i.e. preserves the current default where both indexed and unindexed metadata are allowed. Relates to #6029.
*Resources
FIX: (#6111) Error in the condition to get FlavorName
FIX: (#6110) use identity instead of subject of proxy in CloudCE
FIX: (#6102) remove decode from proxy in CloudCE
FIX: (#6095) use ceType instead of ceName for CE parameters
FIX: (#6088) autodetect cvmfs proxy in default cloudce template
FIX: (#6083) ARCCE make ldapsearch call forward compatible with newer ldapsearch versions
FIX: (#6080) Adapt StompMQConnector to Stomp 8
*tests
FIX: (#6104) typo in testJobDefinitions for new Cloud test
NEW: (#6081) added hackathon test for CloudComputingElement
*WorkloadManagement
FIX: (#6103) Avoid producing massive log output in OptimizationMind
*Elasticsearch
NEW: (#6094) option op_type in index API for Elasticsearch
[v8.0.0a22]
*Web
FIX: (#6046) Fix downloading sandboxes from the Job Manager in the WebApp
*Resources
CHANGE: (#5996) release stomp version
NEW: (#5996) Simple stomp interface
[v8.0.0a21]
*WorkloadManagement
FIX: (#6066) In case of a pilot version not matching the production version, the JobAgent expects the error message to contain the words "Pilot version does not match", so the error message was updated accordingly.
*WorkloadManagementSystem
FIX: (#6065) Don't crash if systemCall fails
*Resources
NEW: (#6061) add SRM+HTTPs as a default TPC protocol in SRM
FIX: (#6060) use ceType instead of ceName for CE parameters
FIX: (#6052) string format for future in PoolCE
*Core
NEW: (#6017) TornadoBaseRequestHandler define a class logger with an extra attribute "tornadoComponent"
NEW: (#6017) DIRACDB can now use a parentLogger instead of direct gLogger
*All
CHANGE: (#6017) services propagate their local loggers to the DB
CHANGE: (#6017) tornado services use local logger
[v8.0.0a20]
*docs
FIX: (#6041) Added (renewed) jobs and pilot State Machines drawings
*DataManagementSystem
NEW: (#6040) bitterly implement multihop
NEW: (#6015) BringOnline setting for FTS transfers can be SE dependant
*Resources
FIX: (#6039) HTCondorCE: make sure proxy is available when running condor_rm
NEW: (#6038) add CloudComputingElement
*WorkloadManagementSystem
CHANGE: (#6038) add hook for pilot cleanup
*RequestManagementSystem
FIX: (#6033) if the request ID is an integer the test `is digit()` throws an exception
*Core
FIX: (#6031) Allow rootPath to be overridden with DIRAC_ROOT_PATH environment variable
*FrameworkSystem
FIX: (#6007) ComponentSupervisionAgent: skip Tornado service
*DMS/WMS/Monitoring
FIX: (#6006) Fixed errors in Data Operation Monitoring and PilotsHistory from hackaton
*MonitoringSystem
CHANGE: (#6002) Replaced the flags for monitoring with a new unified flag system
*Monitoring/Core
CHANGE: (#5944) Divided ComponentMonitoring into AgentMonitoring and ServiceMonitoring
[v8.0.0a19]
*WorkloadManagementSystem
FIX: (#6004) StalledJobAgent can force status from Submitting to Failed
FIX: (#6004) ElasticJobParametersDB: make sure the docID is less than 512 characters
*WMS/Monitoring
NEW: (#5999) Added a PilotsHistory Monitoring
*Subsystem
FIX: (#5994) allow to run Watchdog on non x86 platforms
*WorkloadManagement
CHANGE: (#5993) dirac-wms-get-cpu-normalization depends on db12 multiple_dirac_benchmark
*Core
FIX: (#5985) Set default value of BaseRequestHandler.COMPONENT_NAME
*DataManagementSystem
FIX: (#5984) fix a race condition when running multiple FTS3Agents in parallel
*RequestManagementSystem
FIX: (#5960) moved REA initialization inside the initialize() method
*MonitoringSystem
FIX: (#5955) Fixed data operation and pilot submission monitoring from certif Hackaton
[v8.0.0.a18]
*Core
FIX: (#5983) fix: allow for agents to loop every 10 seconds
FIX: (#5980) Limit reoccurrences of subprocess unicode error
*WorkloadManagementSystem
FIX: (#5983) can't change the polling time of JobAgent from inside an already running agent
FIX: (#5982) Stop fetching jobs when no more slot available in the inner PoolCE
FIX: (#5982) Make sure dirac-jobexec does not use the server certificate to execute the workflow
FIX: (#5973) update PoolCE.InnerCESubmissionType from JobAgent
FIX: (#5950) escape time stamp in setHeartBeatData
*MonitoringSystem
NEW: (#5974) TornadoMonitoring handler
*docs
FIX: (#5969) added better docs for JobShare and priorities
*FrameworkSystem
CHANGE: (#5965) removed gMonitor
FIX: (#5941) dirac-login uses the correct proxy to add VOMS attributes
*RequestManagementSystem
FIX: (#5958) dirac-rms-request takes into account PEP-515
FIX: (#5951) fix variable referenced before assignment in RequestTask
FIX: (#5951) take into account that targetSEList can be empty in the operations
*DataManagementSystem
CHANGE: (#5954) DFC dump feature can dump more than one SE at the same time
*Accounting/Monitoring
FIX: (#5945) Fixed the plotter for pilot submission accounting and other small fixes
[v8.0.0a17]
*WorkloadManagementSystem
FIX: (#5937) using a DErrno for no match found
CHANGE: (#5933) removed last SubmitPool entries
FIX: (#5907) Matcher: fixed parsing of (required) tags
*Resources
FIX: (#5935) File protocol listDirectory returns metadata
CHANGE: (#5919) GFAL2_StorageBase redefine ECOMM if it does not exist
*RequestManagementSystem
FIX: (#5932) filter by SourceSE in ReplicateAndRegister preparation
FIX: (#5932) Change RMS.Operation sourceSEList and targtSEList default to empty list
FIX: (#5926) removing python2 support
*FrameworkSystem
CHANGE: (#5931) DBs stop using BLOB types in favor of TEXT types
NEW: (#5901) move ProxyManager to HTTPs
*DataManagementSystem
NEW: (#5927) TornadoDataIntegrityHandler
*Tornado
NEW: (#5925) added TornadoWMSAdministratorHandler
[v8.0.0a16]
*tests
CHANGE: (#5900) use dirac-login instead of dirac-proxy-init for tests
*FrameworkSystem
NEW: (#5899) dirac-login (AS) added VOMS extension
NEW: (#5896) dirac-login try to connect to DIRAC AS if no local certificate
*Core
FIX: (#5898) print error details/callstack in BaseRequestHandler
[v8.0.0a15]
*docs
CHANGE: (#5895) moved docs of ComputingElement from ConfReference to dirac.cfg
CHANGE: (#5875) add more details about the PushJobAgent configuration
FIX: (#5868) update DFC components description
NEW: (#5867) Added general service configuration options description including Authorization options
*WorkloadManagementSystem
NEW: (#5894) Added Account option for SLURM
CHANGE: (#5876) HeartBeatTime is set every time a status is changed by the job itself
CHANGE: (#5840) Replace BLOBs with TEXTs for JobDB and PilotAgentsDB
*Core
CHANGE: (#5883) Show callstack in service logs when returning S_ERROR
CHANGE: (#5872) Minimum Python version is now 3.9
FIX: (#5870) allow to use `deprecated` decorator with classes without predefined `__init__` function.
*WorkloadManagement
FIX: (#5875) add proxy in PushJobAgent and enhance it
*Accounting
CHANGE: (#5873) decrease some logs verbosity
*All
CHANGE: (#5870) use DIRAC.Core.Base.Script.Script instead of DIRAC.Core.Utilities.DIRACScript.DIRACScript
CHANGE: (#5870) add deprecation message
*FrameworkSystem
CHANGE: (#5865) remove unused 'rfc' argument
NEW: (#5862) add password mask to dirac-login
*ResourceStatusSystem
FIX: (#5864) return S_OK/S_ERROR in PublisherHandler.getSite
*MonitoringSystem
FIX: (#5860) Changed name of methods in PilotSubmissionMonitoringPlotter
NEW: (#5807) Move DataOperation to Monitoring
[v8.0.0a14]
*DataManagementSystem
FIX: (#5858) FileCatalogCLI - removed faulty evaluation of undefined option to the rebuild command. Fixes #5759
*Core
FIX: (#5857) TorandoREST ignore arguments that not defined in the target method
FIX: (#5857) dirac-info show "None found" instead of exception
CHANGE: (#5854) Treat X509 data as str instead of bytes
*FrameworkSystem
FIX: (#5857) use payload keyword argument for handle_response method
CHANGE: (#5857) use DIRAC CAs location to request DIRAC AS
FIX: (#5857) fix message in dirac-login after authorization
FIX: (#5854) UserProfileManager usage with JEncode
*Resources
CHANGE: (#5857) verify CAs when request IdP by default
FIX: (#5856) Changed the nomenclature for PilotSubmissionMonitoring
*WorkloadManagement
FIX: (#5854) JobDB usage with JEncode
*Production
FIX: (#5854) ProductionManager use with JEncode
*WorkloadManagementSystem
FIX: (#5848) WMS DBs: do not try to decode when it's not bytes
CHANGE: (#5846) added Modules and PipInstallOptions to SiteDirector/Pilot option
*ConfigurationSystem
FIX: (#5844) VOMS2CSAgent correctly takes into account multiple CAs for users with multiple DNs
*MonitoringSystem
FIX: (#5838) Fixing nomenclature and other issues with PilotMonitoringSubmission
*Tornado
FIX: (#5837) better logging formatting
*TransformationSystem
NEW: (#5829) Added TornadoTransformationManager service
[v8.0.0a13]
*FrameworkSystem
CHANGE: (#5836) added a flag for disabling the use of SecurityLogging service
*Core
CHANGE: (#5833) JEncode bytes, tests
FIX: (#5810) do not allow arbitrary code execution in JEncode
*ConfigurationSystem
CHANGE: (#5833) align with JEncode changes
*ResourceStatusSystem
NEW: (#5830) Added tornado version for the three RSS services
*TransformationSystem
CHANGE: (#5828) TransformationDB: removing BLOB in favor of TEXT for non-binary data
FIX: (#5822) TransformationDB: removed old compatibility layer with mySAM tables
FIX: (#5796) Convert fileID / taskID to fix issue when task status is updated in transformation treatment
*docs
FIX: (#5825) fix indentation in the doc related to the server installation
CHANGE: (#5823) add further details about the server installation process with python3
*WorkloadManagement
CHANGE: (#5824) enable logs from db12 in dirac-wms-cpu-normalization
*Interface
FIX: (#5818) send only single jobID to checkJobStateTransition
Thank you for writing the text to appear in the release notes. It will show up
exactly as it appears between the two bold lines
Please follow the template:
Thank you for writing the text to appear in the release notes. It will show up
exactly as it appears between the two bold lines
Please follow the template:
*WorkloadManagementSystem
FIX: (#5816) Allow transition Rescheduled->Failed
*Subsystem
NEW/CHANGE/FIX: (#5804) explanation
For examples look into release.notes
*MonitoringSystem
CHANGE: (#5788) moving pilot submission to monitoring
*Docs
FIX: (#5711) updated/simplified the documentation for adding new components
[v8.0.0a12]
*WorkloadManagementSystem
FIX: (#5793) better use the job state machine when setting the job status, in particular when it comes from failover requests that may be inserted between 2 successful updates.
FIX: (#5751) allow Stalled, Running and Matched jobs to go to Reschedule
FIX: (#5751) JobAgent forces the reschedule of jobs in case of exceptions
FIX: (#5746) convert inputs from tuple to list in JobAgent before submitting to a CE
NEW: (#5714) added WMSDashboard and diracLogsDashboard in NDJSON format.
FIX: (#5714) removed old index patterns in WMSDashboard.json, updated documentation.
*DataManagementSystem
FIX: (#5791) Fix pickle error in dirac-dms-directory-sync
*WorkloadManagement
FIX: (#5789) JobSanity doesn't handle input sanboxes named "LFN:"
*TransformationSystem
Fix: Restore task state "Scheduled"
*Resources
CHANGE: (#5774) SingularityComputingElement: install python3 version
NEW: (#5766) allow to overwrite input/output storage plugin parameter with the CS
CHANGE: (#5766) add "xroot" to the list of input protocol for the XROOT plugin
*Interfaces
CHANGE: (#5772) DIRAC APIs check if a JobState transition is allowed
*Subsystem
FIX: (#5768) dirac_production_runjoblocal.py: moved to py3 Pilot install
*All
CHANGE: (#5764) use Client instead of RPCClient
*HTTPS
FIX: (#5762) Replace DIRAC.Core.DISET.TransferClient with DIRAC.Core.Tornado.Client.ClientSelector
*FrameworkSystem
FIX: (#5757) rename downloadablePersonalProxy value to allowProxyDownload
FIX: (#5748) Get Tornado's port from the CS when adding URLs
*docs
CHANGE: (#5756) add document image source files
*Core
FIX: (#5743) removed isPy3VersionNumber function
[v8.0.0a11]
*WorkloadManagement
NEW: (#5736) PushJobAgent to prepare jobs for sites with no external connectivity
NEW: (#5736) RemoteRunner to execute applications remotely (submitting them to remote CE)
*Resources
CHANGE: (#5736) add inputs and outputs parameters in ARC.submitJob()
CHANGE: (#5702) ParallelLibrary: Embed the executable in the srun wrapper
*Workflow
CHANGE: (#5736) integrate RemoteRunner
*WorkloadManagementSystem
FIX: (#5735) securityLogging: zip after 1 day
CHANGE: (#5728) Replace DB12.py with the db12 package
FIX: (#5725) ElasticJobParametersDB: get in output all entries
CHANGE: (#5724) JobAgent and JobWrapper: only reporting changes on major status when needed
*ResourceStatusSystem
FIX: (#5734) CSHelpers: some sites have no resources
*HTTPS
FIX: (#5730) Add URLs to CS when installing HTTPS services
*Core
FIX: (#5726) TornadoBaseClientcorrectly checks CA
FIX: (#5717) encodeDict when the dict keys have mixed types
CHANGE: (#5698) Replacing Elasticsearch python client with OpenSearch.
FIX: (#5694) do no look anymore for __RCSID__
FIX: (#5683) ThreadSafe.Synchronizer: Add target method docstring to preserve docstring of decorated function
*ConfigurationSystem
FIX: (#5701) fix an exception in the VOMS2CSAgent when a user joins a second VO
FIX: (#5684) fix CSGlobals.Extensions class
*TransformationSystem
FIX: (#5690) fix more flaky tests for RequestTasks
[v8.0.0a10]
*ConfigurationSystem
CHANGE: (#5682) Bdii2CSAgent: add InjectSingleCoreQueue option to automatically create single core equivalents for MutliCore Queues , kind of fixes #5582
CHANGE: (#5682) Bdii2CSAgent: CEs in the BannedCEs list are no longer updated by the agent, previously this list only concerned CEs that are not already in the Configuration, fixes #5224
CHANGE: (#5682) dirac-admin-add-resources: add --onecore option to automatically create single core equivalents for MutliCore Queues
FIX: (#5682) dirac-admin-add-resources: fix the query about adding new CEs, this can now be answered in the negative
*Resources
FIX: (#5674) decode the http response in WLCGAccountingHTTPJson
CHANGE: (#5643) HTCondorCE submits jobs with -spool option when a local schedd is used to spool the pilot wrappers that can be deleted afterwards
*Core
FIX: (#5671) DIRAC.isPy3VersionNumber always returns True
FIX: (#5656) Tasks never get removed from the ExecutorState if an UnrecoverableTaskException is raised
CHANGE: (#5629) use DIRAC_DEBUG_M2CRYPTO to enable SSL debugging in tornado
*RMA
FIX: (#5670) If a job was Killed, set it Killed at request completion
*TransformationSystem
FIX: (#5666) fix RequestTasks flaky tests
NEW: (#5642) Allow MultiOperation body to fetch values from the task dict
NEW: (#5642) Introduce Body Plugins for DMS transformations
FIX: (#5630) TransformationCleaningAgent 'deletes', do not 'remove' jobs
*WorkloadManagement
FIX: (#5665) Python 3 support in SSHComputingElement
FIX: (#5657) Bad error handling in OptimizationMindHandler.exec_taskError
*Docs
CHANGE: (#5663) removed a bunch of py2 related documentation
*MonitoringSystem
FIX: (#5658) Wrong link on the docs page for the WMSHistory dashboard. Fixed the link and edited some of the text.
NEW: (#5634) JSON file for WMS monitoring dashboard
*docs
FIX: (#5654) fix trailing-whitespace
*FrameworkSystem
FIX: (#5652) SystemAdministratorCLI: get DIRAC version from Extensions
CHANGE: (#5641) proxy expiration emails are sent from an address taken from the Service/Agent config instead of the DB
NEW/CHANGE/FIX: (#5627) avoid duplication of code in TornadoBundleDeliveryHandler
FIX: (#5619) continue on empty service list in getComponentsStatus
*DataManagementSystem
FIX: (#5651) dirac-dms-directory-sync: The script will now try all files and not end a thread if one of the files assigned to be uploaded or downloaded to a thread fails. The script will also print out all failed files, and their error at the end of a thread. A few log messages have been corrected as well.
*ResourceStatusSystem
FIX: (#5647) specifying the gocDB type for known types
*WorkloadManagementSystem
CHANGE: (#5643) Add functions in WMSUtilities, move content of getPilotLoggingInfo() and getGridJobOutput() in PilotManagerHandler
*RequestManagementSystem
NEW: (#5615) add SweepSize option to the ReqProxy
[v8.0.0a9]
*TransformationSystem
FIX: (#5612) fix bug in TransformationCleaningAgent, that was NOT removing jobs but only deleting them
*WorkloadManagementSystem
FIX: (#5612) removing jobs instead of deleting them when bulk submission fails
CHANGE: (#5570) better logging output in the JobStateUpdate
*WorkloadManagement
FIX: (#5610) Return error if JobPolicy.getControlledUsers returns an empty list
FIX: (#5603) Don't run JobsStateMachine if newStat is ''
FIX: (#5595) UnicodeDecodeErrors in JobDB.getJobParameters
FIX: (#5585) Error logging when optimisation fails
*FrameworkSystem
FIX: (#5609) Cleaning old installations from a Python 3 based server
FIX: (#5604) Issue installing DIRAC and extensions on servers running Python 3
FIX: (#5583) display correctly the help in sysadmin-cli
CHANGE: (#5571) allow DIRAC AS return proxy by default
*ConfigurationSystem
FIX: (#5608) ServiceInterface.py - use ThreadPoolExecutor instead of ThreadPool to avoid thread leaks in the ConfigurationServer
CHANGE: (#5596) do not add the master CS first in the list of CS when new slaves are added
*Test
FIX: (#5605) fix typo in the HTTPs Resources tests
*Core
FIX: (#5594) Extension ordering in DIRACScript
*Resources
FIX: (#5586) cloudinit-template - allow time in the monitor for the pilot to start
FIX: (#5586) cloudinit-template - set DIRACSYSCONFIG to point to the pilot.cfg in order to allow its use in the user applications
*docs
CHANGE: (#5573) move the ConfReference DIRAC section part to dirac.cfg and add there OAuth 2 settings.
FIX: (#5572) update nginx installation instruction
NEW: (#5540) add documentation with information about new system APIs component
Align `WebApp` installation manual with last changes (using BaseRequestHandler in WebAppDIRAC 5.0).
The request Accounting -> Network -> Packet lost rate (magnified) causes an error:
```
ERROR: (#5569) Error while generating the plots Value keys 100 - IF(SUM(PacketLossRate)/SUM(entriesInBucket)*10 > 100, 100, SUM(PacketLossRate)/SUM(entriesInBucket)*10), 100 are not defined
```
This RP should provide correct pattern.
*Accounting
FIX: (#5569) provide correct pattern for reportMagnifiedPacketLossRate
[v8.0.0a8]
[v8.0.0a7]
*WorkloadManagementSystem
CHANGE: (#5566) JobParameters - generalize getting CS parameters for Cloud VMTypes also
FIX: (#5563) PilotCStoJSONSynchronizer: do not look anymore in releases.cfg
CHANGE: (#5555) SiteDirector submits only py3 pilots
FIX: (#5541) old OptimizerModule can commit to ElasticJobParametersDB
FIX: (#5529) QueueUtilities - do not stop queue instantiation in case of failures
FIX: (#5518) CloudDirector - create only one pilot reference
*Core
FIX: (#5561) Fix overwriting files from an X509Chain object
NEW: (#5543) Utility for parsing DIRAC version number to Py3
CHANGE: (#5537) Do not return failures when sending tasks to executors
FIX: (#5530) wait that the CS is loaded before checking for master CS in tornado-start-all
FIX: (#5516) Fix finding Dwatermark.png for the accounting watermark
*Docs
CHANGE: (#5559) Forces docutils to 0.17 because of bug with sphynx
CHANGE: (#5546) do not calculate FQDN unnecessarily
*FrameworkSystem
NEW: (#5543) dirac-admin-update-instance converts version number to PEP-440 style if needed
CHANGE: (#5537) ComponentInstaller: look also for HTTPs services
FIX: (#5482) Allow Python 3 style pre-releases to be installed if the primary extension is a pre-release
*ConfigurationSystem
FIX: (#5535) VOMS2CSAgent: VOMS users that have multiple DNs and are suspended should no longer lead to "User not registered" errors and have their status correctly set
*Resources
FIX: (#5529) ComputingElement - remove unnecessary warning
CHANGE: (#5528) Use PilotManagerClient to get CE status instead of interrogating PilotAgentsDB in HTCondorCE
FIX: (#5518) OpenStackCE - removed buggy line
NEW: (#5518) Utilities - added possibility to specify extra yum installable packages in the configuration
NEW: (#5518) Utilities - added possibility to specify ssh connection to the VM in the configuration
FIX: (#5518) cloudinit.template - use VMType as the Queue analog
FIX: (#5518) dirac_resource_get_parameters - fixed bugs in the VMType getting logic
*Interfaces
FIX: (#5522) Dirac.py - JobMonitoring.getJobsSummary now returns a structure and not a string
*RequestManagementSystem
CHANGE: (#5521) getRequestFileStatus takes better into account cases of multiple operations on the same file
CHANGE: (#5521) physicalRemoval checker expects an LFN and not a PFN
*WorkloadManagement
FIX: (#5516) Ensure valenc is bytes before decoding in retrieveOptimizerParam
*TransformationSystem
NEW: (#5514) Allow GroupSize parameter to be mutable
NEW: (#5514) Add setGroupSize to command transformation CLI
*SecurityLog
CHANGE: (#5513) fed up with unreadable job logs...
[v8.0.0a1]
*FrameworkSystem
FIX: (#5410) ProxyManagerClient - upload proxy without embedded DIRAC group
*DataManagementSystem
CHANGE: (#5391) The LcgFileCatalogClient has been removed
Discard https://github.com/DIRACGrid/DIRAC/commit/dfbb53232dba40aff4bf0f04802877c51d078ab1 that resolve error described in previous hackathon https://trello.com/c/y7TgEuF7/40-multi-vo-tests. Now after the last lot of changes, this key returns correctly, so I canceled the last adjustment.
*Interfaces
FIX: (#5388) Discard https://github.com/DIRACGrid/DIRAC/commit/dfbb53232dba40aff4bf0f04802877c51d078ab1.
[v8r0-pre4]
NEW: Basic support for the OAuth/OIDC tokens
*DataManagement
CHANGE: (#5391) The LcgFileCatalogClient has been removed
[v7r3p4]
FIX: fixes from v7r2p28
*WorkloadManagement
CHANGE: (#5429) Use python3 in VMDIRAC instances
[v7r3p3]
*Resources
NEW: (#5396) Add Emies endpoint to the ARC CE
*docs
FIX: (#5424) some minor fixes for user documentation