This repository has been archived by the owner on Sep 12, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 51
Python3 Migration Notes
Amanda Goldani edited this page Aug 13, 2021
·
9 revisions
This page is intended to show the differences between the old version of the code and the new one.
Note that all Python3 top levels are located in src/python/, this was omitted in the details for convenience.
Includes all cache functions and classes from utils.py.
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/cacheInfo.from_file |
Cache/CacheManager.py/CacheManager._getFromFile |
✅ | * called in get |
utils.py/cacheInfo._file_key |
Cache/CacheManager.py/CacheManager._getKeyFilePath |
✅ | * called in get |
utils.py/cacheInfo.get |
Cache/CacheManager.py/CacheManager.get |
✅ | |
utils.py/cacheInfo.purge |
Cache/CacheManager.py/CacheManager.purge |
❌ | |
utils.py/cacheInfo.store |
Cache/CacheManager.py/CacheManager.set |
❌ | |
utils.py/docCache.get |
Cache/DataCacheLoader.py/DataCacheLoader.get |
❌ | |
utils.py/docCache.__init__.cache |
Cache/DocumentCache.py |
❌ | * each possible document cache key became a class with the corresponding get method. The DataCacheLoader makes the mapping and loads the document data when required |
| — | Cache/CacheManager.py/CacheManager._buildMongoDocument |
✅ | * called in set |
| — | Cache/CacheManager.py/CacheManager._setMongoCollection |
✅ | * required by the MongoClient |
Includes the helper function from utils.py that creates the connection to MongoDB, as well as the migration of the file that creates the connection to OracleDB.
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/mongo_client |
Databases/Mongo/MongoClient.py |
— | * this is an abstract class where the mongo collection must be defined in the child classes |
Unified/assignSession.py |
Databases/Oracle/OracleClient.py |
✅ | * tested through the OracleDB test file |
Unified/assignSchema.py |
Databases/Oracle/OracleDB.py |
✅ |
Includes all the helper classes from utils.py controlling a MongoDB collection.
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/agentInfoDB |
MongoControllers/AgentController.py/AgentController._setMongoCollection |
✅ | * required by the MongoClient |
utils.py/agentInfo.agentStatus |
MongoControllers/AgentController.py/AgentController.get |
✅ | * this will return all the agent's data, use result.get("status") to keep the same behavior |
utils.py/agentInfo.speed_draining |
MongoControllers/AgentController.py/AgentController.getAgents(speeddrain=True) |
✅ | |
utils.py/agentInfo.poll |
MongoControllers/AgentController.py/AgentController.poll |
❌ | |
utils.py/agentInfo.change_status |
MongoControllers/AgentController.py/AgentController.set |
❌ | |
utils.py/agentInfo.getStatus |
MongoControllers/AgentController.py/AgentController.syncToProduction |
❌ | |
utils.py/agentInfo.checkTrello |
MongoControllers/AgentController.py/AgentController.syncToTrelloMongoControllers/AgentController.py/AgentController.updateTrelloBoard
|
❌ | * split because the Trello board was updated or the collection was sync to the board (never both!) |
| — | MongoControllers/AgentController.py/AgentController._areRunningOverThreshold |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._bootAgent |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/AgentController.py/AgentController._countAgentsWithReleaseStatus |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._countJobsAndCpus |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._drainAgent |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._initializePollParams |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._isLight |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._isStuffed |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._newRelease |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._retireAgent |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setCandidatesForDraining |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setCandidatesForOpenDraining |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setCandidatesForSpeedDraining |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setCandidatesForWakingUp |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setLastestAndOldestReleases |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setRecentAgents |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._setSpeedDrainingPriority |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._sumRunningCapacity |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._syncAgentStatus |
❌ | * called in syncToProduction |
| — | MongoControllers/AgentController.py/AgentController._tell |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController._updateAgentConfig |
❌ | * called in poll |
| — | MongoControllers/AgentController.py/AgentController.getAgents |
✅ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/batchInfo.content |
MongoControllers/BatchController.py/BatchController.get |
✅ | |
utils.py/batchInfo.all |
MongoControllers/BatchController.py/BatchController.getBatches |
✅ | |
utils.py/batchInfo.pop |
MongoControllers/BatchController.py/BatchController.pop |
❌ | |
utils.py/batchInfo.updateutils.py/batchInfo.add
|
MongoControllers/BatchController.py/BatchController.set |
❌ | * unified because add is only called inside update |
| — | MongoControllers/BatchController.py/BatchController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/BatchController.py/BatchController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/campaignInfo.content |
MongoControllers/CampaignController.py/CampaignController.get |
✅ | |
utils.py/campaignInfo.all |
MongoControllers/CampaignController.py/CampaignController.getCampaigns |
✅ | |
utils.py/campaignInfo.parameters |
MongoControllers/CampaignController.py/CampaignController.getCampaignParameters |
✅ | |
utils.py/campaignInfo.allSecondaries |
MongoControllers/CampaignController.py/CampaignController.getSecondaries |
✅ | |
utils.py/campaignInfo.get |
MongoControllers/CampaignController.py/CampaignController.getCampaignValue |
✅ | |
utils.py/campaignInfo.go |
MongoControllers/CampaignController.py/CampaignController.go |
✅ | |
utils.py/campaignInfo.pop |
MongoControllers/CampaignController.py/CampaignController.pop |
❌ | |
utils.py/campaignInfo.updateutils.py/campaignInfo.add
|
MongoControllers/CampaignController.py/CampaignController.set |
❌ | * unified because add is only called inside update |
| — | MongoControllers/CampaignController.py/CampaignController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/CampaignController.py/CampaignController._setCampaigns |
✅ | * called when instantiated |
| — | MongoControllers/CampaignController.py/CampaignController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/closeoutInfo.one_line |
MongoControllers/CloseoutController.py/CloseoutController._buildHtmlRow |
❌ | * called in assistance and summary |
utils.py/closeoutInfo.assistance |
MongoControllers/CloseoutController.py/CloseoutController.assistance |
❌ | |
utils.py/closeoutInfo.pop |
MongoControllers/CloseoutController.py/CloseoutController.clean |
❌ | |
utils.py/closeoutInfo.get |
MongoControllers/CloseoutController.py/CloseoutController.get |
✅ | |
utils.py/closeoutInfo.html |
MongoControllers/CloseoutController.py/CloseoutController.html |
❌ | |
utils.py/closeoutInfo.update |
MongoControllers/CloseoutController.py/CloseoutController.set |
❌ | |
utils.py/closeoutInfo.summary |
MongoControllers/CloseoutController.py/CloseoutController.summary |
❌ | |
utils.py/closeoutInfo.table_header |
Included in the CloseoutController templates | — | |
| — | MongoControllers/CloseoutController.py/CloseoutController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/CloseoutController.py/CloseoutController._renderAssistanceHtml |
❌ | * called in assistance |
| — | MongoControllers/CloseoutController.py/CloseoutController._renderSummaryHtml |
❌ | * called in summary |
| — | MongoControllers/CloseoutController.py/CloseoutController._setMongoCollection |
✅ | * required by the MongoClient |
| — | MongoControllers/CloseoutController.py/CloseoutController.getWorkflows |
✅ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/moduleLock.check |
MongoControllers/ModuleLockController.py/ModuleLockController.check |
❌ | |
utils.py/moduleLock.clean |
MongoControllers/ModuleLockController.py/ModuleLockController.clean |
❌ | |
utils.py/moduleLock.all_locks |
MongoControllers/ModuleLockController.py/ModuleLockController.get |
✅ | |
| — | MongoControllers/ModuleLockController.py/ModuleLockController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/ModuleLockController.py/ModuleLockController._setMongoCollection |
✅ | * required by the MongoClient |
| — | MongoControllers/ModuleLockController.py/ModuleLockController.go |
✅ | |
| — | MongoControllers/ModuleLockController.py/ModuleLockController.pop |
❌ | |
| — | MongoControllers/ModuleLockController.py/ModuleLockController.set |
❌ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/remainingDatasetInfo.clean |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.clean |
❌ | |
utils.py/remainingDatasetInfo.get |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.get |
✅ | |
utils.py/remainingDatasetInfo.sites |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.getSites |
✅ | |
utils.py/remainingDatasetInfo.purge |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.purge |
❌ | |
utils.py/remainingDatasetInfo.set |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.set |
❌ | |
utils.py/remainingDatasetInfo.sync |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.syncMongoControllers/RemainingDatasetController.py/RemainingDatasetController.syncSite
|
❌ | * split to let explicit if one site or all sites are being sync |
utils.py/remainingDatasetInfo.tell |
MongoControllers/RemainingDatasetController.py/RemainingDatasetController.tell |
✅ | |
| — | MongoControllers/RemainingDatasetController.py/RemainingDatasetController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/RemainingDatasetController.py/RemainingDatasetController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/reportInfo._convert |
MongoControllers/ReportController.py/ReportController._convertValues |
✅ | * called in set |
utils.py/reportInfo.get |
MongoControllers/ReportController .py/ReportController.get |
✅ | |
utils.py/reportInfo.purge |
MongoControllers/ReportController .py/ReportController.purgeMongoControllers/ReportController .py/ReportController.cleanMongoControllers/ReportController .py/ReportController.cleanWorkflow
|
❌ | * split because the old function would do different things depending os the input params |
utils.py/reportInfo._put |
MongoControllers/ReportController.py/ReportController.set |
❌ | |
utils.py/reportInfo.set_blocks |
MongoControllers/ReportController.py/ReportController.setWorkflowBlocks |
❌ | |
utils.py/reportInfo.set_errors |
MongoControllers/ReportController.py/ReportController.setWorkflowErrors |
❌ | |
utils.py/reportInfo.set_files |
MongoControllers/ReportController.py/ReportController.setWorkflowFiles |
❌ | |
utils.py/reportInfo.set_IO |
MongoControllers/ReportController.py/ReportController.setWorkflowIO |
❌ | |
utils.py/reportInfo.set_logs |
MongoControllers/ReportController.py/ReportController.setWorkflowLogs |
❌ | |
utils.py/reportInfo.set_missing |
MongoControllers/ReportController.py/ReportController.setWorkflowMissing |
❌ | |
utils.py/reportInfo._set_for_task |
MongoControllers/ReportController.py/ReportController.setWorkflowTask |
❌ | |
utils.py/reportInfo.set_ufiles |
MongoControllers/ReportController.py/ReportController.setWorkflowUFiles |
❌ | |
| — | MongoControllers/ReportController.py/ReportController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/ReportController.py/ReportController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/StartStopInfo.get |
MongoControllers/StartStopTimeController.py/StartStopTimeController.get |
✅ | |
utils.py/StartStopInfo.purge |
MongoControllers/StartStopTimeController.py/StartStopTimeController.purge |
❌ | |
utils.py/StartStopInfo.pushStartStopTime |
MongoControllers/StartStopTimeController.py/StartStopTimeController.set |
❌ | |
| — | MongoControllers/StartStopTimeController.py/StartStopTimeController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/StartStopTimeController.py/StartStopTimeController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/statusHistory.py/statusHistory.content |
MongoControllers/StatusHistoryController.py/StatusHistoryController.get |
✅ | |
utils.py/statusHistory.py/statusHistory.trim |
MongoControllers/StatusHistoryController.py/StatusHistoryController.purge |
❌ | |
utils.py/statusHistory.py/statusHistory.add |
MongoControllers/StatusHistoryController.py/StatusHistoryController.set |
❌ | |
| — | MongoControllers/StatusHistoryController.py/StatusHistoryController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/StatusHistoryController.py/StatusHistoryController._setMongoCollection |
✅ | * required by the MongoClient |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/wtcInfo.clean |
MongoControllers/WTCController.py/WTCController.clean |
❌ | |
utils.py/wtcInfo.remove |
MongoControllers/WTCController.py/WTCController.cleanKeyword |
❌ | |
utils.py/wtcInfo._get |
MongoControllers/WTCController.py/WTCController.get |
✅ | |
utils.py/wtcInfo.getBypass |
MongoControllers/WTCController.py/WTCController.getBypass |
✅ | |
utils.py/wtcInfo.getForce |
MongoControllers/WTCController.py/WTCController.getForce |
✅ | |
utils.py/wtcInfo.getHold |
MongoControllers/WTCController.py/WTCController.getHold |
✅ | |
utils.py/wtcInfo.add |
MongoControllers/WTCController.py/WTCController.set |
❌ | |
| — | MongoControllers/WTCController.py/WTCController._buildMongoDocument |
✅ | * called in set |
| — | MongoControllers/WTCController.py/WTCController._setMongoCollection |
✅ | * required by the MongoClient |
Includes all helper functions from utils.py talking to a service.
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/workflowInfo.getRecoveryDoc |
Services/ACDC/ACDCReader.py/ACDCReader.getRecoveryDocs |
✅ | * input arg collection_name was switched by the workflow name because it was never used |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/componentCheck.check_dbs |
Services/DBS/DBSReader.py/DBSReader.check |
❌ | |
utils.py/getFileBlock |
Services/DBS/DBSReader.py/DBSReader.getBlockName |
✅ | |
utils.py/getDatasetBlocks |
Services/DBS/DBSReader.py/DBSReader.getDatasetBlockNamesServices/DBS/DBSReader.py/DBSReader.getDatasetBlockNamesByLumisServices/DBS/DBSReader.py/DBSReader.getDatasetBlockNamesByRuns
|
✅ | * split because the block names were filtered by runs or by lumis (never both!) |
utils.py/getDatasetEventsAndLumis |
Services/DBS/DBSReader.py/DBSReader.getDatasetEventsAndLumisServices/DBS/DBSReader.py/DBSReader.getBlocksEventsAndLumis
|
✅ | * split because events and lumis were got by dataset or blocks (never both!) |
utils.py/getDatasetEventsPerLumis |
Services/DBS/DBSReader.py/DBSReader.getDatasetEventsPerLumis |
✅ | |
utils.py/getDatasetFileArray |
Services/DBS/DBSReader.py/DBSReader.getDatasetFiles |
✅ | * input args use_array and cache_timeout were removed because were not used |
utils.py/getDatasetLumisAndFiles |
Services/DBS/DBSReader.py/DBSReader.getDatasetLumisAndFiles |
✅ | * only does the request, input args runs and lumilist were removed but the helper functions Utilities/DataTools.py/filterLumisAndFilesByRuns or filterLumisAndFilesByLumis can be used if needed |
utils.py/getDatasets |
Services/DBS/DBSReader.py/DBSReader.getDatasetNames |
✅ | |
utils.py/findParent |
Services/DBS/DBSReader.py/DBSReader.getDatasetParent |
✅ | |
utils.py/getDatasetRuns |
Services/DBS/DBSReader.py/DBSReader.getDatasetRuns |
✅ | |
utils.py/getDatasetSize |
Services/DBS/DBSReader.py/DBSReader.getDatasetSize |
✅ | |
utils.py/getDatasetStatus |
Services/DBS/DBSReader.py/DBSReader.getDBSStatus |
✅ | |
utils.py/getFilesWithLumiInRun |
Services/DBS/DBSReader.py/DBSReader.getFilesWithLumiInRun |
✅ | |
utils.py/getLFNbase |
Services/DBS/DBSReader.py/DBSReader.getLFNBase |
✅ | |
utils.py/setDatasetStatus |
Services/DBS/DBSWriter.py/DBSWriter.setDatasetStatus |
❌ | |
utils.py/setFileStatus |
Services/DBS/DBSWriter.py/DBSWriter.setFileStatus |
❌ | |
| — | Services/DBS/DBSReader.py/DBSReader._getBlockFileLumis |
❌ | * called in getBlocksLumisAndFilesForCaching |
| — | Services/DBS/DBSReader.py/DBSReader._getFileLumiArray |
❌ | * called in getFilesWithLumiInRun |
| — | Services/DBS/DBSReader.py/DBSReader.getBlocksLumisAndFilesForCaching |
✅ | |
| — | Services/DBS/DBSReader.py/DBSReader.getRecoveryBlocksAndLocations |
❌ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/eosRead |
Services/EOS/EOSReader.py/EOSReader.read |
✅ | |
utils.py/eosFile.close |
Services/EOS/EOSWriter.py/EOSWriter.save |
❌ | |
utils.py/eosFile.write |
Services/EOS/EOSWriter.py/EOSWriter.write |
❌ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/workflowInfo.getGlideMon |
Services/GWMSMon/GWMSMonReader.py/GWMSMonReader.getRequestSummary |
❌ | * request return an error |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/getAgentConfig |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getAgentConfig |
❌ | |
utils.py/getReqmgrInfo |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getReqmgrInfo |
✅ | |
utils.py/getWorkflowByCampaign |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByCampaign |
✅ | |
utils.py/getWorkflowByName |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByNames |
✅ | |
utils.py/getWorkflowByOutput |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByOutput |
✅ | * endpoint changed from CouchDB to the RESTful api |
utils.py/getWorkflowById |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByPrepId |
✅ | * endpoint changed from CouchDB to the RESTful api |
utils.py/getWorkflows |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByStatus |
✅ | |
utils.py/getWorkload |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowSchema |
✅ | |
utils.py/workflowInfo.getSchema |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowSchema(makeCopy=True) |
✅ | |
utils.py/workflowInfo.getSummary |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkloadSummary |
✅ | |
utils.py/workflowInfo.get_spec |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getSpec |
✅ | |
utils.py/workflowInfo.getSplittingsNew |
Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getSplittingsSchema |
✅ | * only does the request, input args strip and allTasks were removed but the helper functions Utilities/DataTools.py/filterSplittingsParam and filterSplittingsTaskTypes can be used if needed |
utils.py/sendAgentConfig |
Services/ReqMgr/ReqMgrWriter.py/ReqMgrWriter.setAgentConfig |
❌ | |
| — | Services/ReqMgr/ReqMgrReader.py/ReqMgrReader.getWorkflowsByParam |
❌ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
RucioClient.py/getFileCountBlock |
Services/Rucio/RucioReader.py/RucioReader.countBlockFiles |
✅ | |
RucioClient.py/getFileCountDataset |
Services/Rucio/RucioReader.py/RucioReader.countDatasetFiles |
✅ | |
RucioClient.py/getFileCountPerBlock |
Services/Rucio/RucioReader.py/RucioReader.countDatasetFilesPerBlock |
✅ | |
RucioClient.py/getBlockNamesDataset |
Services/Rucio/RucioReader.py/RucioReader.getDatasetBlockNames |
✅ | |
RucioClient.py/getFileNamesDataset |
Services/Rucio/RucioReader.py/RucioReader.getDatasetFileNames |
✅ | |
RucioClient.py/getDatasetLocationsByAccount |
Services/Rucio/RucioReader.py/RucioReader.getDatasetLocationsByAccount |
✅ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
TrelloClient.py/_get |
Services/Trello/TrelloClient.py/TrelloClient._getResponse |
❌ | * called in getBoard and getCard |
TrelloClient.py/_put |
Services/Trello/TrelloClient.py/TrelloClient._sendResponse |
❌ | * called in setList |
TrelloClient.py/getAgents |
Services/Trello/TrelloClient.py/TrelloClient._syncAgents |
❌ | * called when instantiated |
TrelloClient.py/getBoard |
Services/Trello/TrelloClient.py/TrelloClient.getBoard |
❌ | |
TrelloClient.py/getCard |
Services/Trello/TrelloClient.py/TrelloClient.getCard |
❌ | |
TrelloClient.py/changeList |
Services/Trello/TrelloClient.py/TrelloClient.setList |
❌ | |
| — | Services/Trello/TrelloClient.py/TrelloClient.getListId |
❌ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/getAllAgents |
Services/WMStats/WMStatsReader.py/WMStatsReader.getAgents |
✅ | |
utils.py/workflowInfo.getWMErrors |
Services/WMStats/WMStatsReader.py/WMStatsReader.getWMErrors |
❌ | * only include the request of the original function — call WorkflowController.getWMErrors to keep the same behavior |
utils.py/workflowInfo.getWMStats |
Services/WMStats/WMStatsReader.py/WMStatsReader.getWMStats |
✅ | * only include the request of the original function — call WorkflowController.getWMStats to keep the same behavior |
| — | Services/WMStats/WMStatsReader.py/WMStatsReader.getProductionAgents |
✅ |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/workflowInfo.getWorkQueue |
Services/WorkQueue/WorkQueueReader.py/WorkQueueReader.getWorkQueue |
✅ |
Includes all helper functions from utils.py that are used for authentication, configuration, etc.
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/make_x509_conn |
Utilities/Authenticate.py/getX509Conn |
❌ | |
utils.py/unifiedConfiguration |
Utilities/ConfigurationHandler.py/ConfigurationHandler |
❌ | |
utils.py/workflowInfo._taskDescending |
Utilities/DataTools/flattenTaskTree |
❌ | |
utils.py/ThreadHandler |
Utilities/Decorators.py/runWithMultiThreading |
❌ | |
utils.py/runWithRetries |
Utilities/Decorators.py/runWithRetries |
❌ | |
utils.py/display_N |
Utilities/Logging/displayNumber |
❌ | |
utils.py/display_time |
Utilities/Logging/displayTime |
❌ | |
utils.py/GET |
Utilities/WebTools.py/getResponse |
❌ | |
| — | Utilities/DataTools/filterFilesAndLocationsInDBS |
❌ | |
| — | Utilities/DataTools/filterLumisAndFilesByLumis |
❌ | |
| — | Utilities/DataTools/filterLumisAndFilesByRuns |
❌ | |
| — | Utilities/DataTools/filterRecoveryFilesAndLocations |
❌ | |
| — | Utilities/DataTools/filterSplittingsParam |
❌ | |
| — | Utilities/DataTools/filterSplittingsTaskTypes |
❌ | |
| — | Utilities/DataTools/sortByWakeUpPriority |
❌ | |
| — | Utilities/IteratorTools/filterKeys |
❌ | |
| — | Utilities/IteratorTools/mapKeys |
❌ | |
| — | Utilities/IteratorTools/mapValues |
❌ | |
| — | Utilities/IteratorTools/sortByKeys |
❌ | |
| — | Utilities/WebTools.py/sendResponse |
❌ |
Includes all helper functions from utils.py that are used for managing a workflow
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/workflowInfo.checkSplitting |
WorkflowMgmt/WorkflowController.py/WorkflowController.checkSplitting |
✅ | |
utils.py/workflowInfo.getAgents |
WorkflowMgmt/WorkflowController.py/WorkflowController.getAgents |
✅ | |
utils.py/workflowInfo.getAllTasks |
WorkflowMgmt/WorkflowController.py/WorkflowController.getAllTasks |
✅ | |
utils.py/workflowInfo.getBlocks |
WorkflowMgmt/WorkflowController.py/WorkflowController.getBlocks |
✅ | |
utils.py/workflowInfo.getBlockWhiteList |
WorkflowMgmt/WorkflowController.py/WorkflowController.getBlockWhiteList |
✅ | |
utils.py/workflowInfo.getBlowupFactors |
WorkflowMgmt/WorkflowController.py/WorkflowController.getBlowupFactor |
✅ | * return only the factor, since the additional outputs were not used |
utils.py/workflowInfo.getCampaignPerTask |
WorkflowMgmt/WorkflowController.py/WorkflowController.getCampaignByTask |
✅ | |
utils.py/workflowInfo.getConfigCacheID |
WorkflowMgmt/WorkflowController.py/WorkflowController.getConfigCacheID |
✅ | |
utils.py/workflowInfo.getCompletionFraction |
WorkflowMgmt/WorkflowController.py/WorkflowController.getCompletionFraction |
✅ | |
utils.py/workflowInfo.getComputingTime |
`WorkflowMgmt/WorkflowController.py/WorkflowController.getComputingTime | ✅ | |
utils.py/workflowInfo.getCorePerTask |
WorkflowMgmt/WorkflowController.py/WorkflowController.getCoreByTask |
✅ | |
utils.py/workflowInfo.getFamilly |
WorkflowMgmt/WorkflowController.py/WorkflowController.getFamily |
✅ | |
utils.py/workflowInfo.getFilterEfficiency |
WorkflowMgmt/WorkflowController.py/WorkflowController.getFilterEfficiencyByTask |
✅ | |
utils.py/workflowInfo.firstTask |
WorkflowMgmt/WorkflowController.py/WorkflowController.getFirstTask |
✅ | |
utils.py/workflowInfo.getGlideMon |
WorkflowMgmt/WorkflowController.py/WorkflowController.getGlideWMSMonSummary |
❌ | |
utils.py/workflowInfo.getMemoryPerTask |
WorkflowMgmt/WorkflowController.py/WorkflowController.getMemoryByTask |
✅ | |
utils.py/workflowInfo.getNCopies |
WorkflowMgmt/WorkflowController.py/WorkflowController.getNCopies |
✅ | |
utils.py/workflowInfo.getNextVersion |
WorkflowMgmt/WorkflowController.py/WorkflowController.getNextVersion |
✅ | |
utils.py/workflowInfo.getOutputPerTask |
WorkflowMgmt/WorkflowController.py/WorkflowController.getOutputDatasetsPerTask |
✅ | |
utils.py/workflowInfo.getPrepIDs |
WorkflowMgmt/WorkflowController.py/WorkflowController.getPrepIDs |
✅ | |
utils.py/workflowInfo.getRecoveryBlocks |
WorkflowMgmt/WorkflowController.py/WorkflowController.getRecoveryBlocks |
❌ | |
utils.py/workflowInfo.getRecoveryInfo |
WorkflowMgmt/WorkflowController.py/WorkflowController.getRecoveryInfo |
❌ | |
utils.py/workflowInfo.getRunWhiteList |
WorkflowMgmt/WorkflowController.py/WorkflowController.getRunWhiteList |
✅ | |
utils.py/workflowInfo.getArchs |
WorkflowMgmt/WorkflowController.py/WorkflowController.getScramArches |
✅ | |
utils.py/workflowInfo.getSiteWhiteList |
WorkflowMgmt/WorkflowController.py/WorkflowController.getSiteWhiteList |
❌ | |
utils.py/workflowInfo.getSpplitings |
WorkflowMgmt/WorkflowController.py/WorkflowController.getSpplitings |
✅ | |
utils.py/workflowInfo.getSplittingsNew |
WorkflowMgmt/WorkflowController.py/WorkflowController.getSpplitingsSchema |
✅ | |
utils.py/workflowInfo.getSummary |
WorkflowMgmt/WorkflowController.py/WorkflowController.getSummary |
✅ | |
utils.py/workflowInfo.getWMErrors |
WorkflowMgmt/WorkflowController.py/WorkflowController.getWMErrors |
❌ | |
utils.py/workflowInfo.getWMStatus |
WorkflowMgmt/WorkflowController.py/WorkflowController.getWMStatus |
✅ | |
utils.py/workflowInfo.getWorkTasks |
WorkflowMgmt/WorkflowController.py/WorkflowController.getWorkTasks |
✅ | |
utils.py/workflowInfo.go |
WorkflowMgmt/WorkflowController.py/WorkflowController.go |
✅ | |
utils.py/workflowInfo.heavyRead |
WorkflowMgmt/WorkflowController.py/WorkflowController.isHeavyToReady |
✅ | |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._getAllowedSites |
❌ | * called in getSiteWhiteList |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._getFromFile |
✅ | * called in getWMStats and getWMErrors |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._getVersionByConflictingWorkflows |
✅ | * called in getNextVersion |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._getVersionByWildcardPattern |
✅ | * called in getNextVersion |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._getWorkloadHandler |
✅ | * called when instantiated |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._restrictAllowedSitesByBlowUpFactor |
❌ | * called in getSiteWhiteList |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._restrictAllowedSitesByCampaign |
❌ | * called in getSiteWhiteList |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._parseOutputProcessingString |
❌ | * called in getNextVersion |
| — | WorkflowMgmt/WorkflowController.py/WorkflowController._saveInFile |
❌ | * called in getWMStats and getWMErrors |
Details
| Python 2 | Python 3 | tested | notes |
|---|---|---|---|
utils.py/workflowInfo.acquisitionEra |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getAcquisitionEraWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getAcquisitionEra
|
✅ | |
utils.py/workflowInfo.campaigns |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getCampaignsWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getCampaigns
|
✅ | |
utils.py/workflowInfo.getCampaigns |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getCampaigns(details=False)WorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getCampaigns(details=False)
|
✅ | |
utils.py/workflowInfo._collectinchainutils.py/workflowInfo._collectin_uhm_chain
|
WorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getChainValues |
✅ | |
utils.py/workflowInfo.getIO |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getIOWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getIO
|
✅ | |
utils.py/workflowInfo.getLumiWhiteList |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getLumiWhiteListWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getLumiWhiteList
|
✅ | |
utils.py/workflowInfo.getMemory |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getMemoryWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getMemory
|
✅ | |
utils.py/workflowInfo.getMulticore |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getMulticoreWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getMulticore
|
✅ | |
utils.py/workflowInfo.getMulticores |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getMulticore(maxOnly=False)WorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getMulticore(maxOnly=False)
|
✅ | |
utils.py/workflowInfo.processingString |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getProcessingStringWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getProcessingString
|
✅ | |
utils.py/workflowInfo.getRequestNumEvents |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getRequestNumEventsWorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler.getRequestNumEvents
|
✅ | |
utils.py/workflowInfo.isGoodToConvertToStepChain |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.isGoodToConvertToStepChainWorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler.isGoodToConvertToStepChain
|
✅ | |
utils.py/workflowInfo.producePremix |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.isProducingPremix |
✅ | |
utils.py/workflowInfo.isRelVal |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.isRelVal |
✅ | |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getOutputSizeCorrectionFactor |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler._getTaskIO |
❌ | * called in getIO |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTimeInfo |
✅ | * called in isGoodToConvertToStepChain |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTaskEfficiencyFactor |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTaskMaxEventsBySize |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTaskMaxEventsByTime |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTaskMaxEventsPerLumiAndJob |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._getTaskSchema |
❌ | * called in checkSplitting |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._hasAcceptableEfficiency |
✅ | * called in isGoodToConvertToStepChain |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._hasNonZeroEventStreams |
❌ | * called in isGoodToConvertToStepChain |
| — | WorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler._isBelowMinEventsPerLumi |
❌ | * called in checkSplitting |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.checkSplittingWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.checkSplittingWorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler.checkSplitting
|
✅ | |
| — | WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.get |
❌ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getBlowupFactorWorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler.getBlowupFactor
|
✅ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getCampaignsAndLabelsWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getCampaignsAndLabels
|
✅ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getComputingTimeWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getComputingTime
|
✅ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getExpectedEventsPerTaskWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getExpectedEventsPerTask
|
❌ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getParamByTaskWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getParamByTask
|
❌ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getParamListWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getParamList
|
❌ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getOutputDatasetsPerTaskWorkflowMgmt/WorkflowSchemaHandlers/StepChainWfSchemaHandler.py/StepChainWfSchemaHandler.getOutputDatasetsPerTask
|
✅ | |
| — | WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.getTasksPerOutputDatasets |
❌ | |
| — |
WorkflowMgmt/WorkflowSchemaHandlers/BaseWfSchemaHandler.py/BaseWfSchemaHandler.writeDatasetPatternNameWorkflowMgmt/WorkflowSchemaHandlers/TaskChainWfSchemaHandler.py/TaskChainWfSchemaHandler.writeDatasetPatternName
|
❌ |