-
Notifications
You must be signed in to change notification settings - Fork 51
Campaign git workflow
- Get your own copy of the offline code from Github with a fork from the upstream
- Develop, test, develop, retest, … until you are happy
- Commit your code and push it back to Github
- Create a pull request that asks for your code to be added to the main repository
- A human being checks your code - when there are no problems with your code, the merge is accepted and your code has been integrated.
Go to https://github.com/CMSCompOps/WmAgentScripts, click to the Fork button on the upper-right corner of the page. This will create your own copy of the code.
Go wherever you want to make your changes, either to your local computer or some remote machine and clone the forked repository:
git clone https://github.com/<your-username>/WmAgentScripts.git
When code is developed it should begin from the current HEAD version of the main repository. So we add the main repository as a remote repository, called upstream:
cd WmAgentScripts
git remote add upstream https://github.com/CMSCompOps/WmAgentScripts.git # or any other valid URL
Your local checkout now has two remotes, origin (your fork) and upstream (the main repository):
$ git remote -v show
origin https://github.com/haozturk/WmAgentScripts.git (fetch)
origin https://github.com/haozturk/WmAgentScripts.git (push)
upstream https://github.com/CMSCompOps/WmAgentScripts.git (fetch)
upstream https://github.com/CMSCompOps/WmAgentScripts.git (push)
Note that it is enough to do the first 3 steps only once.
Before you start development create a space that will hold your code changes in an easily identifiable way. In git this is done by creating a new branch:
git fetch upstream
git checkout -b master-my-topic upstream/master
Why git fetch upstream first? This is to make sure that your local repository is completely up to date with the main repository before you start work on your patch.
It's very important to create a new branch at this step in order to avoid any conflicts that may arise due to using an old branch.
Now you can do your awesome changes.
Git commits code in two phases:
- You add changed files to a staging area with git add
- You commit the staging area with git commit
Example:
# You make your changes
$ git status
On branch master-my-topic
Your branch is up to date with 'upstream/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: campaigns.json
no changes added to commit (use "git add" and/or "git commit -a")
$ git diff
diff --git a/campaigns.json b/campaigns.json
index ec85ee13..09b8ee93 100755
--- a/campaigns.json
+++ b/campaigns.json
@@ -3,7 +3,7 @@
"custodial_override": [
"DQMIO"
],
- "fractionpass": 0.99,
+ "fractionpass": 0.95,
"go": true,
"lumisize": -1,
"maxcopies": 1,
$ git add campaigns.json
$ git commit -m "Change the fractionpass of X campaign"
[hasan-branch e94d31e0] Change the fractionpass of X campaign
1 file changed, 1 insertion(+), 1 deletion(-)
Please use git diff and git status before adding and committing your changes, i.e. please check the changes that you're made is correct.
Once your development is finished and tested, copy it from your local repository back to your Github fork:
git push origin master-my-topic
Create a PR from <your-fork-WmAgentScripts/your-topic-branch> to upstream-WmAgentScripts/master
If there is change is needed for your PR, go to your own development branch and make your changes. They will be reflected to your PR.
If the PR is good to merge, then either workflow manager or one of the L3s will merge the PR. Then follow these steps:
# ssh into of the integration machine of unified - vocms0275.cern.ch as of now
ssh <username>@lxplus.cern.ch
ssh <username>@vocms0275.cern.ch
# Go to the path where the repository is located:
cd /data/unified/WmAgentScripts
# Check if everything is okay
git status
# Ideally this repo should always be on master, but in case another branch is checked out:
git checkout master
# Pull the changes:
git pull
After the step 6, you should load the most up-to-date changes to the necessary databases. Before that, you should set up your Unified environment. (Go to the main README to see how)
# If the PR only updated some campaigns:
python campaignsConfiguration.py --load campaigns.json
# If there are new campaigns, you should use the --name option with the new changes e.g.
python campaignsConfiguration.py --name RunIIpp5Spring18GSFixBeamspot --configuration '{"custodial": "T1_FR_CCIN2P3_MSS","fractionpass": 0.95,"go": true,"lumisize": -1, "parameters": {"MergedLFNBase": "/store/himc","SiteBlacklist": ["T1_US_FNAL","T2_US_Purdue","T2_US_Caltech","T2_US_Florida","T2_US_Nebraska","T2_US_UCSD","T2_US_Wisconsin"]},"resize": "auto", "tune": true}'
If there are both new and updated changes, then first add the new campaign, then run the load.
8. Check if the changes are successfully passed to Couchdb/WMCore (Workflow manager/Campaign manager/L3s):
Cross check your changes with this page and make sure that everything is consistent.
Reference: https://atlassoftwaredocs.web.cern.ch/gittutorial/
- Get your own copy of the offline code from Github with a new branch on upstream
- Develop, test, develop, retest, … until you are happy
- Commit your code and push it back to Github
- Create a pull request that asks for your code to be added to the main repository
- A human being checks your code - when there are no problems with your code, the merge is accepted and your code has been integrated.
Go wherever you want to make your changes, either to your local computer or some remote machine and clone the original repository:
https://github.com/CMSCompOps/WmAgentScripts.git
Before you start development create a space that will hold your code changes in an easily identifiable way. In git this is done by creating a new branch:
git fetch upstream
git checkout -b master-my-topic upstream/master
Why git fetch upstream first? This is to make sure that your local repository is completely up to date with the main repository before you start work on your patch.
It's very important to create a new branch at this step in order to avoid any conflicts that may arise due to using an old branch.
Now you can do your awesome changes.
Git commits code in two phases:
- You add changed files to a staging area with git add
- You commit the staging area with git commit
Example:
# You make your changes
$ git status
On branch master-my-topic
Your branch is up to date with 'upstream/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: campaigns.json
no changes added to commit (use "git add" and/or "git commit -a")
$ git diff
diff --git a/campaigns.json b/campaigns.json
index ec85ee13..09b8ee93 100755
--- a/campaigns.json
+++ b/campaigns.json
@@ -3,7 +3,7 @@
"custodial_override": [
"DQMIO"
],
- "fractionpass": 0.99,
+ "fractionpass": 0.95,
"go": true,
"lumisize": -1,
"maxcopies": 1,
$ git add campaigns.json
$ git commit -m "Change the fractionpass of X campaign"
[hasan-branch e94d31e0] Change the fractionpass of X campaign
1 file changed, 1 insertion(+), 1 deletion(-)
Please use git diff and git status before adding and committing your changes, i.e. please check the changes that you're made is correct.
Once your development is finished and tested, copy it from your local repository back to the original Github repo:
git push origin master-my-topic
Create a PR from <upstream-WmAgentScripts/your-topic-branch> to upstream-WmAgentScripts/master
If there is change is needed for your PR, go to your own development branch and make your changes. They will be reflected to your PR.
If the PR is good to merge, then either workflow manager or one of the L3s will merge the PR. Then follow these steps:
# ssh into of the integration machine of unified - vocms0275.cern.ch as of now
ssh <username>@lxplus.cern.ch
ssh <username>@vocms0275.cern.ch
# Go to the path where the repository is located:
cd /data/unified/WmAgentScripts
# Check if everything is okay
git status
# Ideally this repo should always be on master, but in case another branch is checked out:
git checkout master
# Pull the changes:
git pull
After the step 6, you should load the most up-to-date changes to the necessary databases. Before that, you should set up your Unified environment. (Go to the main README to see how)
# If the PR only updated some campaigns:
python campaignsConfiguration.py --load campaigns.json
# If there are new campaigns, you should use the --name option with the new changes e.g.
python campaignsConfiguration.py --name RunIIpp5Spring18GSFixBeamspot --configuration '{"custodial": "T1_FR_CCIN2P3_MSS","fractionpass": 0.95,"go": true,"lumisize": -1, "parameters": {"MergedLFNBase": "/store/himc","SiteBlacklist": ["T1_US_FNAL","T2_US_Purdue","T2_US_Caltech","T2_US_Florida","T2_US_Nebraska","T2_US_UCSD","T2_US_Wisconsin"]},"resize": "auto", "tune": true}'
If there are both new and updated changes, then first add the new campaign, then run the load.
5. Check if the changes are successfully passed to Couchdb/WMCore (Workflow manager/Campaign manager/L3s):
Cross check your changes with this page and make sure that everything is consistent.
Delete the branch from both local and remote repository
// delete branch locally
git branch -d localBranchName
// delete branch remotely
git push origin --delete remoteBranchName