Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #365

Closed
wants to merge 90 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
e3cf0f1
Update dependencies to include real estate
AlexAxthelm Sep 26, 2024
24fe92d
Trigger CI
AlexAxthelm Sep 26, 2024
08a9e4c
Use updated testing Peer results
AlexAxthelm Sep 27, 2024
54003c6
Trigger CI
AlexAxthelm Sep 27, 2024
affe89a
Trigger CI
AlexAxthelm Sep 27, 2024
6cd4a5f
Trigger CI
AlexAxthelm Sep 27, 2024
fc0cbae
Trigger CI
AlexAxthelm Sep 30, 2024
49ab4e4
Trigger CI
AlexAxthelm Sep 30, 2024
d4c05de
Reduce Test suite
AlexAxthelm Sep 30, 2024
3fcbd66
Use new `az storage fs upload` command
AlexAxthelm Oct 1, 2024
8b489c7
Trigger CI
AlexAxthelm Oct 1, 2024
9b94288
Get `azcopy` authenticaiton from `az` login
AlexAxthelm Oct 1, 2024
4e7f1e0
Explicitly login with azcopy login
AlexAxthelm Oct 1, 2024
0988daf
Authenticate with MSI + Client ID
AlexAxthelm Oct 1, 2024
157d55b
minimize test workflow
AlexAxthelm Oct 1, 2024
f09e1f1
Actually minimize test workflow
AlexAxthelm Oct 1, 2024
5d571f5
Grant permissions
AlexAxthelm Oct 1, 2024
f9f375a
remove removed dep
AlexAxthelm Oct 1, 2024
de340b3
Use AZCLI auth
AlexAxthelm Oct 1, 2024
c19e92c
Add tenant ID
AlexAxthelm Oct 1, 2024
67966c5
Install azcopy
AlexAxthelm Oct 1, 2024
78e3b86
Explicity use azcopy, not az passthrough
AlexAxthelm Oct 1, 2024
0be1759
Show path
AlexAxthelm Oct 1, 2024
93ecaa1
Use fullpath for azcopy command
AlexAxthelm Oct 1, 2024
4a674ff
Logging
AlexAxthelm Oct 1, 2024
4b4db36
Use standard shell, not az action
AlexAxthelm Oct 1, 2024
7bb2e40
Pass correct envvars to azcopy
AlexAxthelm Oct 1, 2024
bd4b56e
Use new action
AlexAxthelm Oct 2, 2024
dbe13f7
Show files
AlexAxthelm Oct 2, 2024
ca4cd58
fix: duplicate step ID
AlexAxthelm Oct 2, 2024
b6ec484
Login in workflow
AlexAxthelm Oct 2, 2024
8946729
Fix: yaml indentation
AlexAxthelm Oct 2, 2024
461bcd2
Fix: yaml indentation
AlexAxthelm Oct 2, 2024
642b415
Trigger CI
AlexAxthelm Oct 2, 2024
5c2607d
Trigger CI
AlexAxthelm Oct 2, 2024
693d0aa
Trigger CI
AlexAxthelm Oct 2, 2024
fea0a5b
Trigger CI
AlexAxthelm Oct 2, 2024
4bd38b4
specify `--recursive=true`, not just `--recursive`
AlexAxthelm Oct 2, 2024
ac8efaf
Suppress trailing newline
AlexAxthelm Oct 2, 2024
0583751
Pass tenant-id to updated action
AlexAxthelm Oct 2, 2024
ee006e3
Trigger CI
AlexAxthelm Oct 2, 2024
12322b1
Pass client ID to action
AlexAxthelm Oct 2, 2024
d2fd513
Trigger CI
AlexAxthelm Oct 2, 2024
252c4d2
Trigger CI
AlexAxthelm Oct 2, 2024
7ef26ee
Trigger CI
AlexAxthelm Oct 3, 2024
c15c67b
Trigger CI
AlexAxthelm Oct 3, 2024
feaf441
Trigger CI
AlexAxthelm Oct 3, 2024
7cb4e0f
Use updated action with SAS
AlexAxthelm Oct 3, 2024
dbbc5a2
Trigger CI
AlexAxthelm Oct 3, 2024
8b1b7bf
Trigger CI
AlexAxthelm Oct 3, 2024
4728dfb
Trigger CI
AlexAxthelm Oct 3, 2024
c733703
Trigger CI
AlexAxthelm Oct 3, 2024
b1196f1
Remove unused client-id input
AlexAxthelm Oct 3, 2024
1182088
Trigger CI
AlexAxthelm Oct 3, 2024
c9ae487
Trigger CI
AlexAxthelm Oct 3, 2024
e8ca253
Trigger CI
AlexAxthelm Oct 3, 2024
61a6be7
Trigger CI
AlexAxthelm Oct 3, 2024
f38ea29
Trigger CI
AlexAxthelm Oct 3, 2024
cf95dd5
Use python branch of copy action
AlexAxthelm Oct 8, 2024
d5ac436
Update action name
AlexAxthelm Oct 8, 2024
7c604a9
Trigger CI
AlexAxthelm Oct 8, 2024
7efdac4
Trigger CI
AlexAxthelm Oct 8, 2024
b40d820
Trigger CI
AlexAxthelm Oct 8, 2024
9a3294a
Change container for testing
AlexAxthelm Oct 8, 2024
785ee6d
Change step name
AlexAxthelm Oct 8, 2024
e0c0393
Trigger CI
AlexAxthelm Oct 8, 2024
cfefbad
Overwrite existing blobs
AlexAxthelm Oct 8, 2024
eeb3cad
Trigger CI
AlexAxthelm Oct 8, 2024
3bcfb0f
Restore original build-pysh-private workflow
AlexAxthelm Oct 8, 2024
12c3e12
Use new upload action to upload results and reports.
AlexAxthelm Oct 8, 2024
015233c
Restore original Triggers
AlexAxthelm Oct 8, 2024
a0e56d3
Trigger CI
AlexAxthelm Oct 8, 2024
96350b2
Use gh outputs, not env
AlexAxthelm Oct 8, 2024
2c049d1
Remove handling for private report creation
AlexAxthelm Oct 8, 2024
b67c289
Merge branch 'real-estate' into develop
AlexAxthelm Oct 8, 2024
bee93ec
Fix upload Path
AlexAxthelm Oct 8, 2024
8d2e851
Fix upload Path
AlexAxthelm Oct 8, 2024
9cad417
Fix URL for public reports
AlexAxthelm Oct 8, 2024
e9b31bd
Fix Report container URL
AlexAxthelm Oct 8, 2024
60c0a03
DOn't append report URL to upload dir
AlexAxthelm Oct 8, 2024
45d4815
Merge branch 'ci/363-restore-report-links' into develop
AlexAxthelm Oct 8, 2024
45e9cfb
Use updated action with log control
AlexAxthelm Oct 9, 2024
2744eda
Trigger CI
AlexAxthelm Oct 9, 2024
a16b4e7
Trigger CI
AlexAxthelm Oct 9, 2024
ab3013b
Use public reports path
AlexAxthelm Oct 9, 2024
0444301
Trigger CI
AlexAxthelm Oct 9, 2024
47d5b16
Trigger CI
AlexAxthelm Oct 9, 2024
269b409
Remove unused file
AlexAxthelm Oct 9, 2024
b12610b
Merge branch 'ci/363-restore-report-links' into develop
AlexAxthelm Oct 9, 2024
876c3b6
Merge branch 'real-estate' into develop
AlexAxthelm Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 30 additions & 39 deletions .github/workflows/build-push-private.yml
Original file line number Diff line number Diff line change
Expand Up @@ -468,28 +468,23 @@ jobs:
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

# https://github.com/marketplace/actions/azure-cli-action#workflow-to-execute-an-azure-cli-script-of-a-specific-cli-version
- name: Upload results to blob store
uses: azure/CLI@v2
env:
REPORTS_DESTINATION_URL: ${{ needs.read-config.outputs.reports-destination-url }}
RESULTS_DESTINATION_URL: ${{ needs.read-config.outputs.results-destination-url }}
language: ${{ matrix.language }}
peer_group: ${{ matrix.peer_group }}
run_attempt: ${{ github.run_attempt }}
test_dir_parent: ${{ env.TEST_DIR_PARENT }}
- name: Upload results to Blob store
id: upload-results
uses: RMI-PACTA/actions/actions/azure/blob-copy@blob-copy-python
with:
inlineScript: |
az storage copy \
--source "$test_dir_parent" \
--destination "$RESULTS_DESTINATION_URL" \
--recursive

az storage copy \
--source "$test_dir_parent" \
--destination "$REPORTS_DESTINATION_URL" \
--include-path "$language/$peer_group/$run_attempt/working_dir/50_Outputs" \
--recursive
source: ${{ env.TEST_DIR_PARENT }}
destination: https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-full
overwrite: false
log_level: debug

- name: Upload report to Blob store
id: upload-report
uses: RMI-PACTA/actions/actions/azure/blob-copy@blob-copy-python
with:
source: ${{ env.TEST_DIR_PARENT }}/${{ matrix.language }}/${{ matrix.peer_group }}/${{ github.run_attempt }}/working_dir/50_Outputs
destination: https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports
overwrite: false
log_level: debug

- name: Export Outputs
id: export-outputs
Expand All @@ -503,27 +498,23 @@ jobs:
echo "blob-dir=$BLOB_DIR" >> "$GITHUB_OUTPUT"
echo "blob-dir=$BLOB_DIR" >> "$GITHUB_ENV"

PUBLIC_BLOB_URL="https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports"

if [ "$REPORT_URL" = "$PUBLIC_BLOB_URL" ]; then
REPORT_URL="$BLOB_DIR/report/index.html"
echo "report-url=$REPORT_URL"
echo "report-url=$REPORT_URL" >> "$GITHUB_OUTPUT"
echo "report-url=$REPORT_URL" >> "$GITHUB_ENV"

SUMMARY_PATH="${test_dir}/working_dir/50_Outputs/$image_name/executive_summary/template.pdf"
if [ -f "$SUMMARY_PATH" ]; then
SUMMARY_URL="$BLOB_DIR/executive_summary/template.pdf"
echo "summary-url=$SUMMARY_URL"
echo "summary-url=$SUMMARY_URL" >> "$GITHUB_OUTPUT"
echo "summary-url=$SUMMARY_URL" >> "$GITHUB_ENV"
fi
REPORT_URL="$BLOB_DIR/report/index.html"
echo "report-url=$REPORT_URL"
echo "report-url=$REPORT_URL" >> "$GITHUB_OUTPUT"
echo "report-url=$REPORT_URL" >> "$GITHUB_ENV"

SUMMARY_PATH="${test_dir}/working_dir/50_Outputs/$image_name/executive_summary/template.pdf"
if [ -f "$SUMMARY_PATH" ]; then
SUMMARY_URL="$BLOB_DIR/executive_summary/template.pdf"
echo "summary-url=$SUMMARY_URL"
echo "summary-url=$SUMMARY_URL" >> "$GITHUB_OUTPUT"
echo "summary-url=$SUMMARY_URL" >> "$GITHUB_ENV"
fi

- name: Prepare comment artifact
id: prepare-artifact
env:
blob_dir: ${{ env.blob-dir }}
blob_dir: ${{ steps.export-outputs.outputs.blob-dir }}
commit_time: ${{ github.event.pull_request.updated_at }}
full_image_name: ${{ needs.docker-build.outputs.full-image-name }}
git_sha: ${{ github.event.pull_request.head.sha }},
Expand All @@ -533,9 +524,9 @@ jobs:
peer_group: ${{ matrix.peer_group }}
project_code: ${{ needs.read-config.outputs.project-code }}
registry: ${{ inputs.registry }}
report_url: ${{ env.report-url }}
report_url: ${{ steps.export-outputs.outputs.report-url }}
run_attempt: ${{ github.run_attempt }}
summary_url: ${{ env.summary-url }}
summary_url: ${{ steps.export-outputs.outputs.summary-url }}
run: |
mkdir -p /tmp/comment-json
unique_string=
Expand Down
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Remotes:
RMI-PACTA/pacta.portfolio.allocate,
RMI-PACTA/pacta.portfolio.audit,
RMI-PACTA/pacta.portfolio.import,
RMI-PACTA/pacta.portfolio.report,
RMI-PACTA/pacta.portfolio.report@add_real_estate,
RMI-PACTA/pacta.portfolio.utils
Depends:
R (>= 3.5.0)
16 changes: 8 additions & 8 deletions build/config/rmi_pacta_2023q4_pa2024ch.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"data_share_path": "2023Q4_20240718T150252Z",
"pacta_data_quarter": "2023Q4",
"project_code": "PA2024CH",
"templates_ref": "",
"templates_ref": "add_real_estate",
"peer_results": "https://pactadatadev.blob.core.windows.net/project-files/pa2024ch/peer_files",
"net_zero_targets": "https://pactadatadev.blob.core.windows.net/project-files/pa2024ch/fin_data_net_zero_targets.csv",
"test_matrix": {
Expand All @@ -14,7 +14,7 @@
"user_id": 74,
"language": "DE",
"peer_group": "bank",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/bank"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/bank"
},
{
"user_id": 74,
Expand All @@ -25,7 +25,7 @@
"user_id": 74,
"language": "FR",
"peer_group": "bank",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/bank"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/bank"
},
{
"user_id": 74,
Expand All @@ -34,22 +34,22 @@
},
{
"peer_group": "assetmanager",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/assetmanager"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/assetmanager"
},
{
"peer_group": "bank",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/bank"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/bank"
},
{
"peer_group": "insurance",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/insurance"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/insurance"
},
{
"peer_group": "pensionfund",
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240702/pensionfund"
"user_results": "https://pactadatadev.blob.core.windows.net/testing-files/user-results/PA2024CH_peer-test/20240927/pensionfund"
}
]
},
"resultsDestinationURL": "https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-full",
"reportsDestinationURL": "https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private"
"reportsDestinationURL": "https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports"
}
Loading