Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support dataflow affinity #3829

Merged
merged 7 commits into from
May 14, 2024

Conversation

xliuqq
Copy link
Collaborator

@xliuqq xliuqq commented Apr 9, 2024

Ⅰ. Describe what this PR does

support dataflow affinity, design proposal see fluid-cloudnative/community#53

Ⅱ. Does this pull request fix one issue?

fixes #XXXX

Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

api/v1alpha1/status.go Outdated Show resolved Hide resolved
docker/Dockerfile.dataset Outdated Show resolved Hide resolved
xliuqq added 5 commits April 29, 2024 15:51
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
@xliuqq xliuqq force-pushed the dataflow_affinity branch from dacb7f3 to 5bbc2a3 Compare April 29, 2024 15:02
Copy link

codecov bot commented May 1, 2024

Codecov Report

Attention: Patch coverage is 65.35433% with 88 lines in your changes are missing coverage. Please review.

Project coverage is 64.31%. Comparing base (910ace1) to head (5bbc2a3).
Report is 135 commits behind head on master.

❗ Current head 5bbc2a3 differs from pull request most recent head 2e8717f. Consider uploading reports for the commit 2e8717f to get more accurate results

Files Patch % Lines
pkg/utils/kubeclient/job.go 0.00% 15 Missing ⚠️
pkg/utils/affinity.go 44.00% 13 Missing and 1 partial ⚠️
pkg/dataprocess/generate_values.go 0.00% 13 Missing ⚠️
pkg/dataflow/affinity.go 86.44% 6 Missing and 2 partials ⚠️
pkg/dataflow/helper.go 91.17% 3 Missing and 3 partials ⚠️
pkg/ddc/alluxio/load_data.go 55.55% 2 Missing and 2 partials ⚠️
pkg/ddc/goosefs/load_data.go 55.55% 2 Missing and 2 partials ⚠️
pkg/ddc/jindo/load_data.go 55.55% 2 Missing and 2 partials ⚠️
pkg/ddc/jindocache/load_data.go 55.55% 2 Missing and 2 partials ⚠️
pkg/ddc/jindofsx/load_data.go 55.55% 2 Missing and 2 partials ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3829      +/-   ##
==========================================
- Coverage   64.33%   64.31%   -0.03%     
==========================================
  Files         477      480       +3     
  Lines       28444    28668     +224     
==========================================
+ Hits        18300    18437     +137     
- Misses       7968     8036      +68     
- Partials     2176     2195      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cheyang cheyang requested a review from TrafalgarZZZ May 1, 2024 09:14
api/v1alpha1/common.go Outdated Show resolved Hide resolved
api/v1alpha1/common.go Outdated Show resolved Hide resolved
pkg/dataflow/affinity.go Outdated Show resolved Hide resolved
pkg/dataflow/affinity.go Show resolved Hide resolved
pkg/dataflow/affinity.go Outdated Show resolved Hide resolved
pkg/dataflow/helper.go Show resolved Hide resolved
pkg/utils/kubeclient/job.go Show resolved Hide resolved
charts/fluid-dataprocess/common/values.yaml Outdated Show resolved Hide resolved
xliuqq added 2 commits May 6, 2024 22:58
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Copy link

sonarqubecloud bot commented May 8, 2024

Quality Gate Passed Quality Gate passed

Issues
21 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
3.8% Duplication on New Code

See analysis details on SonarCloud

@cheyang cheyang requested review from cheyang and TrafalgarZZZ May 11, 2024 01:51
@TrafalgarZZZ
Copy link
Member

/test fluid-e2e

api/v1alpha1/common.go Show resolved Hide resolved
@cheyang cheyang self-requested a review May 14, 2024 09:34
Copy link
Collaborator

@cheyang cheyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Copy link

fluid-e2e-bot bot commented May 14, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cheyang, TrafalgarZZZ

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [TrafalgarZZZ,cheyang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fluid-e2e-bot fluid-e2e-bot bot merged commit 84029c5 into fluid-cloudnative:master May 14, 2024
9 checks passed
@xliuqq xliuqq deleted the dataflow_affinity branch May 15, 2024 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants