Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce cost for pangeo-hubs, three n2-highmem-8 core nodes running #3379

Closed
consideRatio opened this issue Nov 5, 2023 · 3 comments
Closed
Assignees
Labels
allocation:internal-eng tech:cloud-infra Optimization of cloud infra to reduce costs etc.

Comments

@consideRatio
Copy link
Contributor

consideRatio commented Nov 5, 2023

The pods running could fit on a single node computational capacity wise, I think they don't run on a single node because of a few pods with anti-affinity policies for example. This can be improved on greatly no matter what.

I think this must be related to having many konnectivity-agents that doesn't want to be scheduled together on nodes more than "1 skew", discussed in #2490. I reduced the autoscaling of these pods to not have more than 2 until at least the cluster had 10 nodes running to help avoid the need for more than 1 core node in most situations.

@github-project-automation github-project-automation bot moved this to Needs Shaping / Refinement in DEPRECATED Engineering and Product Backlog Nov 5, 2023
@consideRatio consideRatio added the tech:cloud-infra Optimization of cloud infra to reduce costs etc. label Nov 5, 2023
@consideRatio
Copy link
Contributor Author

I've managed to get this down from 3 to 1 nodes, and it can be further reduced to a single n2-highmem-4 node instead of using the n2-highmem-8 nodes. A comment that this is more suitable is added in #3375.

@consideRatio consideRatio self-assigned this Nov 5, 2023
@consideRatio
Copy link
Contributor Author

Remaining action point

  • Change the core nodes for pangeo-hubs from n2-highmem-8 nodes to n2-highmem-4 nodes

@damianavila damianavila moved this to In Progress ⚡ in Sprint Board Nov 13, 2023
@damianavila damianavila moved this from Needs Shaping / Refinement to In progress in DEPRECATED Engineering and Product Backlog Nov 13, 2023
@consideRatio
Copy link
Contributor Author

@github-project-automation github-project-automation bot moved this from In Progress ⚡ to Done 🎉 in Sprint Board Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
allocation:internal-eng tech:cloud-infra Optimization of cloud infra to reduce costs etc.
Projects
No open projects
Status: Done 🎉
Development

No branches or pull requests

2 participants