Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create resourceQuota in rhods-notebooks namespace (Prod cluster) #377

Closed
dystewart opened this issue Jan 12, 2024 · 1 comment · Fixed by OCP-on-NERC/nerc-ocp-config#341
Assignees
Labels
openshift This issue pertains to NERC OpenShift rhoai RHOAI

Comments

@dystewart
Copy link

We need to create a resourceQuota in this namespace since there isn't one by default.

We will need a pretty hefty quota since both ope courses are using the namespace. Since we can't limit the notebook resource options to the minimal allocation in rhods without removing them as options from all the users, we need to keep in mind that students technically have the ability to choose any of the allocation options for their notebook.

We have in the docs specified the x-small resource option but we can't enforce this in our current setup. That makes it somewhat tricky to nail down an exact hard quota limit Since in the absolute worst case (incredibly unlikely but technically possible) all students could select other bigger options.

We ran the scale test in ope-rhods-testing with 300 instances and didn't hit our quota so maybe that's a good starting point. As for storage we can pretty comfortably set a limit once we know exactly the number of students we expect (all notebooks create a pvc with one Gi).

We can use this as a starting point:

kind: ResourceQuota
apiVersion: v1
metadata:
  name: spring-course-quota
  namespace: rhods-notebooks

spec:
  hard:
    limits.cpu: '1000'
    limits.ephemeral-storage: 50Gi
    limits.memory: 3000000Mi
    persistentvolumeclaims: '1000'
    requests.storage: 100Gi

We should run the test again with the expected number of students to see how it looks against something like the resourceQuota above.

@dystewart
Copy link
Author

PR for this issue

dystewart added a commit to OCP-on-NERC/nerc-ocp-config that referenced this issue Jan 15, 2024
Closes: nerc-project/operations#377

Adding this resourceQuota as a component at the cluster scope and adding it to
the rhods-notebooks namespace in the prod cluster. The quota values themselves were
determined based on our resourceQuota in the namespace where we did scale testing,
with a bump to accomodate all the expected students
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
openshift This issue pertains to NERC OpenShift rhoai RHOAI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant