Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

assigner: implements task assignment for scans and dummy mock test #95

Merged
merged 7 commits into from
Jan 3, 2025

Conversation

ashwiniag
Copy link
Member

@ashwiniag ashwiniag commented Jan 1, 2025

closes: #91

Introduces

  • the task assigner logic in assigner.go, which is designed to distribute pending scans among available agents through round robin fashion. This assignment logic run independently as go routine and periodically at 1min interval
  • adds assigner_test.go which just test interaction with HTTP endpoints and verifies that the assigner.go logic correctly interacts with endpoints. As follow:
  • Basic Task Assignment
  • No Agents Available
  • No Scans Pending
  • adds an endpoint for GET call in agentTasks for robust query
curl -X GET \
-H "Authorization: Bearer letsdoit" \
-H "Content-Type: application/json" \
"http://localhost:8000/api/v1/agents/tasks?scan_id=<scan-uuid>" | jq '.'

curl -X GET \
-H "Authorization: Bearer letsdoit" \
-H "Content-Type: application/json" \
"http://localhost:8000/api/v1/agents/tasks?agent_id=1" | jq '.'

Abstract Assigner Flow Logic

  1. Fetch Pending Scans:
    Retrieve all scans with scans.Status=scan_pending that are waiting for assignment.

  2. Fetch Available Agents:
    Identify agents with agents.Status=connected that are ready to process tasks.

  3. Assign scans to agents via Round-Robin Assignment:
    Each scan is assigned to an agent in sequential order.
    Once the last agent is assigned, the logic loops back to the first agent.
    Used this thinking it ensures even distribution without overloading any single agent and retries any new batch of scans comes in with periodic execution.

  4. Process Assignment
    Before assigning a scan, verify if it is already assigned to an agent in agentTasks to avoid duplication.

  5. Assign Tasks:
    Create a task linking the scan and the assigned agent in in agentTasks.

  6. Periodic Execution:
    The assigner runs at regular intervals to continuously check for new scans and available agents, ensuring no scan is left unprocessed.

Note:
This deosn't handle the logic of if scans have failed which are in scan.Status=error. Need to think on this

@ashwiniag ashwiniag changed the title assigner: implements task assignment for scans assigner: implements task assignment for scans and dummy mock test Jan 1, 2025
@ashwiniag ashwiniag changed the base branch from main to ag-server-enhancement January 1, 2025 13:22
@ashwiniag ashwiniag changed the base branch from ag-server-enhancement to ag-registration January 1, 2025 13:26
@ashwiniag ashwiniag merged commit e21becc into ag-registration Jan 3, 2025
1 check passed
ashwiniag added a commit that referenced this pull request Jan 3, 2025
… reports (#90)

* adds logic to register the agent and poll

* adds field scanner in policies

* adds field scanner in scans tables and updates test cases across

* adds foreign key integrationID to table scans and updates test cases

* adds field scanner in config yaml and supporting logic, adds enum for status fields

* adds field scanner in config yaml and supporting logic, adds enum for status fields

* updates logic to add an image:tag string in scans.image and integrationID in scans.integrationsID rendering its value using policy.image.registery which is equivalent to integrations.name

* fix executing of migration just once at the start of config loads

* restructure code for v0

* updates the get list call logic i.e orders by created_at ASC and adds its test case

* modularizing registries like dockerhub, gcr

* updates scan.Report logic to hold the json result of scanner

* modularizes the scanner tools

* modularizes the scanner tools

* some todo

* updates the get call to query by status

* updates the CRUD to use pointers as  certain fields are optional or conditionally updated

* agent implementation workflow

* assigner: implements task assignment for scans and dummy mock test (#95)

* updates the get call of scan to query by status

* updates the GET call of agents to query by status

* updates the GET call of agents to query by status or scan_id

* adds task assignment logic

* adds task assignment logic

* fix lints

* fix lints
ashwiniag added a commit that referenced this pull request Jan 6, 2025
* adds config parsing and feeds to database at server start

* removes redundency on config reloads

* agent: implements logic to register agents and scan images and upload reports (#90)

* adds logic to register the agent and poll

* adds field scanner in policies

* adds field scanner in scans tables and updates test cases across

* adds foreign key integrationID to table scans and updates test cases

* adds field scanner in config yaml and supporting logic, adds enum for status fields

* adds field scanner in config yaml and supporting logic, adds enum for status fields

* updates logic to add an image:tag string in scans.image and integrationID in scans.integrationsID rendering its value using policy.image.registery which is equivalent to integrations.name

* fix executing of migration just once at the start of config loads

* restructure code for v0

* updates the get list call logic i.e orders by created_at ASC and adds its test case

* modularizing registries like dockerhub, gcr

* updates scan.Report logic to hold the json result of scanner

* modularizes the scanner tools

* modularizes the scanner tools

* some todo

* updates the get call to query by status

* updates the CRUD to use pointers as  certain fields are optional or conditionally updated

* agent implementation workflow

* assigner: implements task assignment for scans and dummy mock test (#95)

* updates the get call of scan to query by status

* updates the GET call of agents to query by status

* updates the GET call of agents to query by status or scan_id

* adds task assignment logic

* adds task assignment logic

* fix lints

* fix lints
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant