Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/36 dag configuration translator #71

Open
wants to merge 32 commits into
base: main
Choose a base branch
from

Conversation

annabeckers
Copy link
Contributor

Converts our project and block model into a python file for airflow.
Also validates the DAG to be acyclic.

@annabeckers annabeckers marked this pull request as ready for review January 13, 2025 17:14
@annabeckers
Copy link
Contributor Author

We need to add an API route to be able to add a selected entrypoint to an block which is already added to a project. The DAG translator needs the block to have a selected entrypoint, but up to now we cannot add this after a block was added to a project.

Comment on lines 1 to 30
"""changes to project and block

Revision ID: d96594e39ae6
Revises: 449a1635be99
Create Date: 2025-01-08 02:05:50.343521

"""
from typing import Sequence, Union

from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision: str = 'd96594e39ae6'
down_revision: Union[str, None] = '449a1635be99'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty migration

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's look into the migrations together as soon as I find some time

core/conftest.py Show resolved Hide resolved
try:
project_uuid = project_controller.create_project(
data.name,
token_data.get("user_uuid")
data.name, data.user_uuid
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not extracting the user_uuid from the token?

Comment on lines 84 to 104
async def add_new_block(data: AddNewBlockRequest):
try:
project_controller.add_new_block(
data.project_uuid,
data.name,
data.priority_weight,
data.retries,
data.retry_delay,
data.custom_name,
data.description,
data.author,
data.docker_image,
data.repo_url,
# data.selected_entrypoint_uuid,
data.x_pos,
data.y_pos,
data.upstream_blocks,
data.downstream_blocks
)
except Exception as e:
raise handle_error(e)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As already written in the corresponding controller function & discussed on Slack. I suppose we do not need this endpoint. Do you agree? Or am I missing some use-cases here?

annabeckers and others added 5 commits January 21, 2025 15:39
Co-authored-by: Paul Kalhorn <67231903+PaulKalho@users.noreply.github.com>
Co-authored-by: Paul Kalhorn <67231903+PaulKalho@users.noreply.github.com>
Co-authored-by: Paul Kalhorn <67231903+PaulKalho@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants