production release (#288)

* [pre-commit.ci] pre-commit autoupdate updates: - [github.com/commitizen-tools/commitizen: v3.29.0 → v3.29.1](commitizen-tools/commitizen@v3.29.0...v3.29.1) - [github.com/astral-sh/ruff-pre-commit: v0.6.7 → v0.6.8](astral-sh/ruff-pre-commit@v0.6.7...v0.6.8) * feat: added updated at field in task events table * fix: handle exception if project not found in db * feat(user-profile): pass watch on form props * fix(user-profile)-#209: able to submit if the password and confirm password not matched * fix(user-profile)-#208: validation issue * feat: added updated_at field when user request, lock, unlock task * feat: added updated_at in task details endpoint * fix: get superuser from user table instead of access token * feat: update file and folder structure and rename `UserProfile` to `Complete UserProfile` * refac: remove super user from AuthUser Schemas * feat: add Breadcrumb component * feat: replace user-profile layout component and and add new route called complete-profile * feat: added two new enums such as IMAGE_UPLOADED & IMAGE_PROCESSED * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(task-description): Remove BLOB middleware for processing result downloads * fix: issues reslove on updated date * feat: added change password based on old and new password * feat(individual-project): post date with other data on task lock/unlock * feat(task-description): show task locked date * feat(update-profile): add BasicDetails component * feat(update-profile): add Header component * feat(update-profile): add OrganizationDetail component * feat(update-profile): add Password component * feat(update-profile): add OtherDetails component * feat: add common css class main-content that gives the full screen height after reducing the nav bar height * feat(user-profile): add UpdateUserProfile component * feat: remove the route `user-profile` add `complete-profile` from the list of routes without navbar * feat(user-profile): update basic details * refactor: remove user data fetching on projects page * refactor(user-profile): revalidate user details data on basic details patch success * feat: store user details on local storage on fetch success * feat(user-profile): create user profile patch service * feat(user-profile): call user details api on userProfile of nav section * feat(user-profile): update organization details * feat(user-profile): update other details * feat(user-profile): update password * fix: typo * fix: minor style * fix: typo * fix(ser-profile): make patch request on profile data change * feat(user-profile): show error message * hot-fix(user-profile): check old & new password before verify the password * fix: added message if user do not sent old and new password * refac: exception raise on password check * feat(user-profile): redirect to complete profile page if the user not completed the signup * feat(user-profile): redirect to complete profile on google auth success * fix(user-profile): minor UI * fix(user-profile): navigate to complete profile only after the profile fetching is completed * fix: typo * used presigned url for map screenshot in projects endpoint * feat: added search features in projects read endpoit * fix(update-userProfile): update background color * feat(dashboard): show user role * feat: updated the task as completed after process from s3 * Feat: updated pagination on list projects (#264) * feat: updated pagination on list projects * refactor(project): make fetching query dyanamic so the querykey, params can be pass dynamically * refactor(project): update project list fetching service params * feat: add usePagination hook * feat: add Pagination component * refactor(dashboard-map-section): remove api call and get project list through props * feat(project-dashboard): implement pagination on project list * feat: update searchInput component * feat(project-dashboard): implement project search * feat(project-dashboard): make responsive header * fix: added new pydentic model class for read the single project --------- Co-authored-by: Sujit <sujitkarki703@gmail.com> * feat: update task state to reflect the start of image processing * fix: remove comment from the task update function * fix: adjust the state based on processing... * fix: refine the state of task processing * Feat: Updated the task as completed after process from S3 (#270) * feat: updated the task as completed after process from s3 * feat: update task state to reflect the start of image processing * fix: remove comment from the task update function * fix: adjust the state based on processing... * fix: refine the state of task processing * feat(projects): Show status on project card * feat(dashboard): update UI and pass other project details on project card props * feat: update the status of each projects on projects list endpoint * feat(dashboard): show slug on project card * feat: refine the status of tasks * feat(project-details): add new status on legend * feat(project-details): show skeleton on project data fetching * feat(project-details): fill different color for completed task and image uploaded task show go to task button * feat(project-details): show loader on table * feat(dashboard): make logs table responsive * feat: add Status chip component * feat(project-dashboard): show complete status on project card * feat: fill circle color code from properties on VectorLayerWithCluster component * feat(project-description): add different color code as per project status on properties of projects geojson * hotfix: Update project/id to project_id in S3 image processing logic * feat: reduce table height * feat(project-details): image processing status text update * fix(project-details): available tasks not showing on table behave the task not having state as available tasks * fix: added asyncio event loop to call async update_task_state in process_images_from_s3 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Feat: Show project status and task status on map and project cards (#272) * fix(update-userProfile): update background color * feat(dashboard): show user role * feat(projects): Show status on project card * feat(dashboard): update UI and pass other project details on project card props * feat(dashboard): show slug on project card * feat(project-details): add new status on legend * feat(project-details): show skeleton on project data fetching * feat(project-details): fill different color for completed task and image uploaded task show go to task button * feat(project-details): show loader on table * feat(dashboard): make logs table responsive * feat: add Status chip component * feat(project-dashboard): show complete status on project card * feat: fill circle color code from properties on VectorLayerWithCluster component * feat(project-description): add different color code as per project status on properties of projects geojson * feat: reduce table height * feat(project-details): image processing status text update * fix(project-details): available tasks not showing on table behave the task not having state as available tasks --------- Co-authored-by: Sujit <sujitkarki703@gmail.com> * fix: resolve RuntimeError for missing event loop in AnyIO worker thread * hot-fix: minor UI issue on chip * feat: added asgiref package to convert the async to sync function * feat(project-description): increase opacity of task completed tasks fill color * fix: completed task status count * fix: update ongoing_task_count to count tasks in specific states of projects * fix: optimize project status calculation logic & update ongoing status count. * fix(task-description): set height to update takeoff point button section * feat: added new assets download endpoint * fix: remove await from assets download endpoint * fix(task-description): set height to update takeoff point button section (#281) Co-authored-by: Sujit <sujitkarki703@gmail.com> * feat: added users authentication * feat(project-dashboard): udpdate row per page options * feat(project-dashboard): update map and project card style * feat(task-description): update assest information api endpoint * feat(individual-project): get assets list * feat(individual-project):fetch all assets and modify project details to insert each assets on respective task detail * Style: update rows per page and set project card column to 3 (#283) * fix(task-description): set height to update takeoff point button section * feat(project-dashboard): udpdate row per page options * feat(project-dashboard): update map and project card style --------- Co-authored-by: Sujit <sujitkarki703@gmail.com> * feat(individual-project):add loader * Feat: Updated endpoints(download-assets) to fetch task assets status. (#282) * feat: added new assets download endpoint * fix: remove await from assets download endpoint * feat: added users authentication * chore: update verison drone_flightplan to v0.3.1 * fix: resolve deprecated async pool initialization in psycopg * Feat: Update assets downloading endpoint on individual project (#284) * fix(task-description): set height to update takeoff point button section * feat(project-dashboard): udpdate row per page options * feat(project-dashboard): update map and project card style * feat(task-description): update assest information api endpoint * feat(individual-project): get assets list * feat(individual-project):fetch all assets and modify project details to insert each assets on respective task detail --------- Co-authored-by: Sujit <sujitkarki703@gmail.com> --------- Co-authored-by: Sujit <90745363+suzit-10@users.noreply.github.com> Co-authored-by: Sujit <sujitkarki703@gmail.com> * fix(project-details): add `/` on api endpoint * feat(project-description): call assets api only on contirbution tab is on view * fix: merge conflict * fix(backend): reslove merge conflict in pyproject.toml * fix(backend): returns values from calculations function * fix(backend): reslove merge conflict in pyproject.toml * fix: update the pdm lock file --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Niraj Adhikari <41701707+nrjadkry@users.noreply.github.com> Co-authored-by: Sujit <sujitkarki703@gmail.com> Co-authored-by: Niraj Adhikari <nrjadkry@gmail.com> Co-authored-by: Sujit <90745363+suzit-10@users.noreply.github.com>
hotosm · Oct 14, 2024 · 5c86a18 · 5c86a18
1 parent 5e8a37a
commit 5c86a18
Show file tree

Hide file tree

Showing 15 changed files with 354 additions and 262 deletions.
diff --git a/src/backend/app/db/database.py b/src/backend/app/db/database.py
@@ -9,7 +9,9 @@
 
 async def get_db_connection_pool() -> AsyncConnectionPool:
     """Get the connection pool for psycopg."""
-    return AsyncConnectionPool(conninfo=settings.DTM_DB_URL.unicode_string())
+    pool = AsyncConnectionPool(conninfo=settings.DTM_DB_URL.unicode_string())
+    await pool.open()  # Explicitly open the pool
+    return pool
 
 
 async def get_db(request: Request) -> AsyncGenerator[Connection, None]:

diff --git a/src/backend/app/main.py b/src/backend/app/main.py
@@ -7,14 +7,13 @@
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.templating import Jinja2Templates
 from fastapi.responses import RedirectResponse, JSONResponse
-
+from psycopg_pool import AsyncConnectionPool
 from app.config import settings
 from app.projects import project_routes
 from app.drones import drone_routes
 from app.waypoints import waypoint_routes
 from app.users import user_routes
 from app.tasks import task_routes
-from app.db.database import get_db_connection_pool
 
 
 root = os.path.dirname(os.path.abspath(__file__))
@@ -105,23 +104,19 @@ def get_application() -> FastAPI:
 
 
 @asynccontextmanager
-async def lifespan(
-    app: FastAPI,
-):
+async def lifespan(app: FastAPI):
     """FastAPI startup/shutdown event."""
     log.debug("Starting up FastAPI server.")
 
-    db_pool = await get_db_connection_pool()
-    await db_pool.open()
-    # Create a pooled db connection and make available in app state
-    # NOTE we can access 'request.app.state.db_pool' in endpoints
-    app.state.db_pool = db_pool
-
-    yield
+    async with AsyncConnectionPool(
+        conninfo=settings.DTM_DB_URL.unicode_string()
+    ) as db_pool:
+        # The pool is now used within the context manager
+        app.state.db_pool = db_pool
+        yield  # FastAPI will run the application here
 
-    # Shutdown events
+    # Pool will be closed automatically when the context manager exits
     log.debug("Shutting down FastAPI server.")
-    await app.state.db_pool.close()
 
 
 api = get_application()

diff --git a/src/backend/app/projects/project_deps.py b/src/backend/app/projects/project_deps.py
@@ -7,13 +7,35 @@
 from fastapi import Depends, HTTPException, Path, File, UploadFile
 from psycopg import Connection
 from geojson import FeatureCollection
-
+from psycopg.rows import dict_row
 from app.db import database
 from app.models.enums import HTTPStatus
 from app.projects.project_schemas import DbProject
 from app.utils import multipolygon_to_polygon
 
 
+async def get_tasks_by_project_id(project_id: UUID, db: Connection):
+    """Get tasks by project id."""
+    try:
+        async with db.cursor(row_factory=dict_row) as cur:
+            await cur.execute(
+                """SELECT id FROM tasks WHERE project_id = %(project_id)s""",
+                {"project_id": project_id},
+            )
+
+            data = await cur.fetchall()
+
+            if data is None:
+                raise HTTPException(
+                    status_code=HTTPStatus.FORBIDDEN,
+                    detail="No tasks found for this project.",
+                )
+            return data
+
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
 async def get_project_by_id(
     project_id: Annotated[
         UUID,
@@ -27,7 +49,7 @@ async def get_project_by_id(
     try:
         return await DbProject.one(db, project_id)
     except KeyError as e:
-        raise HTTPException(status_code=HTTPStatus.NOT_FOUND) from e
+        raise HTTPException(status_code=HTTPStatus.FORBIDDEN) from e
 
 
 async def geojson_upload(

diff --git a/src/backend/app/projects/project_routes.py b/src/backend/app/projects/project_routes.py
@@ -388,17 +388,34 @@ async def process_imagery(
 
 
 @router.get(
-    "/assets/{project_id}/{task_id}/",
+    "/assets/{project_id}/",
     tags=["Image Processing"],
-    response_model=project_schemas.AssetsInfo,
 )
 async def get_assets_info(
+    user_data: Annotated[AuthUser, Depends(login_required)],
+    db: Annotated[Connection, Depends(database.get_db)],
     project: Annotated[
         project_schemas.DbProject, Depends(project_deps.get_project_by_id)
     ],
-    task_id: uuid.UUID,
+    task_id: Optional[uuid.UUID] = None,
 ):
     """
-    Endpoint to get the number of images and the URL to download the assets for a given project and task.
+    Endpoint to get the number of images and the URL to download the assets
+    for a given project and task. If no task_id is provided, returns info
+    for all tasks associated with the project.
     """
-    return project_logic.get_project_info_from_s3(project.id, task_id)
+    if task_id is None:
+        # Fetch all tasks associated with the project
+        tasks = await project_deps.get_tasks_by_project_id(project.id, db)
+
+        results = []
+
+        for task in tasks:
+            task_info = project_logic.get_project_info_from_s3(
+                project.id, task.get("id")
+            )
+            results.append(task_info)
+
+        return results
+    else:
+        return project_logic.get_project_info_from_s3(project.id, task_id)
diff --git a/src/backend/app/projects/project_schemas.py b/src/backend/app/projects/project_schemas.py
@@ -2,7 +2,6 @@
 import uuid
 from typing import Annotated, Optional, List
 from datetime import datetime, date
-from app.projects import project_logic
 import geojson
 from loguru import logger as log
 from pydantic import BaseModel, computed_field, Field, model_validator, root_validator
@@ -147,24 +146,6 @@ class TaskOut(BaseModel):
     image_count: Optional[int] = None
     assets_url: Optional[str] = None
 
-    @model_validator(mode="after")
-    def set_assets_url(cls, values):
-        """Set image_url and image count before rendering the model."""
-        task_id = values.id
-        project_id = values.project_id
-
-        if task_id and project_id:
-            data = project_logic.get_project_info_from_s3(project_id, task_id)
-            if data:
-                return values.copy(
-                    update={
-                        "assets_url": data.assets_url,
-                        "image_count": data.image_count,
-                    }
-                )
-
-        return values
-
 
 class DbProject(BaseModel):
     """Project model for extracting from database."""
@@ -549,12 +530,12 @@ def calculate_status(cls, values):
         completed_task_count = values.completed_task_count
         total_task_count = values.total_task_count
 
-        if ongoing_task_count == 0:
+        if completed_task_count == 0 and ongoing_task_count == 0:
             values.status = "not-started"
-        elif ongoing_task_count > 0 and ongoing_task_count != completed_task_count:
-            values.status = "ongoing"
-        elif ongoing_task_count == total_task_count:
+        elif completed_task_count == total_task_count:
             values.status = "completed"
+        else:
+            values.status = "ongoing"
 
         return values
 

diff --git a/src/backend/app/tasks/task_routes.py b/src/backend/app/tasks/task_routes.py
@@ -96,7 +96,7 @@ async def get_task_stats(
             raw_sql = """
                 SELECT
                     COUNT(CASE WHEN te.state = 'REQUEST_FOR_MAPPING' THEN 1 END) AS request_logs,
-                    COUNT(CASE WHEN te.state = 'LOCKED_FOR_MAPPING' THEN 1 END) AS ongoing_tasks,
+                    COUNT(CASE WHEN te.state IN ('LOCKED_FOR_MAPPING', 'REQUEST_FOR_MAPPING', 'IMAGE_UPLOADED', 'UNFLYABLE_TASK') THEN 1 END) AS ongoing_tasks,
                     COUNT(CASE WHEN te.state = 'IMAGE_PROCESSED' THEN 1 END) AS completed_tasks,
                     COUNT(CASE WHEN te.state = 'UNFLYABLE_TASK' THEN 1 END) AS unflyable_tasks
                 FROM (