Sorting logic in paginate and additional_data

[Mithmi]

Hello, I have a question regarding the paginate method, specifically the additional_data field, and the logic behind sorting their contents.

I have a logic that generates data for the faceted filters.

Final result is something like that:

{
  "items": [
    {
      "doc_id": "some_docutment"
    }
  ],
  "total": 9997,
  "page": 1,
  "size": 50,
  "pages": 200,
  "filters": {
    "year": {
      "2020": 2414,
      "2021": 2814,
      "2022": 2661,
      "2023": 2068,
      "2024": 40
    }
  }
}

The issue is that i cannot change the order of the "year" filter to make it into 2024->2020.

I tried to do simple things like turning my keys into integers and sorting them after that, but apparently some logic in the paginate function still turns them into strings (i suspect pydantic library somewhere down the line is responsible for such behaviour).

Is there some parameter that would allow me to sort those keys, or is there an example of how to deal with this problem with a crutch?
It's also, for some reason, the only filter that is not sorted by value in descending order.

My Page class modified in this way:

from fastapi_pagination.customization import CustomizedPage, UseModelConfig

Page = CustomizedPage[
    Page,
    UseModelConfig(extra="allow"),
]

And my pagination function looks like this (DocumentWithMeta is my pydantic class for validating Documents):

@router.post(
    "/documents", 
    response_model=Page[DocumentWithMeta],
)
async def get_documents(
    current_user: Annotated[User, Depends(get_current_active_user)], 
    request: Request, 
    params: Params = Depends(),
) -> paginate:
    documents = request.app.documents[unique_key]
    filters = calc_filters_function()
    return paginate(
        documents,
        params,
        additional_data={
            "filters": filters
        }
    )

[uriyyo]

Hi @Mithmi,

Interesting case. I guess you can try to define custom Page with additional filters field:

from typing import TypeVar, Generic, Optional, Dict

from pydantic import BaseModel

from fastapi_pagination import Page as BasePage

T = TypeVar("T")
F = TypeVar("F")


class Page(BasePage[T], Generic[T, F]):
    filters: Optional[F] = None


class DocumentsFilter(BaseModel):
    year: Dict[int, int]

Usage example:

from fastapi_pagination import set_page, set_params, Params, paginate
from typing import Any

with set_page(Page[Any, DocumentsFilter]), set_params(Params()):
    page = paginate(
        [],
        additional_data={
            "filters": {
                "year": {
                    "2020": 2414,
                    "2021": 2814,
                    "2022": 2661,
                    "2023": 2068,
                    "2024": 40
                }
            }
        }
    )

print(page.model_dump_json(indent=4))

{
    "items": [],
    "total": 0,
    "page": 1,
    "size": 50,
    "pages": 0,
    "filters": {
        "year": {
            "2020": 2414,
            "2021": 2814,
            "2022": 2661,
            "2023": 2068,
            "2024": 40
        }
    }
}

In your case you will need to create filter class and update route definition:

@router.post(
    "/documents", 
    response_model=Page[DocumentWithMeta, DocumentFilter],
)
async def get_documents(
    current_user: Annotated[User, Depends(get_current_active_user)], 
    request: Request, 
    params: Params = Depends(),
) -> paginate:
    documents = request.app.documents[unique_key]
    filters = calc_filters_function()
    return paginate(
        documents,
        params,
        additional_data={
            "filters": filters
        }
    )

[Mithmi]

Hello, thank you for your reply.
Strangely enough, it doesn't seem to be working in my case.

The code itself works.
If i add sorting on top of it, it will return the expected results, but it doesn't render as such in the swagger generated by the FastApi.

I printed the page object generated by your proposed code:

items=[] total=0 page=1 size=50 pages=0 filters=DocumentsFilter(year={2024: 40, 2023: 2068, 2022: 2661, 2021: 2814, 2020: 2414})

Then i printed mine:

items=[] total=9997 page=1 size=50 pages=200 filters=DocumentsFilter(year={2024: 40, 2023: 2068, 2022: 2661, 2021: 2814, 2020: 2414})

So far, it's correct.

reverse=False - also correct:

items=[] total=9997 page=1 size=50 pages=200 filters=DocumentsFilter(year={2020: 2414, 2021: 2814, 2022: 2661, 2023: 2068, 2024: 40})

However, the swagger looks like this regardless of the sorting method:

I even tried to add the field_validator that would sort values in the required ways to the DocumentFilter model, it works for results, but not swagger render.

I expected that by defining response_model in addition to validating it with pydantic, I would also define swagger render results. Am I misunderstanding this?

[uriyyo]

I guess it's Swagger UI issue.

Here is simple application:

from typing import TypeVar, Generic, Optional, Dict

from fastapi import FastAPI, Query
from pydantic import BaseModel
from typing import Any

from fastapi_pagination import Page as BasePage, add_pagination, paginate

T = TypeVar("T")
F = TypeVar("F")


class Page(BasePage[T], Generic[T, F]):
    filters: Optional[F] = None


class DocumentsFilter(BaseModel):
    year: Dict[int, int]


app = FastAPI()
add_pagination(app)


@app.get("/documents")
async def route(
        reverse: bool = Query(False),
) -> Page[Any, DocumentsFilter]:
    data = [
        (2020, 2414),
        (2021, 2814),
        (2022, 2661),
        (2023, 2068),
        (2024, 40),
    ]

    if reverse:
        data = data[::-1]

    return paginate(
        [],
        additional_data={
            "filters": {
                "year": dict(data),
            }
        }
    )


if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app)

Here are HTTP calls.

$ curl "http://localhost:8000/documents?reverse=false"
{"items":[],"total":0,"page":1,"size":50,"pages":0,"filters":{"year":{"2020":2414,"2021":2814,"2022":2661,"2023":2068,"2024":40}}}

$ curl "http://localhost:8000/documents?reverse=true"
{"items":[],"total":0,"page":1,"size":50,"pages":0,"filters":{"year":{"2024":40,"2023":2068,"2022":2661,"2021":2814,"2020":2414}}}

As you can see, response is in correct order. So I guess this is an issue with Swagger UI rendering mechanism.

[Mithmi]

Found interesting interaction of the print and the pprint with the result. I guess that's the main issue of such behavior. Thanks for help.

print

{'publication_year': {'2024': 40, '2023': 2068, '2020': 2414, '2022': 2661, '2021': 2814}}
{'items': [], 'total': 0, 'page': 1, 'size': 50, 'pages': 0, 'filters': {'year': {'2024': 40, '2023': 2068, '2022': 2661, '2021': 2814, '2020': 2414}}}
{'items': [], 'total': 0, 'page': 1, 'size': 50, 'pages': 0, 'filters': {'year': {'2020': 2414, '2021': 2814, '2022': 2661, '2023': 2068, '2024': 40}}}

pprint

{'publication_year': {'2020': 2414,
                      '2021': 2814,
                      '2022': 2661,
                      '2023': 2068,
                      '2024': 40}}

{'filters': {'year': {'2020': 2414,
                      '2021': 2814,
                      '2022': 2661,
                      '2023': 2068,
                      '2024': 40}},
 'items': [],
 'page': 1,
 'pages': 0,
 'size': 50,
 'total': 0}

{'filters': {'year': {'2020': 2414,
                      '2021': 2814,
                      '2022': 2661,
                      '2023': 2068,
                      '2024': 40}},
 'items': [],
 'page': 1,
 'pages': 0,
 'size': 50,
 'total': 0}