Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[4233] Add Cursor-based pagination in Project related GET REST APIs #4237

Merged
merged 1 commit into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,23 @@ Specifiers can contribute dedicated AQL services for this feature using implemen
- https://github.com/eclipse-sirius/sirius-web/issues/1047[#1047] [sirius-web] In the _Domain_ diagram, when using direct-edit on a relatin edge, the initial text now only includes the name of the relation (without the cardinality)
- https://github.com/eclipse-sirius/sirius-web/issues/4095[#4095] [tree] Add conditional tree item label element description
- https://github.com/eclipse-sirius/sirius-web/issues/4101[#4101] [tree] Add loop tree item label element description
- https://github.com/eclipse-sirius/sirius-web/issues/4233[#4233] [core] Add cursor-based pagination for Project related GET REST APIs.
New optional attributes are available on all Project related GET REST APIs returning a list of objects, allowing to paginate the data returned by those APIs.
The following APIs are concerned:
** getProjects (`GET /api/rest/projects`)
The new optional attributes are:
** `page[size]` specifies the maximum number of records that will be returned per page in the response
** `page[before]` specifies the URL of the page succeeding the page being requested
** `page[after]` specifies the URL of a page preceding the page being requested
If neither `page[before]` nor `page[after]` is specified, the first page is returned with the same number of records as specified in the `page[size]` query parameter.
If the `page[size]` parameter is not specified, then the default page size is used, which is 20.
Example:
** `http://my-sirius-web-server:8080/api/rest/projects?
page[after]=MTYxODg2MTQ5NjYzMnwyMDEwOWY0MC00ODI1LTQxNmEtODZmNi03NTA4YWM0MmEwMjE&
page[size]=3` will ask for the 3 projects following the one identified by the URL of the page succeeding the page being requested `MTYxODg2MTQ5NjYzMnwyMDEwOWY0MC00ODI1LTQxNmEtODZmNi03NTA4YWM0MmEwMjE`.
Note that you can retrieve the URL of a page in the response header of `GET /api/rest/projects`.
Note that you may need to encode special characters like `[`(by `%5B`) and `]` (by `%5D`) in your requests.


== v2025.1.0

Expand Down
6 changes: 3 additions & 3 deletions integration-tests/cypress/support/serverCommands.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (c) 2021, 2024 Obeo.
* Copyright (c) 2021, 2025 Obeo.
* This program and the accompanying materials
* are made available under the erms of the Eclipse Public License v2.0
* which accompanies this distribution, and is available at
Expand All @@ -16,9 +16,9 @@ const url = Cypress.env('baseAPIUrl') + '/api/graphql';

Cypress.Commands.add('deleteAllProjects', () => {
const getProjectsQuery = `
query getProjects($page: Int!, $limit: Int!) {
query getProjects($after: String, $before: String, $first: Int, $last: Int) {
viewer {
projects(page: $page, limit: $limit) {
projects(after: $after, before: $before, first: $first, last: $last) {
edges {
node {
id
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (c) 2024 Obeo.
* Copyright (c) 2024, 2025 Obeo.
* This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v2.0
* which accompanies this distribution, and is available at
Expand All @@ -16,6 +16,7 @@
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.UUID;
Expand All @@ -30,9 +31,12 @@
import org.eclipse.sirius.web.application.project.dto.RenameProjectSuccessPayload;
import org.eclipse.sirius.web.application.project.dto.RestProject;
import org.eclipse.sirius.web.application.project.services.api.IProjectApplicationService;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.KeysetScrollPosition;
import org.springframework.data.domain.ScrollPosition;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.util.MultiValueMap;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
Expand All @@ -41,7 +45,10 @@
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;
import org.springframework.web.util.UriComponents;

import graphql.relay.Relay;
import io.swagger.v3.oas.annotations.Operation;

/**
Expand All @@ -53,6 +60,8 @@
@RequestMapping("/api/rest/projects")
public class ProjectRestController {

private static final int DEFAULT_PAGE_SIZE = 20;

private static final OffsetDateTime DEFAULT_CREATED = Instant.EPOCH.atOffset(ZoneOffset.UTC);

private final IProjectApplicationService projectApplicationService;
Expand All @@ -63,12 +72,26 @@ public ProjectRestController(IProjectApplicationService projectApplicationServic

@Operation(description = "Get all projects.")
@GetMapping
public ResponseEntity<List<RestProject>> getProjects() {
var restProjects = this.projectApplicationService.findAll(PageRequest.of(0, 20))
public ResponseEntity<List<RestProject>> getProjects(@RequestParam(name = "page[size]") Optional<Integer> pageSize, @RequestParam(name = "page[after]") Optional<String> pageAfter, @RequestParam(name = "page[before]") Optional<String> pageBefore) {
final KeysetScrollPosition position;
if (pageAfter.isPresent() && pageBefore.isEmpty()) {
var cursorProjectId = new Relay().fromGlobalId(pageAfter.get()).getId();
position = ScrollPosition.forward(Map.of("id", cursorProjectId));
} else if (pageBefore.isPresent() && pageAfter.isEmpty()) {
var cursorProjectId = new Relay().fromGlobalId(pageBefore.get()).getId();
position = ScrollPosition.backward(Map.of("id", cursorProjectId));
} else if (pageBefore.isPresent() && pageAfter.isPresent()) {
position = ScrollPosition.keyset();
} else {
position = ScrollPosition.keyset();
}
int limit = pageSize.orElse(DEFAULT_PAGE_SIZE);
var window = this.projectApplicationService.findAll(position, limit);
var restProjects = window
.map(project -> new RestProject(project.id(), DEFAULT_CREATED, new Identified(project.id()), null, project.name()))
.toList();

return new ResponseEntity<>(restProjects, HttpStatus.OK);
var headers = this.handleLinkResponseHeader(restProjects, position, window.hasNext(), limit);
return new ResponseEntity<>(restProjects, headers, HttpStatus.OK);
}

@Operation(description = "Get project with the given id (projectId).")
Expand Down Expand Up @@ -130,4 +153,33 @@ public ResponseEntity<RestProject> deleteProject(@PathVariable UUID projectId) {

return new ResponseEntity<>(restProject, HttpStatus.OK);
}

private MultiValueMap<String, String> handleLinkResponseHeader(List<RestProject> projects, KeysetScrollPosition position, boolean hasNext, int limit) {
MultiValueMap<String, String> headers = new HttpHeaders();
int projectsSize = projects.size();
if (projectsSize > 0 && position.scrollsForward() && hasNext) {
var headerLink = this.createHeaderLink(projects, limit, "after", "next");
headers.add(HttpHeaders.LINK, headerLink);
} else if (projectsSize > 0 && position.scrollsBackward() && hasNext) {
var headerLink = this.createHeaderLink(projects, limit, "before", "prev");
headers.add(HttpHeaders.LINK, headerLink);
}
return headers;
}

private String createHeaderLink(List<RestProject> projects, int limit, String beforeOrAfterPage, String relationType) {
var header = new StringBuilder();
var lastProject = projects.get(Math.min(projects.size() - 1, limit - 1));
var cursorId = new Relay().toGlobalId("Project", lastProject.id().toString());
UriComponents uriComponents = ServletUriComponentsBuilder.fromCurrentRequestUri()
.queryParam("page[" + beforeOrAfterPage + "]", cursorId)
.queryParam("page[size]", limit)
.build();
header.append("<");
header.append(uriComponents.toUriString());
header.append(">; rel=\"");
header.append(relationType);
header.append("\"");
return header.toString();
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (c) 2024 Obeo.
* Copyright (c) 2024, 2025 Obeo.
* This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v2.0
* which accompanies this distribution, and is available at
Expand All @@ -12,16 +12,22 @@
*******************************************************************************/
package org.eclipse.sirius.web.application.project.controllers;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.stream.Collectors;

import org.eclipse.sirius.components.annotations.spring.graphql.QueryDataFetcher;
import org.eclipse.sirius.components.core.graphql.dto.PageInfoWithCount;
import org.eclipse.sirius.components.graphql.api.IDataFetcherWithFieldCoordinates;
import org.eclipse.sirius.web.application.project.dto.ProjectDTO;
import org.eclipse.sirius.web.application.project.services.api.IProjectApplicationService;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageRequest;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.Window;
import org.springframework.data.domain.KeysetScrollPosition;
import org.springframework.data.domain.ScrollPosition;

import graphql.relay.Connection;
import graphql.relay.ConnectionCursor;
Expand All @@ -40,9 +46,15 @@
@QueryDataFetcher(type = "Viewer", field = "projects")
public class ViewerProjectsDataFetcher implements IDataFetcherWithFieldCoordinates<Connection<ProjectDTO>> {

private static final String PAGE_ARGUMENT = "page";
private static final int DEFAULT_PAGE_SIZE = 20;

private static final String LIMIT_ARGUMENT = "limit";
private static final String FIRST_ARGUMENT = "first";

private static final String LAST_ARGUMENT = "last";

private static final String AFTER_ARGUMENT = "after";

private static final String BEFORE_ARGUMENT = "before";

private final IProjectApplicationService projectApplicationService;

Expand All @@ -52,24 +64,65 @@ public ViewerProjectsDataFetcher(IProjectApplicationService projectApplicationSe

@Override
public Connection<ProjectDTO> get(DataFetchingEnvironment environment) throws Exception {
int page = Optional.<Integer> ofNullable(environment.getArgument(PAGE_ARGUMENT))
.filter(pageArgument -> pageArgument > 0)
.orElse(0);
int limit = Optional.<Integer> ofNullable(environment.getArgument(LIMIT_ARGUMENT))
.filter(limitArgument -> limitArgument > 0)
.orElse(20);

var pageable = PageRequest.of(page, limit);
var projectPage = this.projectApplicationService.findAll(pageable);
return this.toConnection(projectPage);
Optional<Integer> first = Optional.<Integer> ofNullable(environment.getArgument(FIRST_ARGUMENT));
Optional<Integer> last = Optional.<Integer> ofNullable(environment.getArgument(LAST_ARGUMENT));
Optional<String> after = Optional.<String> ofNullable(environment.getArgument(AFTER_ARGUMENT));
Optional<String> before = Optional.<String> ofNullable(environment.getArgument(BEFORE_ARGUMENT));

final KeysetScrollPosition position;
final int limit;
if (after.isPresent() && before.isEmpty()) {
var projectId = after.get();
var cursorProjectId = new Relay().fromGlobalId(projectId).getId();
position = ScrollPosition.forward(Map.of("id", cursorProjectId));
if (last.isPresent()) {
limit = 0;
} else if (first.isPresent()) {
limit = first.get();
} else {
limit = DEFAULT_PAGE_SIZE;
}
} else if (before.isPresent() && after.isEmpty()) {
var projectId = before.get();
var cursorProjectId = new Relay().fromGlobalId(projectId).getId();
position = ScrollPosition.backward(Map.of("id", cursorProjectId));
if (first.isPresent()) {
limit = 0;
} else if (last.isPresent()) {
limit = last.get();
} else {
limit = DEFAULT_PAGE_SIZE;
}
} else if (before.isPresent() && after.isPresent()) {
position = ScrollPosition.keyset();
limit = 0;
} else {
position = ScrollPosition.keyset();
if (first.isPresent() && last.isPresent()) {
limit = 0;
} else if (first.isPresent()) {
limit = first.get();
} else if (last.isPresent()) {
limit = last.get();
} else {
limit = DEFAULT_PAGE_SIZE;
}
}

var projectPage = this.projectApplicationService.findAll(position, limit);
return this.toConnection(projectPage, position);
}

private Connection<ProjectDTO> toConnection(Page<ProjectDTO> projectPage) {
var edges = projectPage.stream().map(projectDTO -> {
private Connection<ProjectDTO> toConnection(Window<ProjectDTO> projectPage, KeysetScrollPosition position) {
List<Edge<ProjectDTO>> edges = projectPage.stream().map(projectDTO -> {
var globalId = new Relay().toGlobalId("Project", projectDTO.id().toString());
var cursor = new DefaultConnectionCursor(globalId);
return (Edge<ProjectDTO>) new DefaultEdge<>(projectDTO, cursor);
}).toList();
}).collect(Collectors.toCollection(ArrayList::new));

if (position.scrollsBackward()) {
Collections.reverse(edges);
}

ConnectionCursor startCursor = edges.stream().findFirst()
.map(Edge::getCursor)
Expand All @@ -78,7 +131,18 @@ private Connection<ProjectDTO> toConnection(Page<ProjectDTO> projectPage) {
if (!edges.isEmpty()) {
endCursor = edges.get(edges.size() - 1).getCursor();
}
var pageInfo = new PageInfoWithCount(startCursor, endCursor, projectPage.hasPrevious(), projectPage.hasNext(), projectPage.getTotalElements());

boolean hasNextPage = false;
boolean hasPreviousPage = false;
if (position.scrollsForward()) {
hasNextPage = projectPage.hasNext();
hasPreviousPage = projectPage.hasPrevious();
}
if (position.scrollsBackward()) {
hasNextPage = projectPage.hasNext();
hasPreviousPage = projectPage.hasPrevious();
}
var pageInfo = new PageInfoWithCount(startCursor, endCursor, hasPreviousPage, hasNextPage, projectPage.size());
return new DefaultConnection<>(edges, pageInfo);
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (c) 2024 Obeo.
* Copyright (c) 2024, 2025 Obeo.
* This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v2.0
* which accompanies this distribution, and is available at
Expand Down Expand Up @@ -28,14 +28,14 @@
import org.eclipse.sirius.web.application.project.services.api.IProjectApplicationService;
import org.eclipse.sirius.web.application.project.services.api.IProjectMapper;
import org.eclipse.sirius.web.domain.boundedcontexts.project.Project;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.Window;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.api.IProjectCreationService;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.api.IProjectDeletionService;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.api.IProjectSearchService;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.api.IProjectUpdateService;
import org.eclipse.sirius.web.domain.services.Failure;
import org.eclipse.sirius.web.domain.services.Success;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.KeysetScrollPosition;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

Expand Down Expand Up @@ -72,8 +72,9 @@ public Optional<ProjectDTO> findById(UUID projectId) {

@Override
@Transactional(readOnly = true)
public Page<ProjectDTO> findAll(Pageable pageable) {
return this.projectSearchService.findAll(pageable).map(this.projectMapper::toDTO);
public Window<ProjectDTO> findAll(KeysetScrollPosition position, int limit) {
var window = this.projectSearchService.findAll(position, limit);
return new Window<>(window.map(this.projectMapper::toDTO), window.hasPrevious());
}

@Override
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (c) 2024 Obeo.
* Copyright (c) 2024, 2025 Obeo.
* This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v2.0
* which accompanies this distribution, and is available at
Expand All @@ -20,8 +20,8 @@
import org.eclipse.sirius.web.application.project.dto.DeleteProjectInput;
import org.eclipse.sirius.web.application.project.dto.ProjectDTO;
import org.eclipse.sirius.web.application.project.dto.RenameProjectInput;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.eclipse.sirius.web.domain.boundedcontexts.project.services.Window;
import org.springframework.data.domain.KeysetScrollPosition;

/**
* Application services used to manipulate projects.
Expand All @@ -31,7 +31,7 @@
public interface IProjectApplicationService {
Optional<ProjectDTO> findById(UUID id);

Page<ProjectDTO> findAll(Pageable pageable);
Window<ProjectDTO> findAll(KeysetScrollPosition position, int limit);

IPayload createProject(CreateProjectInput input);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
extend type Viewer {
project(projectId: ID!): Project
projects(page: Int!, limit: Int!): ViewerProjectsConnection!
AxelRICHARD marked this conversation as resolved.
Show resolved Hide resolved
projects(after: String, before: String, first: Int, last: Int): ViewerProjectsConnection!
projectTemplates(page: Int!, limit: Int!): ViewerProjectTemplatesConnection!
}

Expand All @@ -11,6 +11,7 @@ type ViewerProjectsConnection {

type ViewerProjectsEdge {
node: Project!
cursor: String!
}

type Project {
Expand Down
Loading
Loading