From a54e0beabbb9c59d56c282e7dc39450adbdbf66e Mon Sep 17 00:00:00 2001 From: lgrd Date: Thu, 11 May 2023 17:15:32 +0200 Subject: [PATCH 01/12] [github] add templates for issues and PR (#60) --- .github/ISSUE_TEMPLATE/bug_report.md | 43 +++++++++++++++++++ .github/ISSUE_TEMPLATE/feature_request.md | 24 +++++++++++ .../pull_request_template.md | 21 +++++++++ 3 files changed, 88 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md create mode 100644 .github/PULL_REQUEST_TEMPLATE/pull_request_template.md diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..0f2968d --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,43 @@ +--- +name: Bug report +about: Create a report to help us improve this project +title: '' +labels: bug +assignees: '' + +--- + +### Bug + +#### Describe the bug +A clear and concise description of what the bug is. + +#### Type of use (choose between): +- [ ] `full generation` +- [ ] `pivot generation` +- [ ] `graph generation` +- [ ] `configuration generation` + +#### To Reproduce +Steps to reproduce the behavior: +1. Start a generation with configuration + +#### Expected behavior +A clear and concise description of what you expected to happen. + +#### Environment (please complete the following information): + - OS: [e.g. Debian] + - OS Version: [e.g. 11] + - Python version: [e.g. 3.10] + - R2GG commit: [sha] + +#### Configuration: +- the configuration was inside the docker image: [eg. yes/no] +- if no, please, provide us the configuration + +#### Data: +Type of data concerned by the generation : [eg. osrm, pgrouting, valhalla] +System used : [eg. official docker image of route-graph-generator, other] + +#### Additional context +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..6f305b4 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,24 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: '' +assignees: '' + +--- + +### Feature + +Please, make one issue per asked feature. + +#### Is your feature request related to a problem? Please describe. +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +#### Describe the solution you'd like +A clear and concise description of what you want to happen. + +#### Describe alternatives you've considered +A clear and concise description of any alternative solutions or features you've considered. + +#### Additional context +Add any other context or screenshots about the feature request here. \ No newline at end of file diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md new file mode 100644 index 0000000..3b2608d --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md @@ -0,0 +1,21 @@ +### Need + +Please, make one PR per feature or fix, and describe the reason of it. + +What is appreciated is to link this PR to an existing issue created with a template of this project. + +### Tasks + +- [ ] Code the feature on a new branch created from develop (see next paragraph) +- [ ] Update tests +- [ ] Run tests + - [ ] Inside the docker provided by the project + - [ ] Manually if necessary +- [ ] Update the documentation + - [ ] changelog.md + - [ ] documentation directory +- [ ] If possible, rebase your branch from develop + +### Code + +Please, describe what has been done to achieve this feature or fix. \ No newline at end of file From 7b96785a6b61a27870f6544b7901b5ec120fd484 Mon Sep 17 00:00:00 2001 From: lgrd Date: Tue, 23 May 2023 10:43:23 +0200 Subject: [PATCH 02/12] [doc] add a code of conduct (#61) --- CODE_OF_CONDUCT.md | 128 +++++++++++++++++++++++++++++++++++++++++++++ README.md | 4 ++ changelog.md | 3 ++ 3 files changed, 135 insertions(+) create mode 100644 CODE_OF_CONDUCT.md diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..4562593 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,128 @@ + +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, caste, color, religion, or sexual +identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people by being friendly, patient and careful in used words +* Being welcoming, respectful of differing opinions, viewpoints, and experiences in your speech and actions +* Attempting collaboration before conflict +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall + community +* Alerting community leaders if you notice a dangerous situation, someone in distress, or violations of this Code of Conduct, even if they seem inconsequential + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of + any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, + without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, +commits, code, wiki edits, issues, pull requests, discussions and other contributions +that are not aligned to this Code of Conduct, and will communicate reasons for +moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of community spaces are issues, pull request and discussions of +the Github project. +Examples of representing our community include using an official e-mail address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement at +report.road2@ign.fr or by a report on the Github project. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: If possible, a private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. This warning could be public otherwise. +A public apology may be requested. +When written, impacts may be reported and moderated. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of +actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or permanent +ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the +community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), +version [2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html). +Elements were taken from the Django Code of Conduct and the Citizen Code of Conduct. + + diff --git a/README.md b/README.md index 4e4ee06..8b54749 100644 --- a/README.md +++ b/README.md @@ -93,3 +93,7 @@ Elle est indiquée dans le `__init__.py` et dans le `setup.py`. ## Licence Route-graph-generator est diffusé sous la licence GPL v3. + +## Participer aux développements + +Les participations à ce projet sont encouragées (votre notre [charte](./CODE_OF_CONDUCT.md) à ce sujet). Il vous est demandé de réaliser vos développements en partant de la branche *develop*. diff --git a/changelog.md b/changelog.md index 40b5c91..4922273 100644 --- a/changelog.md +++ b/changelog.md @@ -7,6 +7,9 @@ CHANGED: - modification de la ci github pour prendre en compte la branche master - suppression du déploiement de github pages lors d'un tag +ADDED: +- A code of conduct was adapted from the contributor covenant + ## 2.2.0 FIX: From f3e70cfc4fb1764c1fa6fc205ab9c5bf8dd7ec41 Mon Sep 17 00:00:00 2001 From: lgrd Date: Mon, 5 Jun 2023 13:57:28 +0200 Subject: [PATCH 03/12] [doc] add a contributing and the DCO (#62) --- CONTRIBUTING.md | 217 +++++++++++++++++++++++++++++++ README.md | 2 +- changelog.md | 1 + docs/development/DCO.md | 34 +++++ docs/development/conduct.md | 2 + docs/development/contributing.md | 2 + docs/index.md | 2 + 7 files changed, 259 insertions(+), 1 deletion(-) create mode 100644 CONTRIBUTING.md create mode 100644 docs/development/DCO.md create mode 100644 docs/development/conduct.md create mode 100644 docs/development/contributing.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..cea0df6 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,217 @@ +# Welcome to our contributing guide + +Thank you for your interest and for contributing to this project ! + +Before contributing, please, read and adhere to : +- the [Code of Conduct](./CODE_OF_CONDUCT.md) to keep this community healthy, approachable and respectable; +- and the [Developer's Certificate of Origin](./documentation/developers/DCO.md) if you want to add code or documentation to this project. + +In this guide you will get an overview of different contribution workflows : +- Bug reports +- Feature requests +- No-Code contributions +- Code contributions +- Pull Requests + +Following these guidelines helps to communicate that you respect the time of the developers managing and developing this open source project. In return, they should reciprocate that respect in addressing your issue, assessing changes, and helping you finalize your pull requests. + +## Report bugs + +If you spot a bug in the code or in any other part of this project, you can create an issue. + +But before you open a new issue, please [search](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests#search-by-the-title-body-or-comments) for older ones that cover the same issue. If you find one, please, avoid "me too" comments. You can add a +1 emoji reaction to the issue if you want to express interest in this. + +When using the issue tracker, you will have an issue template for bugs. Don't worry if you can't answer every detail, just fill in what you can. + +When possible, try to recreate the bug inside a docker container and give us your instructions to do it. Clear and easily reproducible instructions are your best gift for us <3. + +## Request new features + +Feature requests are welcome. + +If you would love to have something new in this projet, or if you want to code it yourself, please : +- take a look to our [roadmap](https://github.com/orgs/IGNF/projects/3/views/1), +- search for existing issues about this feature, +- if there is no issue for your need, please open an issue asking for feedback. It is really important before starting any implementation. Otherwise, you risk spending a lot of time working on something that the the project's developers might not want to merge into the project. + +When using the issue tracker, you will have an issue template for features. Please, fill in what you can. It will be an area of discussions between you and the project's developers. + +But take a moment to find out whether your idea fits within the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Please provide as much detail and context as possible. + +## No-Code contributions + +Improving documentation or writing tutorials are all examples of helpful contributions that make us thankful. It is a good way to discover the project and its community. + +## Code contributions + +We really appreciate the community picking up and fixing bugs or even implementing new features. As said before, following these guidelines helps to communicate that you respect the time of the developers managing and developing this open source project. In return, we should reciprocate that respect in addressing your issue, assessing changes, and helping you finalize your pull requests. + +## Pull Requests + +### Setting up your local environment + +To get started, you will need to have `git`, `docker` and `docker-compose` installed locally. The last two are used for tests. + +#### Step 1 : Fork + +[Fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the [project](https://github.com/IGNF/route-graph-generator) on GitHub and clone your fork locally : + +``` +git clone git@github.com:username/route-graph-generator.git +cd route-graph-generator +git remote add upstream https://github.com/IGNF/route-graph-generator.git +git pull upstream +``` + +Don't forget to configure `git` so that it knows who you are : +``` +git config user.name "Random User" +git config user.email "random.user@example.com" +``` + +You can use any name/email address you prefer here. We only use the metadata generated by git using this configuration for properly attributing your changes to you in the AUTHORS file and the changelog. + +If you would like for the GitHub UI to link the commit to your account and award you the Contributor label after the changes have been merged, make sure this local email is also added to your GitHub email list. + +#### Step 2 : Branch + +As a best practice to keep your development environment as organized as possible, create local branches to work within. When you create a branch, do it directly off of the upstream **develop** branch. + +``` +git checkout upstream develop +git checkout -b my-branch +``` + +Please, create one branch (so one Pull Request) per feature or fix. + +##### Branch name guidelines + +The branch name depends on the contribution you want to do : +- `doc/*` for modifying or adding docs only, +- `feat/*` for coding new functionnalities, +- `fix/*` for fixing the code, +- `docker/*` for modifying the docker part only, +- `test/*` for modifying or adding tests only, +- `ci/*` for modifying or adding Github CI only + +### The process of making changes + +#### Step 3 : Code + +Modifying route-graph-generator (r2gg) typically involves changes to one or more of a few places in the repository. + +- Python code contained in the `/src` directory, +- SQL code contained in the `/sql` directory, +- Documentation in `/docs` directory and in possibly the `/readme.md`, +- Tests within the `/tests` directory, +- Dockerfiles and configuration in `/docker` directory, +- the `/changelog.md` must be updated. + +##### Code guidelines + +Route-graph-generator is coded in Python 3. + +Let's dive into some specificities of this project. + +###### Comments + +Comment the code as much as possible to facilitate its understanding. + +###### Logs + +There are different levels of logs. You can use each of them. The easiest way is to take a look at what is currently being done in the code. + +###### SQL scripts + +A SQL script will only be added if the concerned format is widely used by a strong community. + +###### Format + +A format will only be added if the concerned format is widely used by a strong community. + +#### Step 4 : Commit + +It is a best practice to keep your changes as logically grouped as possible within individual commits. There is no limit to the number of commits any single pull request may have, and many contributors find it easier to review changes that are split across multiple commits. + +``` +git add my/changed/files +git commit -m "[type of change] explicit change" +``` + +Note that multiple commits often will get squashed when they are landed. + +#### Step 5 : Rebase + +As a best practice, once you have committed your changes, it is a good idea to use `git rebase` (not `git merge`) to synchronize your work with the main repository. + +``` +git checkout upstream develop +git pull upstream develop +git checkout my-branch +git rebase develop +``` + +This ensures that your working branch has the latest changes from the develop branch of the origin repository. Moreover, your Pull Request will be easier to merge. + +#### Step 6 : Test + +Bug fixes and features should always come with tests. + +There are not a lot of tests in this project. Help is needed by the way. ;) + +The `/tests` directory contains : +- functionnal tests : those tests must all be validated. To run them, you can see this [doc](./docs/development/testing.md). + +#### Step 7 : Push + +Once you are sure your commits are ready to go, with passing tests, begin the process of opening a pull request by pushing your working branch to your fork on GitHub. + +``` +git push origin my-branch +``` + +#### Step 8 : Opening the Pull Request + +From within GitHub, opening a new pull request will present you with a pull request template. Please, try to do your best at filling out the details, but feel free to skip parts if you're not sure what to put. + +Once opened, pull requests are usually reviewed within a few days. + +To get feedback on your proposed change even though it is not ready to land, use the `Convert to draft` option in the GitHub UI instead of the `wip` label. + +Breaking changes need to have a BREAKING prefix. + +#### Step 9 : Discuss and update + +You will probably get feedback or requests for changes to your pull request. This is a big part of the submission process so don't be discouraged! Some contributors may sign off on the pull request right away, others may have more detailed comments or feedback. This is a necessary part of the process in order to evaluate whether the proposed changes are correct and necessary. + +To make changes to an existing pull request, make the changes to your local branch, add a new commit with those changes, and push those to your fork. GitHub will automatically update the pull request. + +``` +git add my/changed/files +git commit +git push origin my-branch +``` + +If a git conflict arises, it is necessary to synchronize your branch with other changes that have landed upstream by using git `rebase`: + +``` +git checkout upstream develop +git pull upstream develop +git checkout my-branch +git rebase develop +git push --force origin my-branch +``` + +Important: The git push `--force` command is one of the few ways to delete history in `git`. It also complicates the review process, as it won't allow reviewers to get a quick glance on what changed. Before you use it, make sure you understand the risks. If in doubt, you can always ask for guidance in the pull request. + +There are a number of more advanced mechanisms for managing commits using `git rebase` that can be used, but are beyond the scope of this guide. + +All pull requests require "sign off" in order to land. Whenever a contributor reviews a pull request they may find specific details that they would like to see changed or fixed. These may be as simple as fixing a typo, or may involve substantive changes to the code you have written. While such requests are intended to be helpful, they may come across as abrupt or unhelpful, especially requests to change things that do not include concrete suggestions on how to change them. + +Try not to be discouraged. If you feel that a particular review is unfair, say so, or contact one of the other contributors in the project and seek their input. Often such comments are the result of the reviewer having only taken a short amount of time to review and are not ill-intended. Such issues can often be resolved with a bit of patience. That said, reviewers should be expected to be helpful in their feedback, and feedback that is simply vague, dismissive, and unhelpful is likely safe to ignore. + +#### Step 10 : Landing + +In order to land, a pull request needs to be reviewed and approved by at least two administrators and pass a CI (Continuous Integration) test run. + +When an administrator lands your pull request, GitHub might show the pull request as Closed at this point, but don't worry. If you look at the branch you raised your pull request against, you should see a commit with your name on it. Congratulations and thanks for your contribution! diff --git a/README.md b/README.md index 8b54749..c8a948e 100644 --- a/README.md +++ b/README.md @@ -96,4 +96,4 @@ Route-graph-generator est diffusé sous la licence GPL v3. ## Participer aux développements -Les participations à ce projet sont encouragées (votre notre [charte](./CODE_OF_CONDUCT.md) à ce sujet). Il vous est demandé de réaliser vos développements en partant de la branche *develop*. +Les participations à ce projet sont encouragées (votre notre [charte](./CODE_OF_CONDUCT.md) à ce sujet). Nous avons mis en place un [guide](./CONTRIBUTING.md) des contributions pour vous accompagner dans cette démarche. diff --git a/changelog.md b/changelog.md index 4922273..63b30c2 100644 --- a/changelog.md +++ b/changelog.md @@ -9,6 +9,7 @@ CHANGED: ADDED: - A code of conduct was adapted from the contributor covenant +- A contributing was added ## 2.2.0 diff --git a/docs/development/DCO.md b/docs/development/DCO.md new file mode 100644 index 0000000..49b8cb0 --- /dev/null +++ b/docs/development/DCO.md @@ -0,0 +1,34 @@ +Developer Certificate of Origin +Version 1.1 + +Copyright (C) 2004, 2006 The Linux Foundation and its contributors. + +Everyone is permitted to copy and distribute verbatim copies of this +license document, but changing it is not allowed. + + +Developer's Certificate of Origin 1.1 + +By making a contribution to this project, I certify that: + +(a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + +(b) The contribution is based upon previous work that, to the best + of my knowledge, is covered under an appropriate open source + license and I have the right under that license to submit that + work with modifications, whether created in whole or in part + by me, under the same open source license (unless I am + permitted to submit under a different license), as indicated + in the file; or + +(c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified + it. + +(d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including all + personal information I submit with it, including my sign-off) is + maintained indefinitely and may be redistributed consistent with + this project or the open source license(s) involved. diff --git a/docs/development/conduct.md b/docs/development/conduct.md new file mode 100644 index 0000000..cc6912b --- /dev/null +++ b/docs/development/conduct.md @@ -0,0 +1,2 @@ +```{include} ../../CODE_OF_CONDUCT.md +``` diff --git a/docs/development/contributing.md b/docs/development/contributing.md new file mode 100644 index 0000000..ef6daa8 --- /dev/null +++ b/docs/development/contributing.md @@ -0,0 +1,2 @@ +```{include} ../../CONTRIBUTING.md +``` diff --git a/docs/index.md b/docs/index.md index 0878c7c..4e9ef13 100644 --- a/docs/index.md +++ b/docs/index.md @@ -45,4 +45,6 @@ caption: Développement --- Documentation Changelog +Contribuer +Code de conduite ``` From faceefb25b3b3c687bf62b7299dd4295f7d54d04 Mon Sep 17 00:00:00 2001 From: "amaury.zarzelli" Date: Mon, 19 Jun 2023 15:08:25 +0200 Subject: [PATCH 04/12] fix(osrm): incoherent travel time between car-shortest and car-fastest --- changelog.md | 13 +++++++++---- r2gg/_lua_builder.py | 7 +++---- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/changelog.md b/changelog.md index 63b30c2..9c72cd1 100644 --- a/changelog.md +++ b/changelog.md @@ -1,6 +1,11 @@ # CHANGELOG -## 2.2.1 +## NEXT-VERSION + +FIX: +- Durée de parcours incohérente sur OSRM entre car-fastest et car-shortest + +## 2.2.1 CHANGED: - reference de la doc à la branche master @@ -8,7 +13,7 @@ CHANGED: - suppression du déploiement de github pages lors d'un tag ADDED: -- A code of conduct was adapted from the contributor covenant +- A code of conduct was adapted from the contributor covenant - A contributing was added ## 2.2.0 @@ -36,7 +41,7 @@ ADD: - Ajout support des schémas d'entrée pour la conversion vers la base pivot - Ajout d'un script pour la conversion d'une BDTOPO -UPDATE: +UPDATE: - Mise à jour version python en 3.10 dans l'image docker - Mise à jour dockerfile pour meilleur gestion des requirements - Mise à jour dockerfile pour utiliser la version bullseye de debian @@ -53,7 +58,7 @@ CHANGE: ADD: - Ajout du fichier licence -UPDATE: +UPDATE: - Mise à jour de la documentation des fichiers de génération - Mise à jour des fichiers de génération dans la partie docker (wkt, renommage de fichiers, contraintes sur valhalla) diff --git a/r2gg/_lua_builder.py b/r2gg/_lua_builder.py index 5ad8db3..5d6d94b 100644 --- a/r2gg/_lua_builder.py +++ b/r2gg/_lua_builder.py @@ -172,10 +172,9 @@ def _build_process_way(costs_config, output_cost): process_way_string += "\n" # durée - if output_cost["cost_type"] == 'duration': - process_way_string += "\n -- durée\n" - process_way_string += " result.duration = {}\n".format(compute_operations_string(output_cost["operations"])) - process_way_string += "\n" + process_way_string += "\n -- durée\n" + process_way_string += " result.duration = {}\n".format(compute_operations_string(output_cost["operations"])) + process_way_string += "\n" # gestion du sens direct process_way_string += " -- gestion du sens direct\n" From 61abe0233da329283b4f63e59581c15d1aac1d80 Mon Sep 17 00:00:00 2001 From: "amaury.zarzelli" Date: Tue, 18 Jul 2023 10:34:51 +0200 Subject: [PATCH 05/12] feat(bdtopo): restrict access to pedestrian ways according to bdtopo --- docker/config/costs_calculation.json | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docker/config/costs_calculation.json b/docker/config/costs_calculation.json index aa9207d..2cbc50a 100644 --- a/docker/config/costs_calculation.json +++ b/docker/config/costs_calculation.json @@ -55,8 +55,8 @@ { "name": "cost_m_pedestrian", "speed_value": 4, - "direct_conditions" : "nature~='Type autoroutier';nature~='Bretelle'", - "reverse_conditions" : "nature~='Type autoroutier';nature~='Bretelle'", + "direct_conditions" : "nature~='Type autoroutier';nature~='Bretelle';acces_pieton~='Restreint aux ayants droit'", + "reverse_conditions" : "nature~='Type autoroutier';nature~='Bretelle';acces_pieton~='Restreint aux ayants droit'", "turn_restrictions": false, "cost_type": "distance", "operations": [ @@ -66,8 +66,8 @@ { "name": "cost_s_pedestrian", "speed_value": 4, - "direct_conditions" : "nature~='Type autoroutier';nature~='Bretelle'", - "reverse_conditions" : "nature~='Type autoroutier';nature~='Bretelle'", + "direct_conditions" : "nature~='Type autoroutier';nature~='Bretelle';acces_pieton~='Restreint aux ayants droit'", + "reverse_conditions" : "nature~='Type autoroutier';nature~='Bretelle';acces_pieton~='Restreint aux ayants droit'", "turn_restrictions": false, "cost_type": "duration", "operations": [ From 90b9661c0639dc07d04b034770410f4db77145eb Mon Sep 17 00:00:00 2001 From: Amaury Zarzelli Date: Fri, 4 Aug 2023 14:33:09 +0200 Subject: [PATCH 06/12] fix(costs calculation): adding the acces_pieton variable --- docker/config/costs_calculation.json | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docker/config/costs_calculation.json b/docker/config/costs_calculation.json index 2cbc50a..7a1f7b7 100644 --- a/docker/config/costs_calculation.json +++ b/docker/config/costs_calculation.json @@ -20,6 +20,11 @@ "column_name": "direction", "mapping": "value" }, + { + "name": "acces_pieton", + "column_name": "acces_pieton", + "mapping": "value" + }, { "name": "urbain", "column_name": "urbain", From 346d02452218110184ac71bc82046cd8347ed94a Mon Sep 17 00:00:00 2001 From: "amaury.zarzelli" Date: Thu, 14 Sep 2023 16:48:35 +0200 Subject: [PATCH 07/12] fix(restrictions): wrong data types --- r2gg/_pivot_to_pgr.py | 10 +++++----- sql/bdtopo_v3.3.sql | 11 ++++++----- sql/bduni_convert.sql | 10 +++++----- 3 files changed, 16 insertions(+), 15 deletions(-) diff --git a/r2gg/_pivot_to_pgr.py b/r2gg/_pivot_to_pgr.py index 64934d6..644d1d2 100644 --- a/r2gg/_pivot_to_pgr.py +++ b/r2gg/_pivot_to_pgr.py @@ -71,11 +71,11 @@ def pivot_to_pgr(source, cost_calculation_file_path, connection_work, connection urbain boolean, acces_pieton text, nature_de_la_restriction text, - restriction_de_hauteur text, - restriction_de_poids_total text, - restriction_de_poids_par_essieu text, - restriction_de_largeur text, - restriction_de_longueur text, + restriction_de_hauteur double precision, + restriction_de_poids_total double precision, + restriction_de_poids_par_essieu double precision, + restriction_de_largeur double precision, + restriction_de_longueur double precision, matieres_dangereuses_interdites boolean, cpx_gestionnaire text, cpx_numero_route_europeenne text, diff --git a/sql/bdtopo_v3.3.sql b/sql/bdtopo_v3.3.sql index 1f001cf..51e2f5d 100644 --- a/sql/bdtopo_v3.3.sql +++ b/sql/bdtopo_v3.3.sql @@ -74,11 +74,11 @@ CREATE TABLE IF NOT EXISTS {output_schema}.edges ( urbain boolean, acces_pieton text, nature_de_la_restriction text, - restriction_de_hauteur text, - restriction_de_poids_total text, - restriction_de_poids_par_essieu text, - restriction_de_largeur text, - restriction_de_longueur text, + restriction_de_hauteur double precision, + restriction_de_poids_total double precision, + restriction_de_poids_par_essieu double precision, + restriction_de_largeur double precision, + restriction_de_longueur double precision, matieres_dangereuses_interdites boolean, cpx_gestionnaire text, cpx_numero_route_europeenne text, @@ -162,6 +162,7 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS t.sens_de_circulation as sens_de_circulation, (CASE WHEN t.vitesse_moyenne_vl=1 THEN 0 + WHEN t.vitesse_moyenne_vl IS NULL THEN 0 ELSE t.vitesse_moyenne_vl::integer END) as vitesse_moyenne_vl, diff --git a/sql/bduni_convert.sql b/sql/bduni_convert.sql index e75bda0..022c041 100644 --- a/sql/bduni_convert.sql +++ b/sql/bduni_convert.sql @@ -73,11 +73,11 @@ CREATE TABLE IF NOT EXISTS {output_schema}.edges ( urbain boolean, acces_pieton text, nature_de_la_restriction text, - restriction_de_hauteur text, - restriction_de_poids_total text, - restriction_de_poids_par_essieu text, - restriction_de_largeur text, - restriction_de_longueur text, + restriction_de_hauteur double precision, + restriction_de_poids_total double precision, + restriction_de_poids_par_essieu double precision, + restriction_de_largeur double precision, + restriction_de_longueur double precision, matieres_dangereuses_interdites boolean, cpx_gestionnaire text, cpx_numero_route_europeenne text, From a6888bdf360091fe1ccca47d105aa526b48f9977 Mon Sep 17 00:00:00 2001 From: "amaury.zarzelli" Date: Tue, 19 Sep 2023 15:45:22 +0200 Subject: [PATCH 08/12] fix(cost-calc): urbain can now be NULL in bduni --- sql/bdtopo_v3.3.sql | 5 ++++- sql/bduni_convert.sql | 6 +++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/sql/bdtopo_v3.3.sql b/sql/bdtopo_v3.3.sql index 51e2f5d..dfeb7c7 100644 --- a/sql/bdtopo_v3.3.sql +++ b/sql/bdtopo_v3.3.sql @@ -190,7 +190,10 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS -- TODO: remove t.bande_cyclable as bande_cyclable, t.reserve_aux_bus as reserve_aux_bus, - t.urbain as urbain, + (CASE + WHEN t.urbain IS NULL THEN 0 + ELSE t.urbain + END) as urbain, t.acces_pieton as acces_pieton, t.nature_de_la_restriction as nature_de_la_restriction, t.restriction_de_hauteur as restriction_de_hauteur, diff --git a/sql/bduni_convert.sql b/sql/bduni_convert.sql index 022c041..0751a15 100644 --- a/sql/bduni_convert.sql +++ b/sql/bduni_convert.sql @@ -161,6 +161,7 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS t.sens_de_circulation as sens_de_circulation, (CASE WHEN t.vitesse_moyenne_vl=1 THEN 0 + WHEN t.vitesse_moyenne_vl IS NULL THEN 0 ELSE t.vitesse_moyenne_vl::integer END) as vitesse_moyenne_vl, @@ -188,7 +189,10 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS -- TODO: remove t.bande_cyclable as bande_cyclable, t.reserve_aux_bus as reserve_aux_bus, - t.urbain as urbain, + (CASE + WHEN t.urbain IS NULL THEN 0 + ELSE t.urbain + END) as urbain, t.acces_pieton as acces_pieton, t.nature_de_la_restriction as nature_de_la_restriction, t.restriction_de_hauteur as restriction_de_hauteur, From faa97930cfcef8ec26950b93b4f9d78eb170c77c Mon Sep 17 00:00:00 2001 From: "amaury.zarzelli" Date: Thu, 21 Sep 2023 09:01:55 +0200 Subject: [PATCH 09/12] fix(sql): urbain 0 when null cast to boolean --- sql/bdtopo_v3.3.sql | 2 +- sql/bduni_convert.sql | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sql/bdtopo_v3.3.sql b/sql/bdtopo_v3.3.sql index dfeb7c7..e42af3b 100644 --- a/sql/bdtopo_v3.3.sql +++ b/sql/bdtopo_v3.3.sql @@ -191,7 +191,7 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS t.bande_cyclable as bande_cyclable, t.reserve_aux_bus as reserve_aux_bus, (CASE - WHEN t.urbain IS NULL THEN 0 + WHEN t.urbain IS NULL THEN 0::boolean ELSE t.urbain END) as urbain, t.acces_pieton as acces_pieton, diff --git a/sql/bduni_convert.sql b/sql/bduni_convert.sql index 0751a15..d20fd01 100644 --- a/sql/bduni_convert.sql +++ b/sql/bduni_convert.sql @@ -190,7 +190,7 @@ CREATE TEMP TABLE IF NOT EXISTS bduni_troncon AS t.bande_cyclable as bande_cyclable, t.reserve_aux_bus as reserve_aux_bus, (CASE - WHEN t.urbain IS NULL THEN 0 + WHEN t.urbain IS NULL THEN 0::boolean ELSE t.urbain END) as urbain, t.acces_pieton as acces_pieton, From 28c867ab749219bdc91ff0e58269c8e9f25afe03 Mon Sep 17 00:00:00 2001 From: XavDmz <30922860+XavDmz@users.noreply.github.com> Date: Thu, 18 Jan 2024 11:10:24 +0100 Subject: [PATCH 10/12] Update Dockerfile Update prime_server version references in Dockerfile to 0.7.1. --- docker/debian/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/debian/Dockerfile b/docker/debian/Dockerfile index 18ddda9..d039b06 100644 --- a/docker/debian/Dockerfile +++ b/docker/debian/Dockerfile @@ -55,7 +55,7 @@ COPY --from=build /home/osrm/osrm-backend-5.26.0/build/OSRM-0.1.1-Linux.tar.gz . RUN tar -xzvf OSRM-0.1.1-Linux.tar.gz && cd OSRM-0.1.1-Linux && cp -r bin/* /usr/bin/ && cp -r lib/* /usr/lib/ && cp -r include/* /usr/include/ && cp -r share/* /usr/share/ ### Installation prime-server -COPY --from=build /usr/local/lib/libprime_server.so.0.7.0 /usr/lib/libprime_server.so.0.0.0 +COPY --from=build /usr/local/lib/libprime_server.so.0.7.1 /usr/lib/libprime_server.so.0.0.0 COPY --from=build /usr/local/lib/libprime_server.so.0 /usr/lib/libprime_server.so.0 COPY --from=build /usr/local/lib/libprime_server.so /usr/lib/libprime_server.so From 3a5dc66ac5daef4690a6194ed82a2ba252194007 Mon Sep 17 00:00:00 2001 From: jmkerloch <53606373+jmkerloch@users.noreply.github.com> Date: Thu, 1 Feb 2024 17:53:46 +0100 Subject: [PATCH 11/12] Feat/apply vacuum to schema only (#69) * feat(vacuum): analyse only created pivot table * feat(vacuum): analyse only created pgr tables * feat(changelog): update for VACUUM ANALYSE --- changelog.md | 3 +++ r2gg/_pivot_to_pgr.py | 21 ++++++++++++++++++++- sql/bdtopo_v3.3.sql | 4 +++- sql/bduni_convert.sql | 4 +++- 4 files changed, 29 insertions(+), 3 deletions(-) diff --git a/changelog.md b/changelog.md index 9c72cd1..c2e2975 100644 --- a/changelog.md +++ b/changelog.md @@ -2,6 +2,9 @@ ## NEXT-VERSION +ADDED: +- VACUUM ANALYSE is done only on created tables + FIX: - Durée de parcours incohérente sur OSRM entre car-fastest et car-shortest diff --git a/r2gg/_pivot_to_pgr.py b/r2gg/_pivot_to_pgr.py index 644d1d2..ded60ff 100644 --- a/r2gg/_pivot_to_pgr.py +++ b/r2gg/_pivot_to_pgr.py @@ -356,12 +356,31 @@ def pivot_to_pgr(source, cost_calculation_file_path, connection_work, connection old_isolation_level = connection_out.isolation_level connection_out.set_isolation_level(0) - vacuum_query = "VACUUM ANALYZE;" + + # VACCUM ANALYZE for ways + vacuum_query = f"VACUUM ANALYZE {ways_table_name};" + logger.info("SQL: {}".format(vacuum_query)) + st_execute = time.time() + cursor_out.execute(vacuum_query) + et_execute = time.time() + logger.info("Execution ended. Elapsed time : %s seconds." %(et_execute - st_execute)) + + # VACCUM ANALYZE for ways_vertices_pgr + vacuum_query = f"VACUUM ANALYZE {ways_table_name}_vertices_pgr;" logger.info("SQL: {}".format(vacuum_query)) st_execute = time.time() cursor_out.execute(vacuum_query) et_execute = time.time() logger.info("Execution ended. Elapsed time : %s seconds." %(et_execute - st_execute)) + + # VACCUM ANALYZE for turn_restrictions + vacuum_query = f"VACUUM ANALYZE {schema}.turn_restrictions;" + logger.info("SQL: {}".format(vacuum_query)) + st_execute = time.time() + cursor_out.execute(vacuum_query) + et_execute = time.time() + logger.info("Execution ended. Elapsed time : %s seconds." %(et_execute - st_execute)) + connection_out.set_isolation_level(old_isolation_level) connection_out.commit() diff --git a/sql/bdtopo_v3.3.sql b/sql/bdtopo_v3.3.sql index e42af3b..8e4dd0e 100644 --- a/sql/bdtopo_v3.3.sql +++ b/sql/bdtopo_v3.3.sql @@ -347,4 +347,6 @@ DROP FOREIGN TABLE IF EXISTS {output_schema}.troncon_de_route CASCADE; DROP FOREIGN TABLE IF EXISTS {output_schema}.non_communication CASCADE; END TRANSACTION; -VACUUM ANALYZE; +VACUUM ANALYZE {output_schema}.non_comm; +VACUUM ANALYZE {output_schema}.edges; +VACUUM ANALYZE {output_schema}.nodes; diff --git a/sql/bduni_convert.sql b/sql/bduni_convert.sql index d20fd01..5e7f2dd 100644 --- a/sql/bduni_convert.sql +++ b/sql/bduni_convert.sql @@ -343,4 +343,6 @@ DROP FOREIGN TABLE IF EXISTS {output_schema}.troncon_de_route CASCADE; DROP FOREIGN TABLE IF EXISTS {output_schema}.non_communication CASCADE; END TRANSACTION; -VACUUM ANALYZE; +VACUUM ANALYZE {output_schema}.non_comm; +VACUUM ANALYZE {output_schema}.edges; +VACUUM ANALYZE {output_schema}.nodes; From a9fd9dd3f080e153d045b7f81e80e9a4816ef723 Mon Sep 17 00:00:00 2001 From: lgrd Date: Fri, 2 Feb 2024 10:33:23 +0100 Subject: [PATCH 12/12] doc : updates for a 2.2.2 (#70) * doc : update changelog for a 2.2.2 * doc : update version and readme for a 2.2.2 --- README.md | 2 +- changelog.md | 14 +++++++++----- r2gg/__about__.py | 2 +- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index c8a948e..d6a0ace 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ r2gg-road2config config.json ## Version -Elle est indiquée dans le `__init__.py` et dans le `setup.py`. +Elle est indiquée dans le `__about__.py`. ## Licence diff --git a/changelog.md b/changelog.md index c2e2975..7ad3c08 100644 --- a/changelog.md +++ b/changelog.md @@ -1,12 +1,19 @@ # CHANGELOG -## NEXT-VERSION +## 2.2.2 -ADDED: +ADD: - VACUUM ANALYSE is done only on created tables +- Templates for issues and PR +- A code of conduct was adapted from the contributor covenant +- A contributing was added +- The DCO was added +- Restrict access to pedestrian ways according to BDTOPO +- Better handling of urbain column inside the BDTOPO FIX: - Durée de parcours incohérente sur OSRM entre car-fastest et car-shortest +- Wrong data types for some restrictions of the BDTOPO ## 2.2.1 @@ -15,9 +22,6 @@ CHANGED: - modification de la ci github pour prendre en compte la branche master - suppression du déploiement de github pages lors d'un tag -ADDED: -- A code of conduct was adapted from the contributor covenant -- A contributing was added ## 2.2.0 diff --git a/r2gg/__about__.py b/r2gg/__about__.py index 91bbbf0..f2a2887 100644 --- a/r2gg/__about__.py +++ b/r2gg/__about__.py @@ -34,7 +34,7 @@ __uri_tracker__ = f"{__uri_repository__}issues/" __uri__ = __uri_repository__ -__version__ = "2.2.1" +__version__ = "2.2.2" __version_info__ = tuple( [ int(num) if num.isdigit() else num