Merge pull request #43 from LocationMind/update-script-omf-new-release

Update script omf new release

.gitignore

@@ -5,7 +5,6 @@ cache/
 tiles
 other.py
 .temp
-.vscode
 
 # ignore data dir
 Bash/**/data/**

.vscode/settings.json

@@ -0,0 +1,7 @@
+// .vscode/settings.json
+{
+    "[python]": {
+        "editor.defaultFormatter": "ms-python.black-formatter",
+        "editor.formatOnSave": true,
+    },
+}

Data/bboxs.json

@@ -27,6 +27,10 @@
         {
             "bbox":"2.289261,48.828241,2.395691,48.899046",
             "area":"Paris"
+        },
+        {
+            "bbox":"46.62470434,24.56755775,46.78428180,24.70689399",
+            "area":"Riyadh"
         }
     ]
-}
+}

Documentation/user-doc.md

@@ -50,6 +50,8 @@ The database used in this repository is:
 - `user`: `postgres`
 - `password`: `postgres`
 
+These information can be changed by modifying the [.env](../.env) file directly.
+
 ### Common model
 
 To compare OSM and OMF data, a common model was created (for transportation data), mainly based on the OMF transportation model. The model is as follows:
@@ -106,7 +108,7 @@ CREATE SCHEMA IF NOT EXISTS omf;
 CREATE SCHEMA IF NOT EXISTS results;
 ```
 
-*Note*: It is sufficient to create the database, as in the [main.py](../Python/Assessment/main.py#L24), these SQL commands are executed (inside the `utils.initialisePostgreSQL(connection)` function).
+*Note*: It is sufficient to create the database, as in the [data_integration.py](../src/Assessment/data_integration.py#L24), these SQL commands are executed (inside the `utils.initialisePostgreSQL()` function).
 
 ## Python
 
@@ -214,7 +216,7 @@ In the Python files, the database connection can take one of three forms:
 
 - `initialiseDuckDB()`: Initializes DuckDB and connects it to a PostgreSQL database.
 
-These functions are defined in the [utils.py](../Python/Utils/utils.py) script.
+These functions are defined in the [utils.py](../src/Utils/utils.py) script.
 They require one argument, a path to a `.env` file. If no path is provided, the default path will be:
 
 ```python
@@ -242,45 +244,45 @@ The same environment file is used for the data integration, quality assessment,
 
 ## Download and Process Data
 
-Data corresponding to the chosen areas can be downloaded using the [main.py](../Python/Assessment/main.py) script.
+Data corresponding to the chosen areas can be downloaded using the [data_integration.py](../src/Assessment/data_integration.py) script.
 Certain attribute values in this file can be modified, such as:
 
-- [`createBoundingBoxTable`](../Python/Assessment/main.py#L19): If `True`, it will create the bounding box table, even if it has already been created. Defaults to `True`.
+- [`createBoundingBoxTable`](../src/Assessment/data_integration.py#L20): If `True`, it will create the bounding box table, even if it has already been created. Defaults to `True`.
 
-- [`skip<theme>Check`](../Python/Assessment/main.py#L57): If `True`, it will recreate all layers of each area for the specified theme (one of `Graph`, `Building`, or `Place`). Otherwise, layers will be created only if they have not been created yet. Defaults to `False`.
+- [`skip<theme>Check`](../src/Assessment/data_integration.py#L58): If `True`, it will recreate all layers of each area for the specified theme (one of `Graph`, `Building`, or `Place`). Otherwise, layers will be created only if they have not been created yet. Defaults to `False`.
 
-- [`ox.settings.overpass_settings`](../Python/Assessment/main.py#L38): This setting is used to limit OSM data to a specific date. The default date is `2024-06-07T23:59:59Z`. This date can be changed if desired, but it is preferable to select a date approximately the same as the one used by the `overturemaps.py` tool (the default date corresponds to the 2024-06-13-beta.1 release).
+- [`ox.settings.overpass_settings`](../src/Assessment/data_integration.py#L38): This setting is used to limit OSM data to a specific date. The default date is `2024-06-07T23:59:59Z`. This date can be changed if desired, but it is preferable to select a date approximately the same as the one used by the `overturemaps.py` tool (the default date corresponds to the 2024-06-13-beta.1 release).
 
 Normally, no additional changes should be necessary (such as altering template names for the layer, schema names, or the path to the bbox file, etc.).
 If changes are required, they should be made consistently across the different files.
 
 The script can be run with this command:
 
 ```cmd
-python Python\Assessment\main.py
+python Python\Assessment\data_integration.py
 ```
 
 Or directly by running the script in an IDE of choice.
 
 ## Quality Assessment Criteria
 
 Once the data has been downloaded, the necessary scripts can be run to assess the quality of the different layers.
-Currently, the criteria are focused on graph data. The Python script is [graph_analysis.py](../Python/Assessment/graph_analysis.py).
+Currently, the criteria are focused on graph data. The Python script is [quality_assessment.py](../src/Assessment/quality_assessment.py), that uses the function in the [quality.py](../src/Assessment/quality.py) script.
 
 Certain attribute values can be modified, such as:
 
-- [`fileName`](../Python/Assessment/graph_analysis.py#L943): The name of the markdown file that will be produced. Defaults to `Automatic_result.md`.
+- [`fileName`](../src/Assessment/quality_assessment.py#L20): The name of the markdown file that will be produced. Defaults to `Automatic_result.md`.
 
-- [`pathSave`](../Python/Assessment/graph_analysis.py#L945): The path to save the markdown file. If the folder does not exist, an error will be returned. Defaults to `./Data/Results/<fileName>`, where `<fileName>` is the previously mentioned variable.
+- [`pathSave`](../src/Assessment/quality_assessment.py#L22): The path to save the markdown file. If the folder does not exist, an error will be returned. Defaults to `./Data/Results/<fileName>`, where `<fileName>` is the previously mentioned variable.
 
-- [`bounding_box_table`](../Python/Assessment/graph_analysis.py#L975): The name of the bounding box table in the public schema of the database. Defaults to `bounding_box`.
+- [`bounding_box_table`](../src/Assessment/quality_assessment.py#L59): The name of the bounding box table in the public schema of the database. Defaults to `bounding_box`.
 
 Other variables can also be changed, but this is not recommended, especially since changes would need to be updated in the dashboard as well.
 
 As with the previous script, this script can be run with this command:
 
 ```cmd
-python Python\Assessment\graph_analysis.py
+python Python\Assessment\quality_assessment.py
 ```
 
 Or directly by running the script in an IDE of choice.
@@ -300,4 +302,4 @@ Then, open your browser and navigate to [http://127.0.0.1:8000](http://127.0.0.1
 
 ## Use the Application
 
-To use the application, refer to the `Help` panel within the application or consult the [help.md](../Python/GeoDataCompare/help.md) file directly.
+To use the application, refer to the `Help` panel within the application or consult the [help.md](../src/GeoDataCompare/help.md) file directly.

Readme.md

@@ -1,12 +1,12 @@
-# GeoDataCompare: Visualisation system to compare OpenStreetMap and Overture Maps Fundation data
+# GeoDataCompare: Visualisation system to compare OpenStreetMap and Overture Maps Foundation data
 
 |   |   |
 |:---:|:---:|
 | Main  | [![Test](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml/badge.svg?branch=main)](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml?query=branch%3Amain)  |
 | Dev  | [![Test](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml/badge.svg?branch=dev)](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml?query=branch%3Adev)  |
 | Last commit | [![Test](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml/badge.svg)](https://github.com/LocationMind/OSM_Overture_Works/actions/workflows/action.yml) |
 
-- [GeoDataCompare: Visualisation system to compare OpenStreetMap and Overture Maps Fundation data](#geodatacompare-visualisation-system-to-compare-openstreetmap-and-overture-maps-fundation-data)
+- [GeoDataCompare: Visualisation system to compare OpenStreetMap and Overture Maps Foundation data](#geodatacompare-visualisation-system-to-compare-openstreetmap-and-overture-maps-foundation-data)
 - [Documentation](#documentation)
 - [Install](#install)
   - [Necessary components](#necessary-components)
@@ -20,7 +20,7 @@
 - [Licenses](#licenses)
 - [Credits](#credits)
 
-[Overture Maps Fundation](https://overturemaps.org/) (OMF) released their first official release in July 2024, and as their schemas is structured and their data come from different sources, it is interesting to know what one can do with this data.
+[Overture Maps Foundation](https://overturemaps.org/) (OMF) released their first official release in July 2024, and as their schemas is structured and their data come from different sources, it is interesting to know what one can do with this data.
 To answer this question, a comparison between them and [OpenStreetMap](https://www.openstreetmap.org/) (OSM), probably the most famous Open Source dataset for geoinformation data has been made.
 
 This project provides Python scripts to download and integrate data into a PostgreSQL database under a common model and assess their quality according to specific criteria and a visualisation system based on Shiny for Python and LonBoard, two Python packages, to compare data with a DashBoard.
@@ -113,24 +113,24 @@ This file is used in all scripts, whether it is for the quality assessment or th
 
 ## Download data
 
-Use the [main.py](Python\Assessment\main.py) file to download and integrate OSM and OMF data:
+Use the [data_integration.py](Python\Assessment\data_integration.py) file to download and integrate OSM and OMF data:
 
 ```cmd
-python Python\Assessment\main.py
+python Python\Assessment\data_integration.py
 ```
 
 It uses the file containing the bounding box: [bboxs.json](./Data/bboxs.json).
 Refer to the [user documentation](./Documentation/user-doc.md#adding-areas) for more information about how to add new areas.
-You will also find information about how to configurate the [main.py](Python\Assessment\main.py) file for forcing the data download or preventing the bounding box table to be recreated again.
+You will also find information about how to configurate the [data_integration.py](src/Assessment/data_integration.py/) file for forcing the data download or preventing the bounding box table to be recreated again.
 
 ## Quality assessment
 
 This script needs to be run after the data integration process, but before running the DashBoard.
-It is contained in the [graph_analysis.py](./Python/Assessment/graph_analysis.py)
+It is contained in the [quality_assessment.py](./src/Assessment/quality_assessment.py)
 To run this script:
 
 ```cmd
-python Python\Assessment\graph_analysis.py
+python Python\Assessment\quality_assessment.py
 ```
 
 This script will create layers in the database for the visible ones and also a sumup result located in a folder `.temp` in the `Data` folder of the repository.
@@ -151,7 +151,7 @@ You can find more information about how to use the application in the `Help` sec
 
 # Licenses
 
-OpenStreetMap data is under the ([Open Data Commons Open Database License](https://opendatacommons.org/licenses/odbl/)), while Overture Maps Fundation might have different licenses (see [here](https://docs.overturemaps.org/attribution/)), but for buildings and transportation theme, as OSM data is used, it is under the same license.
+OpenStreetMap data is under the ([Open Data Commons Open Database License](https://opendatacommons.org/licenses/odbl/)), while Overture Maps Foundation might have different licenses (see [here](https://docs.overturemaps.org/attribution/)), but for buildings and transportation theme, as OSM data is used, it is under the same license.
 Place data of OvertureMap is under [Community Data License Agreement – Permissive – Version 2.0](https://cdla.dev/permissive-2-0/).
 
 The visualisation platform and this project in general is under the [MISSING LICENSE!](.License.md).

Requirements/requirements.in

@@ -3,10 +3,7 @@ duckdb==0.10.2
 geoalchemy2==0.15.1
 matplotlib==3.9.0
 osmnx==1.9.3
-# 0.7.0 = 2024-06-13-beta.1 release
-overturemaps==0.7.0
-# 0.5.0 = 2024-05-16-beta.0 release
-# overturemaps==0.5.0
+overturemaps==0.9.0
 psycopg2==2.9.9
 pytest==8.2.1
 shapely==2.0.4

Requirements/requirements.txt

@@ -120,6 +120,7 @@ numpy==1.26.4
     #   geopandas
     #   matplotlib
     #   osmnx
+    #   overturemaps
     #   pandas
     #   patsy
     #   pyarrow
@@ -130,7 +131,7 @@ numpy==1.26.4
     #   statsmodels
 osmnx==1.9.3
     # via -r Requirements\requirements.in
-overturemaps==0.7.0
+overturemaps==0.9.0
     # via -r Requirements\requirements.in
 packaging==24.1
     # via