Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTML outputs for GTFS data. #112

Merged
merged 40 commits into from
Sep 15, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7d2d483
Add code for producing HTML outputs for GTFS data
CBROWN-ONS Aug 30, 2023
fc1d255
Update requirements.txt
CBROWN-ONS Aug 30, 2023
6d32925
fix: Update tests to reflect changes
CBROWN-ONS Aug 30, 2023
a26ad76
Merge branch 'gtfs-html-new' of https://github.com/datasciencecampus/…
CBROWN-ONS Aug 30, 2023
a71ccad
Added better file ext checking; update tests; added new tests
CBROWN-ONS Aug 31, 2023
2e7f5e1
chore: Clean up docstrings to comply with numpy formatting
CBROWN-ONS Sep 4, 2023
cbad501
chore: Add additional raises in docstrings
CBROWN-ONS Sep 4, 2023
c5f9b80
chore: Bring up to date with dev
Sep 11, 2023
9c3f1b4
chore: Breaking changes with dev resolved
Sep 11, 2023
d9da952
refactor: Several type utils -> _type_defence
Sep 11, 2023
7a728b7
fix: Undo accidental overwrite with review branch
Sep 11, 2023
5743d11
Merge pull request #123 from datasciencecampus/review-_type_defence
CBROWN-ONS Sep 11, 2023
b649429
Merge branch 'gtfs-html-new' of https://github.com/datasciencecampus/…
CBROWN-ONS Sep 12, 2023
7ce1b9e
test: _type_defence raises with single or multiple types
Sep 12, 2023
a0ef054
test: _type_defence pass on single & multiple values to types
Sep 12, 2023
2f46b63
Merge branch 'gtfs-html-new' of https://github.com/datasciencecampus/…
Sep 12, 2023
1ca03b0
Refactor save paths for plotting summaries (#124)
CBROWN-ONS Sep 12, 2023
d613772
feat: Tests for _check_column_in_df()
CBROWN-ONS Sep 13, 2023
c32af5d
fix: Added another stage of testing for set_up_report_dir() which imp…
CBROWN-ONS Sep 13, 2023
acaf557
Updated gtfs_utils.py and corresponding tests. Removed scheme param. …
CBROWN-ONS Sep 14, 2023
5c4b07d
Make TemplateHtml methods private; Update docstring
CBROWN-ONS Sep 14, 2023
bbd5830
Updated error message for replace_multiple=False and multiple placeho…
CBROWN-ONS Sep 14, 2023
ae17f6d
Updated test_report_utils.py to reflect private functions
CBROWN-ONS Sep 14, 2023
ce0113e
Updated _set_up_report_dir() to better suite check_parent_dir_exists(…
CBROWN-ONS Sep 14, 2023
2d6c957
Update tests in test_report_utils
CBROWN-ONS Sep 14, 2023
cf1ed87
Only .lower() orientation in one place, rather than 2
CBROWN-ONS Sep 14, 2023
151d357
Create small helper function to remove duplicate logic; added testing
CBROWN-ONS Sep 14, 2023
47a35e9
Move test to test__plot_summary_defences() as it is more appropriate
CBROWN-ONS Sep 14, 2023
291b3b5
Refactore validation.py to only use _plot_summary();Added additional …
CBROWN-ONS Sep 15, 2023
c1d415e
Update check_attribute() due to save error
CBROWN-ONS Sep 15, 2023
14442ff
Update defence.py
CBROWN-ONS Sep 15, 2023
8fed07d
Update defence.py for missing char
CBROWN-ONS Sep 15, 2023
3dfef83
chore: Run pre-commit against all files
Sep 15, 2023
c45e4ed
docs: Update private methods in docstring
Sep 15, 2023
2b6fd16
chore: Remove lint
Sep 15, 2023
77d8c6e
refactor: Param formatting in Error message
Sep 15, 2023
93545dc
Merge branch 'dev' into gtfs-html-new
CBROWN-ONS Sep 15, 2023
6adc245
refactor: Rm str replace carriage returns logic in test
Sep 15, 2023
5e07121
Merge branch 'gtfs-html-new' of https://github.com/datasciencecampus/…
Sep 15, 2023
b516ce5
chore: Remove filterwarning
Sep 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,18 @@
# moved blanket rules above specific exceptions for test fixtures
*.zip
*.pkl
# except test fixtures
*.html

# exception for test fixtures
!tests/data/newport-2023-06-13.osm.pbf
!tests/data/newport-20230613_gtfs.zip
!tests/data/gtfs/route_lookup.pkl
!tests/data/gtfs/report/html_template.html

# exception for html templates
!src/transport_performance/gtfs/report/html_templates/evaluation_template.html
!src/transport_performance/gtfs/report/html_templates/stops_template.html
!src/transport_performance/gtfs/report/html_templates/summary_template.html

### Project structure ###
data/*
Expand All @@ -35,7 +43,6 @@ outputs/*

*.Rproj

*.html
*.pdf
*.csv
*.rds
Expand Down
2 changes: 2 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,7 @@ cartopy
folium
mapclassify
seaborn
pretty_html_table
kaleido
numpy>=1.25.0 # test suite will fail if user installed lower than this
-e .
85 changes: 85 additions & 0 deletions src/transport_performance/gtfs/gtfs_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@
import geopandas as gpd
from shapely.geometry import box
from pyprojroot import here
import plotly.graph_objects as go
import pandas as pd

from transport_performance.utils.defence import (
_is_expected_filetype,
_check_list,
_type_defence,
)


Expand Down Expand Up @@ -60,3 +63,85 @@ def bbox_filter_gtfs(
print(f"Filtered feed written to {out_pth}.")

return None


# NOTE: Possibly move to a more generalised utils file
def convert_pandas_to_plotly(
df: pd.DataFrame, return_html: bool = False
) -> go.Figure:
"""Convert a pandas dataframe to a visual plotly figure.

Parameters
----------
df : pd.DataFrame
A pandas dataframe to convert to plotly
(single index only)
return_html : bool, optional
Whether or not to return the html element,
by default False

Returns
-------
go.Figure
A plotly figure containing the drawn dataframe

Raises
------
LookupError
An error raised if an invalid colour scheme is passed
TypeError
An error raised if the given pandas dataframe is MultiIndex

"""
# pre-defined colour schemes
schemes = {
"dsc": {
"header_fill": "#12436D",
"header_font_colour": "white",
"cell_fill": "#A285D1",
"cell_font_colour": "black",
"font_family": "sans-serif",
"line_colour": "black",
}
}
# defences
_type_defence(df, "df", pd.DataFrame)
_type_defence(return_html, "return_html", bool)
if isinstance(df.columns, pd.MultiIndex) or isinstance(
df.index, pd.MultiIndex
):
raise TypeError(
"Pandas dataframe must have a singular index, not MultiIndex. "
"This means that 'df.columns' or 'df.index' does not return a "
"MultiIndex."
)
# harcoding scheme for now. Could be changed to param if more are added
scheme = "dsc"
# create plotly df
fig = go.Figure(
data=go.Table(
header=dict(
values=df.columns.values,
fill_color=schemes[scheme]["header_fill"],
font=dict(
color=schemes[scheme]["header_font_colour"],
family=schemes[scheme]["font_family"],
),
line_color=schemes[scheme]["line_colour"],
),
cells=dict(
values=[df[col_name] for col_name in df.columns],
fill_color="#A285D1",
font=dict(
color=schemes[scheme]["cell_font_colour"],
family=schemes[scheme]["font_family"],
),
align="left",
line_color=schemes[scheme]["line_colour"],
),
)
)

if return_html:
return fig.to_html(full_html=False)
return fig
195 changes: 195 additions & 0 deletions src/transport_performance/gtfs/report/css_styles/styles.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
* {
margin: 0;
padding: 0;
box-sizing: border-box;
font-family: 'Poppins', sans-serif;
}

body {
min-height: 100vh;
}

a {
text-decoration: none;

}

li {
list-style: none;
}

h1,
h2 {
color: black;
}

h3 {
color: #999;
}

table {
overflow: hidden;
overflow-x: scroll;
max-width: 1500px;
display: block;
overflow-y: scroll;
max-height: 800px;
}

thead {
position: sticky;
top: -10px;
}

.btn {
background: #f05462;
color: white;
padding: 5px 10px;
text-align: center;
}

.btn:hover {
color: #f05462;
background: white;
padding: 3px 8px;
border: 2px solid #f05462;
}

.title {
display: flex;
align-items: center;
justify-content: space-around;
padding: 15px 10px;
border-bottom: 2px solid #999;
}

table {
padding: 10px;
}

th,
td {
text-align: left;
padding: 8px;
}

.side-menu {
position: fixed;
background: #28A197;
width: 20vw;
min-height: 100vh;
display: flex;
flex-direction: column;
}

.side-menu .side-menu-title {
height: 10vh;
display: flex;
align-items: center;
justify-content: center;
font-weight: bold
}

.side-menu li {
font-size: 24px;
padding: 10px 40px;
display: flex;
align-items: center;
}

.side-menu li:hover {
background: white;
color: #28A197;

}

.side-menu li:hover .option{
color: #28A197;
background-color: white;
}

.side-menu .option {
color: white;
}

.side-menu .option:hover{
color: #28A197;
background-color: white;
}

.container {
position: absolute;
right: 0;
width: 80vw;
height: 100vh;
background: #f1f1f1;
}

.container .content {
position: relative;
background: #f1f1f1;
padding-left: 20px;
padding-right: 20px;
}

.container .content .analysis-cont {
display: flex;
justify-content: space-around;
align-items: flex-start;
flex-wrap: wrap;
background-color: #f1f1f1;
padding: 20px;
}

.container .content .analysis-cont .summary div {
display: block;
}


.container .content .analysis-cont .summary dd {
font-weight: bold;
margin-inline-start: 0;
display: inline-block;
min-width: 130px;
}

.container .content .analysis-cont .summary dt {
display: inline-block;
}

.analysis-title {
font-weight: bold;
font-size: large;
margin-bottom: 10px;
}

hr {
border: 0;
clear:both;
display:block;
width: 100%;
background-color: black;
height: 5px;
}

.container .header {
position: fixed;
top: 0;
right: 0;
width: 80vw;
height: 10vh;
background: #801650;
display: flex;
align-items: center;
justify-content: center;
box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
z-index: 1;
}

.container .header .header-title {
display: flex;
align-items: center;
color: #F46A25;
font-weight: bold;
font-size: xx-large;
}
Loading