OpenQuake Engine 3.24.1

[Paul Henshaw (@pslh), Paolo Tormene (@ptormene)]

• Force use of utf8 encoding when writing non-binary files.
• Update AreaSource demo to use UTF8 chars that are obviously not ASCII.
• Use utf-8-sig when reading so as to handle BOM

[Michele Simionato (@micheles)]

• Removed the BASE33489 hack and made the first branch of the source
  model logic tree regular, with a single letter abbreviation

[Paolo Tormene (@ptormene)]

• When running a job via the WebUI it is possible to select multiple input
  files or a single .zip file
• Added a whitelist of resources that can be extracted from the datastore
  via the '/v1/calc/<calc_id>/extract/' and the
  '/v1/calc/<calc_id>/extract_html_table/' API endpoints. All
  resources can still be extracted if the WebUI authentication is disabled
  or if the current user has a high profile level (>1) or has the permission
  'auth.can_view_'

[Michele Simionato (@micheles)]

• Reduced the number of tiles
• Optimized the management of cluster of sources (the array loop)
• Changed the ps_grid_spacing approximation to use average occurrence rates

[Claudia Mascandola (@mascandola)]

• Added class "LanzanoEtAl2019_RJB_OMO_NESS2" to lanzano_2019 gsim

[Michele Simionato (@micheles)]

• Reduced the memory consumption of multifault sources for the USA model
• Improved the parallelization of disagg_source in AELO calculations
• Fixed the OverflowError: Python integer out of bounds for uint32
  happening for extra-large calculations when saving source_info

[Guillermo Aldama-Bustos (@aldabus)]

• Fixed the unit of measure in the Douglas et al (2024) GMM and updated
  coefficients for the 3- and 5-branch models to fix an error

[Michele Simionato (@micheles), Kendra Johnson (@kejohnso)]

• Internal: parallelized the AELO mosaic tests and made it possible to pass
  the asce_version and vs30 parameters

[Michele Simionato (@micheles)]

• Extended the source_id-filtering feature to use .startswith
• Added a flag "aggregate_exposure" (false by default) to aggregate
  the assets with the same taxonomy on the same hazard site
• Optimized "building cmakers" and the RateMap in the case of clusters
  of sources (260x for the latest USA model)

[Christopher Brooks (@CB-quakemodel)]

• Added BSSA14 site term capability to Modifiable GMPE + unit tests

[Richard Styron (@cossatot)]

• Added method from_moment to the TaperedGRMFD class
• Fixed KiteSurface.get_middle_point to manage NaNs

[Michele Simionato (@micheles)]

• Doing the check on the export_dir only if --exports is passed
• Fixed a NameError: Missing liquefaction:LiqProb in gmf_data
  when using the TodorovicSilva2022NonParametric model
• gzipping the avg_losses reducing the disk space by half or so
• Added an extractor losses_by_location
• Filtering the assets around the rupture according to the maximum distance
  in scenario calculations

[Claudia Mascandola (@mascandola)]

• Added a Ramadan et al (2023) gsim

[Michele Simionato (@micheles)]

• Disabled the threads in PicklableInferenceSession, making it possible
  to use the TodorovicSilva2022NonParametric model in high concurrency
  situations
• Made it possible to merge pre_job.ini and job.ini in ShakeMap calculations
• Internal: changed Exposure.read_around to work in terms of h3 hexagons;
  changed the exposure.hdf5 accordingly
• Added a private endpoint /v1/calc/run_ini to run remote calculations
• Added a private endpoint /v1/calc/jobs_from_inis

[Myles Turner (@mylesturner99), Guillermo Aldama-Bustos (@aldabus)]

• Added GMM of Douglas et al. (2024)

[Michele Simionato (@micheles)]

• Added a parameter config.dbserver.cache to avoid repeating calculations
• Removed the --reuse-input flag, since it was not used (use --hc instead)
• Changed install.py to exit if any of the tools is not installed
• Fixed small negative damages due to numeric errors
• Fixed the MIMETYPE to application/x-zip when needed in the WebUI
• Removed the parameter site_effects since it is useless
• Changed num_cores to be equal to the pool size when explicit
• On Windows use as default num_cores
  num_threads/2 if num_threads > 8 else num_threads
• Extended the aggregation by site_id to multiple tags
• Fixed an extract bug in disagg-traditional with realizations
• Fixed a bug with the quantiles in event_based_damage
• Improved loggin the progress in event based risk
• Reduced slow tasks in the USA model
• Added output mce_default in AELO calculations
• Extended oq export_mean_disagg_by_src to multiple sites
• Extended oq export_mag_dst_eps_sig to multiple sites
• Extended aggexp_tags and aggrisk_tags to multi-tags

[Paolo Tormene (@ptormene)]

• The OQ-Impact workflow option 'Use finite rupture from the USGS' was
  replaced by two options 'Use ShakeMap fault rupture from the USGS'
  and 'Use finite-fault model from the USGS', making it possible to convert
  a wider set of ruptures from the USGS data
• A sanity check was added to make sure that the hypocenter is inside the
  surface of the rupture. Otherwise, it is relocated to the middle point of
  the surface and a warning indicates the original and translated
  coordinates.

[Christopher Brooks (@CB-quakemodel)]

• Fixed bug in ParkerEtAl2020 epistemic uncertainty scaling factors.

[Michele Simionato (@micheles)]

• Fixed indentation errors in nearly all secondary peril classes
• Reduced the data transfer in avg_losses (20x for Philippines),
  thus solving an out-of-memory error; also improved the performance
  (25% for Philippines) and reduced the memory occupation on the workers
• Print a warning if there are vs30 values distant more than 10% from
  DEFINED_FOR_REFERENCE_VELOCITY

[Anirudh Rao (@raoanirudh)]

• Fixed a bug when generating the aggregate curves for affectedpop and
  injured loss types, due to missing units

[Michele Simionato (@micheles)]

• Extended override_vs30 to multiple values
• Forbidden GMF correlation with more then max_sites_correl=1000
• Made minimum_intensity and minimum_magnitude mandatory with event based
  calculations with many sites

[Paolo Tormene (@ptormene)]

• When a job is deleted, also the corresponding calc_CALCID_tmp.hdf5 file is
  removed, if present

[Astha Poudel (@asthapoudel), Catarina Costa (@catarinaquintela)]

• Modified infrastructure risk to avoid calculating efficiency loss for
  large networks

[Paolo Tormene (@ptormene)]

• AELO: the management of warnings was made more robust

[Michele Simionato (@micheles)]

• OQImpact: reading the sites around the rupture and not around the hypocenter
• Raised an error if the ShakeMap does not contain SA(0.6) when required
• Fixed the multi-peril implementation (imt_by_lt)
• Raised an error for risk IMTs not covered by the secondary IMTs
• Optimized build_global_exposure

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

• OQ-Impact: fixed aggregation by ID_2, which was using indices instead
  of strings

[Marco Pagani (@mmpagani)]

• Storing also tau in the mean spectrum calculator

[Christopher Brooks (@CB-quakemodel)]

• Added ability to specify z1pt0 and z2pt5 as -999 within site model
  to inform (basin term considering) GMMs to use their own vs30 to
  z1pt0 or z2pt5 relationship to estimate these site parameters, in
  effect preventing additional adjustment to GMM basin terms based
  on unknown site information. QA tests (classical case_89) + unit
  tests (utils_infer_basin_params_test.py) added
• Added validation checks for core site params (lon, lat, vs30, z1pt0
  and z2pt5) when building the site collection within a calculation

[Paolo Tormene (@ptormene)]

• In the API endpoint listing the outputs for a job, the first available
  export_type is specified in each output url

[Michele Simionato (@micheles)]

• Added a new uncertainty abMaxMagAbsolute to the logic tree
• Internal: refactored lt.Branch, lt.BranchSet and lt.CompositeLogicTree
• Avoided out of memory in calculations starting from the global SES

[Lana Todorovic (@LanaTodorovic93)]

• Updated units of slope (in m/m) in site model.

[Michele Simionato (@micheles)]

• Improved the validation of area source polygons
• Internal: added method CompositeLogicTree.apply_all
• Fixed downloading of uncertainty.xml in OQImpact calculations

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

• Added a route /v1/calc/XXX/job_zip and a button to download the archive
  from the WebUI, visible only in OQImpact for level 2 users
• Fixed an error in oq plot examples

[Baran Güryuva (@BaranGuryuva), Christopher Brooks (@CB-quakemodel)]

• Implemented the EMME24 backbone GMC for crustal
  earthquakes (emme24.py)

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

• Added uncertainties for area geometry and seismogenic depths

[Michele Simionato (@micheles)]

• Added a check on duplicate branches in the source model logic tree
• Internal: added methods CompositeLogicTree.to_node and
  CompositeLogicTree.to_nrml

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

• Added extractor for 'damages-stats'

[Kendra Johnson (@kejohnso)]

• Updated the mosaic geometries for AELO year 4

[Matteo Nastasi (@nastasi-oq)]

• Allow free mapping between standalone apps and their url path prefix
  (regardless of their name)

[Claudia Mascandola (@mascandola)]

• Added GMPEs of ambraseys_1996 and sabetta_pugliese_1996

[Graeme Weatherill (@g-weatherill), Michele Simionato (@micheles)]

• Implemented the MacedoEtAl2019SInter and MacedoEtAl2019SSlab conditional
  GMMs

[Anirudh Rao (@raoanirudh)]

• In the stations parser, make sure to skip all "vertical" ('z' or 'Z') or
  "up" ('u' or 'U') components when extracting the max horizontal component

[Manuela Villani (@ManuelaVillani)]

• Implementation of ASCE design parameters Sd, Sm in the AELO engine

[Paolo Tormene (@ptormene)]

• Added to oq-impact the possibility to upload a rupture in csv format
• Added to oq-impact the possibility to choose between different ShakeMap
  versions for the given USGS identifier
• Added to oq-impact the possibility to edit the preset job description

[Michele Simionato (@micheles)]

• Internal: added utility get_ruptures(file_csv)
• Optimized memory consumption in conditioned GMFs calculations

[Anirudh Rao (@raoanirudh)]

• Sort observed imtls and imts and ensure all of the target imts go
  into the context maker for the target sites in the conditioned gmfs
  calculator - helps in debugging mode to prevent accidental
  reordering of the imts

[Michele Simionato (@micheles)]

• Managed better the low hazard case in AELO calculations
• Optimized get_taxidx, thus making reading the USA exposure 10x faster
• Internal: restored the view portfolio_damage
• Too many post_classical tasks were generated with an artificially large
  parameter max_sites_disagg
• Internal: added a flag --wfp to the script utils/build_global_exposure

[Paolo Tormene (@ptormene)]

• Added oq plot "sites?" and added to the AELO advanced outputs page a
  button to visualize the site of interest

[Michele Simionato (@micheles)]

• Implemented gsim_aliases as thunks
• Forced the consequence files to contain risk IDs and removed
  utils/fix_consequences, now useless
• OQImpact: now aggregating by ID_2 and not ID_1
• Removed the --config-file option from the oq engine command
• Fixed numeric error in high intensity hazard curves produced by
  multifault source by using 64 bit PoEs
• Removed the dependency from pyshp
• Improved the task distribution
• Exposed the output avg_gmf
• Internal: enhanced build_global_exposure to work at admin level 2
• Do not store the backarc parameter when not needed
• Fixed call to v1/calc/list breaking the DbServer
• Extended the exposure reader to accept .csv.gz files
• Fixed oq info calculators that was still displaying the obsolete and now
  removed ucerf calculators
• Fixed oq show rlz: when the gsim logic tree was not properly reduced
• Internal: turned multi_fault.BLOCKSIZE into a configurable parameter
  config.memory.max_multi_fault_ruptures
• Added command oq info apply_uncertainty
• Added an extractor losses_by_site
• Improved oq info sources and oq show relevant_sources:<IMT>
• Internal: accepted source_model_file in place of
  source_model_logic_tree_file
• Extended source_model.serialise_to_nrml to accept a mesh_spacing
  parameter

[Marco Pagani (@mmpagani)]

• Added a module to the modifiable GMPE computing site response according
  to Stewart et al. (2020) and Hashash et al. (2020).
• Added a model for AUS23 that replaces the fixed Vs30 ones currently in
  the engine

[Marco Pagani (@mmpagani), Valeria Cascone (@ValeriaCascone)]

• Added an get_width and get_length methods to Strasser et al. (2010) MSR
• Added methods get_length and get_width to the Thingbaijam et al. scaling
  relationship
• Fixed a bug in the Thingbaijam et al. (2017) MSR

[Michele Simionato (@micheles)]

• Added an input mmi_shapes_file for use in OQ Impact and then
  a JSON output /v1/calc/:ID/mmi_tags

[Christopher Brooks (@CB-quakemodel)]

• Added Boore and Atkinson (2008) site term to the ModifiableGMPE
  capabilities and as an input arg for the Atkinson and Macias (2009)
  GMM as required for the US 2023 model + added unit tests.

[Michele Simionato (@micheles)]

• Extended avg_gmf to the secondary perils

[Kendra Johnson (@kejohnso), Michele Simionato (@micheles)]

• Implemented splitting of multifault sources into faults when using
  disagg_by_src=true and use_rates=true

[Mara Mita (@MaraMita), Catarina Costa (@catarinaquintela)]

• Added new landslide models and replaced NewmarkDisplacement,
  GrantEtAl2016RockSlopeFailure, NowickiJessee2018Landslides

[Michele Simionato (@micheles)]

• Extended the output loss types to affectedpop and injured
• Fixed oq plot?hcurves: the disaggregation PoEs were plotted in linear
  scale and not in logarithmic scale
• Internal: introduced parameter config.memory.gmf_data_rows

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

• Fixed ArtetaEtAl2021SlabVs30 by adding backarc to the site parameters

[Manuela Villani (@ManuelaVillani), Michele Simionato (@micheles)]

• Extended the global_ses script to override several input parameters;
  extended oq show_attrs to work with any HDF5 file

[Christopher Brooks (@CB-quakemodel)]

• Fixed modified_gsim (valid.py) to work with more complex
  GMMs (e.g. NGAEast GMPE Tables) + added a unit test