Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMPORT][SYNTHESE] Sélection du JDD ligne par ligne #2943

Commits on Mar 8, 2024

  1. import schema

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    5a13925 View commit details
    Browse the repository at this point in the history
  2. import to synthese

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    accae28 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3cb2f60 View commit details
    Browse the repository at this point in the history
  4. import to occhab

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d7049df View commit details
    Browse the repository at this point in the history
  5. rework uuid checks

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    ceef3de View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c1090e7 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    160557b View commit details
    Browse the repository at this point in the history
  8. bump pandas 1.3.5 → 2.1.4

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    9867b32 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    2beea9a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    4c5c293 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    9df80b7 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    3886646 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    d6bdde0 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    c69baf8 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    4f80359 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    0a7818c View commit details
    Browse the repository at this point in the history
  17. fix(occhab): defaults nomenclature for 'ALL' orga

    Correctly handle the 'ALL' organism.
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    18aba42 View commit details
    Browse the repository at this point in the history
  18. chore(occhab): complete models

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    889156f View commit details
    Browse the repository at this point in the history
  19. fix(tests/imports): determinist ordering

    Add entity_source_pk_value column in test file for determinist ordering.
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    74b9e3e View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    64a03ac View commit details
    Browse the repository at this point in the history
  21. fix(imports): several checks corrections

    - check_dataset
    - check_duplicate_uuid
    - check_duplicate_source_pk
    - check_orphan_rows
    - check_mandatory_fields
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    20e0d94 View commit details
    Browse the repository at this point in the history
  22. feat(imports): add parent/child related checks

    - set_id_parent_from_destination
    - set_parent_line_no
    - check_no_parent_entity
    - check_erroneous_parent_entities
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    e0e376b View commit details
    Browse the repository at this point in the history
  23. feat(imports): add parent related error code

    - NO_PARENT_ENTITY
    - ERRONEOUS_PARENT_ENTITY
    
    Thus, set WARNING error level for ORPHAN_ROW errors
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    3e6c00f View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    756287c View commit details
    Browse the repository at this point in the history
  25. chore(imports): concat_dates takes columns

    instead of fields
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    8748af5 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    49ec338 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    c21020f View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    208c381 View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    f4e44a7 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    ba28a58 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    4502684 View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    98cd3ba View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    9a901ee View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    3a531fc View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    aea31f2 View commit details
    Browse the repository at this point in the history
  36. feat(back): list all imports

    Change route `get_import_list`to get all imports without specified
    destination
    
    Change `url_value_preprocessor` to allow no `destination` in endpoint
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    e4047ea View commit details
    Browse the repository at this point in the history
  37. test(back): get list all import

    Add new file for test for all import
    
    Add fixture to insert all destination available in TImports
    
    Create test to get list of all import
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6489172 View commit details
    Browse the repository at this point in the history
  38. refact: remove useless return

    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    25ecd67 View commit details
    Browse the repository at this point in the history
  39. fix(tests/import): fixtures

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    57b9559 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    c3908de View commit details
    Browse the repository at this point in the history
  41. fix(occhab): fix edit habitats

    Fix the persistence of an habitat after an edit canceled.
    
    Signed-off-by: Pierre-Narcisi <pierre.narcisi@mnhn.fr>
    Pierre-Narcisi authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    f5c3276 View commit details
    Browse the repository at this point in the history
  42. Fix(occhab) remove import error du to default value creating a non su…

    …bquery select in a select
    Pierre-Narcisi authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    f222089 View commit details
    Browse the repository at this point in the history
  43. Configuration menu
    Copy the full SHA
    990e72a View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    3cd69da View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    c037349 View commit details
    Browse the repository at this point in the history
  46. Configuration menu
    Copy the full SHA
    5860933 View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    56c34e3 View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    492d40c View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    8e39af4 View commit details
    Browse the repository at this point in the history
  50. feat(notifications): make notifs for import-done destination-generic

    Update DB and MAIL notifications for IMPORT-DONE to make them generic as respect to destination.
    
    Signed-off-by: VincentCauchois <vincent.cauchois@mnhn.fr>
    VincentCauchois authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    4b5ac9a View commit details
    Browse the repository at this point in the history
  51. Configuration menu
    Copy the full SHA
    66e6314 View commit details
    Browse the repository at this point in the history
  52. Configuration menu
    Copy the full SHA
    2fda2b6 View commit details
    Browse the repository at this point in the history
  53. fix(import): access to import report from several places

    Right routing from summary before import, after import done, from list of imports, in notif.
    
    Signed-off-by: Pierre-Narcisi <pierre.narcisi@mnhn.fr>
    Pierre-Narcisi authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    0aafeb5 View commit details
    Browse the repository at this point in the history
  54. fix(import): download file and delete import from imports list

    Set the destination in the import data-service so as to have correct calls to API routes.
    
    Signed-off-by: Pierre-Narcisi <pierre.narcisi@mnhn.fr>
    Pierre-Narcisi authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6100cb3 View commit details
    Browse the repository at this point in the history
  55. feat: remove specific id_source col

    Remove id_source column to list import
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    ad898b4 View commit details
    Browse the repository at this point in the history
  56. feat: add optional destination to getImportList

    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    f5d1910 View commit details
    Browse the repository at this point in the history
  57. feat: add select destination component

    Add destination component to filter imports on destination
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    3915d42 View commit details
    Browse the repository at this point in the history
  58. feat: add column destination to list import

    Add column "Destination" to list import (front)
    Make column sortable (backend)
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    b8fa0f7 View commit details
    Browse the repository at this point in the history
  59. test(back): add test on sort destination.code

    test changes apply to route get_import_list
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    112495f View commit details
    Browse the repository at this point in the history
  60. style: change ux filter destination and language

    Add language for international
    Add class bootstrap for ux
    
    Reviewed-by: andriac
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6782bee View commit details
    Browse the repository at this point in the history
  61. feat: fieldmapping synchro

    edelclaux authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    c4e1b52 View commit details
    Browse the repository at this point in the history
  62. fix backend test

    jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    2d9da5b View commit details
    Browse the repository at this point in the history
  63. feat: remove taxa_count and add stats in report

    Use data.statistics (no more data.taxa_account)
    Change the way to display statistic according to
    different stats from different entity
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    df89603 View commit details
    Browse the repository at this point in the history
  64. front: add upload to config i18n

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    379f338 View commit details
    Browse the repository at this point in the history
  65. feat: add btn "Upload" to synthese component

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6588df2 View commit details
    Browse the repository at this point in the history
  66. fix(style): html and css for occhab page

    -Remove position absolute
    - Add class container,row, col
    - Place Btn Download at bottom
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    293ee45 View commit details
    Browse the repository at this point in the history
  67. feat: add btn "Upload" to OCCHAB page

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    494cf2d View commit details
    Browse the repository at this point in the history
  68. style(front): apply prettier

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    105f9fc View commit details
    Browse the repository at this point in the history
  69. refact: use different way to use module_code

    Use ModuleService to get current_module.module_code
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    66050cd View commit details
    Browse the repository at this point in the history
  70. feat: add conditional display btn based on cruved

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6ab2255 View commit details
    Browse the repository at this point in the history
  71. fix: add comma for css

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    b22ef55 View commit details
    Browse the repository at this point in the history
  72. style(front): apply prettier

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d19d2a8 View commit details
    Browse the repository at this point in the history
  73. feat(front): add cruved Import srvc into occhab

    Check permission Import in order to hidde/display Import btn
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    5dd96d2 View commit details
    Browse the repository at this point in the history
  74. feat(front): add cruved Import srvc into synthese

    Check permission Import in order to hidde/display Import btn
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    612c00f View commit details
    Browse the repository at this point in the history
  75. Feat(destination) getting destination in modules

    And removed unused dataService
    
    Author commit cherry-picked: @Pierre-Narcisi
    (cherry picked from commit 234bd2c5f11ca76d2e71d276d3ba3573b23d5d83)
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    30ae245 View commit details
    Browse the repository at this point in the history
  76. Configuration menu
    Copy the full SHA
    96e6444 View commit details
    Browse the repository at this point in the history
  77. Configuration menu
    Copy the full SHA
    39a31a6 View commit details
    Browse the repository at this point in the history
  78. Configuration menu
    Copy the full SHA
    1951f1c View commit details
    Browse the repository at this point in the history
  79. feat: group field by entity in report

    Call getBibField
    Match Import Data fields with response from getBibField
    Create function to match object
    
    Add one level of directive ngFor to create table as many as entity exists
    Use existing type in match object function
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    e65268e View commit details
    Browse the repository at this point in the history
  80. Configuration menu
    Copy the full SHA
    b751ffe View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    b3d632f View commit details
    Browse the repository at this point in the history
  82. Configuration menu
    Copy the full SHA
    dfc46ab View commit details
    Browse the repository at this point in the history
  83. feat(import): add counting statistics in import report

    Destination-specific stats.: taxa count for Synthese, habitats count and stations count for OccHab.
    
    Signed-off-by: VincentCauchois <vincent.cauchois@mnhn.fr>
    VincentCauchois authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    935472c View commit details
    Browse the repository at this point in the history
  84. import: geo checks fixes

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    ab79dc7 View commit details
    Browse the repository at this point in the history
  85. Configuration menu
    Copy the full SHA
    8530035 View commit details
    Browse the repository at this point in the history
  86. Configuration menu
    Copy the full SHA
    7720654 View commit details
    Browse the repository at this point in the history
  87. Configuration menu
    Copy the full SHA
    9856ac0 View commit details
    Browse the repository at this point in the history
  88. feat: add test for line number errors

    Change one csv to get column with line number
    Test return rows error are the same based on import from db and csv
    
    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d69b7cb View commit details
    Browse the repository at this point in the history
  89. import: simplify errors csv test

    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    4b102c1 View commit details
    Browse the repository at this point in the history
  90. Configuration menu
    Copy the full SHA
    d67c253 View commit details
    Browse the repository at this point in the history
  91. fix(import): entity & errors

    - small fix regarding report of entity in sql checks
    - replace unique constraint on (id_import, id_error, column_error) by two partial unique indexes:
      - (id_import, id_error, column_error) WHEN id_entity IS NULL
      - (id_import, id_entity, id_error, column_error) WHEN id_entity IS NOT NULL
    bouttier authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    8ab25ef View commit details
    Browse the repository at this point in the history
  92. Configuration menu
    Copy the full SHA
    66b1a11 View commit details
    Browse the repository at this point in the history
  93. Configuration menu
    Copy the full SHA
    cee819e View commit details
    Browse the repository at this point in the history
  94. Configuration menu
    Copy the full SHA
    f582f88 View commit details
    Browse the repository at this point in the history
  95. Configuration menu
    Copy the full SHA
    9382151 View commit details
    Browse the repository at this point in the history
  96. fix lint

    jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    36801e9 View commit details
    Browse the repository at this point in the history
  97. Configuration menu
    Copy the full SHA
    80c53a3 View commit details
    Browse the repository at this point in the history
  98. fix(import) fix design flaws

    Pierre-Narcisi authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    2658f5e View commit details
    Browse the repository at this point in the history
  99. fix lint

    jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    58106f5 View commit details
    Browse the repository at this point in the history
  100. Configuration menu
    Copy the full SHA
    d4a3178 View commit details
    Browse the repository at this point in the history
  101. fix: label stats in report pdf

    Reviewed-by: andriacap
    andriacap authored and jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    3b3c90a View commit details
    Browse the repository at this point in the history
  102. merge alembic branch

    jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    7bee951 View commit details
    Browse the repository at this point in the history
  103. fix rebase error

    jacquesfize committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d87e880 View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. feat(back):add unique_dataset_id to imprt synthese

    - Revision alembic to add column unique_dataset_id and src_unique_dataset_id
    - Add into bib_fields and into cor_entity_fields
    - change the model Synthese Geonature to add unique_dataset_id
    
    Reviewed-by: andriacap
    andriacap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    6dc944e View commit details
    Browse the repository at this point in the history
  2. fix: change type of "eng_label" and "desc_label"

    In database these two fields are nullable then
    change type to allow "None" value
    
    Reviewed-by: andriacap
    andriacap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    7acc901 View commit details
    Browse the repository at this point in the history
  3. feat: add check_dataset for synthese

    Reviewed-by: andriacap
    andriacap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    cbf51e4 View commit details
    Browse the repository at this point in the history
  4. fix(test): errors with unique_dataset_id

    Fix errors based on adding unique_dataset_id
    
    Reviewed-by: andriacap
    andriacap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    7a411f9 View commit details
    Browse the repository at this point in the history
  5. fix: change literal(impr.id_dataset)

     utiliser la colonne id_dataset de la table t_import_synthese plutôt que d’utiliser la valeur litéral imprt.id_dataset
    andriacap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    a254824 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. fix: change revision id (after rebase feat/import)

    Reviewed-by: andriacap
    andriacap committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    60ded8a View commit details
    Browse the repository at this point in the history
  2. refactor: id_dataset field in synthese

    Reviewed-by: andriac
    andriacap committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    42b0cf8 View commit details
    Browse the repository at this point in the history
  3. fix: nomenclature raise load error

    Reviewed-by: andriac
    andriacap committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    f88d45d View commit details
    Browse the repository at this point in the history
  4. refact: test synthese import jdd

    Based on Elie's review
    
    Reviewed-by: andriac
    andriacap committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    a2cb5a9 View commit details
    Browse the repository at this point in the history