-
Notifications
You must be signed in to change notification settings - Fork 5
Review - VITO / CDSE #95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5f28a6f
f5e6baf
09ed240
d2c2a84
d65f77f
dc85ebb
802683e
6d76324
4a23e47
c5a5cee
48beddb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,7 +66,7 @@ Go to the [**Open Science Catalog**](https://opensciencedata.esa.int/) to start | |
| ## Choosing a Platform | ||
| EarthCODE provides you *single‑sign‑on* access to a growing number of EO platforms that host EO data, development tools and compute. Once you log in with your EarthCODE account, you carry the same identity across the **EarthCODE Portal**, the **EarthCODE Forum**, each **integrated platform**, and the **Open Science Catalog**. | ||
|
|
||
| For all of the EarthCODE integrated platforms, you can apply for Network‑of‑Resources (NoR) sponsorship to cover compute costs. Every platform comes with FAIR tooling that lets you publish results to the Open Science Catalog in an automated manner. Additionally, some platforms support creating and executing several types of workflows, which means that if you develop a workflow on them, it will be cross-compatible with other platforms. | ||
| For all of the EarthCODE integrated platforms, you can apply for Network‑of‑Resources (NoR) sponsorship to cover compute costs. Every platform comes with FAIR tooling that lets you publish results to the Open Science Catalog in an automated manner. Additionally, some platforms support creating and executing several types of workflows, which means that if you develop a workflow on them, it will be cross-compatible with other platforms supporting the same type of workflow. | ||
|
|
||
|  | ||
|
|
||
|
|
@@ -86,12 +86,12 @@ Each platform provides a variety of EO data stored on the cloud and expose acces | |
| --- | ||
|
|
||
| ### Workflow Development | ||
| EarthCODE platforms differ in interface and how you build and run algorithms: some offer visual process designers, others provide Jupyter Notebooks with extensions, and several combine both. For example OpenEO provides both the OpenEO API and a graphical interface. Many platforms also ship pre‑configured machine‑learning environments and ready‑made domain algorithms. Pick the platform whose tooling matches the tools you already use and project goals; cloud execution keeps everything reproducible and scalable without local setup hassles. By using platforms to develop code you can directly publish end-to-end FAIR workflows that enable users to reproduce and reuse result datasets and code. | ||
| EarthCODE platforms differ in interface and how you build and run workflows: some offer visual process designers, others provide Jupyter Notebooks with extensions, and several combine both. For example OpenEO provides an API, a set of client libraries, supporting modern programming languages such as Python, and a graphical interface. Many platforms also ship pre‑configured machine‑learning environments and ready‑made domain workflows. Pick the platform whose tooling matches the tools you already use and project goals; cloud execution keeps everything reproducible and scalable without local setup hassles. By using platforms to develop code you can directly publish end-to-end FAIR workflows that enable users to reproduce and reuse result datasets and code. | ||
|
|
||
| <!-- :::tip Note to self turn this to a gif of how the editor produces something | ||
| ::: --> | ||
|  | ||
| *Open EO Process Editor GUI* | ||
|  | ||
| *OpenEO Web Editor GUI* | ||
|
|
||
| --- | ||
|
|
||
|
|
@@ -137,11 +137,12 @@ Platforms differ in how they let you visualize published results from others and | |
|
|
||
| Visit the [Working with Platforms](../Technical%20Documentation/Platforms/) page to explore EarthCODE platform and for help to choose the platform that best suits your research objectives. | ||
|
|
||
| ## Exploring the EarthCODE Open Science Catalog | ||
| After completing your research and preparing your final results for journal publication, your validated data products and code can be published in the EarthCODE Catalog — the [Open Science Catalog](https://opensciencedata.esa.int/). Only finalized outputs that are ready for publication and long-term storage should be submitted. The catalog includes datasets, workflows, and software that follow EarthCODE best practices, most of which originate from integrated platforms. It's your gateway to a wide range of reusable scientific resources, from environmental data to executable workflows.ß | ||
| ## Exploring the Open Science Catalog | ||
| After completing your research and preparing your final results for journal publication, your validated data products and code can be published in the [Open Science Catalog](https://opensciencedata.esa.int/). Only finalized outputs that are ready for publication and long-term storage should be submitted. The catalog includes datasets, workflows, and software that follow EarthCODE best practices, most of which originate from integrated platforms. It's your gateway to a wide range of reusable scientific resources, from environmental data to executable workflows. | ||
|
|
||
| ### Validating and Publishing Data to the Catalog {#publishing-anchor} | ||
|
|
||
| EarthCODE allows you to validate products and experiments, ensuring they meet reproducibility and scientific standards. If you have an ESA-funded project, you can contribute to the platform by publishing your datasets, software, or experiments, making your work available for the broader community to use. | ||
| EarthCODE allows you to validate products, workflows and experiments, ensuring they meet reproducibility and scientific standards. If you have an ESA-funded project, you can contribute to the platform by publishing your datasets, workflows, or experiments, making your work available for the broader community to use. | ||
|
|
||
| #### How data is published | ||
| The [Open Science Catalog](https://opensciencedata.esa.int/) is built on the [SpatioTemporal Asset Catalog (STAC)](https://stacspec.org/en), a standardised format for describing geospatial data. To contribute your research, you need to create valid STAC objects and commit them to the [`open-science-catalog-metadata-staging`](https://github.com/EOEPCA/open-science-catalog-metadata-staging) repository on GitHub. These STAC objects, stored as JSON files, will be automatically processed and rendered in the catalog. | ||
|
|
@@ -166,15 +167,15 @@ You can read more on how to publish data in the [Uploading Your Data](../Technic | |
| #### Best Practices | ||
| EarthCODE makes FAIR publishing the default. Scientists are expected to: | ||
|
|
||
| - Deliver publication‑ready outputs: final datasets, workflows, and models, kept in persistent cloud storage with open, clearly stated licences. | ||
| - Deliver publication‑ready outputs: final datasets, workflows, and models, kept in persistent cloud storage with open, clearly stated licenses. | ||
| - Attach rich, standards‑based metadata: STAC/ISO 19115 descriptions with DOIs, spatial‑temporal extents, provenance, and links to source data and code. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should the user provide DOIs or is this done by EarthCODE? -> mentioned on line 196
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the user already has a DOI it should be reused. If none is available EarthCODE will automatically during publishing provide one. |
||
| - Maintain reproducibility: specify software environments, parameters, and lineage, ensuring others can rerun and verify results. | ||
|
|
||
| These practices align with the FAIR pillars: | ||
|
|
||
| | Findable | Accessible | Interoperable | Reusable | | ||
| |----------|------------|---------------|----------| | ||
| | DOIs, indexed metadata | HTTPS, STAC & OGC APIs | NetCDF, GeoTIFF, Zarr, CF vocabularies | Open licences, provenance, community standards | | ||
| | DOIs, indexed metadata | HTTPS, STAC & OGC APIs | NetCDF, GeoTIFF, Zarr, CF vocabularies | Open licenses, provenance, community standards | | ||
|
|
||
| For detailed guide on best practices refer to the [Best Practices Page](../Community%20and%20Best%20Practices/). | ||
|
|
||
|
|
@@ -191,7 +192,7 @@ EarthCODE integrates FAIR tools and standards at every stage of the research lif | |
| | **Development** | EarthCODE provides integrated platforms that provide different options for developing scientific workflows | | ||
| | **Data, Workflows and Platforms** | EarthCODE describes Data, workflows and platforms as FAIR objects. All items published on the Open Science Catalog follow EarthCODE metadata standards (STAC, OGC) for consistency across platforms and tools. | | ||
| | **EarthCODE Standard Variable Dictionary** | Shared [vocabulary](https://opensciencedata.esa.int/variables/catalog) for EO variables to ensure semantic consistency across datasets and workflows. | | ||
| | **Publishing Automation** | EarthCODE platforms provide automation to support publishing of finalized data and workflows to the EarthCODE catalog | | ||
| | **Publishing Automation** | EarthCODE platforms provide automation to support publishing of finalized data and workflows to the Open Science Catalog | | ||
| | **Publishing and Maintenance**| Users can maintain (or publish) FAIR data and workflows through the EarthCODE publishing GUI. | | ||
| | **Data Storage** | EarthCODE provides persistent cloud storage via ESA PRR which ensures accessibility and long-term preservation. | | ||
| | **Finding Products and Workflows** | The Open Science Catalog makes metadata discoverable via STAC API and its webpage search. | | ||
|
|
@@ -202,9 +203,9 @@ EarthCODE integrates FAIR tools and standards at every stage of the research lif | |
| | **Re-Using Workflows** | Users can directly re-use published workflows on the platform used to develop them, compatible platforms (or locally). | | ||
| | **Code Quality** | Automated pipelines integrate linting, code quality checking and recommendation, and security scanning to ensure code is robust and reusable. | | ||
| | **Reproducibility** | Published workflows are described in full as experiments, where you have information about inputs and configurations. EarthCODE provides the means to directly execute these experiments and reproduce data products. | | ||
| | **Intraoperability** | Shared metadata schemas and API protocols ensure you can execute worklows across platforms and published daat on EarthCODE follows common standards (Zarr, NetCDF) with standard variables (e.g. CF-Conventions). | | ||
| | **Intraoperability** | Shared metadata schemas and API protocols ensure you can execute workflows across platforms and published data on EarthCODE follows common standards (Zarr, NetCDF) with standard variables (e.g. CF-Conventions). | | ||
| | **Visualisation and Findability** | Dashboards and linked visual previews make datasets easier to interpret and explore. | | ||
| | **Contacts and Communication**| Published items have contact details in the metadata, EarthCODE additionally provides a chat functionallity in Discourse to connect and discuss. | | ||
| | **Contacts and Communication**| Published items have contact details in the metadata, EarthCODE additionally provides a chat functionality in Discourse to connect and discuss. | | ||
| | **Community** | Discourse forum, with published items linked directly to posts, and best practice documentation foster open collaboration. | | ||
|
|
||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the publishing happen to the staging environment? If so and if my understanding is correct, their records will only be visualised in the staging catalog. If correct, it should be clear to the reader that this is not rendered in the main catalogue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, this should be changed to the production repository URL.