diff --git a/pages/Community and Collaboration/Getting Started With The EarthCODE Discourse/Join the Conversation.md b/pages/Community and Collaboration/Getting Started With The EarthCODE Discourse/Join the Conversation.md index f20a6a6f..98c86ed5 100644 --- a/pages/Community and Collaboration/Getting Started With The EarthCODE Discourse/Join the Conversation.md +++ b/pages/Community and Collaboration/Getting Started With The EarthCODE Discourse/Join the Conversation.md @@ -132,7 +132,7 @@ By following these guidelines, you help keep the forum organized, useful, and we **Starting a Discussion About an Item in the Open Science Catalog** -If your discussion relates to an item in the **Open Science Catalog**, it's best to use the [integrated features on the catalog](../../Technical%20Documentation/Data/Discovering%20Resources%20in%20The%20EarthCODE%20Catalog.md#start-discussions-directly-from-the-catalog): +If your discussion relates to an item in the **Open Science Catalog**, it's best to use the [integrated features on the catalog](../../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Discovering%20Resources.md#start-discussions-directly-from-the-catalog): 1. Navigate to the catalog and locate the item. 2. Use the **"Start a new topic on the EarthCODE forum"** button. diff --git a/pages/Getting started with EarthCODE/10 minutes to EarthCODE.md b/pages/Getting started with EarthCODE/10 minutes to EarthCODE.md index f384e170..22bb8792 100644 --- a/pages/Getting started with EarthCODE/10 minutes to EarthCODE.md +++ b/pages/Getting started with EarthCODE/10 minutes to EarthCODE.md @@ -25,17 +25,14 @@ See [Accessing EarthCODE](../Getting%20started%20with%20EarthCODE/Accessing%20Ea Different users of EarthCODE have different needs and goals. To help you find the most relevant information quickly, we've outlined suggested paths based on your role and intentions. -### Researchers Ready to Upload Their Data +### Researchers looking to contribute to the Open Science Catalog -If you've completed your research and are looking to **publish your data to the Open Science Catalog (OSC)**, your main priority is to ensure your data is **persistent, properly formatted and publicly available**. +If you have 1) completed your research; 2) have your data/workflow prepared in a persitent online repository; and 3) are looking to ** contribute to the Open Science Catalog (OSC)**, head to the [**Contributing to the OSC section**](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md)). -Head directly to the [**Uploading Data**](../Technical%20Documentation/Data/Contributing%20to%20the%20EarthCODE%20Catalog.md) section to learn how to: -- Prepare your data for publication. -- Create valid STAC objects and metadata. -- Publish data products to the Open Science Catalog. -This section will guide you through the entire upload process, including validating your metadata and ensuring your data assets are accessible. +### Researchers looking for persistent storage +If you've 1) completed your research; and 2) want to store your data/workflow prepared in a persitent online repository you can use the ESA Project Results Repository to do so. Head to the [**ESA Project Results Repository (PRR) section**](../Technical%20Documentation/ESA%20Project%20Results%20Repository%20(PRR)/index.md)) to for instructions how to do this. After you have stored your data/workflow in the PRR, add an entry for it in the [Open Science Catalog](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md)). ### New Researchers Starting a Project @@ -51,7 +48,7 @@ Visit the [**Platform Selection**](../Technical%20Documentation/Platforms/) sect - Understanding which tools will streamline your workflow. - Setting up your environment for collaborative or individual work. -If you plan to publish your results later, it’s a good idea to also familiarize yourself with the [**Uploading Data**](../Technical%20Documentation/Data/Contributing%20to%20the%20EarthCODE%20Catalog.md) section to understand the requirements early on. +It’s also a good idea to also familiarize yourself with the [**Contributing to the OSC section**](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md)) to understand the requirements early on. ### Users Exploring Published Research @@ -161,7 +158,7 @@ To improve discoverability, STAC objects also include metadata fields such as: Because these objects must follow a strict predefined schema, ensuring compliance can be complex. However, EarthCODE provides example guides to help you structure your data correctly. Additionally, there are automated tools that simplify the creation of STAC objects, reducing manual effort. -You can read more on how to publish data in the [Uploading Your Data](../Technical%20Documentation/Data/Contributing%20to%20the%20EarthCODE%20Catalog.md) section, and check out our [examples](https://esa-earthcode.github.io/examples/). +You can read more on how to publish entities in the [Contributing to the Open Science Catalog](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md) section, and check out our [examples](https://esa-earthcode.github.io/examples/). #### Best Practices diff --git a/pages/Getting started with EarthCODE/index.md b/pages/Getting started with EarthCODE/index.md index 85db72da..0626ee92 100644 --- a/pages/Getting started with EarthCODE/index.md +++ b/pages/Getting started with EarthCODE/index.md @@ -86,7 +86,7 @@ When your research is complete, you: :::tip *Only ESA-funded projects can publish to EarthCODE.* ::: -The **ESA Project Results Repository (ESA PRR)** is a long-term storage service provided by ESA to persistently host results from ESA-funded projects. Once users have completed their work, they can upload their datasets and workflows to the PRR to ensure they remain accessible and preserved over time. While publishing to the PRR is optional, it is strongly recommended for long-term availability and compliance with FAIR principles. A detailed guide to this process is available in the [publication section](../Technical%20Documentation/Data/Contributing%20to%20the%20EarthCODE%20Catalog.md). +The **ESA Project Results Repository (ESA PRR)** is a long-term storage service provided by ESA to persistently host results from ESA-funded projects. Once users have completed their work, they can upload their datasets and workflows to the PRR to ensure they remain accessible and preserved over time. While publishing to the PRR is optional, it is strongly recommended for long-term availability and compliance with FAIR principles. A detailed guide to this process is available in the [publication section](../Technical%20Documentation/ESA%20Project%20Results%20Repository%20(PRR)/index.md). The [**Open Science Catalog (OSC)**](https://opensciencedata.esa.int/catalog) is a public, web-based application that allows users to explore and access scientific datasets, workflows, and documentation produced through ESA-funded Earth Observation research. Researchers publish metadata describing their outputs to the Open Science Catalog, and after review, this metadata is made discoverable through the OSC. The catalog uses STAC to describe datasets and OGC API - Records for workflows. Through this rich metadata on the Open Science Catalog, resources are made FAIR. Additionally, the open science catalog is integrated with the EarthCODE platforms and allows users to reproduce experiments (run workflows that produce products) that they find on the catalog. @@ -106,9 +106,9 @@ Each published item in the OSC can be linked to a discussion thread, enabling ex This step will help you become familiar with the platform's tools and resources, and how they integrate to support your research. The best way to follow our tutorials and learn about EarthCODE is by using one of the EarthCODE platforms. We recommend visiting the page where we explain how [to choose the right tools and platforms for your work](../Technical%20Documentation/Platforms/). EarthCODE offers a variety of features, in the portal you will be able to find: -- [Datasets](../Technical%20Documentation/Data/): Access a variety of datasets available for your research needs and discovery. +- [Datasets](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/): Access a variety of datasets available for your research needs and discovery. - [Computational Research](../Technical%20Documentation/Platforms/): Use EarthCODE integrated platforms with advanced computational tools for Earth System Science on collaborative research environment -- [Workflows](../Technical%20Documentation/Workflows/) : Easily find, publish and re-use FAIR workflows to enhance reproducibility and collaboration in research +- [Workflows](../Technical%20Documentation/Workflows%20and%20Experiments/index.md) : Easily find, publish and re-use FAIR workflows to enhance reproducibility and collaboration in research - [Community](../Community%20and%20Collaboration/Getting%20Started%20With%20The%20EarthCODE%20Discourse/): Fostering a collaborative community dedicated to FAIR Open Science and sustainable innovation By exploring these sections, you’ll better understand how to leverage EarthCODE’s resources and tools to support your research. diff --git a/pages/Integrating New Platforms to EarthCODE/Deep Dive into EarthCODE.md b/pages/Integrating New Platforms to EarthCODE/Deep Dive into EarthCODE.md index b1b4bf1a..470d11c2 100644 --- a/pages/Integrating New Platforms to EarthCODE/Deep Dive into EarthCODE.md +++ b/pages/Integrating New Platforms to EarthCODE/Deep Dive into EarthCODE.md @@ -189,7 +189,7 @@ For some examples of what these might look like in practice, refer to the [10 mi Usage of these platforms, and the data, compute and storage resources on these platforms can be funded through the [ESA Network of Resources (NoR)](https://eo4society.esa.int/network-of-resources/), depending on the project type and eligibility. -Users can also decide to use their own local environments instead of the platforms. For users working outside the integrated platforms—such as on institutional infrastructure or personal systems—EarthCODE supports **manual publishing** of datasets and workflows to the catalog, through the EarthCODE publishing GUI or manually via git pull requests (see more at [Working with Data](../Technical%20Documentation/Data/)). +Users can also decide to use their own local environments instead of the platforms. For users working outside the integrated platforms—such as on institutional infrastructure or personal systems—EarthCODE supports **manual publishing** of datasets and workflows to the catalog, through the EarthCODE publishing GUI or manually via git pull requests (see more in the [**Contributing to the OSC section**](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md)). ### Network of Resources @@ -255,7 +255,7 @@ The EarthCODE project runs standard best practice procurement cycles on esa-star More information about how integrating with EarthCODE works, where and how to apply, can be found at the [Integrating New Platforms Page](../Integrating%20New%20Platforms%20to%20EarthCODE/). ### Publishing Experiments Data and Workflows -Once a research activity is complete, the results can be published to the EarthCODE ecosystem, making them findable, reproducible, and reusable by the broader scientific community. If you are working on an integrated platform, publishing is typically automated. When the experiment is finalized, the platform can generate the appropriate metadata and push it to the Open Science Catalog. If you're working outside of an integrated platform—on institutional infrastructure or locally—you can still publish your results manually by following the Open Science Catalog submission guidelines [Working with Data](../Technical%20Documentation/Data/index.md). +Once a research activity is complete, the results can be published to the EarthCODE ecosystem, making them findable, reproducible, and reusable by the broader scientific community. If you are working on an integrated platform, publishing is typically automated. When the experiment is finalized, the platform can generate the appropriate metadata and push it to the Open Science Catalog. If you're working outside of an integrated platform—on institutional infrastructure or locally—you can still publish your results manually by following the [Open Science Catalog submission guidelines](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Contributing%20to%20the%20Open%20Science%20Catalog.md). At the time of publishing, it is expected that the outputs have been finalized and, where applicable, reviewed and approved by the ESA project’s Technical Officer. When you're ready to publish you: 1. (Optional) Upload your **datasets and workflows** to the **ESA Project Results Repository (PRR)** for long-term preservation. @@ -273,7 +273,7 @@ The core units of publication in EarthCODE are data and workflows (combined in e In EarthCODE, the final outputs of your research—referred to as **Products** or data products—are stored, described, and published in a way that ensures long-term FAIRness and availability. -These data products can be hosted on the ESA Project Results Repository (PRR) or an external repository of your choice. The PRR is ESA’s dedicated long-term storage service for project results. Uploading to the PRR is optional but recommended, especially for ensuring compliance with FAIR principles and facilitating persistent access to your work. You simply provide a link to the data location as part of the product’s metadata. For detailed instructions on uploading to the PRR, refer to the [Contributing to the Open Science Catalog guide](../Technical%20Documentation/Data/Contributing%20to%20the%20EarthCODE%20Catalog.md). +These data products can be hosted on the ESA Project Results Repository (PRR) or an external repository of your choice. The PRR is ESA’s dedicated long-term storage service for project results. Uploading to the PRR is optional but recommended, especially for ensuring compliance with FAIR principles and facilitating persistent access to your work. You simply provide a link to the data location as part of the product’s metadata. For detailed instructions on uploading to the PRR, refer to the [**ESA Project Results Repository (PRR) section **](../Technical%20Documentation/ESA%20Project%20Results%20Repository%20(PRR)/index.md). Each product is described using **STAC (SpatioTemporal Asset Catalog) metadata**, specifically through a `Collection` that captures key attributes like the spatial and temporal extent, scientific context, provenance, and more. @@ -340,7 +340,7 @@ The [Open Science Catalog (OSC)](https://opensciencedata.esa.int/catalog) is the Metadata describing datasets (Products) is published using the **SpatioTemporal Asset Catalog (STAC)** specification. Workflows and experiments are described using the **OGC API - Records** standard. These two metadata models are connected by references. The Open Science Catalog is also integrated with EarthCODE's computing infrastructure. Users with appropriate access—such as NoR-sponsored compute—can reproduce experiments directly from the catalog. This means running the same workflow with the same input and configuration on a compatible platform, enabling reproducible results. -The catalog acts as both a registry of published research artifacts and an operational gateway for executing FAIR experiments across federated EO platforms. More information about the catalog and how it works can be found at [Data - Discovering Resources in The Open Science Catalog)](../Technical%20Documentation/Data/Discovering%20Resources%20in%20The%20EarthCODE%20Catalog) +The catalog acts as both a registry of published research artifacts and an operational gateway for executing FAIR experiments across federated EO platforms. More information about the catalog and how it works can be found at [Data - Discovering Resources in The Open Science Catalog)](../Technical%20Documentation/Open%20Science%20Catalog%20(OSC)/Discovering%20Resources) +See the [PRR page](../ESA%20Project%20Results%20Repository%20(PRR)/) for a detailed, interactive introduction and a bank of examples of how different ESA projects have generated their collections. + +::: details Requesting PRR Storage +If you have any questions or require suppport please email the EarthCODE support team: [earth-code@esa.int](mailto:earth-code@esa.int) . +::: + + + +## Step 2: Creating OSC Entries + +### How to publish new data to the catalog? + +The Open Science Catalog is built on the Spatio Temporal Asset Catalog (STAC), which is a standardised format for describing geospatial data. Therefore new entries must conform to its specification. There are three ways to create the entries: + +### 1: Using the OSC Publishing GUI + +- The [Open Science Catalog Editor](https://workspace.earthcode.eox.at/) is graphical user interface for automatically creating OSC entries and review requests. + +### 2: Manual creation +- [Directly creating/editing STAC files](https://esa-earthcode.github.io/examples/osc-pr-manual/) - A guide for manually creating OSC entries. Requires knowledge of git. + +- [Generating OSC files using pystac](https://esa-earthcode.github.io/examples/osc-pr-pystac/) - A guide for creating OSC entries using pystac. Requires knowledge of git and Python. + +### 3: Using platform tools and support +- [DeepCode](https://github.com/deepesdl/deep-code) - An example using DeepCode: a library for automatically generating product entries for DeepESDL datasets. +- Additionally, you can contact your platform supplier for support. + + + +## Step 3: Review & Publishing + +Regardless of what option for creating OSC Entries you choose, the generated data will be reviewed by EarthCODE team before it is accepted into the PRR. The review process will take place on GitHub via its [pull request functionality](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests). During the review the EarthCODE team will: +- check the accuracy and completeness of descriptions, links and information +- ask for a code snippet that shows how to read the data (if applicable) +- ask for a code snippet that demonstrates how to run the code (if applicable) +After any required changes are made, the OSC entries are ingested in the catalog. + +When a new product or workflow is ingested in the OSC, the team will encourage you to promote it on the (EarthCODE forum)[https://discourse-earthcode.eox.at/]. \ No newline at end of file diff --git a/pages/Technical Documentation/Data/Discovering Resources in The EarthCODE Catalog.md b/pages/Technical Documentation/Open Science Catalog (OSC)/Discovering Resources.md similarity index 99% rename from pages/Technical Documentation/Data/Discovering Resources in The EarthCODE Catalog.md rename to pages/Technical Documentation/Open Science Catalog (OSC)/Discovering Resources.md index e0eeccab..964c7200 100644 --- a/pages/Technical Documentation/Data/Discovering Resources in The EarthCODE Catalog.md +++ b/pages/Technical Documentation/Open Science Catalog (OSC)/Discovering Resources.md @@ -88,7 +88,7 @@ The OSC Catalog page is built upon the open-source STAC Browser application, all - Workflows - Experiments -Each entry lists the number of objects of the selected type, which are represented as a STAC Catalog, STAC Collection, or OGC API Record. These objects use the OSC STAC extension to reference elements of other groups they are associated with, e.g., a Product has an `osc:variables` field, that lists the measurement variables this product is comprised of. More information on how the items are linked is provided in the [Uploading Your Data](Contributing%20to%20the%20EarthCODE%20Catalog) section. +Each entry lists the number of objects of the selected type, which are represented as a STAC Catalog, STAC Collection, or OGC API Record. These objects use the OSC STAC extension to reference elements of other groups they are associated with, e.g., a Product has an `osc:variables` field, that lists the measurement variables this product is comprised of. More information on how the items are linked is provided in the [**Contributing to the OSC section**](./Contributing%20to%20the%20Open%20Science%20Catalog.md) ![STACBrowser-OSC](https://github.com/EOEPCA/open-science-catalog-metadata/assets/120453810/257daa0c-a567-4ed2-a8bc-3fb594079b2c) diff --git a/pages/Technical Documentation/Open Science Catalog (OSC)/Open Science Catalog Overview.md b/pages/Technical Documentation/Open Science Catalog (OSC)/Open Science Catalog Overview.md new file mode 100644 index 00000000..e8de54a9 --- /dev/null +++ b/pages/Technical Documentation/Open Science Catalog (OSC)/Open Science Catalog Overview.md @@ -0,0 +1,154 @@ +--- +order: 1 +--- +# Open Science Catalog (OSC) + +## Introduction + +The Open Science Catalog (OSC) is a key component of the ESA EO Open Science framework. It is built on the Spatio Temporal Asset Catalog (STAC), which is a standardised format for describing geospatial data. The catalog captures information about Projects, Products, Workflows, and Experiments, and their relationships to ESA Themes, Variables, EO missions. These elements contain **information and direct links** to the corresponding research outcomes, which are themselves located in external storage providers. + +Users can browse and explore these interlinked elemennts throguht the webrowser, API or directly through the data itself. See [Data Discovery and Access](Discovering%20Resources) for more information. + + +## Adding / updating entries. + +The different ways to add/update/remove entries from the catalog are described in the [**Contributing to the OSC section**](./Contributing%20to%20the%20Open%20Science%20Catalog.md). + +At a core level, each update to metadata is handled via a [Pull Request (PR)](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests). + +This Pull Request allows for reviewers to see the changes to be applied in advance, to check for validity of the requested changes (via an automated validation script) and to provide reviews as comments. + + +## Structure + +The Open Science Catalog is a deployment of several EOEPCA components, in combination with additional supplementary components. In this section we focus on the metadata and its structure, as that is what most users will need to work with. You can see the full technical architechture of the Open Science Catalog [here](https://github.com/ESA-EarthCODE/open-science-catalog-metadata/wiki/System-Design-Document-%E2%80%90-v1.0.0). + +The Open Science Catalog metadata is a STAC catalog comprised of json files, with specific attributes and structure that together describe its elements - Themes, Variables, EO Missions, Projects, Products, Workflows, and Experiments. All files are stored directly on Github [here] (https://github.com/ESA-EarthCODE/open-science-catalog-metadata/tree/main), as they only contain metadata and links and not the actual data in the products, or the code in the workflows. + +Detailed information about Projects, Products, Workflows, and Experiments is available in the tutorial which shows how to manually create files for the OSC - [here](https://esa-earthcode.github.io/examples/osc-pr-manual/) . + + +### Projects + +Projects are the containers that have the top level information about your work. It is the first type of information you should provide. Typically an OSC project corresponds to a project financed by the European Space Agency - Earth Observation programme. Before creating new project, check if your project is not already on the [list of onboarded projects](https://opensciencedata.esa.int/projects/catalog). In such case you can use your project entry and only update it where needed. + + +| **Field** | **Description** | **STAC representation** | +|--------------------|--------------------------|------------------------------------| +| Project_ID | Numeric identifier | | +| Status | “ongoing” or “completed” | osc:status property | +| Project_Name | Name | title property | +| Short_Description | | description property | +| Website | | link | +| Eo4Society_link | | link | +| Consortium | | contacts[].name property | +| Start_Date_Project | | extent.temporal[] property | +| End_Date_Project | | extent.temporal[] property | +| TO | | contacts[].name property | +| TO_E-mail | | contacts[].emails[].value property | +| Theme1 - Theme6 | Theme identifiers | osc:themes property | + + +Metadata of each project is stored in a folder named after their unique id `(collectionid)`. Each folder has one file - collection.json that has all the project information (metadata). + +In addition to specifying the links within the project collection.json entry (created above), you should also add an entry in the parent catalog, listing all projects to be correclty rendered into STAC Browser. + +### Products + +Products represent the outputs of you projects and typically reference datasets. Similarly to Projects, they are STAC items and follow similar structure, with some additional fields, improving their findability. + +| **Field** | **Description** | **STAC representation** | +|---------------------|---------------------------------------|---------------------------------------| +| **ID** | Numeric identifier | | +| **Status** | “ongoing” or “completed” | osc:status property | +| **Project** | The project identifier | osc:project property, collection link | +| **Website** | | link | +| **Product** | Name | link | +| **Short_Name** | | identifier | +| **Description** | | description property | +| **Access** | URL | link | +| **Documentation** | URL | link | +| **Version** | | version property | +| **DOI** | Digital Object Identifier | sci:doi property and cite-as link | +| **Variable** | Variable identifier | collection link | +| **Start** | | extent.temporal[] | +| **End** | | extent.temporal[] | +| **Region** | | osc:region property | +| **Polygon** | | geometry | +| **Released** | | created property | +| **Theme1 - Theme6** | Theme identifiers | osc:themes property | +| **EO_Missions** | Semi-colon separated list of missions | osc:missions property | +| **Standard_Name** | | cf:parameter.name property | + + +In addition to specifying the links from the product to other parts of the catalog, **it is required** to add the reverse links, as in case of the Project to following elements: +- From the Product Collection.json to the Catalog.json (listing all products in the OSC) +- From the associated Project to the Product +- From the associated EO-Missions catalog to the Product +- From the associated Variables Catalog to the Product +- From the associated Themes Catalog to the Product + + +## Workflows + +Workflows are the code and workflows associated with a project, that have been used to generate a specific product. Workflows follow `OGC record specifications` in contrast to OSC Projects and Products entries. However, the metadata of a workflow is also expressed in JSON format. + +| Field Name | Description | +| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `conformsTo` | An array of URIs indicating which OGC API Records specifications this record conforms to. | +| `type` | Indicates the GeoJSON object type. Required to be `"Feature"` for OGC compliance. | +| `geometry` | Spatial representation of the item. Set to `None` here, as it may not be spatially explicit. | +| `linkTemplates` | An array of link templates as per the OGC API. Used for dynamic link generation. | +| `id` | Unique identifier for the workflow STAC item (`'worldcereal-workflow2'`). | +| `links` | List of external and internal references including catalog navigation, project association, theme association, process graph, source code, and service endpoint. | +| `properties.contacts` | List of individuals or organizations associated with the workflow. Each contact may include name, email, and roles such as `technical_officer` or `consortium_member`. | +| `properties.created` | Timestamp representing when the workflow was first created (`2025-07-14T18:02:13Z`). | +| `properties.updated` | Timestamp of the most recent update to the workflow (`2025-07-14T18:02:13Z`). | +| `properties.version` | The version number of the workflow (`1`). | +| `properties.title` | A concise, descriptive title of the workflow: *"ESA worldcereal global crop extent detector2"*. | +| `properties.description` | A summary of what the workflow does: *"Detects crop land at 10m resolution, trained for global use..."*. | +| `properties.keywords` | Array of keywords to support discoverability (e.g., `agriculture`, `crops`). | +| `properties.themes` | Array of themes the workflow relates to. Each entry includes a `concepts` array with IDs (e.g., `'land'`) and a `scheme` URL. | +| `properties.formats` | Output formats of the workflow (e.g., `GeoTIFF`). | +| `properties.osc:project` | Project ID associated with the workflow (`worldcereal2`). | +| `properties.osc:status` | Current status of the workflow (e.g., `completed`). | +| `properties.osc:type` | Type of OSC object, expected to be `workflow`. | +| `properties.license` | License for the workflow (e.g., `'varuious'` – likely a typo for `various`). | + + + +In addition to specifying the links from the workflow to other parts of the catalog, **it is required** to add the reverse links: + +- From the Workflow record.json to the workflows/catalog.json (listing all workflows in the OSC) +- From the associated Project to the Workflow +- From the associated Themes to the Workflow + + +### Themes + +Themes describe the Earth Science topics linked to the grand science challenges set in the ESA strategy. This is a fixed list. + +Field | Description | STAC representation +-- | -- | -- +theme | Theme name | id +description | Theme description | description +link | Link to further resources | link + +### Variables + +The variables field describes the Geoscience, climate and environmental variables that products and workflows model. There is a fixed list of variables, however if your variable is missing from it you can add it in your Pull Request. + +Field | Description | STAC representation +-- | -- | -- +theme | The associated theme name | osc:theme property +theme_description |   |   +link | Link to further resources | link +variable | The variable name | id +domain | The variables domain |   +variable description |   | description + + + + + + diff --git a/pages/Technical Documentation/Open Science Catalog (OSC)/index.md b/pages/Technical Documentation/Open Science Catalog (OSC)/index.md new file mode 100644 index 00000000..cdd218ab --- /dev/null +++ b/pages/Technical Documentation/Open Science Catalog (OSC)/index.md @@ -0,0 +1,15 @@ +--- +order: 1 +--- +# Open Science Catalog + +The Open Science Catalog Section provides detailed instructions on how to efficiently access, manage, and use data from EarthCode. This includes discovering and accessing datasets through the Open Science Catalog, uploading and managing your own data, and incorporating datasets into your workflows for analysis and processing. + +## [Open Science Catalog Overivew](Open%20Science%20Catalog%20Overview.md) +This subsection gives an overview of the Open Science Catalog, its core entries - products, workflows, projects and experiments, and how they are structured and stored. + +## [Data Discovery and Access](Discovering%20Resources.md) +You will learn how to search for and access data through the Open Science Catalog. It covers instructions on using metadata and dependencies, helping you locate relevant datasets for your research. Additionally, you will gain insight into how to utilize APIs for automated access to data. + +## [Contributing to the Open Science Catalog](Contributing%20to%20the%20Open%20Science%20Catalog.md) +Here, you will find guidelines on how to incorporate your own datasets, code and documentation to the Open Science Catalog. diff --git a/pages/Technical Documentation/Data/Using Data in Workflows.md b/pages/Technical Documentation/Workflows and Experiments/Using Data in Workflows.md similarity index 100% rename from pages/Technical Documentation/Data/Using Data in Workflows.md rename to pages/Technical Documentation/Workflows and Experiments/Using Data in Workflows.md diff --git a/pages/Technical Documentation/Workflows and Experiments/index.md b/pages/Technical Documentation/Workflows and Experiments/index.md new file mode 100644 index 00000000..27f3a72e --- /dev/null +++ b/pages/Technical Documentation/Workflows and Experiments/index.md @@ -0,0 +1,2 @@ +## [Using Data in Workflows](Using%20Data%20in%20Workflows) +This subsection explains how to incorporate data into your workflows and experiments within EarthCODE. It provides step-by-step instructions on integrating external datasets into your research workflows, enabling you to perform analysis, processing, and visualization tasks in an efficient and reproducible manner. \ No newline at end of file diff --git a/pages/Technical Documentation/Workflows/index.md b/pages/Technical Documentation/Workflows/index.md deleted file mode 100644 index df2089c8..00000000 --- a/pages/Technical Documentation/Workflows/index.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -order: 1 ---- -# Working with Workflows -:::warning 🛠️ Page Under Development -Content is being actively developed and updated for this page. EarthCODE's documentation is a living document and will be continuously updated with detailed reviews. -::: -## Workflow Management - -Guidance on using EarthCODE's workflow tools, such as interactive notebooks, containers, and pipeline orchestration for reproducible research. - -!(cross platform)[/img/terms/cross-platform-reusability.png] - -## Creating Workflows -How to develop research workflows using EarthCODE’s interactive graphical tooling, including workflow design and validation. - -## Running Workflows -How to execute workflows using the platform’s cloud-based resources, including how to monitor progress and manage execution. - -## Automation and Continuous Integration - -Setting up automated processes for testing, deployment, and data processing using continuous integration/continuous deployment (CI/CD) pipelines. - - -## Using Machine Learning Tools -Instructions on setting up and running machine learning models within EarthCODE’s scalable environment. - - diff --git a/pages/Technical Documentation/index.md b/pages/Technical Documentation/index.md index 2c8f2b08..e5bdf051 100644 --- a/pages/Technical Documentation/index.md +++ b/pages/Technical Documentation/index.md @@ -10,5 +10,5 @@ Content is being actively developed and updated for this page. EarthCODE's docum The **Technical Documentation** page offers comprehensive guidance on how to configure and personalize your EarthCODE environment to meet your specific research needs. This section provides step-by-step instructions on accessing and integrating the tools, data, and services required for your projects. Whether you're setting up your workspace, connecting to cloud platforms, or ensuring access to the Open Science Catalog, this page will equip you with everything you need to create an efficient and effective research environment. It also covers best practices for managing data, code, and workflows, helping you establish a seamless and collaborative research process. - [Working with Platforms](./Platforms/) -- [Working with Data](./Data/) -- [Working with Workflows](./Workflows/) +- [Working with the ESA Project Results Repository (PRR)](./ESA%20Project%20Results%20Repository%20(PRR)/) +- [Working with the Open Science Catalog (OSC)](./Open%20Science%20Catalog%20(OSC)/) \ No newline at end of file