-
Notifications
You must be signed in to change notification settings - Fork 34
/
Copy pathglossary.qmd
107 lines (56 loc) · 11 KB
/
glossary.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
---
title: 'Glossary, Cheatsheets, & Guides'
---
## Glossary
A new paradigm brings new terminology you will encounter while you learn about cloud data access and computing.
This handy cheatsheet, **Cloud Terminology 101**, defines commonly used cloud computing terms and phrases.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vSryqtYkNSuw7wgg8m5Vdi1oaBRN9gIIO-LXlaOWMzHJFc8v9nPeXAbKkKtcbS2n8BOwPBaJjPFy-uv/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vSryqtYkNSuw7wgg8m5Vdi1oaBRN9gIIO-LXlaOWMzHJFc8v9nPeXAbKkKtcbS2n8BOwPBaJjPFy-uv/pub?start=false&loop=false&delayms=3000){target="_blank"}
More specific to working with NASA Earthdata in the cloud, this **Workflow Terminology Cheatsheet** defines terms associated with NASA Earthdata, cloud-optimized data, and some open software science tools you may encounter. A full list of NASA Eathdata-specific terms are found in the [NASA Earthdata Glossary](https://earthdata.nasa.gov/learn/user-resources/glossary).
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vR7Iap03ogOenimB9aoMy0rxB2J2VWcNhXlE_pIabKaCj9VoNNdxuc6IK0A_WjJVEEYZtmzkMn4GHTY/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vR7Iap03ogOenimB9aoMy0rxB2J2VWcNhXlE_pIabKaCj9VoNNdxuc6IK0A_WjJVEEYZtmzkMn4GHTY/pub?start=false&loop=false&delayms=3000){target="_blank"}
## Cheatsheets & Guides
How are all of these terms and concepts related? And how can you begin to put them to work?
Here are some cheatsheets and guides to help visualize what working with NASA Earthdata in the cloud can look like, and how to get started.
_All slides and cheatsheets are available for re-use and remix! Let us know what you think! We welcome your input so we can continue to improve and update these guides. Slides are credited for each deck; Cheatsheets development have been led by Catalina Oaida Taglialatela and Cassie Nickles (PO.DAAC) in Spring 2022._
([Internal link](https://docs.google.com/presentation/d/1PIWlAHkfn17txYAzPT06RyFL9qvekK33Gw0TH4MsBxQ/edit){target="_blank"}).
### What is the NASA Earthdata Cloud?
NASA Earthdata Cloud is the NASA archive of Earth observations and is hosted in Amazon Web Services (AWS) cloud with DAAC tools and services built for use "next to the data." The NASA DAACs (data centers) are currently transitioning to this cloud-based environment. The cloud offers a scalable and effective way to address storage, network, and data movement concerns while offering a tremendous amount of flexibility to the user. Particularly if working with large data volumes, data access and processing would be more efficient if workflows are taking place in the cloud, avoiding having to download large data volumes. Data download will continue to be freely available to users, from the Earthdata Cloud archive.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vS2qii_wY_hzDzamTqCWCU2SfLNG-Zh68uVzklwE1EJLyPi-xV1jdrSZKA_035D2nYIQnuHfE77gbFA/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vS2qii_wY_hzDzamTqCWCU2SfLNG-Zh68uVzklwE1EJLyPi-xV1jdrSZKA_035D2nYIQnuHfE77gbFA/pub?start=false&loop=false&delayms=3000){target="_blank"}
### Cloud Access Pathways
Three pathway examples, to interact and access data (and services) from and within the NASA Earthdata Cloud, are illustrated in the diagram. Green arrows and icons indicate working locally, after downloading data to your local machine, servers, or compute/storage space. Orange arrows and icons highlight a workflow within the cloud, setting up your own AWS EC2 cloud instance, or virtual machine, in the cloud next to the data. Blue arrows and icons also indicate a within the cloud workflow, through shareable cloud environments such as Binder or JupyterHub set up in an AWS cloud region. Note that each of these may have a range of cost models. EOSDIS data are being stored in the *us-west-2* region of AWS cloud; we recommend setting up your cloud computing environment in the same region as the data for free and easy in-cloud access.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vTFrXw0UQGB5cv6apCyv6VYDmfw0ZW_1seD6h61LmhfeqGmNdon-aU_Lo8IkE06z3ms0bD-sj831j5i/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vTFrXw0UQGB5cv6apCyv6VYDmfw0ZW_1seD6h61LmhfeqGmNdon-aU_Lo8IkE06z3ms0bD-sj831j5i/pub?start=false&loop=false&delayms=3000){target="_blank"}
A note on *costing*: What is free and what do I have to budget for, now that data is archived in the cloud?
- Downloading data from the Earthdata Cloud archive in AWS, to your local computer environment or local storage (e.g. servers) is and will continue to be free for the user.
- Accessing the data directly in the cloud (from *us-west-2* S3 region) is free. Users will need a NASA Earthdata Login account and AWS credentials to access, but there is no cost associated with these authentication steps, which are in place for security reasons.
- Accessing data in the cloud via EOSDIS or DAAC cloud-based tools and services such as the CMR API, Harmony API, OPenDAP API (from *us-west-2* S3 region) is free to the user. Having the tools and services "next to the data" in the cloud enables DAACs to support data reduction and transformation, more efficiently, on behalf of the user, so users only access the data they need.
- Cloud computing environments (i.e. virtual machines in the cloud) for working with data in the cloud (beyond direct or via services provided access) such as data analysis or running models with the data, is user responsibility, and should be considered in budgeting. I.e. User would need to set up a cloud compute environment (such as an EC2 instance or JupyterLab) and are responsible for any storage and computing costs.
- This means that even though direct data access in the cloud is free to the user, they would first need to have a cloud computing environment/machine to execute the data access step from, and then continue their analysis.
- Depending on whether that cloud environment is provided by the user themselves, user's institution, community hubs like Pangeo or NASA Openscapes JupyterLab sandbox, this element of the workflow may require user accountability, budgeting and user financial maintenance.
### Getting Started Roadmap
#### Cloud Workflow
The following is a conceptual roadmap for users getting started with NASA Earth Observations cloud-archived data using an in-cloud workflow (i.e. bringing user code into the cloud, avoiding data download and performing data workflows "next to the data").
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vR1ReEM5OSyjGqdOu6C_6KFCtiCL6cF_Y4-06uZnCs_PAiMyp4cwEpaA4iRsroLL39J1hSw27Co1-bX/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vR1ReEM5OSyjGqdOu6C_6KFCtiCL6cF_Y4-06uZnCs_PAiMyp4cwEpaA4iRsroLL39J1hSw27Co1-bX/pub?start=false&loop=false&delayms=3000){target="_blank"}
#### Local Workflow
The following is a conceptual roadmap for users getting started with NASA Earth Observations cloud-archived data using a local machine (e.g. laptop) workflow, as data storage and computational work.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vSpmQBM-BfRMdtut9qT2oc4uxxTDFpulXMCN6h_9TviDlujn-MZJ4gwX8ilHK1qg1hgvwrt_JXwlaVF/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vSpmQBM-BfRMdtut9qT2oc4uxxTDFpulXMCN6h_9TviDlujn-MZJ4gwX8ilHK1qg1hgvwrt_JXwlaVF/pub?start=false&loop=false&delayms=3000){target="_blank"}
### Tools & Services Roadmap
Below is a practical guide for learning about and selecting helpful tools or services for a given use case, focusing on how to find and access NASA Earthdata Cloud-archived data from local compute environment (e.g. laptop) or from a cloud computing workspace, with accompanying example tutorials. Once you follow your desired pathway, click on the respective blue notebook icon to get to the example tutorial. Note: these pathways are not exhaustive, there are many ways to accomplish these common steps, but these are some of our recommendations.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vQZluRH_3t4UsGM-g0LntY9txqQpx3Nr_AJ_D0IQjdrXMadE694Acsef9uCUANHPafsT16WdpQ8daFZ/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vQZluRH_3t4UsGM-g0LntY9txqQpx3Nr_AJ_D0IQjdrXMadE694Acsef9uCUANHPafsT16WdpQ8daFZ/pub?start=false&loop=false&delayms=3000){target="_blank"}
### Workflow Cheatsheet
The following is a practical reference guide with links to tutorials and informational websites for users who are starting to take the conceptual pieces and explore and implement in their own workflows.
<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vRR8Vi1K_c82ztwuNcZPMnD3CYZBtMo-NGPngpjh80UlrV3he6Y_mdtt5azNhdfN-lzQQrrqvBzDQSM/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
[Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vRR8Vi1K_c82ztwuNcZPMnD3CYZBtMo-NGPngpjh80UlrV3he6Y_mdtt5azNhdfN-lzQQrrqvBzDQSM/pub?start=false&loop=false&delayms=3000){target="_blank"}
## Slides
Selected presentations about working with NASA Earthdata on the Cloud; for all presentations see [nasa-openscapes.github.io > slides](https://nasa-openscapes.github.io/about.html#slides).
### [NASA Earthdata Cloud: Myths, Truths, Questions](https://docs.google.com/presentation/d/1rmzodGujlOQxy4ocl7g2BEqVxMOQoIS_Fw48rifvh0M/edit?usp=sharing)
by Amy Steiker, Kate Heightley (NSIDC) September 7, 2022
### [NSIDC DAAC User Working Group](https://docs.google.com/presentation/d/1Wtlgt-wUx8HgeWjoxvvz4BwknLlKDmGP2F-pSuI5utQ/edit?usp=sharing){target="_blank"}
by Andrew Barrett, Amy Steiker, Walt Meier, Jennie Roebuck, Mikala Beig, Luis Lopez, (NSIDC) May 20, 2022.
### [NASA Earthdata Cloud & The Cloud Paradigm](https://docs.google.com/presentation/d/1D-d3VQDG-KlRt6gnyfj5aN3yVo1FBsbYTaQInRB4eMw/edit#slide=id.p){target="_blank"}
by Aaron Friesz (LP DAAC), April 2022.