Welcome to NPSdataverse Discussions!

[RobLBaker]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[HeartlandNetwork]

Hi Rob, everyone:

I am the data manager at HTLN. Below is a bit of a narrative from my first-time experience with packaging data (a work in progress). I would be very interested in the experiences that others have to share about the data packaging process.

I am slowly working through the data packaging EML process for our breeding bird dataset. Getting the complete taxonomy, Kingdom - > species for all our bird species has been pretty time consuming. Once I finish that, we have about ten data tables of habitat data along with supporting LUTs to include in the data package. The NPS dataverse installed just fine for me after I had RTools installed. I am trying to make the call as to whether to just join the LUT tables to the data or include them as separate LUTs. Either way, I want to represent them as categorical data. As you might expect, the real work in creating a data package is the data processing, cleaning and transformations.

[RobLBaker]

Gareth,
I'm glad to hear the NPSdataverse installed and seems to be working well. Plese do let us know if you run into complications.

That sounds like an impressive data package with a lot of different components. This is certainly the early stages for most (all?) of us in terms of data package creation. I think for the next little bit there is likely to be a lot of back-and-forth and we bounce ideas for best practices and useful tools off each other.

From the metadata perspective, EMLassemblyline should be able to ingest a column of scientific names (typically genus + species) and generate the taxonomic hierarchy all the way through Kingdom for you. It's one of the cooler parts of the package. If you want to include that taxonomic hierarchy in the data files themselves, that's a different problem. We could certainly look into some function to reverse EMLassembyline's process and extract all of the taxonomy into different data columns, or we could see if there are other existing functions to perform a similar task. I hope I correctly understood the issue, but if not feel free to elaborate.

In terms of how to structure the data within a data package - whether to lump it or split it - that's purposefully left up to the data package creator. It's a bit of an artform. I suggest you think about how to set up a future data user (perhaps yourself or your immediate collaborators) to most easily access and use the data. It would be nice to have data that was coherently organized and easily accessible to a downstream user without a lot of complicated joins or merges. That said, it's impossible to anticipate all potential future uses for a given data package and so also (I think) impossible to design a single data package that is perfectly organized for all potential users. And, of course, all of this takes time and effort. So it's also worth balancing the cost and time associated with producing a very structured data package with the ultimate goal of publishing the data in a timely fashion (and with completing all the competing tasks that I'm sure are on your plate).

[HeartlandNetwork]

I was not aware that EMLassemblyline generates the higher taxonomy from a Genus species list. I need to go back and read up on that. It would be super helpful if that step was automated.

[RobLBaker]

Well, I'm not sure if this is what you're looking for, but if you have a column of scientificNames (say, genus + species or genus + species + variety or even, I think, something where they are only ID'd to family) it should auto-fill the metadata. You definitely don't want to do it by hand! For instance, in an example dataset where the scientificName column included "Paspalum monostachyum", EML editor will generate the following (I hope the spacing/indentation holds up) and insert it into the taxonomic coverage component of the .xml file:

<taxonomicClassification>
 <taxonRankName>kingdom</taxonRankName>
 <taxonRankValue>Plantae</taxonRankValue>
 <commonName>plants</commonName>
 <taxonId provider="https://itis.gov">202422</taxonId>
 <taxonomicClassification>
  <taxonRankName>subkingdom</taxonRankName>
  <taxonRankValue>Viridiplantae</taxonRankValue>
  <commonName>green plants</commonName>
  <taxonId provider="https://itis.gov">954898</taxonId>
  <taxonomicClassification>
   <taxonRankName>infrakingdom</taxonRankName>
   <taxonRankValue>Streptophyta</taxonRankValue>
   <commonName>land plants</commonName>
   <taxonId provider="https://itis.gov">846494</taxonId>
   <taxonomicClassification>
    <taxonRankName>superdivision</taxonRankName>
    <taxonRankValue>Embryophyta</taxonRankValue>
    <taxonId provider="https://itis.gov">954900</taxonId>
    <taxonomicClassification>
     <taxonRankName>division</taxonRankName>
     <taxonRankValue>Tracheophyta</taxonRankValue> 
     <commonName>vascular plants</commonName>
     <commonName>tracheophytes</commonName>
     <taxonId provider="https://itis.gov">846496</taxonId>
     <taxonomicClassification>
      <taxonRankName>subdivision</taxonRankName>
      <taxonRankValue>Spermatophytina</taxonRankValue>
      <commonName>spermatophytes</commonName>
      <commonName>seed plants</commonName>
      <taxonId provider="https://itis.gov">846504</taxonId>
      <taxonomicClassification>
       <taxonRankName>class</taxonRankName>
       <taxonRankValue>Magnoliopsida</taxonRankValue>
       <taxonId provider="https://itis.gov">18063</taxonId>
       <taxonomicClassification>
        <taxonRankName>superorder</taxonRankName>
        <taxonRankValue>Lilianae</taxonRankValue>
        <commonName>monocots</commonName>
        <commonName>monocotyledons</commonName>
        <taxonId provider="https://itis.gov">846542</taxonId>
        <taxonomicClassification>
         <taxonRankName>order</taxonRankName>
         <taxonRankValue>Poales</taxonRankValue>
         <taxonId provider="https://itis.gov">846620</taxonId>
         <taxonomicClassification>
          <taxonRankName>family</taxonRankName>
          <taxonRankValue>Poaceae</taxonRankValue>
          <taxonId provider="https://itis.gov">40351</taxonId>
          <taxonomicClassification>
           <taxonRankName>genus</taxonRankName> 
           <taxonRankValue>Paspalum</taxonRankValue>
           <taxonId provider="https://itis.gov">40990</taxonId>
           <taxonomicClassification>
            <taxonRankName>species</taxonRankName>
            <taxonRankValue>Paspalum monostachyum</taxonRankValue>
            <taxonId provider="https://itis.gov">41030</taxonId>
            </taxonomicClassification>
           </taxonomicClassification>
          </taxonomicClassification>
         </taxonomicClassification>
        </taxonomicClassification>
       </taxonomicClassification>
      </taxonomicClassification>
     </taxonomicClassification>
    </taxonomicClassification>
   </taxonomicClassification>
  </taxonomicClassification>
</taxonomicClassification>

[HeartlandNetwork]

Thanks Rob! I will definitely give this a try.

[juddpatterson]

Great discussion, and then once EMLassemblyline (or ezEML) have helped automate the taxonomic section in metadata, the next DataStore release will be able to harvest that XML and populate the "Taxonomy" tab too! Definitely reach out if you have other questions or want us to take a look at an interim product. We are all learning together.