Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data table previews #1758

Closed
laurenwalker opened this issue May 18, 2021 · 11 comments · Fixed by #2523
Closed

Data table previews #1758

laurenwalker opened this issue May 18, 2021 · 11 comments · Fixed by #2523
Assignees
@robyngit
Labels
ADC CI-14 Data search and display improvements (ADC deliverable) arctic data center enhancement Epic ESS-DIVE Issues associated with the ESS-DIVE project
Milestone
2.32.0

Comments

@laurenwalker
Copy link
Member

Display a preview of the data table so researchers can browse a few rows of data before downloading.

Milestone CI-14 in ADC proposal.
This was referenced May 18, 2021
Closed
Closed
@robyngit
Copy link
Member

As part of the markdown editor, I created a TableEditorView. Just wanted to point this out in case the view can be re-purposed as a data table previewer. The view would need to be updated to include a non-editable mode, and to parse csv data (in addition to the currently supported markdown and JSON formats).

@amoeba
Copy link
Contributor

amoeba commented May 18, 2021

This would be a great addition.

Seems like we might want to tweak the behavior of MNRead.get to support either HTTP range requests and/or streaming out responses. For very small files, grabbing the entire thing in order to build a preview isn't terrible but the experience gets worse as the size of the previewed file grows. Range requests would allow us to do a lot of neat things beyond just previews too. If we did this, we might consider whether or not partial reads count as read events.

@laurenwalker
Copy link
Member Author

I think partial reads should definitely count as view events, but not as download events, which they currently would be.

@laurenwalker laurenwalker changed the title Data table previews on the dataset landing page Data table previews Dec 21, 2021
@laurenwalker laurenwalker mentioned this issue Mar 22, 2022
Closed
@laurenwalker
Copy link
Member Author

We also recently had a request via #1969 to display previews of .txt files as well. We should include that in our overall design for previewing data tables, too.

@robyngit robyngit added arctic data center ADC CI-14 Data search and display improvements (ADC deliverable) labels May 17, 2023
@vchendrix
Copy link
Collaborator

Hi there @robyngit I would like to +1 this issue ticket.

@robyngit robyngit added the ESS-DIVE Issues associated with the ESS-DIVE project label Oct 2, 2023
@robyngit
Copy link
Member

robyngit commented Oct 2, 2023

Thanks @vchendrix, looking into what would be involved to implement this

@robyngit robyngit mentioned this issue Oct 2, 2023
Open
@robyngit
Copy link
Member

robyngit commented Oct 2, 2023

@vchendrix we discussed this as a team today. There are some backend changes required before we're able to move forward with this in MetacatUI. These are implied in the comments above, but to summarize:

  • Metacat doesn't support downloading a portion of a file, so to preview files we'd need to download the entire thing. This would be very slow, especially for large files.
  • Other than performance, the issue is that there is currently no way to download a file for preview without counting it as a download event. So a preview feature like this would skew our metrics pretty badly.

I've made in new a new Metacat issue to cover this. It's on the roadmap, but not currently a high priority for us.

@vchendrix
Copy link
Collaborator

vchendrix commented Oct 5, 2023 via email •
edited
Loading

@robyngit
Copy link
Member

robyngit commented Oct 5, 2023

Previewing only smaller and simple files was my initial thought as well @vchendrix! There would still be a problem with inflating the metrics though. We can't retrieve files without adding a "download" event to the metrics counts. So anytime someone viewed a dataset with some of the data previewed, download events would be recorded, and eventually those statistics would be inflated and not very representative anymore.

If the preview wasn't automatic, for example, a user had to click a button to explore a file, then perhaps there's an argument for counting that as a download... But probably something to get some feedback on and discuss. What do you think?

@vchendrix
Copy link
Collaborator

Previewing only smaller and simple files was my initial thought as well @vchendrix! There would still be a problem with inflating the metrics though. We can't retrieve files without adding a "download" event to the metrics counts. So anytime someone viewed a dataset with some of the data previewed, download events would be recorded, and eventually those statistics would be inflated and not very representative anymore.

If the preview wasn't automatic, for example, a user had to click a button to explore a file, then perhaps there's an argument for counting that as a download... But probably something to get some feedback on and discuss. What do you think?

That is what I had in mind. Not an automatic preview but an intentional one. And maybe "preview" is the wrong term if they are downloading and viewing the file in their browser. I was thinking that there doesn't need to be any additional javascript for viewing the file just use the browser capabilities as most browsers can render files of different types in their native format.

@robyngit
Copy link
Member

robyngit commented Jul 8, 2024

I think the MVP for this feature is:

  • Include a "preview" button for files under a maximum file size and of certain file types
  • This will open a modal with the data displayed

@robyngit robyngit self-assigned this Sep 9, 2024
@robyngit robyngit added this to the 2.32.0 milestone Sep 9, 2024
@robyngit robyngit mentioned this issue Sep 10, 2024
Open
7 tasks
robyngit added a commit that referenced this issue Sep 10, 2024
@robyngit
Fix eslint issues in MetadataView
e682b3c 
Issue #1758
@robyngit robyngit mentioned this issue Sep 10, 2024
Merged
2 tasks
robyngit added a commit that referenced this issue Sep 10, 2024
@robyngit
Fix eslint issues in MetadataView
351b21c 
and remove unused imports

Issue #1758
robyngit added a commit that referenced this issue Sep 10, 2024
@robyngit
Fix & add JS docs to MetadataView
a007aec 
Issue #1758
robyngit added a commit that referenced this issue Sep 10, 2024
@robyngit
Add papaparse and update app.js
5dd57a8 
Issue #1758
robyngit added a commit that referenced this issue Sep 10, 2024
@robyngit
Support rendering TableEditorView from csv
cba360e 
Issue #1758
robyngit added a commit that referenced this issue Sep 11, 2024
@robyngit
Add view only mode to table editor
b84570d 
Issue #1758
robyngit added a commit that referenced this issue Sep 11, 2024
@robyngit
Keep sort function in table editor view mode
72b0634 
Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
@robyngit
Modularize downloadWithCredentials in SolrResult
1f72aed 
- Separate out the logic for getting the data from DataONE from the logic for downloading that data onto the user's machine.
- Allows us to use the same data fetching logic in other parts of the app, i.e. the DataONEObjectView.
- Also separate out the logic for getting the file name.
- Add a tests for the new functions

Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
@robyngit
Add a View for getting & displaying Data Objects
646cd14 
- MVP, shows CSVs in a table (no other file types yet, no loading spinner, or error handling)
- Ensure TableEditorView returns the view from render()

Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
@robyngit
Create a button to open modal & view data object
b28a85c 
Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
@robyngit
Render the view data button in the metadata view
04a3c8b 
- Move rendering of the download button & view data button to a new method
- Render both buttons in a container div

Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
@robyngit
Display loading & error msgs in data object view
9088638 
Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Speed up rendering large tables in TableEditorView
4445212 
- Instead of creating an empty table, then populating it, we now add table cell content as it's created
- Instead of rendering the table in editing mode then removing editing elements if the table is in view-mode, we now render the table without editing elements if the table is in view-mode
- Also use document fragment to append table rows to the table element, which is faster than appending each row individually

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Set max num of cells to display in TableEditorView
73fcc25 
- Truncate data and show a message when the table is too large

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Row numbering & parsing fixes for TableEditorView
26e7a58 
- Add row numbers to parsed data rather than assuming the first column is the row number
- Handle parsing errors more gracefully
- Don't show empty CSV lines as data rows
- Allow multiple TableEditorView instances on the same page (Fix issues with dropdowns & unique IDs)
- Fix issue with adding new rows & cols in editor mode

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Set supported file types & sizes in DataObjectView
d213d71 
- Show error message when file type is not supported or file size is too large

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Enable initializing DataObjectView w/ PID
21a6e20 
Can now initialize DataObjectView with a object ID or SolrModel

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Hide ViewObjectButton for invalid filetype or size
cc43393 
Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Return response from Solr downloadWithCredentials
9580676 
Fixes SolrResult test
Fix a typo

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Show icon & name in Data Object modal view
2b6139d 
Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit
Show icon & name in Data Object modal view
7ef5093 
Issue #1758
@robyngit robyngit linked a pull request Sep 18, 2024 that will close this issue
Final misc. tasks to support data previews (Step 6 of issue #1758) #2523
Merged
4 tasks
robyngit added a commit that referenced this issue Sep 19, 2024
@robyngit
Show icon & name in Data Object modal view
d2df808 
Issue #1758
This was referenced Sep 20, 2024
Open
Open
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
@rushirajnenuji
Merge pull request #2518 from NCEAS/feature-1758-data-table-previews-1
e1e8264 
MetadataView & TableEditorView cleanup (Step 1 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
@rushirajnenuji
Merge pull request #2519 from NCEAS/feature-1758-data-table-previews-2
337975e 
Support rendering TableEditorView from csv (Step 2 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
@rushirajnenuji
Merge pull request #2520 from NCEAS/feature-1758-data-table-previews-3
c23f645 
Add a view-only mode to the Table Editor View (Step 3 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
@rushirajnenuji
Merge pull request #2521 from NCEAS/feature-1758-data-table-previews-4
8f3de73 
Create the DataObject View (Step 4 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 26, 2024
@rushirajnenuji
Merge pull request #2522 from NCEAS/feature-1758-data-table-previews-5
0990e79 
Create the ViewObjectButtonView and use in Metadata View (Step 5 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 26, 2024
@rushirajnenuji
Merge pull request #2523 from NCEAS/feature-1758-data-table-previews-6
85d4801 
Final misc. tasks to support data previews (Step 6 of issue #1758)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@robyngit robyngit

Labels
ADC CI-14 Data search and display improvements (ADC deliverable) arctic data center enhancement Epic ESS-DIVE Issues associated with the ESS-DIVE project
Projects
Arctic Data Center Management Plan Deliverables
Status: Done
Milestone
2.32.0
Development

Successfully merging a pull request may close this issue.

Final misc. tasks to support data previews (Step 6 of issue #1758) NCEAS/metacatui
4 participants
@amoeba @vchendrix @laurenwalker @robyngit