Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement datasetType metadata block support (at global level) #10519

Closed
cmbz opened this issue Apr 22, 2024 · 6 comments · Fixed by #11001
Closed

Implement datasetType metadata block support (at global level) #10519

cmbz opened this issue Apr 22, 2024 · 6 comments · Fixed by #11001
Labels
FY25 Sprint 8 FY25 Sprint 8 (2024-10-09 - 2024-10-23) FY25 Sprint 9 FY25 Sprint 9 (2024-10-23 - 2024-11-06) GREI Year 3 Year 3 GREI task GREI 6 Connect Digital Objects Requested SPA Improvement Desired improvement for SPA, do not duplicate JSF UI Size: 50 A percentage of a sprint. 35 hours. Type: Feature a feature request
Milestone
6.6

Comments

@cmbz
Copy link

cmbz commented Apr 22, 2024 •
edited by pdurbin
Loading

Overview

Proposed release not snippet (scope of work)

(See also notes where this snippet was iterated on.)

Metadata Blocks Can Be Associated with Dataset Types

Metadata blocks (e.g. "codemeta") can now be associated with dataset types (e.g. "software") using new superuser APIs.

This will have the following effects for the APIs used by the new Dataverse UI:

  • The list of fields shown when creating a dataset will include fields marked as "displayoncreate" (in the tsv/database) for metadata blocks (e.g. "codemeta") that are associated with the dataset type (e.g. "software") that is passed to the API.
  • The metadata blocks shown when editing a dataset will include metadata blocks (e.g. "codemeta") that are associated with the dataset type (e.g. "software") that is passed to the API.

For more information, see the guides and #10519.

Depends on

See also
@cmbz cmbz added Type: Feature a feature request Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) GREI 6 Connect Digital Objects labels Apr 22, 2024
@cmbz cmbz mentioned this issue Apr 22, 2024
Open
18 tasks
@cmbz cmbz moved this to SPRINT READY in IQSS Dataverse Project May 6, 2024
@qqmyers qqmyers mentioned this issue May 9, 2024
Open
9 tasks
@cmbz cmbz added the GREI Year 3 Year 3 GREI task label May 9, 2024
@cmbz cmbz mentioned this issue May 9, 2024
Open
@cmbz cmbz added FY25 Sprint 5 FY25 sprint 5 Status: Waiting for Related Issues/PRs This issue depends upon the completion of one or more issues/PRs labels Aug 29, 2024
@pdurbin pdurbin removed the Status: Waiting for Related Issues/PRs This issue depends upon the completion of one or more issues/PRs label Sep 6, 2024
@pdurbin
Copy link
Member

pdurbin commented Sep 11, 2024 •
edited
Loading

As discussed at tech hours yesterday, I pulled this into FY25 Sprint 5 two weeks ago without realizing how poorly defined this issue is and without grasping that the proposal, as written, implies a lot of work on the JSF code, which we intend to dispose of. 🗑️

I liked the suggestion from @qqmyers that a possible direction would be to create a series of superuser-only APIs that allows an installation to be configured to associate a dataset type with metadata fields. However, what should the JSON look like that will ultimately be used by the SPA? Should we hide this output behind the "admin" API for now and check in with the frontend team (when they are ready) to see if the JSON is agreeable? In short, there would be no value to the end user yet, but we could make progress on database tables and business logic.

@scolapasta suggested that a smaller group (than all of tech hours) could discuss this issue further and unblock it. Meanwhile, perhaps the size should be removed and it should go back into "need sizing" for this discussion to happen (heads up to @cmbz).

@pdurbin pdurbin added the Status: Needs Input Applied to issues in need of input from someone currently unavailable label Sep 11, 2024
@cmbz cmbz added the FY25 Sprint 6 FY25 Sprint 6 label Sep 11, 2024
@pdurbin pdurbin removed FY25 Sprint 5 FY25 sprint 5 FY25 Sprint 6 FY25 Sprint 6 labels Sep 11, 2024
@cmbz
Copy link
Author

cmbz commented Sep 25, 2024 •
edited
Loading

2024/09/25:

@cmbz cmbz added the Requested SPA Improvement Desired improvement for SPA, do not duplicate JSF UI label Sep 30, 2024
@cmbz
Copy link
Author

cmbz commented Sep 30, 2024 •
edited
Loading

2024/09/30: See discussion and decisions here

@pdurbin
Copy link
Member

pdurbin commented Sep 30, 2024

Notes from our discussion (thanks, all).

In the issue description I'll copy the important bits and might try to future clarify things.

Updated: Sep 30, 2024
datasetType Discussion
2024/09/30

Attendees: Ceilyn Boyd Gustavo Durand Philip Durbin Jim Myers

Notes

  • Implement datasetType metadata block support (at global level) #10519
  • The API-only option for configuration will meet GREI needs for Year 3, along with the Java/back-end changes to show the configured blocks when creating/editing a dataset. We will address the UI in Year 4 when the SPA beta is completed.
  • The current UI / SPA should already be leveraging this code in the way they get the metadata blocks, or should only require small tweaks.
  • A similar plan should work for configuring licenses for dataset types. Implement list of allowed datasetType licenses (at global level) · Issue Implement list of allowed datasetType licenses (at global level) #10520 · IQSS/dataverse (github.com)
  • Templates are out of scope and not expected to change
  • Tasks
    • Create a series of superuser-only APIs that allows an installation to be configured to associate a dataset type with metadata blocks (fields in the future) at the global/installation level
    • Modify underlying code to get fields that's used by JSF and API
    • Modify the API the SPA uses to get the fields, allow type=software to be passed
    • Change displayOnCreate to true for CodeMeta fields
  • Test case 1
    • don't enable CodeMeta for a collection
    • create dataset with type=software and ensure that CodeMeta fields show up
  • Ask SPA team to add a query parameter for type=software?

Action items
*

@pdurbin pdurbin added Size: 50 A percentage of a sprint. 35 hours. and removed Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) Status: Needs Input Applied to issues in need of input from someone currently unavailable labels Sep 30, 2024
@pdurbin pdurbin self-assigned this Oct 9, 2024
@pdurbin pdurbin moved this from SPRINT READY to In Progress 💻 in IQSS Dataverse Project Oct 9, 2024
@cmbz cmbz added the FY25 Sprint 8 FY25 Sprint 8 (2024-10-09 - 2024-10-23) label Oct 9, 2024
@cmbz cmbz added the FY25 Sprint 9 FY25 Sprint 9 (2024-10-23 - 2024-11-06) label Oct 23, 2024
@pdurbin pdurbin mentioned this issue Oct 28, 2024
Open
pdurbin added a commit that referenced this issue Nov 4, 2024
@pdurbin
allow links between dataset types and metadata blocks #10519
264d648
@pdurbin pdurbin mentioned this issue Nov 4, 2024
Merged
pdurbin added a commit that referenced this issue Nov 5, 2024
@pdurbin
Merge branch 'develop' into 10519-dataset-types #10519
1b45992
@pdurbin pdurbin moved this from In Progress 💻 to Done 🧹 in IQSS Dataverse Project Nov 5, 2024
@pdurbin pdurbin removed their assignment Nov 5, 2024
pdurbin added a commit that referenced this issue Nov 5, 2024
@pdurbin
add missing "Command" suffix #10519
399fe8d
pdurbin added a commit that referenced this issue Nov 5, 2024
@pdurbin
remove debug line #10519
be83ada
pdurbin added a commit that referenced this issue Nov 5, 2024
@pdurbin
improve docs #10519
a7eee45
pdurbin added a commit that referenced this issue Nov 5, 2024
@pdurbin
rename "ownerDataverse" to "dataverse", remove TODOs #10519
4c45fc0
pdurbin added a commit that referenced this issue Nov 6, 2024
@pdurbin
enable codeMeta20 test, disable size=6 test #10519
4f77a3b
pdurbin added a commit that referenced this issue Nov 8, 2024
@pdurbin
load codemeta metadata block in docker dev persona #10519
5afad8e 
It's used in API tests.
pdurbin added a commit that referenced this issue Dec 13, 2024
@pdurbin
Merge branch 'develop' into 10519-dataset-types #10519
1ffeee7
@cmbz
Copy link
Author

cmbz commented Jan 22, 2025

2025-01-22: Closing as complete: #10519.

@cmbz cmbz closed this as completed Jan 22, 2025
pdurbin added a commit that referenced this issue Jan 23, 2025
@pdurbin
Merge branch 'develop' into 10519-dataset-types #10519
a9ed64e
pdurbin added a commit that referenced this issue Jan 23, 2025
@pdurbin
fix JsonPrinter and failing tests #10519
faf1d37 
JsonPrinter was edited in #10764 and #11066 which were merged five and
two weeks ago, respectively. This caused merge conflicts which should
be resolved correctly now. In this commit, we are starting with
JsonPrinter as of the most recent "develop" branch commit (7fdb21a)
and adding in our DatasetType changes.

PR #10764 resulted in the number of metadata fields being reduced from
80 to 35 and this commit make the same change in an assertion.
pdurbin added a commit that referenced this issue Jan 23, 2025
@pdurbin
cleanup #10519
2773c2e
pdurbin added a commit that referenced this issue Jan 23, 2025
@pdurbin
remove unnecessary sending of content type #10519
e52052a
pdurbin added a commit that referenced this issue Jan 23, 2025
@pdurbin
clarify docs #10519
57e0c71
pdurbin added a commit that referenced this issue Jan 24, 2025
@pdurbin
stop loading Codemeta in Docker, add setDisplayOnCreate API #10519
9b2fb8d 
In Docker, we were testing with the Codemeta block. This made for a nice
real-world story of creating a "software" dataset type and using it with
the Codemeta block.

The Codemeta block also has the advantage of having some fields that are
set to displayOnCreate, which helped us make assertions that the code is
working properly. It's the only non-citation block with fields set to
displayOnCreate.

However, Jenkins doesn't have the Codemeta block, meaning that tests are
failing. Also, we aren't ready to promote the Codemeta block to be
shipped with Dataverse because a new version is out:
#10859

So, we are switching from Codemeta to the Astrophysics block. We create
an "instrument" dataset type. Like all non-citation blocks (except for
Codemeta), there are no fields that are set to displayOnCreate=true.
Therefore, we added an API for this.
@pdurbin
Copy link
Member

pdurbin commented Jan 24, 2025

Reopening. This issue will be auto-closed when this PR is merged:

@pdurbin pdurbin reopened this Jan 24, 2025
pdurbin added a commit that referenced this issue Feb 3, 2025
@pdurbin
Merge branch 'develop' into 10519-dataset-types #10519
b5fffaa 
Conflicts:
src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java
pdurbin added a commit that referenced this issue Feb 5, 2025
@pdurbin
fix typo in docs (PUT vs. POST) #10519
ce03b30
@pdurbin pdurbin added this to the 6.6 milestone Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
FY25 Sprint 8 FY25 Sprint 8 (2024-10-09 - 2024-10-23) FY25 Sprint 9 FY25 Sprint 9 (2024-10-23 - 2024-11-06) GREI Year 3 Year 3 GREI task GREI 6 Connect Digital Objects Requested SPA Improvement Desired improvement for SPA, do not duplicate JSF UI Size: 50 A percentage of a sprint. 35 hours. Type: Feature a feature request
Projects
None yet
Milestone
6.6
Development

Successfully merging a pull request may close this issue.

allow links between dataset types and metadata blocks IQSS/dataverse
2 participants
@pdurbin @cmbz