Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature-11: storeMetadata Public API method #27

Merged
merged 27 commits into from
Jun 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
4e995ae
Fix bug in junit test for 'writeTo...Checksums()' where assertion for…
doulikecookiedough Jun 22, 2023
78ae4f5
Fix latent bug for 'storeObject' & related methods when checksumAlgor…
doulikecookiedough Jun 23, 2023
1b6bc08
Refactor 'FileHashStore' initialization process to accept new key 'st…
doulikecookiedough Jun 23, 2023
5dbfaa5
Update 'HashStore' interface for revised methods 'storeMetadata()', r…
doulikecookiedough Jun 23, 2023
baaf828
Fix incorrect comparison operators used when comparing additionalAlgo…
doulikecookiedough Jun 23, 2023
b409bb5
Code check-in: add Public API method 'storeMetadata()', missing javad…
doulikecookiedough Jun 23, 2023
527eb27
Add new junit tests and javadocs for 'writeToTmpMetadataFile()' and a…
doulikecookiedough Jun 26, 2023
dca7b76
Add new junit tests and javadocs for 'putMetadata()'
doulikecookiedough Jun 26, 2023
415a570
Add new junit tests for 'storeMetadata()' and clean-up junit tests in…
doulikecookiedough Jun 26, 2023
380e141
Fix bug in 'storeMedata()' missing header section with 'formatId' and…
doulikecookiedough Jun 26, 2023
798a15a
Add new junit tests to verify 'storeMetadata()' content - body and he…
doulikecookiedough Jun 26, 2023
64598a5
Refactor .move() for new 'entity' parameter which provides control to…
doulikecookiedough Jun 27, 2023
b4b9fd2
Add new junit test to check 'storeMetadata()' metadataLockedIds handl…
doulikecookiedough Jun 27, 2023
5afe784
Add 'printStackTrace()' statement to failing test 'storeObject_object…
doulikecookiedough Jun 27, 2023
28cd9a3
Clean up .move() method to assert string with .equals() and .putMetad…
doulikecookiedough Jun 27, 2023
9112cbd
Revise 'storeObject_objectLockedIds()' junit test to assert 'PidObjec…
doulikecookiedough Jun 27, 2023
7f90049
Update javadoc description for 'storeObject_objectLockedIds' to expla…
doulikecookiedough Jun 27, 2023
e6e7816
Re-implement previous move() method via moveObject() to test exceptio…
doulikecookiedough Jun 27, 2023
e23d431
Remove debug method .moveObject()
doulikecookiedough Jun 27, 2023
de37d93
Fix typo in javadoc for 'storeObject_objectLockedIds'
doulikecookiedough Jun 27, 2023
32e2709
Add new junit test to test 'storeObject()' with two threads
doulikecookiedough Jun 27, 2023
58f9b74
Revise original 'storeObject_objectLockedIds' test to execute 5 threa…
doulikecookiedough Jun 27, 2023
de5d52a
Add debug statement to 'storeObject_objectLockedIds_FiveThreads' to v…
doulikecookiedough Jun 27, 2023
bfd8cfa
Re-add or assertion statement to 'storeObject_objectLockedIds_TwoThre…
doulikecookiedough Jun 27, 2023
2425e0a
Update javadoc for junit test 'storeObject_objectLockedIds_TwoThreads'
doulikecookiedough Jun 27, 2023
b920585
Update HashStore interface javadoc and exceptions for 'storeMetadata()'
doulikecookiedough Jun 27, 2023
b012841
Add missing exception catch blocks in 'storeMetadata()', add new juni…
doulikecookiedough Jun 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 40 additions & 23 deletions src/main/java/org/dataone/hashstore/HashStore.java
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package org.dataone.hashstore;

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchAlgorithmException;
Expand Down Expand Up @@ -62,24 +63,37 @@ HashAddress storeObject(InputStream object, String pid, String additionalAlgorit
throws NoSuchAlgorithmException, IOException, PidObjectExistsException, RuntimeException;

/**
* The `storeSysmeta` method is responsible for adding and/or updating metadata
* (`sysmeta`) to disk using a given InputStream and a persistent identifier
* (pid). The metadata object consists of a header and body portion. The header
* is formed by writing the namespace/format (utf-8) of the metadata document
* followed by a null character `\x00` and the body follows immediately after.
* The `storeMetadata` method is responsible for adding/updating metadata
* (ex. `sysmeta`) to disk using a given InputStream, a persistent identifier
* (pid) and metadata format (formatId). The metadata object consists of a
* header and body portion. The header is formed by writing the namespace/format
* (utf-8) of the metadata document followed by a null character `\u0000` and
* the body (metadata content) follows immediately after.
*
* Upon successful storage of sysmeta, the method returns a String that
* represents the file's permanent address, and similarly to storeObject, this
* permanent address is determined by calculating the SHA-256 hex digest of the
* provided pid. Finally, sysmeta are stored in parallel to objects in the
* `/[...storeDirectory]/sysmeta/` directory.
* The permanent address of the metadata document is determined by calculating
* the SHA-256 hex digest of the provided `pid` + `format_id`; and the body
* contains the metadata content (ex. `sysmeta`).
*
* @param sysmeta Input stream to metadata document
* @param pid Authority-based identifier
* @return String representing metadata address
* @throws Exception TODO: Add specific exceptions
* Upon successful storage of metadata, `store_metadata` returns a string that
* represents the file's permanent address. Lastly, the metadata objects are
* stored in parallel to objects in the `/store_directory/metadata/` directory.
*
* @param metadata Input stream to metadata document
* @param pid Authority-based identifier
* @param formatId Metadata namespace/format
* @return Metadata content identifier (string representing metadata address)
* @throws IOException When there is an error writing the metadata
* document
* @throws IllegalArgumentException Invalid values like null for metadata, or
* empty pids and formatIds
* @throws FileNotFoundException When temp metadata file is not found
* @throws InterruptedException metadataLockedIds synchronization issue
* @throws NoSuchAlgorithmException Algorithm used to calculate permanent
* address is not supported
*/
String storeSysmeta(InputStream sysmeta, String pid) throws Exception;
String storeMetadata(InputStream metadata, String pid, String formatId)
throws IOException, IllegalArgumentException, FileNotFoundException, InterruptedException,
NoSuchAlgorithmException;

/**
* The `retrieveObject` method retrieves an object from disk using a given
Expand All @@ -94,14 +108,15 @@ HashAddress storeObject(InputStream object, String pid, String additionalAlgorit
BufferedReader retrieveObject(String pid) throws Exception;

/**
* The 'retrieveSysmeta' method retrieves the metadata content from disk and
* returns it in the form of a String using a given persistent identifier.
* The 'retrieveMetadata' method retrieves the metadata content of a given pid
* and metadata namespace from disk and returns it in the form of a String.
*
* @param pid Authority-based identifier
* @param pid Authority-based identifier
* @param formatId Metadata namespace/format
* @return Sysmeta (metadata) document of given pid
* @throws Exception TODO: Add specific exceptions
*/
String retrieveSysmeta(String pid) throws Exception;
String retrieveMetadata(String pid, String formatId) throws Exception;

/**
* The 'deleteObject' method deletes an object permanently from disk using a
Expand All @@ -114,14 +129,16 @@ HashAddress storeObject(InputStream object, String pid, String additionalAlgorit
boolean deleteObject(String pid) throws Exception;

/**
* The 'deleteSysmeta' method deletes an metadata document (sysmeta) permanently
* from disk using a given persistent identifier.
* The 'deleteMetadata' method deletes a metadata document (ex. `sysmeta`)
* permanently from disk using a given persistent identifier and its respective
* metadata namespace.
*
* @param pid Authority-based identifier
* @param pid Authority-based identifier
* @param formatId Metadata namespace/format
* @return
* @throws Exception TODO: Add specific exceptions
*/
boolean deleteSysmeta(String pid) throws Exception;
boolean deleteMetadata(String pid, String formatId) throws Exception;

/**
* The 'getHexDigest' method calculates the hex digest of an object that exists
Expand Down
Loading