The catalogue is IUDXs data discovery and dataset metadata publishing portal. It allows data providers to publish their data resources by making an IUDX vocabulary annotated meta-data document describing their datasource and affiliated terminologies. The datasources publish their data to the IUDX Resource Server. It allows consumers of such data to easily discover such resources by means of powerful queries and consume the data from Resource Servers in an automated and machine interpretable way.
- Search and discovery of data resources hosted on IUDX platform
- Support for text, geo-spatial, relationship, list and attributes searches
- Upload, delete and modify operations on catalogue objects (meta-information corresponding to resources)
- Stores meta-information as JSON-LD documents using published vocabulary and attributes
- Rate and review data resources hosted on IUDX platform
- Scalable, service mesh architecture based implementation using open source components: Vert.X API framework, Elasticsearch for data-base
- Hazelcast and Zookeeper based cluster management and service discovery
The live running instance of the IUDX catalogue can be found here.
The api docs can be found here.
Make a config file based on the template in ./configs/config-example.json
- Generate a certificate using Lets Encrypt or other methods
- Make a Java Keystore File and mention its path and password in the appropriate sections
- Modify the database url and associated credentials in the appropriate sections
- Install docker and docker-compose
- Clone this repo
- Build the images
./docker/build.sh - Modify the
docker-compose.ymlfile to map the config file you just created - Start the server in production (prod) or development (dev) mode using docker-compose
docker-compose up prod
- Install java 13 and maven
- Use the maven exec plugin based starter to start the server
mvn clean compile exec:java@catalogue-server
A hot-swappable redeployer is provided for quick development
./redeploy.sh
The server requires certificates to be stored in Java keystore format.
- Obtain certs for your domain using Letsencrypt. Note: Self signed certificates using openssl will also work.
- Concat all pems into one file
sudo cat /etc/letsencrypt/live/demo.example.com/*.pem > fullcert.pem - Convert to pkcs format
openssl pkcs12 -export -out fullcert.pkcs12 -in fullcert.pem - Create new temporary keystore using JDK keytool, will prompt for password
keytool -genkey -keyalg RSA -alias mykeystore -keystore mykeystore.ks
keytool -delete -alias mykeystore -keystore mykeystore.ks - Make JKS, will prompt for password
keytool -v -importkeystore -srckeystore fullcert.pkcs12 -destkeystore mykeystore.ks -deststoretype JKS - Store JKS in config directory and edit the keyfile name and password entered in previous step
- Mention keystore mount path (w.r.t docker-compose) in config.json
- Run the server through either docker, maven or redeployer
- Run the unit tests and generate a surefire report
mvn clean test-compile surefire:test surefire-report:report - Reports are stored in
./target/
Integration tests are through Postman/Newman whose script can be found from here.
- Install prerequisites
- Example Postman environment can be found here
- Run the server through either docker, maven or redeployer
- Run the integration tests and generate the newman report
newman run <postman-collection-path> -e <postman-environment> --insecure -r htmlextra --reporter-htmlextra-export . - Reports are stored in
./target/
We follow Git Merge based workflow
- Fork this repository
- Create a new feature branch in your fork. Multiple features must have a hyphen separated name, or refer to a milestone name as mentioned in Github -> Projects
- Commit to your fork and raise a Pull Request with upstream