Development

README.md

@@ -1,59 +1,82 @@
 # Media Data
 
-This repository is a large dataset of records for testing against 
-the [vtmedia-schema](https://github.com/cjcodeproj/vtmedia-schema) project and the 
-[medialibrary](https://github.com/cjcodeproj/medialibrary) project.  It is available for test cases 
-where a larger sample of data may be required, or for importing data to other 
-systems, or it could be used as reference material for anyone looking to use the 
-schema to catalog their own media library.
+This repository is a large dataset suitable for testing either 
+the [vtmedia-schema](https://github.com/cjcodeproj/vtmedia-schema) project,  the 
+[medialibrary](https://github.com/cjcodeproj/medialibrary) project, or both.  It can
+be used for software testing, schema testing, testing XML tools, or as a reference for authoring
+your own data.
 
-Everything here is licensed under Creative Commons.
+![License](https://img.shields.io/github/license/cjcodeproj/mediadata)
 
+Everything here is licensed under a Creative Commons license.
 
-# Getting started
+# Examples
 
-Using the data just requires cloning this repository.  
-Depending on what you want to do with the data, you may want to clone one or both of 
-the other repositories listed below.
+Here are two examples that could be done with the test data.  In both cases, everything is done as a non-privileged
+user and the repository (or software) can be deleted if it's no longer required.
 
-| Repository | Purpose |
-| --- | --- |
-| [vtmedia-schema](https://github.com/cjcodeproj/vtmedia-schema) | Validation of data using XML tools |
-| [medialibrary](https://github.com/cjcodeproj/medialibrary) | Examination of data using Python code |
+## XMLSchema Testing
 
-Both repositories include README.md files that cover installation and management of the files.
+| Repository | Release | Purpose |
+| --- | --- | -- |
+| [vtmedia-schema](https://github.com/cjcodeproj/vtmedia-schema) | [![Release Download](https://img.shields.io/github/v/release/cjcodeproj/vtmedia-schema?display_name=tag)](https://github.com/cjcodeproj/vtmedia-schema/releases) | XSD/XML Validation Schema |
 
-Downloading the schema will let you use tools like xmllint, xmlstarlet, VS Code, or any other 
-tool that can work with XMLSchema.
+Download the schema repository using either a release, or the development branch.  The download
+should be in a separate directory path.
 
-Downloading the Python code will pull in the data and organize it into data structures.  If you don't intend to 
-modify the Python code base, just install it as a module into your local module path.
+There is a command line tool in the bin directory to run a validation check; it just needs to
+know where the schema is.  The tool requires `xmllint`, which most operating systems already
+have installed.
 
+The schema should also be usable by other tools, including xmlstarlet, PowerShell, VS Code,
+and most IDE development tools.
 
-# Branches
+```
+$ mkdir git
+$ cd git
+$ git clone https://github.com/cjcodeproj/mediadata.git
+$ git clone https://github.com/cjcodeproj/vtmedia-schema.git
+$ cd mediadata
+$ export XSDPATH=~/git/vtmedia-schema/schemas
+$ ./bin/movietest.sh
+```
 
-The `main` branch of the repository will always be usable with the `main` branches of the schema and 
-Python code repository.  The `development` branch will almost always be consistent with the development
-branches of the code.
+## Python Software Testing
 
-# Validation testing code
+| Repository | PyPi Release | Purpose |
+| --- | --- | -- |
+| [medialibrary](https://github.com/cjcodeproj/medialibrary) | [![PyPi version](https://img.shields.io/pypi/v/medialibrary)](https://pypi.org/project/medialibrary/) | Media API package |
 
-The `bin` directory includes one program for testing the XML files.  If you download the XML Schema, you 
-should change the `XPATH` variable to point to the location of the schema files.
+Download the code repository from PyPi, or pull down the development branch if you are testing
+unreleased features.  When the software is installed you can set an environmental variable 
+with the path of the dataset.
 
-# Examination code
 
-If you download the Python code repository, you can reference the location of the dataset by 
-defining an environmental variable called `MEDIAPATH`.  All the tools include command line
-help information.
+```
+$ mkdir git
+$ cd git
+$ git clone https://github.com/cjcodeproj/mediadata.git
+$ python3 -m pip install --user medialibrary
+$ export MEDIAPATH=~/git/mediadata
+$ python3 -m media.tools.movies.list
+```
 
-# Contributing
+# Repository Notes
+
+## Branches
+
+The `main` branch of this repository will always be usable with the `main` branches of the schema and 
+Python code repository; meaning it will always be suitable for testing production code.
+
+The `development` branch of this repository will almost always be consistent with the development
+branches of the schema or the Python code.
+
+## Contributing
 
 Contributions for corrections to existing data will be accepted.  New title submissions will not
-be accepted.
+be accepted at this time.
 
 If you want to generate new data for a given film, it should go in your own fork.
 
 Future updates will be posted to this repository, but there is no set schedule.  New titles 
 will be added in batches, under a separate branch, and then merged by a pull request.
-

bin/movietest.sh

@@ -5,7 +5,7 @@ CPATH=`dirname $0`
 . $CPATH/test_include.sh
 
 FILELIST=`find . -type f -name "*-bluray*.xml" -o -name "*-dvd.xml" -o -name "*-ultrahd.xml"  -o -name "*-vhs.xml"`
-# XPATH
+# XSDPATH
 
 COUNT=0
 ERRORS=0
@@ -17,8 +17,8 @@ CLEAN=()
 # Main Block
 #
 
-if [ -z $XPATH ]; then
-	echo "Need schema location in order for XML validation tests"
+if [ -z $XSDPATH ]; then
+	echo "Define XSDPATH if you want to do schema validation"
 fi
 
 for FI in $FILELIST
@@ -28,10 +28,11 @@ do
 	if [ $DEBUG ]; then
 		echo $FI
 	fi
-	if [ ! -z $XPATH ]; then
+	if [ ! -z $XSDPATH ]; then
 		testxml $FI
 	fi
 	testquotes $FI
+	# testtrailingws $FI
 	if [ -z $SKIPIT ]; then
 		testinstances $FI
 	fi
@@ -59,4 +60,4 @@ fi
 
 echo "Clean count: $CC"
 
-# echo ${CLEAN[*]}
+exit

bin/test_include.sh

@@ -1,34 +1,63 @@
 #!/bin/sh
 
+
+#
+# Test XML for validity
 testxml() 
 {
-        xmllint --xinclude --path $XPATH --noout --schema media-schema.xsd $1 --noout >/dev/null 2>&1
+        xmllint --xinclude --path $XSDPATH --noout --schema media-schema.xsd $1 --noout >/dev/null 2>&1
 	if [ $? -ne 0 ]; then
 		echo " ERROR -- Validation error found in $1"
 		((ERRORS = $ERRORS +1))
+		xmllint --xinclude --path $XSDPATH --noout --schema media-schema.xsd $1
 	fi
 
 }
 
+
+#
+# Tests for presence of double quotes vs. single quotes
+# (They're not bad, it's more of just a style issue)
 testquotes()
 {
 	grep -q '"' $1
 	if [ $? -eq 0 ]; then
-		echo "   ERROR --  Double quote found in $1"
-		((ERRORS = $ERRORS +1))
+		echo "   WARNING --  Double quote found in $1"
+		((WARNINGS = $WARNINGS +1))
 		((IC = $IC + 1))
 	fi
 }
 
+
+#
+# Test for trailing whitespace
+# (not bad, but technically increases the size of the file needlessly)
+testtrailingws()
+{
+	egrep -q " +$" $1
+	if [ $? -eq 0 ]; then
+		echo "  WARNING -- Trailing whitespace found in $1"
+		((WARNINGS = $WARNINGS +1))
+	fi
+}
+
+
+#
+# Test for instance element in the XML
+# (prevents user from accidentally saving possible personal data)
 testinstances()
 {
 	xmllint --xpath "//*[local-name()='instances']" --noout $1 >/dev/null 2>&1
 	if [ $? -eq 0 ]; then
 		echo "   ERROR -- Instances element found in $1"
-		((ERRORS = $ERRORS +1))
+		((WARNINGS = $WARNINGS +1))
 	fi
 }
 
+
+#
+# Test for comments
+# (Comments aren't bad, but you may not want to commit them)
 testcomments()
 {
 	grep -q '<!--' $1
@@ -38,9 +67,13 @@ testcomments()
 	fi
 }
 
+
+#
+# Test for keywords (experimental)
+# See how many keywords the movie had
 testkeywords()
 {
-	xmllint --xpath $XPATH --noout $1 >/dev/null 2>&1
+	xmllint --xpath $XSDPATH --noout $1 >/dev/null 2>&1
 	KWCOUNT=`xmllint --xinclude --xpath "count(//*[local-name()='keywords']//*)" $1`
 	if (( KWCOUNT < 4)); then
 		echo " WARNING -- Less than 4 keywords in $1 ($KWCOUNT)"