Merge pull request bioboxes#189 from pbelmann/fix/profiling-evaluatio…

…n-interface

profiling evaluation interface

container/profiling-evaluation/rfc.mkd

@@ -0,0 +1,65 @@
+## Profiling evaluation container
+
+ * Version:    0.1.0
+ * Maintainer: Peter Belmann <pbelmann@cebitec.uni-bielefeld.de>
+
+### Contents
+
+### Outline
+
+This specification describes the interface for containerised profiling evaluation applications. 
+In addition to the specifications described below, this container MUST implement the
+specifications defined in ['Generic bioinformatics container'](https://github.com/bioboxes/rfc/blob/master/rfc.mkd#generic-bioinformatics-container). 
+
+### Input
+
+#### General Definition
+
+A biobox requires an input YAML with the following definition 
+
+```YAML
+---
+version: NUMBER.NUMBER.NUMBER
+arguments:
+  prediction:
+    path: STRING
+    format: bioboxes.org:/profiling:0.9
+  ground_truth:
+    path: STRING
+    format: bioboxes.org:/profiling:0.9
+```
+
+##### Description:
+
+* **version**: The current version is specified directly under the heading.
+* **arguments**: The arguments field consists of the following fields 
+       * **predictions**: Profiling prediction in bioboxes.org profiling format.
+       * **ground_truth**: Profiling ground truth/gold standard in bioboxes.org profiling format.
+* **path**: Path MUST begin with a slash ('/'), which points to a profiling file. This file has to be mounted to a path that is prefixed by `/bbx/mnt/input`.
+
+##### Mounts:
+ * Your output directory MUST be mounted to /bbx/mnt/output
+ * Your input files MUST be mounted to /bbx/mnt/input
+ * The .yaml MUST be placed as /bbx/mnt/input/biobox.yaml
+ * If the directory /bbx/mnt/metadata is mounted then the following files should be placed inside the directory:
+   log.txt Logging information that is generated by the application inside the container.
+
+##### Formats
+* `bioboxes.org:/profiling:0.9`: bioboxes.org profiling file in version 0.9
+
+### Output
+The biobox produces on a successful run a `biobox.yaml` and can be found in your mounted output directory. The output biobox.yaml must be in an evaluation specific [format](https://github.com/bioboxes/rfc/blob/master/data-format/evaluation.mkd).
+
+### Example
+
+```YAML
+---
+version: 0.1.0
+arguments:
+  prediction:
+    path: /bbx/mnt/input/prediction.txt
+    format: bioboxes.org:/profiling:0.9
+  ground_truth:
+    path: /bbx/mnt/input/ground_truth.txt
+    format: bioboxes.org:/profiling:0.9
+```

container/profiling-evaluation/schema.yaml

@@ -0,0 +1,39 @@
+---
+  $schema: "http://json-schema.org/draft-04/schema#"
+  title: "Bioboxes profiling benchmark input file validator"
+  type: "object"
+  additionalProperties: false
+  required: 
+    - "version"
+    - "arguments"
+  properties: 
+    version: 
+      type: "string"
+      pattern: "^0.1.\\d+$"
+    arguments: 
+      additionalProperties: false
+      type: "object"
+      required: 
+        - "ground_truth"
+        - "prediction"
+      properties: 
+        prediction: 
+          type: "object"
+          required: 
+            - "path"
+            - "format"
+          properties: 
+            format: 
+              enum: 
+                - "bioboxes.org:/profiling:0.9"
+            path: {}
+        ground_truth: 
+          type: "object"
+          required: 
+            - "path"
+            - "format"
+          properties: 
+            format: 
+              enum: 
+                - "bioboxes.org:/profiling:0.9"
+            path: {}