add details about sample target (#8)

CDCgov · Nov 5, 2024 · 7e87866 · 7e87866
1 parent 9927b37
commit 7e87866
Show file tree

Hide file tree

Showing 2 changed files with 49 additions and 6 deletions.
diff --git a/hub-config/tasks.json b/hub-config/tasks.json
@@ -132,12 +132,14 @@
                         "sample": {
                             "output_type_id_params": {
                                 "is_required": false,
-                                "type": "integer",
-                                "min_samples_per_task": 100,
-                                "max_samples_per_task": 100
+                                "type": "character",
+                                "max_length": 15,
+                                "min_samples_per_task": 200,
+                                "max_samples_per_task": 200,
+                                "compound_taskid_set": ["reference_date"]
                             },
                             "value": {
-                                "type": "integer",
+                                "type": "double",
                                 "minimum": 0
                             }
                         }

diff --git a/model-output/README.md b/model-output/README.md
@@ -169,7 +169,7 @@ Values in the `output_type` column are either
 -   "quantile" or
 -   "samples".
 
-This value indicates whether that row corresponds to a quantile forecast or sample trajectories for weekly incident hospital admissions.
+This value indicates whether that row corresponds to a quantile forecast or sample trajectories for weekly incident hospital admissions. Samples can be submitted either for individual modeling tasks, where each `horizon` and `location` is treated independently, or as a part of a compound modeling task that encodes predictive statistical dependency across forecast `horizon`s and/or `location`s.
 
 ### `output_type_id`
 Values in the `output_type_id` column specify identifying information for the output type.
@@ -218,7 +218,48 @@ Teams must provide the following 23 quantiles:
 #### sample output
 
 When the predictions are samples, values in the `output_type_id` column are indexes for the samples. 
-*More details to be added here*
+The `output_type_id` is used to indicate the dependence across multiple task id variables when samples
+ come from a joint predictive distribution. For example, samples from a joint predictive distribution 
+ across `horizon`s for a given `location`, will share `output_type_id` for predictions for different
+  `horizon`s within a same `location`, as shown in the table below:
+
+| origin_date|horizon| location | output_type| output_type_id | value |
+|:---------- |:-----:|:-----:| :-------- | :------------ | :---- |
+| 2024-10-15 | -1      |  MA | sample | s0 | - |
+| 2024-10-15 |  0      |  MA | sample | s0 | - |
+| 2024-10-15 |  1      |  MA | sample | s0 | - |
+| 2024-10-15 | -1      |  NH | sample | s1 | - |
+| 2024-10-15 |  0      |  NH | sample | s1 | - |
+| 2024-10-15 |  1      |  NH | sample | s1 | - |
+| 2024-10-15 | -1      |  MA | sample | s2 | - |
+| 2024-10-15 |  0      |  MA | sample | s2 | - |
+| 2024-10-15 |  1      |  MA | sample | s2 | - |
+| 2024-10-15 | -1      |  NH | sample | s3 | - |
+| 2024-10-15 |  0      |  NH | sample | s3 | - |
+| 2024-10-15 |  1      |  NH | sample | s3 | - |
+
+Here, `output_type_id = s0` and `output_type_id = s1` specifies that the predictions
+ for horizons -1, 0, and 1 are part of the same joint distribution. Samples from joint 
+ distribution across `horizon`s and `location`s can be specified by shared `output_type_id`
+  across `location`s and `horizon`s as shown in the example below:
+
+| origin_date|horizon| location | output_type| output_type_id | value |
+|:---------- |:-----:|:-----:| :-------- | :------------ | :---- |
+| 2024-10-15 | -1      |  MA | sample | S0 | - |
+| 2024-10-15 |  0      |  MA | sample | S0 | - |
+| 2024-10-15 |  1      |  MA | sample | S0 | - |
+| 2024-10-15 | -1      |  NH | sample | S0 | - |
+| 2024-10-15 |  0      |  NH | sample | S0 | - |
+| 2024-10-15 |  1      |  NH | sample | S0 | - |
+| 2024-10-15 | -1      |  MA | sample | S1 | - |
+| 2024-10-15 |  0      |  MA | sample | S1 | - |
+| 2024-10-15 |  1      |  MA | sample | S1 | - |
+| 2024-10-15 | -1      |  NH | sample | S1 | - |
+| 2024-10-15 |  0      |  NH | sample | S1 | - |
+| 2024-10-15 |  1      |  NH | sample | S1 | - |
+
+The above table shows two samples indexed by `output_type_id:` `S1` and `S2` from a joint predictive distribution across `location`s and `horizon`s.
+More details on sample output can be found in the [hubverse documentation of sample output type](https://hubverse.io/en/latest/user-guide/sample-output-type.html).
 
 ### `value`