-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
819fdf9
commit e0067ff
Showing
159 changed files
with
10,093 additions
and
45 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 3b7c129bcacffd00dd24cc29b6c50999 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# Changelog | ||
|
||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). | ||
|
||
## \[Unreleased\] | ||
|
||
## \[0.0.3\] - 2024-10-24 | ||
|
||
### Added | ||
|
||
- Specify environment variables for docker container in spec. | ||
|
||
## \[0.0.2\] - 2024-10-23 | ||
|
||
### Added | ||
|
||
- App spec supports changing docker entrypoint. | ||
- `bfabric-app-runner inputs check` to validate the local files | ||
|
||
### Fixed | ||
|
||
- `bfabric-app-runner inputs list` does not fail anymore if resources have no "name" field value. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
```{toctree} | ||
:glob: | ||
specs/input_specification | ||
specs/output_specification | ||
specs/app_specification | ||
changelog | ||
* | ||
``` | ||
|
||
## Install App Runner | ||
|
||
```bash | ||
pipx install app_runner@git+https://github.com/fgcz/bfabricPy.git@main#egg=app_runner&subdirectory=app_runner | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
## App specification | ||
|
||
TODO: not clear if this same document should also explain the individual steps, or if it would make sense to first | ||
describe the app anatomy in a separate document with figures etc. and then list how to specify it |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
## Input specification | ||
|
||
The inputs module provides a specification schema to define the inputs required by an app. | ||
You can also use this functionality interactively while prototyping. | ||
The file is usually called `inputs.yml` and lists the different inputs, with information and how to retrieve them and | ||
the filename to save them as. | ||
|
||
### General structure | ||
|
||
Generally the structure is a yaml file containing a key `inputs` which is a list of dictionaries, each representing an | ||
input file. | ||
Each input has a `type` key which identifies the input type. | ||
This will allow us to extend this logic to different sources in the future. | ||
|
||
In general the only other input key that will be available for all types is `filename`, which is the name of the file to | ||
save the input as. | ||
Fields like `id` might not be relevant for all types in the future, and depending on the type more specific options | ||
might exist. | ||
|
||
An example file could look like this: | ||
|
||
```yaml | ||
# file: inputs.yml | ||
inputs: | ||
- type: bfabric_dataset | ||
id: 53706 | ||
filename: test.csv | ||
- type: bfabric_resource | ||
id: 2700958 | ||
filename: test.zip | ||
``` | ||
## Commands | ||
### Validation | ||
The input file can be validated with the command: | ||
```bash | ||
bfabric-app-runner validate inputs-spec inputs.yml | ||
``` | ||
|
||
Which on success will output a pretty-printed version of the inputs file. | ||
Validation will also be performed by all other commands, so this is not strictly necessary. | ||
|
||
For instance, in the above case this would print: | ||
|
||
``` | ||
InputsSpec( | ||
│ inputs=[ | ||
│ │ DatasetSpec(type='bfabric_dataset', id=53706, filename='test.csv', separator=','), | ||
│ │ ResourceSpec(type='bfabric_resource', id=2700958, filename='test.zip', check_checksum=True) | ||
│ ] | ||
) | ||
``` | ||
|
||
Here you can also see all the extra parameters which were implicitly set. | ||
|
||
### Prepare files | ||
|
||
The prepare command downloads your files and requires two arguments. | ||
The first is the input file, and the second is the directory to save the files to. | ||
In general to download to the current directory simply use `.` as the second argument: | ||
|
||
```bash | ||
bfabric-app-runner inputs prepare inputs.yml . | ||
``` | ||
|
||
If your files already exist and are up-to-date, it will not download them again. | ||
|
||
### List files | ||
|
||
You can list the files that are present or will be downloaded: | ||
|
||
```bash | ||
bfabric-app-runner inputs list inputs.yml . | ||
``` | ||
|
||
If you also want to check whether the files are up-to-date, you can pass the `--check` flag: | ||
|
||
```bash | ||
bfabric-app-runner inputs list --check inputs.yml . | ||
``` | ||
|
||
## Reference | ||
|
||
```{eval-rst} | ||
.. automodule:: app_runner.input_preparation.spec | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
## Output specification | ||
|
||
The outputs module provides a specification schema to define the outputs that were created by an app and should be registered. | ||
The file is usually called `outputs.yml` and lists the different output files, with information how to register them. | ||
|
||
### General structure | ||
|
||
To be described. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
.autodoc_pydantic_validator_arrow { | ||
padding-left: 8px; | ||
} | ||
|
||
.autodoc_pydantic_collapsable_json { | ||
cursor: pointer; | ||
} | ||
|
||
.autodoc_pydantic_collapsable_erd { | ||
cursor: pointer; | ||
} |
Oops, something went wrong.