-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Atlas packaging template #400
Open
PolarBean
wants to merge
37
commits into
brainglobe:main
Choose a base branch
from
PolarBean:atlas_packaging_template
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
fcb1bf9
First version of the template script
PolarBean 6dc6052
make example_mouse follow the new template
PolarBean 37a75fb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 193a78b
Fix error in example mouse
PolarBean cd9bafe
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] d96ed9d
mixed up reference and annotation
PolarBean 2254886
added mandatory import to template
PolarBean 813b251
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] c521a2f
remove import from example
PolarBean 602dbef
added mandatory import to template
PolarBean f4b0452
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 9009dc2
include case where variables should be passed between functions
PolarBean 1383b14
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean 2b0d42e
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean c47ca80
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean 262a07c
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean c51231c
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean fca9c7b
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean 8052298
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 91c09f7
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean 3ebdbd4
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean 6cc3222
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean 73387c0
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean 23b3a16
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 4740c06
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean f6253e0
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean af0f450
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean 91b97c8
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/template_sc…
PolarBean d9bc6e3
keep lines shorter than limit and only run code in __main__
PolarBean 2ee5c7b
reformat example mouse
PolarBean 1e68bed
use pooch to validate hash
PolarBean 924e484
add bg_root_dir global
PolarBean d643653
fix references to annotation and reference
PolarBean 78fe6d5
fix error in the way structure id paths were shown in the example
PolarBean 642e0bb
fix typo
PolarBean 536281f
Update brainglobe_atlasapi/atlas_generation/atlas_scripts/example_mou…
PolarBean df352c3
adjust comment suggestion to be compliant with the 79 character line …
PolarBean File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
147 changes: 147 additions & 0 deletions
147
brainglobe_atlasapi/atlas_generation/atlas_scripts/template_script.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
from pathlib import Path | ||
|
||
from brainglobe_atlasapi.atlas_generation.wrapup import wrapup_atlas_from_data | ||
PolarBean marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
PolarBean marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# Copy-paste this script into a new file and fill in the functions to package | ||
# your own atlas. | ||
|
||
### Metadata ### | ||
|
||
# The minor version of the atlas in the brainglobe_atlasapi, this is internal, | ||
# if this is the first time this atlas has been added the value should be 0 | ||
# (minor version is the first number after the decimal point, ie the minor | ||
# version of 1.2 is 2) | ||
__version__ = 0 | ||
|
||
# The expected format is FirstAuthor_SpeciesCommonName, e.g. kleven_rat, or | ||
# Institution_SpeciesCommonName, e.g. allen_mouse. | ||
ATLAS_NAME = "example_mouse" | ||
|
||
# DOI of the most relevant citable document | ||
CITATION = None | ||
|
||
# The scientific name of the species, ie; Rattus norvegicus | ||
SPECIES = None | ||
|
||
# The URL for the data files | ||
ATLAS_LINK = None | ||
|
||
# The orientation of the **original** atlas data, in BrainGlobe convention: | ||
# https://brainglobe.info/documentation/setting-up/image-definition.html#orientation | ||
ORIENTATION = "asr" | ||
|
||
# The id of the highest level of the atlas. This is commonly called root or | ||
# brain. Include some information on what to do if your atlas is not | ||
# hierarchical | ||
ROOT_ID = None | ||
|
||
# The resolution of your volume in microns. Details on how to format this | ||
# parameter for non isotropic datasets or datasets with multiple resolutions. | ||
RESOLUTION = None | ||
|
||
|
||
def download_resources(): | ||
""" | ||
Download the necessary resources for the atlas. | ||
|
||
If possible, please use the Pooch library to retrieve any resources. | ||
""" | ||
pass | ||
|
||
|
||
def retrieve_reference_and_annotation(): | ||
""" | ||
Retrieve the desired reference and annotation as two numpy arrays. | ||
|
||
Returns: | ||
tuple: A tuple containing two numpy arrays. The first array is the | ||
reference volume, and the second array is the annotation volume. | ||
""" | ||
reference = None | ||
annotation = None | ||
return reference, annotation | ||
|
||
|
||
def retrieve_hemisphere_map(): | ||
""" | ||
Retrieve a hemisphere map for the atlas. | ||
|
||
If your atlas is asymmetrical, you may want to use a hemisphere map. | ||
This is an array in the same shape as your template, | ||
with 0's marking the left hemisphere, and 1's marking the right. | ||
|
||
If your atlas is symmetrical, ignore this function. | ||
|
||
Returns: | ||
numpy.array or None: A numpy array representing the hemisphere map, | ||
or None if the atlas is symmetrical. | ||
""" | ||
return None | ||
|
||
|
||
def retrieve_structure_information(): | ||
""" | ||
This function should return a pandas DataFrame with information about your | ||
atlas. | ||
|
||
The DataFrame should be in the following format: | ||
|
||
╭────┬───────────────────┬─────────┬───────────────────┬─────────────────╮ | ||
| id | name | acronym | structure_id_path | rgb_triplet | | ||
| | | | | | | ||
├────┼───────────────────┼─────────┼───────────────────┼─────────────────┤ | ||
| 997| root | root | [997] | [255, 255, 255] | | ||
├────┼───────────────────┼─────────┼───────────────────┼─────────────────┤ | ||
| 8 | Basic cell groups | grey | [997, 8] | [191, 218, 227] | | ||
├────┼───────────────────┼─────────┼───────────────────┼─────────────────┤ | ||
| 567| Cerebrum | CH | [997, 8, 567] | [176, 240, 255] | | ||
╰────┴───────────────────┴─────────┴───────────────────┴─────────────────╯ | ||
|
||
Returns: | ||
pandas.DataFrame: A DataFrame containing the atlas information. | ||
""" | ||
return None | ||
|
||
|
||
def retrieve_or_construct_meshes(): | ||
""" | ||
This function should return a dictionary of ids and corresponding paths to | ||
mesh files. Some atlases are packaged with mesh files, in these cases we | ||
should use these files. Then this function should download those meshes. | ||
In other cases we need to construct the meshes ourselves. For this we have | ||
helper functions to achieve this. | ||
""" | ||
meshes_dict = {} | ||
return meshes_dict | ||
|
||
|
||
### If the code above this line has been filled correctly, nothing needs to be | ||
### edited below (unless variables need to be passed between the functions). | ||
if __name__ == "__main__": | ||
bg_root_dir = Path.home() / "brainglobe_workingdir" / ATLAS_NAME | ||
bg_root_dir.mkdir(exist_ok=True) | ||
download_resources() | ||
reference_volume, annotated_volume = retrieve_reference_and_annotation() | ||
hemispheres_stack = retrieve_hemisphere_map() | ||
structures = retrieve_structure_information() | ||
meshes_dict = retrieve_or_construct_meshes() | ||
|
||
output_filename = wrapup_atlas_from_data( | ||
atlas_name=ATLAS_NAME, | ||
atlas_minor_version=__version__, | ||
citation=CITATION, | ||
atlas_link=ATLAS_LINK, | ||
species=SPECIES, | ||
resolution=(RESOLUTION,) * 3, | ||
orientation=ORIENTATION, | ||
root_id=ROOT_ID, | ||
reference_stack=reference_volume, | ||
annotation_stack=annotated_volume, | ||
structures_list=structures, | ||
meshes_dict=meshes_dict, | ||
working_dir=bg_root_dir, | ||
hemispheres_stack=None, | ||
cleanup_files=False, | ||
compress=True, | ||
scale_meshes=True, | ||
) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worry that this is
a) not the way we typically use
pooch
and therefore confusingb) more importantly, might look scary to new contributors coming more from the biology side
I'm tempted to remove any
pooch
-related things, and mention it in the template script only (which we already do in this PR)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point, the only thing is that pooch here actually serves a purpose, to ensure that the files from the Allen don't change without notifying us. In this instance we would want pooch to confirm the hash of the file. I understand that it's non standard but I think that just comes with using the Allen API in the example.
An alternative would be to not use the allen API at all, instead just use the files from the allen informatics archive. The advantage to this is the example would become more of a standard atlas integration allowing us to use pooch, etc, instead of so much allen specific logic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this would be great if you have the patience to implement this? 😁 Would this be possible/easy on the
OntologiesAPI
side for the meshes too?The alternative is that we make a new issue to at some point move to a more standard atlas (e.g. we have recently generated a bird template for which we will host the data ourselves, and which has ITK snap annotations - our standard - so we could use that when the preprint is published.)