Skip to content

Commit

Permalink
Merge pull request #136 from radionets-project/update_readme
Browse files Browse the repository at this point in the history
Update readme
  • Loading branch information
Kevin2 authored Jan 30, 2023
2 parents 08f5593 + 5f34801 commit 99d346f
Show file tree
Hide file tree
Showing 9 changed files with 103 additions and 26 deletions.
72 changes: 72 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
Radionets 0.1.18 (2023-01-30)
=============================


API Changes
-----------


Bug Fixes
---------


New Features
------------

- added creation of uncertainty plots
changed creation and saving/reading of predictions to ``dicts``
prediction ``dicts`` have 3 or 4 entries depending on uncertainty
added scaled option to ``get_ifft``
created new dataset class for sampled images
created option for sampling and saving the whole test dataset
updated and wrote new tests [`#129 <https://github.com/radionets-project/radionets/pull/129>`__]


Maintenance
-----------

- Add and enable ``towncrier`` in CI. [`#130 <https://github.com/radionets-project/radionets/pull/130>`__]

- publish radionets on pypi [`#134 <https://github.com/radionets-project/radionets/pull/134>`__]

- Update README, use figures from the paper, minor text adjustments [`#136 <https://github.com/radionets-project/radionets/pull/136>`__]


Refactoring and Optimization
----------------------------


Radionets 0.1.16 (2023-01-30)
=============================


API Changes
-----------


Bug Fixes
---------


New Features
------------

- added creation of uncertainty plots
changed creation and saving/reading of predictions to ``dicts``
prediction ``dicts`` have 3 or 4 entries depending on uncertainty
added scaled option to ``get_ifft``
created new dataset class for sampled images
created option for sampling and saving the whole test dataset
updated and wrote new tests [`#129 <https://github.com/radionets-project/radionets/pull/129>`__]


Maintenance
-----------

- Add and enable ``towncrier`` in CI. [`#130 <https://github.com/radionets-project/radionets/pull/130>`__]

- publish radionets on pypi [`#134 <https://github.com/radionets-project/radionets/pull/134>`__]


Refactoring and Optimization
----------------------------
54 changes: 29 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@

## Imaging Radio Interferometric Data with Neural Networks

Executables to simulate and analyze radio interferometric data in python. The goal is to reconstruct calibrated observations with
Deep-learning framework for the simulation and analysis of radio interferometric data in Python. The goal is to reconstruct calibrated observations with
convolutional Neural Networks to create high-resolution images. For further information, please have a look at our [paper](https://www.aanda.org/component/article?access=doi&doi=10.1051/0004-6361/202142113).

Analysis strategies leading to a reproducible processing and evaluation of data recorded by radio interferometers:
* Simulation of datasets (will be shifted to `vipy` repository)
Analysis strategies leading to reproducible processing and evaluation of data recorded by radio interferometers:
* Simulation of datasets (see also the `radiosim` [repository](https://github.com/radionets-project/radiosim))
* Simulation of radio interferometer observations (see also the `pyvisgen` [repository](https://github.com/radionets-project/pyvisgen))
* Training of deep learning models
* Reconstruction of radio interferometric data

## Installation (must be reworked after shifting simulation part to `vipy` repository)
## Installation

This repository is build up as a python package. It is recommended to create an own conda environment to handle the dependencies of the packages.
This repository is built as a python package. We recommend creating a mamba environment to handle the dependencies of all packages.
You can create one by running the following command in this repository:
```
$ conda env create -f environment.yml
$ mamba env create -f environment.yml
```
Depending on your `cuda` version you have to specify the `cudatoolkit` version used by `pytorch`. If you are working on machines
with `cuda` versions < 10.2, please change the version number in the environment.yml file. Since the package `pre-commit` is used, you need to execute
Expand All @@ -28,19 +29,19 @@ after the installation.

## Usage

For each task, executables are installed to your `PATH`. Each of them takes `toml` configuration files as input, to manage data paths and options.
Simulated data is saved in `hdf5`, trained models are saved as `pickle` files.
For each task, executables are installed to your `PATH`. Each takes `toml` configuration files as input to manage data paths and options.
Simulated data is saved in `hdf5`; trained models are saved as `pickle` files.

* `radionets_simulations <...>`
This script is used to simulate radio interferometric datasets for the training of deep learning models.
This script is used to simulate radio interferometric data sets for the training of deep learning models.
* `radionets_training <...>`
This script is used to train a model on events with known truth
values for the target variable, usually monte carlo simulations.
values for the target variable, usually Monte Carlo simulations.
* `radionets_evaluation <...>`
This script is used to evaluate the performance of the trained deep learning models.
This script is used to evaluate the performance of the trained deep-learning models.

Default configuration files can be found in the examples directory. Additionally, the examples directory contains jupyter notebooks, which show an example
analysis pipeline and the corresponding commands.
Default configuration files can be found in the examples directory. The examples directory contains `jupyter notebooks`, which show an example
analysis pipeline and the corresponding commands. (need a rework)

## Structure of the Repository

Expand All @@ -51,29 +52,29 @@ An introduction to Neural Networks and an overview of the use of fastai to train

### dl_training

Functions for handling the different training options. Currently, there are the training, the learning rate finder, and the loss plotting mode.
Functions for handling the different training options. Currently, there are the training, the learning rate finder, and the loss plotting mode available.

### simulations (further developed in [vipy](https://github.com/radionets-project/vipy) repository)
### simulations (further developed in [pyvisgen](https://github.com/radionets-project/pyvisgen) repository)

Functions to simulate and illustrate radio interferometric observations. At the moment simulations based on the MNIST dataset and
simulations of Gaussian sources are possible. We are currently working on simulating visibilities directly in Fourier space.
For more information, visit our corresponding repository [vipy](https://github.com/radionets-project/vipy). In the future, the simulations will be created
using the `vipy` repository, while the `radionets` repository contains the training and evaluation methods.
For more information, visit our corresponding repository [pyvisgen](https://github.com/radionets-project/pyvisgen). In the future, the simulations will be created
using the `pyvisgen` repository, while the `radionets` repository contains the training and evaluation methods.

### evaluation

Functions for the evaluation of the training sessions. The available options reach from single, exemplary plots in (u, v) space and image space to
methods computing characteristic values on large test datasets. In detail:

* Amplitude and phase for the prediction and the truth. Example image below including the difference between prediction and truth.
* Amplitude and phase for the prediction and the truth. Example image below includes the difference between prediction and truth.
![](resources/amp_phase.png)
* Reconstructed source images with additional features, such as MS-SSIM values or the viewing angle. Example image below.
![](resources/source_plot.png)
* Histogram of differences between predicted and true viewing angles. Image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
* Histogram of differences between predicted and true viewing angles. The image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
![](resources/hist_jet_offsets.png)
* Histogram of ratio between predicted and true source areas. Image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
* Histogram of the ratio between predicted and true source areas. The image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
![](resources/hist_area_ratios.png)
* Histogram of flux difference in core component. Image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
* Histogram of flux difference in the core component. The image includes a comparison with [wsclean](https://gitlab.com/aroffringa/wsclean).
![](resources/hist_mean_diffs.png)
* Included, but not yet fully operational
* Histogram of differences between predicted and true MS-SSIM values on a dedicated test dataset
Expand All @@ -85,15 +86,18 @@ All histograms are created on a dedicated test dataset.

* Kevin Schmidt [@Kevin2](https://github.com/Kevin2)
* Felix Geyer [@FeGeyer](https://github.com/FeGeyer)
* Arne Poggenpohl [@ArnePoggenpohl](https://github.com/ArnePoggenpohl)
* Stefan Fröse [@StFroese](https://github.com/StFroese)
* Paul-Simon Blomenkamp [@PBlomenkamp](https://github.com/PBlomenkamp)
* Olivia Locke [@olivialocke](https://github.com/olivialocke)
* Kevin Laudamus [@K-Lauda](https://github.com/K-Lauda)
* Emiliano Miranda [@emilianozm24](https://github.com/emilianozm24)
* Maximilian Büchel [@MaxBue](https://github.com/MaxBue)
* Rune Dominik [@RuneDominik](https://github.com/RuneDominik)

## Versions used and tested

* Python >= 3.6
* pyTorch >= 1.2.0
* torchvision >= 0.4.0
* cudatoolkit >= 9.2
* Python >= 3.8
* pyTorch >= 1.11.0
* torchvision >= 0.12.0
* cudatoolkit >= 11.3
1 change: 1 addition & 0 deletions docs/changes/136.maintenance.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update README, use figures from the paper, minor text adjustments
Binary file modified resources/amp_phase.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified resources/hist_area_ratios.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified resources/hist_jet_offsets.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified resources/hist_mean_diffs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified resources/source_plot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

setup(
name="radionets",
version="0.1.17",
version="0.1.18",
description="Imaging radio interferometric data with neural networks",
url="https://github.com/radionets-project/radionets",
author="Kevin Schmidt, Felix Geyer",
Expand Down

0 comments on commit 99d346f

Please sign in to comment.