Fix pytype static checks (Project-MONAI#449)

* Fix pytype warnings

File "monai/monai/inferers/utils.py", line 67, in sliding_window_inference: Cannot unpack 2 values into 3 variables [bad-unpacking]
File "monai/monai/inferers/utils.py", line 97, in sliding_window_inference: Cannot unpack 2 values into 3 variables [bad-unpacking]

* Avoid pytypes warning

File "monai/monai/data/dataset.py", line 29, in <module>: No attribute 'Dataset' on module 'torch.utils.data' [module-attr]
File "monai/monai/data/dataset.py", line 293, in <module>: No attribute 'Dataset' on module 'torch.utils.data' [module-attr]

* Multiple inheritance demands identifying superclass

pytype monai/engines/workflow.py

File "monai/monai/engines/workflow.py", line 57, in __init__: Function Workflow.__init__ expects 1 arg(s), got 2 [wrong-arg-count]
         Expected: (self)
  Actually passed: (self, _)

Disable failing pytype test.  This needs further investigation by
someone who better understands the intended behavior.

The expected behavior is unclear.  Which superclass is intended
to be called for this multiple inheritance case?

For the superclass ABC and Engine, why is Workflow the function that
gives the warning?

FIXME:
pytype .
File "monai/engines/workflow.py",
      line 57: Invalid directive syntax. [invalid-directive]
File "monai/engines/workflow.py",
      line 57, in __init__: Function Workflow.__init__ expects 1 arg(s), got 2 [wrong-arg-count]
         Expected: (self)
  Actually passed: (self, _)
For more details, see https://google.github.io/pytype/errors.html.

* Import as local name for pytypes warning suppression

File "monai/monai/transforms/adaptors.py", line 102, in <module>: No attribute 'export' on module 'monai.utils' [module-attr]
File "monai/monai/transforms/adaptors.py", line 187, in <module>: No attribute 'export' on module 'monai.utils' [module-attr]
File "monai/monai/transforms/adaptors.py", line 208, in <module>: No attribute 'export' on module 'monai.utils' [module-attr]

* Resolve conflicting type definitions

File "monai/monai/visualize/img2tensorboard.py", line 23, in <module>: Invalid type annotation '<instance of Callable>' for image [invalid-annotation] Not a type
File "monai/monai/visualize/img2tensorboard.py", line 55, in <module>: Invalid type annotation '<instance of Callable>' for image [invalid-annotation] Not a type
File "monai/monai/visualize/img2tensorboard.py", line 90, in make_animated_gif_summary: Invalid type annotation '<instance of Callable>' for one_channel_img [invalid-annotation] Not a type
File "monai/monai/visualize/img2tensorboard.py", line 101, in <module>: Invalid type annotation '<instance of Callable>' for image_tensor [invalid-annotation] Not a type
File "monai/monai/visualize/img2tensorboard.py", line 128, in <module>: Invalid type annotation '<instance of Callable>' for image_tensor [invalid-annotation] Not a type

* Fix callable object type matching

File "monai/monai/networks/nets/densenet.py", line 169, in __init__: Built-in function isinstance was called with the wrong arguments [wrong-arg-types]
         Expected: (object, class_or_type_or_tuple: Union[Tuple[Union[Tuple[type, ...], type], ...], type])
  Actually passed: (object, class_or_type_or_tuple: Callable)
File "monai/monai/networks/nets/densenet.py", line 171, in __init__: Built-in function isinstance was called with the wrong arguments [wrong-arg-types]
         Expected: (object, class_or_type_or_tuple: Union[Tuple[Union[Tuple[type, ...], type], ...], type])
  Actually passed: (object, class_or_type_or_tuple: Callable)

 T484 Returning Any from function declared to return Callable[..., Any]

* ENH: Support pytype static analysis.

* Remove unused import

monai/monai/transforms/adaptors.py:99:1: F401 'monai' imported but unused

* ENH: Adding .pytype and .mypy_cache to ignored black file list.

* add type hints

Co-authored-by: Cameron Trentz <cameron-trentz@uiowa.edu>

* Annotate dtype as np.dtype

* add type hints [second pass]

Co-authored-by: Cameron Trentz <cameron-trentz@uiowa.edu>

* Suppress some pytype warnings

Some pytype warnings will require more extensive corrections to fully
resolve them.  These few cases are suppressed in the interest of
allowing more static checks without imposing a huge burden at the
moment.

File "monai/monai/transforms/io/array.py", line 93, in __call__: No attribute 'dtype' on bool [attribute-error]
  In Union[Any, bool]
For more details, see https://google.github.io/pytype/errors.html#attribute-error.

File "monai/monai/transforms/intensity/array.py", line 142, in __call__: Function ScaleIntensity.__init__ was called with the wrong arguments [wrong-arg-types]
         Expected: (self, minv, maxv: Union[float, int] = ..., ...)
  Actually passed: (self, minv, maxv: None, ...)
For more details, see https://google.github.io/pytype/errors.html#wrong-arg-types.

File "monai/monai/transforms/intensity/dictionary.py", line 190, in __call__: Function ScaleIntensity.__init__ was called with the wrong arguments [wrong-arg-types]
         Expected: (self, minv: Union[float, int] = ..., ...)
  Actually passed: (self, minv: None, ...)
For more details, see https://google.github.io/pytype/errors.html#wrong-arg-types.

File "monai/monai/data/dataset.py", line 124, in _pre_first_random_transform: No attribute 'transforms' on Callable [attribute-error]
  In Union[Any, Callable]
File "monai/monai/data/dataset.py", line 140, in _first_random_and_beyond_transform: No attribute 'transforms' on Callable [attribute-error]
  In Union[Any, Callable]
File "monai/monai/data/dataset.py", line 285, in __getitem__: No attribute 'transforms' on Callable [attribute-error]
  In Union[Any, Callable]
For more details, see https://google.github.io/pytype/errors.html#attribute-error.

* Fix type mismatch warning for range object as list

monai/monai/transforms/spatial/array.py:913:22: T484 Invalid index type "range" for
   "Union[Any, Tensor]"; expected type "Union[None, int, slice, Tensor, List[Any], Tuple[Any, ...]]"

Co-authored-by: Benjamin Gorman <benjamin-gorman@uiowa.edu>
Co-authored-by: Cameron Trentz <cameron-trentz@uiowa.edu>

Author: 3 people

.gitignore

@@ -100,6 +100,9 @@ venv.bak/
 # mkdocs documentation
 /site
 
+# pytype cache
+.pytype/
+
 # mypy
 .mypy_cache/
 examples/scd_lvsegs.npz

examples/notebooks/3d_image_transforms.ipynb

@@ -156,7 +156,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "loader = LoadNifti(dtype=np.float32)"
+    "loader = LoadNifti(dtype: np.dtype = np.float32)"
    ]
   },
   {

monai/data/csv_saver.py

@@ -9,11 +9,13 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from collections import OrderedDict
+from typing import Union
+
 import os
 import csv
 import numpy as np
 import torch
-from collections import OrderedDict
 
 
 class CSVSaver:
@@ -24,7 +26,7 @@ class CSVSaver:
     the cached data into CSV file. If no meta data provided, use index from 0 to save data.
     """
 
-    def __init__(self, output_dir="./", filename="predictions.csv", overwrite=True):
+    def __init__(self, output_dir: str = "./", filename: str = "predictions.csv", overwrite: bool = True):
         """
         Args:
             output_dir (str): output CSV file directory.
@@ -34,7 +36,7 @@ def __init__(self, output_dir="./", filename="predictions.csv", overwrite=True):
 
         """
         self.output_dir = output_dir
-        self._cache_dict = OrderedDict()
+        self._cache_dict: OrderedDict = OrderedDict()
         assert isinstance(filename, str) and filename[-4:] == ".csv", "filename must be a string with CSV format."
         self._filepath = os.path.join(output_dir, filename)
         self.overwrite = overwrite
@@ -60,7 +62,7 @@ def finalize(self):
                     f.write("," + str(result))
                 f.write("\n")
 
-    def save(self, data, meta_data=None):
+    def save(self, data: np.ndarray, meta_data=None):
         """Save data into the cache dictionary. The metadata should have the following key:
             - ``'filename_or_obj'`` -- save the data corresponding to file name or object.
         If meta_data is None, use the default index from 0 to save data instead.
@@ -76,7 +78,7 @@ def save(self, data, meta_data=None):
             data = data.detach().cpu().numpy()
         self._cache_dict[save_key] = data.astype(np.float32)
 
-    def save_batch(self, batch_data, meta_data=None):
+    def save_batch(self, batch_data: Union[torch.Tensor, np.ndarray], meta_data=None):
         """Save a batch of data into the cache dictionary.
 
         args:

monai/data/dataset.py

@@ -9,8 +9,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from typing import Optional, Callable
 import sys
-
 import hashlib
 import json
 from pathlib import Path
@@ -25,8 +25,10 @@
 from monai.transforms.utils import apply_transform
 from monai.utils import process_bar, get_seed
 
+from torch.utils.data import Dataset as _TorchDataset
+
 
-class Dataset(torch.utils.data.Dataset):
+class Dataset(_TorchDataset):
     """
     A generic dataset with a length property and an optional callable data transform
     when fetching a data sample.
@@ -39,7 +41,7 @@ class Dataset(torch.utils.data.Dataset):
          },                           },                           }]
     """
 
-    def __init__(self, data, transform=None):
+    def __init__(self, data, transform: Optional[Callable] = None):
         """
         Args:
             data (Iterable): input data to load and transform to generate dataset for model.
@@ -51,7 +53,7 @@ def __init__(self, data, transform=None):
     def __len__(self):
         return len(self.data)
 
-    def __getitem__(self, index):
+    def __getitem__(self, index: int):
         data = self.data[index]
         if self.transform is not None:
             data = self.transform(data)
@@ -93,7 +95,7 @@ class PersistentDataset(Dataset):
     followed by applying the random dependant parts of transform processing.
     """
 
-    def __init__(self, data, transform=None, cache_dir=None):
+    def __init__(self, data, transform: Optional[Callable] = None, cache_dir=None):
         """
         Args:
             data (Iterable): input data to load and transform to generate dataset for model.
@@ -119,7 +121,7 @@ def _pre_first_random_transform(self, item_transformed):
             the transformed element up to the first identified
             random transform object
         """
-        for _transform in self.transform.transforms:
+        for _transform in self.transform.transforms:  # pytype: disable=attribute-error
             # execute all the deterministic transforms before the first random transform
             if isinstance(_transform, Randomizable):
                 break
@@ -135,7 +137,7 @@ def _first_random_and_beyond_transform(self, item_transformed):
             the transformed element through the random transforms
         """
         start_post_randomize_run = False
-        for _transform in self.transform.transforms:
+        for _transform in self.transform.transforms:  # pytype: disable=attribute-error
             if start_post_randomize_run or isinstance(_transform, Randomizable):
                 start_post_randomize_run = True
                 item_transformed = apply_transform(_transform, item_transformed)
@@ -226,7 +228,9 @@ class CacheDataset(Dataset):
     and the outcome not cached.
     """
 
-    def __init__(self, data, transform, cache_num=sys.maxsize, cache_rate=1.0, num_workers=0):
+    def __init__(
+        self, data, transform: Callable, cache_num: int = sys.maxsize, cache_rate: float = 1.0, num_workers: int = 0
+    ):
         """
         Args:
             data (Iterable): input data to load and transform to generate dataset for model.
@@ -278,7 +282,7 @@ def __getitem__(self, index):
             # load data from cache and execute from the first random transform
             start_run = False
             data = self._cache[index]
-            for _transform in self.transform.transforms:
+            for _transform in self.transform.transforms:  # pytype: disable=attribute-error
                 if not start_run and not isinstance(_transform, Randomizable):
                     continue
                 else:
@@ -290,7 +294,7 @@ def __getitem__(self, index):
         return data
 
 
-class ZipDataset(torch.utils.data.Dataset):
+class ZipDataset(_TorchDataset):
     """
     Zip several PyTorch datasets and output data(with the same index) together in a tuple.
     If the output of single dataset is already a tuple, flatten it and extend to the result.
@@ -323,7 +327,7 @@ def __init__(self, datasets, transform=None):
     def __len__(self):
         return self.len
 
-    def __getitem__(self, index):
+    def __getitem__(self, index: int):
         def to_list(x):
             return list(x) if isinstance(x, (tuple, list)) else [x]
 
@@ -375,7 +379,13 @@ def __call__(self, input_):
     """
 
     def __init__(
-        self, img_files, img_transform=None, seg_files=None, seg_transform=None, labels=None, label_transform=None
+        self,
+        img_files,
+        img_transform: Optional[Callable] = None,
+        seg_files=None,
+        seg_transform: Optional[Callable] = None,
+        labels=None,
+        label_transform: Optional[Callable] = None,
     ):
         """
         Initializes the dataset with the filename lists. The transform `img_transform` is applied
@@ -396,7 +406,7 @@ def __init__(
     def randomize(self):
         self.seed = self.R.randint(np.iinfo(np.int32).max)
 
-    def __getitem__(self, index):
+    def __getitem__(self, index: int):
         self.randomize()
         for dataset in self.datasets:
             if isinstance(dataset.transform, Randomizable):

monai/data/grid_dataset.py

@@ -22,7 +22,7 @@ class GridPatchDataset(IterableDataset):
     Yields patches from arrays read from an input dataset. The patches are chosen in a contiguous grid sampling scheme.
     """
 
-    def __init__(self, dataset, patch_size, start_pos=(), pad_mode="wrap", **pad_opts):
+    def __init__(self, dataset, patch_size, start_pos=(), pad_mode: str = "wrap", **pad_opts):
         """
         Initializes this dataset in terms of the input dataset and patch size. The `patch_size` is the size of the 
         patch to sample from the input arrays. Tt is assumed the arrays first dimension is the channel dimension which

monai/data/nifti_reader.py

@@ -9,6 +9,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from typing import Optional, Callable
+
 import numpy as np
 from torch.utils.data import Dataset
 from monai.transforms import LoadNifti
@@ -27,11 +29,11 @@ def __init__(
         image_files,
         seg_files=None,
         labels=None,
-        as_closest_canonical=False,
-        transform=None,
-        seg_transform=None,
-        image_only=True,
-        dtype=np.float32,
+        as_closest_canonical: bool = False,
+        transform: Optional[Callable] = None,
+        seg_transform: Optional[Callable] = None,
+        image_only: bool = True,
+        dtype: Optional[np.dtype] = np.float32,
     ):
         """
         Initializes the dataset with the image and segmentation filename lists. The transform `transform` is applied
@@ -67,7 +69,7 @@ def __len__(self):
     def randomize(self):
         self.seed = self.R.randint(np.iinfo(np.int32).max)
 
-    def __getitem__(self, index):
+    def __getitem__(self, index: int):
         self.randomize()
         meta_data = None
         img_loader = LoadNifti(

monai/data/nifti_saver.py

@@ -9,6 +9,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from typing import Union, Optional
+
 import numpy as np
 import torch
 from monai.data.nifti_writer import write_nifti
@@ -25,14 +27,14 @@ class NiftiSaver:
 
     def __init__(
         self,
-        output_dir="./",
-        output_postfix="seg",
-        output_ext=".nii.gz",
-        resample=True,
-        interp_order=3,
-        mode="constant",
-        cval=0,
-        dtype=None,
+        output_dir: str = "./",
+        output_postfix: str = "seg",
+        output_ext: str = ".nii.gz",
+        resample: bool = True,
+        interp_order: int = 3,
+        mode: str = "constant",
+        cval: Union[int, float] = 0,
+        dtype: Optional[np.dtype] = None,
     ):
         """
         Args:
@@ -63,7 +65,7 @@ def __init__(
         self.dtype = dtype
         self._data_index = 0
 
-    def save(self, data, meta_data=None):
+    def save(self, data: Union[torch.Tensor, np.ndarray], meta_data=None):
         """
         Save data into a Nifti file.
         The metadata could optionally have the following keys:
@@ -108,7 +110,7 @@ def save(self, data, meta_data=None):
             dtype=self.dtype or data.dtype,
         )
 
-    def save_batch(self, batch_data, meta_data=None):
+    def save_batch(self, batch_data: Union[torch.Tensor, np.ndarray], meta_data=None):
         """Save a batch of data into Nifti format files.
 
         args:

monai/data/png_saver.py

@@ -9,6 +9,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from typing import Union
+
 import torch
 import numpy as np
 from monai.data.png_writer import write_png
@@ -25,14 +27,14 @@ class PNGSaver:
 
     def __init__(
         self,
-        output_dir="./",
-        output_postfix="seg",
-        output_ext=".png",
-        resample=True,
-        interp_order=3,
-        mode="constant",
-        cval=0,
-        scale=False,
+        output_dir: str = "./",
+        output_postfix: str = "seg",
+        output_ext: str = ".png",
+        resample: bool = True,
+        interp_order: int = 3,
+        mode: str = "constant",
+        cval: Union[int, float] = 0,
+        scale: bool = False,
     ):
         """
         Args:
@@ -61,7 +63,7 @@ def __init__(
         self.scale = scale
         self._data_index = 0
 
-    def save(self, data, meta_data=None):
+    def save(self, data: Union[torch.Tensor, np.ndarray], meta_data=None):
         """
         Save data into a png file.
         The metadata could optionally have the following keys:
@@ -108,7 +110,7 @@ def save(self, data, meta_data=None):
             scale=self.scale,
         )
 
-    def save_batch(self, batch_data, meta_data=None):
+    def save_batch(self, batch_data: Union[torch.Tensor, np.ndarray], meta_data=None):
         """Save a batch of data into png format files.
 
         args:

monai/data/png_writer.py

@@ -14,7 +14,15 @@
 
 
 def write_png(
-    data, file_name, output_shape=None, interp_order=3, mode="constant", cval=0, scale=False, plugin=None, **plugin_args
+    data,
+    file_name,
+    output_shape=None,
+    interp_order=3,
+    mode="constant",
+    cval=0,
+    scale=False,
+    plugin=None,
+    **plugin_args,
 ):
     """
     Write numpy data into png files to disk.