Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mypy implementation for adapters in tiled #700

Merged
merged 45 commits into from
Apr 26, 2024
Merged
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
fe79576
some more changes
Apr 4, 2024
1038553
typed zarr.py
Apr 4, 2024
7595639
typed tiff.py
Apr 4, 2024
06ac19c
xarray.py typed
Apr 4, 2024
a80caba
some more typing added
Apr 5, 2024
786a275
some more typing. still few problems in mapping.py
Apr 5, 2024
82a3d01
more typing
Apr 5, 2024
c5ac22b
still 1 mypy problem left
Apr 8, 2024
1a0b167
docstring skeletons added
Apr 8, 2024
2265002
some more improvement
Apr 9, 2024
c6358fd
solved some docs build problems
Apr 9, 2024
2777e3c
few more changes
Apr 9, 2024
84e64ac
change type_alliases.py
Apr 9, 2024
9a9486f
some more changes
Apr 9, 2024
64a3fdb
some more changes
Apr 9, 2024
8c32d0b
update python to 3.11 in doc compilation
Apr 10, 2024
090eda3
fixed failing tests
Apr 10, 2024
1f4536a
Add protocols
danielballan Apr 10, 2024
6a20cb9
change docs python to 3.10
Apr 10, 2024
be14c77
preliminary addressing to the comments
Apr 12, 2024
8509083
some tests for protocols
Apr 15, 2024
1be2018
some corrections in protocols tests
Apr 15, 2024
9331763
fix small bug in test_protocols.py
Apr 15, 2024
edf4e1f
try to solve EllipsisType that only exists in python 3.10
Apr 15, 2024
9c7aa41
change python version check
Apr 15, 2024
b066f41
some more fix
Apr 15, 2024
5c05e13
Bump Python version of docs build.
danielballan Apr 23, 2024
ed2a731
tests
Apr 24, 2024
95ffb51
some more unit tests for protocols
Apr 25, 2024
4d1f6b0
add accesspolicy protocol tests
Apr 25, 2024
bb0f108
few more fixes
Apr 25, 2024
809089c
python 3.8 MutableMapping problem: try to replacse with the one from …
Apr 26, 2024
126eed2
try to change collections.abc.MApping with typing.Mapping if python<3.8
Apr 26, 2024
b268500
small fixes for typing errors appeared in python 3.8
Apr 26, 2024
e9dfa6d
one more typing fix for python 3.8
Apr 26, 2024
07f717a
some more fix python3.8
Apr 26, 2024
6f4e773
Type awkward buffers dict more strictly
danielballan Apr 26, 2024
7038db5
Tighten typing on data_uris.
danielballan Apr 26, 2024
9f10eb9
Fix typing of partition parameter in read_partition.
danielballan Apr 26, 2024
d923f4f
Zarr accepts array data, not dataframe/tabular.
danielballan Apr 26, 2024
557e32f
Remove commented unused code
danielballan Apr 26, 2024
7d4a34e
Fix typo
danielballan Apr 26, 2024
d000794
Fix type of partition
danielballan Apr 26, 2024
453b807
added a changelog entry
Apr 26, 2024
e863422
Make python-version consistent for docs
danielballan Apr 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add protocols
danielballan committed Apr 26, 2024
commit 1f4536a1b01dc8046b4a8c716e1fe70748a8389e
118 changes: 118 additions & 0 deletions tiled/adapters/protocols.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
import collections.abc
from typing import Any, Dict, List, Literal, Optional, Set, Tuple, TypedDict, Union
from protcols import Protocol, abstractproperty

from numpy.typing import NDArray
import pandas
import sparse

from ..server.schemas import Principal
from ..structures.core import StructureFamily
from ..structures.array import ArrayStructure
from ..structures.awkward import AwkwardStructure
from ..structures.sparse import SparseStructure
from ..structures.table import TableStructure
from .type_aliases import JSON


Spec = TypedDict({"name": str, "version": str})
Slice = Any # TODO Replace this with our Union for a slice/tuple/.../etc.


class BaseAdapter(Protocol):
structure_family: StructureFamily

def metadata(self) -> JSON:
pass

@abstractproperty
def specs(self) -> List[Spec]:
pass


class ContainerAdapter(collections.abc.Mapping[str, "AnyAdapter"], BaseAdapter):
structure_family = Literal[StructureFamily.container]

def structure(self) -> None:
pass


class ArrayAdapter(BaseAdapter):
structure_family = Literal[StructureFamily.array]

def structure(self) -> ArrayStructure:
pass

# TODO Fix slice
def read(self, slice: Slice) -> NDArray:
pass

# TODO Fix slice
def read_block(self, block: Tuple[int, ...]) -> NDArray:
pass


class AwkwardAdapter(BaseAdapter):
structure_family = Literal[StructureFamily.awkward]

def structure(self) -> AwkwardStructure:
pass

def read(self) -> NDArray: # Are Slice and Array defined by numpy somewhere?
pass

def read_buffers(self, form_keys: Optional[List[str]] = None) -> Dict[str, Any]:
pass


class SparseAdapter(BaseAdapter):
structure_family = Literal[StructureFamily.sparse]

def structure(self) -> SparseStructure:
pass

# TODO Fix slice (just like array)
def read(
self, slice: Slice
) -> sparse.COO: # Are Slice and Array defined by numpy somewhere?
pass

# TODO Fix slice (just like array)
def read_block(self, block: Tuple[int, ...]) -> sparse.COO:
pass


class TableAdapter(BaseAdapter):
structure_family = Literal[StructureFamily.table]

def structure(self) -> TableStructure:
pass

def read(self, fields: list[str]) -> pandas.DataFrame:
pass

def read_partition(self, partition: int) -> pandas.DataFrame:
pass

def get(self, key: str) -> ArrayAdapter:
pass


AnyAdapter = Union[
ArrayAdapter, AwkwardAdapter, ContainerAdapter, SparseAdapter, TableAdapter
]


Scopes = Set[str]
Query = Any # for now...
Filters = List[Query]


class AccessPolicy(Protocol):
def allowed_scopes(self, node: BaseAdapter, principal: Principal) -> Scopes:
pass

def filters(
self, node: BaseAdapter, principal: Principal, scopes: Scopes
) -> Filters:
pass