Skip to content

Commit

Permalink
split linux_aio.raw package into another package (linux_aio_bind)
Browse files Browse the repository at this point in the history
  • Loading branch information
isac322 committed Jan 29, 2019
1 parent dfe3f6c commit be65221
Show file tree
Hide file tree
Showing 22 changed files with 34 additions and 646 deletions.
32 changes: 6 additions & 26 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,34 +4,14 @@ notifications:
dist: xenial

language: python
python:
- '3.7'

matrix:
include:
- sudo: required
services:
- docker
env: DOCKER_IMAGE=quay.io/pypa/manylinux1_x86_64
python:
- '3.7'
- sudo: required
services:
- docker
env: DOCKER_IMAGE=quay.io/pypa/manylinux1_i686
PRE_CMD=linux32
python:
- '3.7'

install:
- docker pull $DOCKER_IMAGE
- pip install codecov
- pip install -r requirements.txt
env:
- PYVER=37

script:
- docker run --rm -v `pwd`:/io $DOCKER_IMAGE $PRE_CMD /io/.travis/build-wheels.sh
- mkdir dist -p
- cp wheelhouse/* dist
- ls dist/
- python setup.py build_ext --inplace
- pip install codecov
- coverage run -m test

after_success:
Expand All @@ -44,6 +24,6 @@ deploy:
secure: "WHQGmYXBaNZrD817kt0/EBHBAZxYnhffMajVi9J3XZArtcpmEiHcy1aKUm6eKZWcoE//8libBNddNdEEqaW74uHc8EWZENVXlAzSYZlgQhRwd4f2USPZ7ZSODHZAvpCglMDhW7jtRwXBRlNyajfCLgF9Rf2yerweIQ/JTWA1afIPN1vniioiZjnJswq7/pzeRuE3RoVsX76961HE/PhB4mqAdQKzbFKtNkU2XtLexrpjR2PkYk46659en6OdHzd3eFd6/UTk4MvIICSfMn5nXwAi8ZHzW2TaJ+iQb5KACgpXLd6PP2QaewacB2gFZumGt0+VHGK4gNt8/KUQKQ17cTYk0+KFlZkB5DP2HjL05dWQFl4RC1B0qbPBPwXYBW8ZcjfFFqUNfzvwueZ3EHooyJw+ZITfEdCYG3UR1t6q/MJTGssRctDiUCSvuw1xJm32iDNLMgOdn9XiC0Jes11EHH9kh+oQCI/+ENUxAi5Vhsyev+dylRHF/3TQxf0heQSYerVty1zQ44+ImrJu/TbEQFVPGhJWEIFPd9BDaC9MW34S4y9ZDB8JAbU5hv9U1ut6pAfd/ODoOTXl+K7Qgzu9mZWGiiORGX5BRXtQewlCOG0RZZtcACEvMdofMoZht6+D0OlLwqa62e9j2S3bdD1eOeXKclhRiLAVeVcRrtPpEMw="
on:
all_branches: true
distributions: 'sdist'
distributions: 'bdist_wheel --python-tag=py${PYVER}'
skip_cleanup: true
skip_existing: true
22 changes: 0 additions & 22 deletions .travis/build-wheels.sh

This file was deleted.

22 changes: 6 additions & 16 deletions README.kor.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@
[![](https://img.shields.io/pypi/wheel/linux_aio.svg?style=flat-square)](https://pypi.org/project/linux-aio/)
[![](https://img.shields.io/pypi/l/linux_aio.svg?style=flat-square)](https://pypi.org/project/linux-aio/)

[Linux Kernel AIO](http://lse.sourceforge.net/io/aio.html)를 직접 사용하는 Python wrapper 모듈
[Linux Kernel AIO](http://lse.sourceforge.net/io/aio.html)를 직접 사용하는 Python High-level wrapper 모듈.

See [linux_aio_bind](https://pypi.org/project/linux-aio-bind) if you want to handle the API directly at low-level.

## Linux Kernel AIO이란?

Expand All @@ -38,27 +39,16 @@ POSIX AIO의 API들은 `aio_` 접두사를 가지지만, Linux Kernel AIO는 `io

## 구현 및 구조

### `linux_aio.raw` 패키지

- [Python의 ctypes 모듈](https://docs.python.org/ko/3/library/ctypes.html) 사용
- Linux AIO의 C header와 1:1 대응되는 정의 모음
- C를 사용했을 때의 기능을 100% 옮김
- [Linux man pages (4.16)](http://man7.org/linux/man-pages/man2/io_submit.2.html)기준으로 man page에 보이는 모든 기능과, [4.20.3 소스 코드](https://elixir.bootlin.com/linux/v4.20.3/source/include/uapi/linux/aio_abi.h#L71)에서 추가된 기능들을 발견할 수 있는 한 모두 옮김
- [ctypes 모듈](https://docs.python.org/ko/3/library/ctypes.html)을 사용하여 포인터 단위로 연산할줄 안다면, 이 패키지를 기반으로하여 다른 형태의 wrapper도 제작 가능
- ABI 호출은 `syscall`을 사용하며, [아키텍쳐별로 다른 syscall number](https://fedora.juszkiewicz.com.pl/syscalls.html)를 모듈 설치시에 얻기위해 [cffi](https://pypi.org/project/cffi/)를 사용
- [소스 코드 참조](linux_aio/raw/syscall.py)
- [python stub](https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules) (`pyi` 파일 - for type hint) 포함

### `linux_aio` 패키지

- `linux_aio.raw` 패키지를 기반으로 구현
- `linux_aio.raw`와 달리 `ctypes`에대한 지식 없이도 사용 가능
- [test codes](test)의 코드들에서 예제 확인 가능
- Linux kernel AIO의 Low-level binding인 [linux_aio_bind](https://pypi.org/project/linux-aio-bind) 패키지를 기반으로 구현
- [linux_aio_bind](https://pypi.org/project/linux-aio-bind)와 달리 `ctypes`에대한 지식 없이도 사용 가능
- [test codes](https://github.com/isac322/linux_aio/tree/master/test)의 코드들에서 예제 확인 가능


## 예제

[test](test)의 코드들에서 예제 확인 가능
[test](https://github.com/isac322/linux_aio/tree/master/test)의 코드들에서 예제 확인 가능


## Notes & Limits
Expand Down
19 changes: 4 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,27 +37,16 @@ So, as you can see from the experiment below, it's much worse than using the blo

## Implementation & Structure

### Package `linux_aio.raw`

- [ctypes module](https://docs.python.org/3/library/ctypes.html) is used.
- It is defined to correspond `1:1` with the C header of Linux AIO.
- Implemented 100% of the functionality when using C.
- All the functions shown in the man page based on [Linux man pages (4.16)](http://man7.org/linux/man-pages/man2/io_submit.2.html), and all the functions added in [4.20.3 source code](https://elixir.bootlin.com/linux/v4.20.3/source/include/uapi/linux/aio_abi.h#L71), as far as I can find them.
- If you know how to use the [ctypes module](https://docs.python.org/3/library/ctypes.html) to operate on pointers, you can also build other types of wrappers based on this package.
- It uses `syscall` for invoking ABI and [cffi](https://pypi.org/project/cffi/) for gathering [different syscall number by architecture](https://fedora.juszkiewicz.com.pl/syscalls.html) on module installation.
- [refer the code](linux_aio/raw/syscall.py)
- [python stub](https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules) (`pyi` files - for type hint) are included.

### Package `linux_aio`

- It based on package `linux_aio.raw`.
- Unlike `linux_aio.raw`, it can be used without knowledge of `ctypes`
- Examples can be found in the code in the [test directory](test).
- Implemented based on [linux_aio_bind](https://pypi.org/project/linux-aio-bind) package which is low-level binding of Linux kernel AIO.
- Unlike [linux_aio_bind](https://pypi.org/project/linux-aio-bind), it can be used without knowledge of `ctypes`
- Examples can be found in the code in the [test directory](https://github.com/isac322/linux_aio/tree/master/test).


## Example

Examples can be found in the code in the [test directory](test).
Examples can be found in the code in the [test directory](https://github.com/isac322/linux_aio/tree/master/test).


## Notes & Limits
Expand Down
7 changes: 4 additions & 3 deletions linux_aio/aio_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,13 @@
from types import TracebackType
from typing import Optional, Tuple, Type

from .aio_event import AIOEvent
from .block import AIOBlock
from .raw import (
from linux_aio_bind import (
IOEvent, Timespec, aio_context_t, create_c_array, io_cancel, io_destroy, io_getevents, io_setup, io_submit, iocb_p
)

from .aio_event import AIOEvent
from .block import AIOBlock


class AIOContext:
"""
Expand Down
3 changes: 2 additions & 1 deletion linux_aio/aio_event.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
from ctypes import py_object
from typing import Tuple, Union

from linux_aio_bind import IOEvent

from .block import AIOBlock, NonVectorBlock, ReadBlock, ReadVBlock, VectorBlock, WriteBlock, WriteVBlock
from .raw import IOEvent


class AIOEvent:
Expand Down
4 changes: 2 additions & 2 deletions linux_aio/block/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@
from ctypes import addressof, py_object
from typing import TYPE_CHECKING, overload

from ..raw import IOCB, IOCBCMD, IOCBFlag, IOPRIO_CLASS_SHIFT, gen_io_priority
from linux_aio_bind import IOCB, IOCBCMD, IOCBFlag, IOPRIO_CLASS_SHIFT, gen_io_priority

if TYPE_CHECKING:
from typing import Any, Optional
from linux_aio_bind import IOCBPriorityClass, IOCBRWFlag
from .non_rw import FDsyncBlock, FsyncBlock, PollBlock
from .non_vector import ReadBlock, WriteBlock
from .vector import ReadVBlock, WriteVBlock
from ..raw import IOCBPriorityClass, IOCBRWFlag


class AIOBlock(metaclass=ABCMeta):
Expand Down
3 changes: 2 additions & 1 deletion linux_aio/block/non_rw.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
import select
from typing import Any

from linux_aio_bind import IOCBCMD, IOCBPriorityClass

from .base import AIOBlock
from ..raw import IOCBCMD, IOCBPriorityClass


class NonRWBlock(AIOBlock):
Expand Down
3 changes: 2 additions & 1 deletion linux_aio/block/non_vector.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

from typing import Any, Union

from linux_aio_bind import IOCBCMD, IOCBPriorityClass, IOCBRWFlag

from .rw import RWBlock
from ..raw import IOCBCMD, IOCBPriorityClass, IOCBRWFlag


class NonVectorBlock(RWBlock):
Expand Down
3 changes: 2 additions & 1 deletion linux_aio/block/rw.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
from ctypes import c_char, c_char_p, c_void_p, cast
from typing import Any, ClassVar, Tuple, Type, TypeVar, Union

from linux_aio_bind import IOCBCMD, IOCBPriorityClass, IOCBRWFlag

from .base import AIOBlock
from ..raw import IOCBCMD, IOCBPriorityClass, IOCBRWFlag

_NAT_BUF_TYPE = TypeVar('_NAT_BUF_TYPE', bytearray, bytes)

Expand Down
3 changes: 2 additions & 1 deletion linux_aio/block/vector.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
from ctypes import addressof
from typing import Any, Iterable, Tuple, Union

from linux_aio_bind import IOCBCMD, IOCBPriorityClass, IOCBRWFlag, IOVec, create_c_array

from .rw import RWBlock, _NAT_BUF_TYPE
from ..raw import IOCBCMD, IOCBPriorityClass, IOCBRWFlag, IOVec, create_c_array


class VectorBlock(RWBlock):
Expand Down
1 change: 0 additions & 1 deletion linux_aio/py.typed
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
partial
13 changes: 0 additions & 13 deletions linux_aio/raw/__init__.py

This file was deleted.

9 changes: 0 additions & 9 deletions linux_aio/raw/_syscall.pyi

This file was deleted.

103 changes: 0 additions & 103 deletions linux_aio/raw/aio.py

This file was deleted.

Loading

0 comments on commit be65221

Please sign in to comment.