Skip to content

Latest commit

 

History

History
47 lines (35 loc) · 1.64 KB

TODO.md

File metadata and controls

47 lines (35 loc) · 1.64 KB

TODO list

ProcessPoolExecutor

  • cancel_on_shutdown
  • robustification dans backend
  • idle workers should shutdown while executor should restart missing worker on submit
  • modification parameters ==> recreate pool
  • debug lock on start executor
  • support python2.7 (kwargs.pop ? ) regardé concurrent future package
  • remove appveyor logs
  • cutsomizable pickler for the queues in ProcessPoolExecutor
  • cleanup compat files
  • refactor thread to optimize communication (_feeder thread) #12

Test

  • Test for errors and crashes in job dispatch by the apply async callback
  • Configure Appveyor to run the test under windows (see joblib one)
  • Add travis to run the test under linux/ mac
  • Unskip numpy freeze test once Spawn Processes are implemented
  • Test the callbacks call with ProcessPoolExecutor
  • Unit test semaphore and fd managment
  • Make cython model only built in test that highlight the OpenMP freeze with multiprocessing fork

Python upstream

  • See if Rpool pass Pool test suite from cpython
  • Do not use the result queue to wakeup _queue_management
  • Perfomance optimization for _queue_management_thread wakeup
  • Add broken process pool detection thread (after #12 is solved)

Spawn Processes

  • Create process with subprocess.Popen and os.pipe for posix
  • Test processes with cpython test suite
  • Pass test rpool with such processes
  • [-] automatic benchmark script launched in CI after test (#11)
  • benchmark large messages (string)

Joblib

  • Implement new backend based on loky
  • Make new backend auto mmemap large numpy