CUDA-compatibility

[hakkelt]

I started working on making operators device-independent. For CUDA (and I think other GPU libs as well), scalar indexing is the cause of incompatibility, so rewriting the operators to do their tasks with high-level matrix operations and range indexing would allow the use of the same source code for CPU and GPU arrays.

In the case of Conv and Eye, it was enough to extend the list of type parameters and replace for loops within mul! using scalar indexing with a slightly more sophisticated array indexing. Also, I created the storageTypeDisplayString property and AbstractOperatorsCudaExt extension so that printing an operator would optionally display their storage device. E.g.:

julia> Eye(ones(4))
I  ℝ^4 -> ℝ^4 

julia> Eye(CUDA.ones(4))
Iᶜᵘ  ℝ^4 -> ℝ^4 

Finally, I created an abstract type for both Eye and Conv to allow the specialization of one or more functions for other GPU array types (e.g., some GPU libraries might not support real FFTs, only complex FFTs, and a custom Conv struct subtyping AbstractConv along with a customized constructor might be enough to add support for that GPU library).

Is it a good idea to do the following steps for the rest of the operators as well?

• Create abstract supertype for all linear and non-linear operators
• Re-write operators to remove all scalar indexing (I think, it is fairly straightforward except for Filt and MIMOFilt, which should be left CPU-only)
• Add a type parameter specifying the storage type for all operators which are free of scalar indexing