Fix yolo configuration with PipeCNN to get correct outputs

[META-DREAMER]

Need to play with precision config for fixed point representation as well as fixing pooling issues

[META-DREAMER]

Finally figured out how to configure the fixed point params for the weights file as well as how to merge the batch norm and scaling layers into the convolution layers.

Also figured out that the data we get out of PipeCNN needs to be multipled by 2^-f_out, where f_out is the number of fractional bits configured for the last layers output.

So I figured out the precision_config for frac_w (number of fractional bits for the weights in each layer), problem now is how to figure out the precision_config for frac_din and frac_dout (number of fractional bits for the input and output data of each layer). The weights were easier to figure out since we have the weights file data and can just look at the range of values in each layer. The data in/out is tricker however, since we somehow have to figure out the range of values outputted from each layer. Opened an issue for it here: doonny/PipeCNN#64

The paper does describe the technique they use to determine the fixed point values, but I dont know how we would go about doing this ourselves:

[META-DREAMER]

Ok so I converted our weights file to Keras and used this implementation of tiny-yolo to get the data output of each layer and see what the ranges of values were, and then came up with the appropriate fractional widths for each layer in our network:

LayerName	frac_w	frac_din	frac_dout
conv1	3	7	4
relu1		4	5
pool1		5	5
conv2	10	5	4
relu2		4	5
pool2		5	5
conv3	9	5	4
relu3		4	5
pool3		5	5
conv4	10	5	4
relu4		4	5
pool4		5	5
conv5	11	5	4
relu5		4	7
pool5		7	7
conv6	11	7	4
relu6		4	7
pool6		7	7
conv7	10	7	3
relu7		3	5
conv8	14	5	6
relu8		6	6
conv9	11	6	5

I set the precision config according to these values and ran the classifier again and passed the results to YOLO.cpp. The values outputted from the network seem more "real" now, but the result is still incorrect. I think this is because the input/output of each layer is very different when using fixed point representation (will be large whole integers instead of floats), so looking at the data from keras which is floating point doesnt help. We will need to read the values outputted in each layer from PipeCNN directly. @nocduro do you have any idea how we can do this? Maybe adding something to conv_pipe.cl that will save the output of each layer to a file or something?

[META-DREAMER]

On the bright side, our weights and anchors and the rest of our config is working well! I recreated our PipeCNN network in Keras, used our weights, and got these results:

[nocduro]

Nice work!

do you have any idea how we can do this? Maybe adding something to conv_pipe.cl that will save the output of each layer to a file or something?

I'll take a look at this tomorrow, I think we can actually do it right from main.cpp by reading out the buffers (?) before going onto the next layer. It might be a bit tricky since I believe some layers save their results different buffers.

Relevant loop: https://github.com/hammadj/Qwik-e-Classifier/blob/yolo/PipeCNN/project/host/main.cpp#L411

[META-DREAMER]

Yea, might be a bit tricky. main.cpp mostly just configures all the arguments for the cl kernel and sends it the input data and output data, doesnt really read the data back for intermediate layers. Here's the part where they actually extract data from the kernel: https://github.com/hammadj/Qwik-e-Classifier/blob/yolo/PipeCNN/project/host/main.cpp#L866

[doonny]

Truncating weight is easy, it is hard to truncate the featuremap, as it should be done at runtime. You need modification on Darknet itself.
I don't think either Keras nor TensorFlow support fixed-point convolutions, you really need to write your own code for fixed-point computations. We have done this in Matlab, writing everything from scratch using fixed-point toolbox.