SoftwareSerial Debugging

[beau-seidon]

https://github.com/nickgammon/SendOnlySoftwareSerial

Although the UART is occupied by MIDI, it might be possible implement serial debugging using this library.

[beau-seidon]

One problem I encountered while attempting this tonight is that the library uses a class which is a C++ feature. I wasnt able to get it to compile with a .cpp source file. I'm still a Make / compiler n00b. Not sure if I just need to fiddle around with the Makefile, or give up and just try to disassemble the SoftwareSerial class into a struct and functions that C can use. Will try again tomorrow.

[beau-seidon]

IT WORKS!

I had to make some slight modifications to a few source files so it would compile in VSCode+PlatformIO (which I will elaborate on later), then I included the SendOnlySoftwareSerial library as a dependency and had to wrap the calls to the C++ functions, that use the class methods, with extern "C" { } (I will post the references I used later too). Then I wrote some basic debug test code in src/midi/midi.c.

Now, each time the NESizer receives a NoteOn message, it sends the MIDI Command+Channel, Data1, and Data2 bytes as debug messages via the ISP header's MOSI pin (11) on the Atmega328p at 9600 baud. However, after I entered 9600 in the initialization, I had to use my oscilloscope to find the actual baud rate of the signal transmitted. Because of clock scaling or something, the output signal is 1.25x the entered value, so to get 9600, I had to enter 7680.

I connected the MOSI pin to the RX pin of another microcontroller's UART, and am using a basic serial pass-through program to monitor input via the VSCode serial monitor, but this could also be done with a serial to USB adapter into some application like Putty.

Gotta work early, but I will play with this more tomorrow night and poast results and bugs!

[beau-seidon]

Still playing with it. At low baud rates, it screws with the timing of the task manager or something. It slows down the code, which makes the LED's freak out every time a MIDI message is received, using the simple debug messages mentioned above.

Another packet capture showed that the baud rate issue scales consistently. Serial transmits 1.25x faster than expected, so you have to enter 0.8x the desired baud. To connect it to my serial console at 115200, I had to tell it to transmit at 92160 then everything communicated smoothly.

At this faster speed, the LED's don't flicker like before, and I still receive all messages even when feeding the NESizer notes at 240 BPM 1/32T from an arp. And they all appear to be coherent! Not sure about MIDI CC messages yet.

[beau-seidon]

uploaded code changes to my NESizer2 fork.

it's in a proof-of-concept state, not exactly production quality.

[Jaffe-]

Great that you're looking into this. It is something that is nice to have, but which is hard to add to the system without affecting the performance. But it can be extremely useful to debugging some of the issues that are not real time critical. For instance I actually developed and tested the whole rewrite of the sample 'file system' on my PC and the ported it to the NESizer because it would be a nightmare to debug it. But still I had to use the LEDs to aid in debugging.

I have a feeling that if we want to get this in it might be better to implement the UART functionality within the task system. The NESizer firmware is a cooperative multitasking system, so each task handler is expected to run for less than one 16 kHz 'tick' (62.5 us or 1250 clock cycles) before returning. If these deadlines are not met, then other tasks will be starved of runtime. So if a task is now spending time calling into code that has a loop that bit-bangs bytes out on the GPIO, it will very quickly run for much longer than this. This is also why increasing the baud rate improves the situation, since the calls to the SendOnlySoftwareSerial::write then will take less time.

Ideally the way this should be implemented is using a task handler for bit banging data out on the GPIO. From what it looks like, the easiest is to have a task handler that bit bangs only one byte and then returns. If that can be made to run reliably at 16 kHz rate, then the great thing about this is that the UART bit banging will be decoupled from the tasks that use the debug functionality. The way it would work then, is that a call to debug() would just put the data into a queue/buffer and return, while the task handler would be consuming one byte from the buffer and bit-gang it out each time it is called. This can probably be achieved by reworking the code from the SendOnlySoftwareSerial library.

[beau-seidon]

Yea I figured the debug messages I shoehorned into the midi_channel_apply() function made the midi_handler() task spend too long printing bytes, so the next task started late, which is likely what caused the LED's to flicker. I do like the idea of using debug calls throughout the code to fill a buffer, and if the buffer is not empty, the task manager could print a byte or several from the buffer each cycle.

The ESP32 I'm using to receive and print the serial data could also be programmed to handle the incoming bytes and load them to a buffer of its own and print them to the monitor only when the message is coherent, instead of printing a raw byte stream. Debug could load messages to the NESizer buffer followed by a STOP message, which the ESP32 could use to trigger the print after everything has been received.

I was going to create a serial_debug_handler task in the NESizer to do that, but the main thing I don't understand yet is how to determine the .period and .counter for a new task, so that it runs for just as long as it needs to, to keep the rest of the processes happening when they should.

For example: {.handler = &midi_handler, .period = 10, .counter = 5},
Where did 10, and 5 come from?

[beau-seidon]

This is effing brilliant.
I made a spreadsheet like your diagram above for the actual tasks in the code, to help me visualize it.
task manager.xlsx

The only thing that is still slightly unclear is the midi_io_handler, which is called every 5 ticks, but has an initial counter of 0. All the rest are at least 1. Is this to offset it back by one period to align with the midi_handler and sequencer_handler?

Also, I built a ring buffer class with a uint8_t member array, which can be loaded with debug message bytes throughout the main code (this should hardly impact the task timing), as well as an extra task with period=1 and lowest priority, which checks to see if there is anything in the print queue and if there is, it bitbangs out 1 byte each tick. It's working great.

The weird issue with the clock scaler affecting the baud rate is no longer present. Not sure why.
Might be something to do with the changes I made in my platformio.ini file since last I tried the serial debug.
Perhaps the F_CPU config or compiler optimization level. Don't know.

I'm not 100% ready to put in a pull request, but I'm fixing to commit my latest revisions to the serial-debug branch of my fork.
Should be ready very soon!

[Jaffe-]

That visualization in the excel sheet is cool, I think you've understood it very well! And actually I see some problems in the code now that I didn't see before... I'm not sure that the initial task 'timeslot placement' will stay stable over time, it seems to me that many of the same tasks can end up with the same count if they get starved for too long.

Anyway, regarding count = 0, that just puts it in 'timeslot 0'. And for the tasks with period = 1, that would actually be a more correct count setting. In practice it doesn't matter, if count starts as 1, then it gets incremented to 2 and the task is run (because 2 >= 1), the same would happen if count = 0, then it gets incremented to 1, and the task is also run (because 1 >= 1).

I'll take a closer look at the PR soon. I don't like dragging in 3rd party libraries for this, but it might be the pragmatic solution here :p

[beau-seidon]

This PR is just for debugging purposes, it doesn't need to be part of the Master branch. It includes a couple extra modules but doesn't modify the existing code base, except in the couple examples I left in. The new code is compartmentalized so the whole branch can be easily updated as Master changes. So maybe add it as a separate Debug branch?

[beau-seidon]

OR decline it, and I'll just keep it up to date in my own fork if you ever need it to play with.

[Jaffe-]

That's what I need to look at! As long as we can build the 'official build' using make and this is not included, then it is fine by me! That would reduce the need for you to manually merge stuff if master changes.