Skip to content

Latest commit

 

History

History
134 lines (77 loc) · 8.24 KB

serial-buffer-adjustment.md

File metadata and controls

134 lines (77 loc) · 8.24 KB
Raw

Updating buffer size

This project requires the modification of serial buffers to work. Read this document to find out how.

Background

Serial buffers are interesting creatures. Normally, they just sit in the background and don't bother anyone. But, in this case, we have to stuff a metric ton of data per second through them. The Arduino can deal with the influx of data, but the buffer needs to be big enough to withstand holding data long enough for the short period of time that the code is not reading.

Arduinos use a ring buffer for their serial port. A ring buffer allows data to be added to the end and read off the beginning in an efficient manner, without having to reallocate memory

The default buffer size is usually 64 bytes, or 16 bytes in a low-RAM scenario. These directions show how to increase the buffer size to 256 bytes, the maximum allowed in an 8-bit variable. We can go higher than this, but most Arduinos have an 8-bit processor width (like the Arduino Uno, Arduino Nano, Lilypad, and Teensy 2.x), meaning that larger variables take more CPU cycles to process, and contribute to less efficient code. 32-bit SAM (Arduino Due, Sparkfun SAMD) and 32-bit AVR (Teensy 3.x) have a 32-bit wide processor, so integers up to 32 bits can still fit on one processor cycle.

Process

To do this, you have manually edit the source files in the Arduino IDE. There's no getting around that (that I've found). Here I will provide a base example that can be followed for most, if not all, boards. File locations and specific board quirks will be provided after.

The buffer size will be a #define with a name like SERIAL_TX_BUFFER_SIZE. It will look something like the following code block. Note that these are all preprocessor directives, so they have to be changed at compile time and cannot be dynamically modified by the code.

#if !defined(SERIAL_TX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64
#endif
#endif
#if !defined(SERIAL_RX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_RX_BUFFER_SIZE 16
#else
#define SERIAL_RX_BUFFER_SIZE 64
#endif
#endif

In this example, the Arduino defaults to a buffer size of 64 bytes, falling back to 16 bytes if the Arduino has a low RAM size. We don't care about the low RAM Arduino, since we're drastically increasing the buffer size. Update both SERIAL_TX_BUFFER_SIZE and SERIAL_RX_BUFFER_SIZE from 64 to 256. Note that 256 is the largest value that fits in a ring buffer controlled by an 8-bit size (a few lines down the code bumps the buffer size from an 8-bit to a 16-bit value if the buffer size exceeds 256). When you've made this changes, save and close the file.

Base directories

Files love to hide! For this process, you will need to know two base directories: the location of the main Arduino IDE, and the location of the Arduino packages directory. I will refer to these as <arduino base> and <arduino pkgs>, respectfully.

The 3 major platforms put their base directories in different places. Because of course they do...

macOS

First, syntax heads up: I will use the term "right click" to refer to the action generated by the right mouse button, Control+click, or two-finger trackpad click. Basically, anything Apple calls a secondary click.

The Arduino base directory is actually inside your Arduino.app package. Locate this package (can be done easily by cmd+clicking the Dock icon), right click on it, and show package contents. Navigate to ./Contents/Java/. This is your <arduino base>. Mine was located in /Applications/Arduino.app/Contents/Java/. In this folder, you should be able to locate a subfolder named hardware.

The packages directory is elusive. Arduino appears to spread out its config between a couple different directories, so be prepared to improvise if you can't find it. Anyways, here goes. First, you need to navigate to your Library folder in your home directory. If you can see a folder called Library inside your home directory, click it and skip the end of the numbered list. If you can't see it right away, you have a number of options:

  1. Alt+click the Go menu in Finder, select Library
  2. Hit the key combo Shift+Cmd+L from any Finder window to move that Finder window to the Library
  3. Open the Go to folder... menu, accessible from the right click menu on the Finder icon, the Go menu, or Shift+Cmd+G. Type ~/Library and press Go

SKIP TO HERE! Now that you're in your Library folder, we can find the Arduino directories. On my computer, they were right in the Library folder going by the names Arduino, Arduino15, and Arduino17. Depending on your system config, they might also be in the Application Support folder.

On my computer, the packages directory was in Arduino15. Your results may vary. This packages folder is the location where the Arduino IDE downloads new boards and board updates from the Boards Manager. My full <arduino pkgs> folder path was ~/Library/Arduino15/packages

Windows (win32 download from Arduino.cc)

TODO

Windows (Microsoft Store)

(yes it's different than if you downloaded from the store)

TODO

Linux

TODO

Files to change

Here you will find the locations of the serial buffers that I have located. If you find others or find any other issue, please open a Github issue so that I can update this.

Arduino 8-bit AVR

Including Arduino Uno, Arduino Nano, and Arduino Mega

There are two places to find these:

  • <arduino base>/hardware/arduino/avr/cores/arduino/HardwareSerial.h
  • <arduino pkgs>/arduino/hardware/arduino/avr/<avr version>/cores/arduino/HardwareSerial.h

If the file exists in both locations, the second one appears to take precedence. The second file may also appear during board updates that do not update the entire IDE.

Arduino 32-bit SAM

Including Arduino Due

  • <arduino pkgs>/arduino/hardware/arduino/sam/<sam version>/cores/arduino/RingBuffer.h

The SAM boards stuff the #define SERIAL_BUFFER_SIZE into RingBuffer instead of HardwareSerial for some reason. Go figure. This value replaces SERIAL_RX_BUFFER_SIZE and SERIAL_TX_BUFFER_SIZE in the base example.

Arduino 32-bit SAMD

Including Arduino Zero, Arduino M0, and Arduino M0 Pro

  • <arduino pkgs>/arduino/hardware/arduino/samd/<samd version>/cores/arduino/RingBuffer.h

Appears very similar to Aruduino 32-bit SAM at the time of this writing. See those directions, and use the file(s) listed here.

SparkFun 32-bit SAMD

Including SparkFun SAMD21 [Dev, Mini] Breakout, SparkFun Wireless Joystick, and SparkFun 9DoF Razor IMU M0

  • <arduino pkgs>/SparkFun/hardware/arduino/samd/<samd version>/cores/arduino/RingBuffer.h

This library appears to be based on the Arduino 32-bit SAMD board definition. See those directions, and use the file(s) listed here.

Teensy 3.x (32-bit AVR)

Including Teensy 3.0, Teensy 3.1, Teensy 3.2, Teensy 3.5, Teensy 3.6, and Teensy LC

  • <arduino base>/hardware/teensy/avr/cores/teensy3/serialX.c where X is a number [1-6]

Teensy is a strange duck. They have their serial buffers defined in the actual C code on a per-port basis, versus the global defines used in the other boards. This gives the advantage of being able to control each serial buffer individually, but can make it a hassle if we just want to update all the ports. But whatever...

Each of the files serial1.c ... serial6.c corresponds to a particular UART port on the Teensy board. The numbers match with the file name. Teensy might set the bar for bumping the port to a 16-bit size at 255 for some reason, or maybe my computer's just buggy. Anyway, I don't mind this on a 32-bit processor since it still fits in 1 CPU cycle. Note that there's a seventh file serial6_lpuart.c that also has to be changed.

Teensy 2.x (8-bit AVR)

Including Teensy 2.0, Teensy++ 2.0

  • <arduino base>/hardware/teensy/avr/cores/teensy/HardwareSerial.cpp

Again, the buffer is in the C code, not the header file. Similar to Arduino 8-bit AVR. Here, the defines are named RX_BUFFER_SIZE and TX_BUFFER_SIZE

Arduino Updates

Updates will probably break these. If the Arduino IDE, Teensyduino loader, or any of the boards from the Board Manager update, check these files as they were probably overwritten in the process.