Acquiring DC input into the pycontrol board from Bonsai

[msoheib]

Dear All,

I have been trying to obtain signals from a pycontrol board into bonsai-rx using a serial micro cable. I tried to create a serial port node in Bonsai-rx, but I was not able to get any readings with DC input. Outside bonsai I am able to do this via mpremote python library but would be nice to work within Bonsai. Is there a firmware similar to firmata for ardiuno for micropython that can help with direct acquisition from within bonsai without an additional acquisition board?

thank you.

[ThomasAkam]

Hi @msoheib,

If I understand correctly your aim is to use pyControl hardware with Bonsai, but without using the pyControl software, either on the pyboard or on the host computer. Is that correct, or are you wanting to run a pyControl task on the pyboard but aquire the data directly into Bonsai?

[alustig3]

@msoheib I'm not sure if it is what you are looking for, but it is possible to create a serial communication channel between pyControl and Bonsai. I am unsure if the latencies involved are acceptable for your application (I haven't measured, but it could be in the 10s of ms)

I got started by modifying the code found on this bonsai discussion

You will need to communicate from the Bonsai computer through a USB port which will need to be converted into a serial UART that can be fed into one of the pyControl breakout boards ports that have a UART.

You'll need a USB to serial UART converter (like this or this). From the converter, you'll need to connect the GND, TX, and RX lines to the pyControl breakout. This can be accomplished by connecting the wires to a port adaptor (can be purchased from open-ephys), then connecting the adaptor to the breakout boards Port 1,3 or 4 (ports that have UART TX/RX lines)

Attached are some files to get started: pycontol_bonsai_communication.zip

For the Bonsai project, you will likely need to change the COM port. When running, pressing the "A" key on your keyboard will send a serial message to the USB COM port. It will also write any message received from that same COM port, into a .txt file

The bonsai_serial_task.py should go in your task folder, and the bonsai.py should go in your devices folder. You will need to "load framework" onto your pyboard so that the new device file gets transferred. The example task uses port 1.

I have never used Bonsai before, so I'm not sure if I can help much more on that front, but the basic example I made seems to work. It sounds like @horsto also uses Bonsai with pyControl, so maybe he can provide some help as well.

[msoheib]

@ThomasAkam @alustig3 Thanks for the answers. @alustig3 that's exactly what I was trying to do but it sounds more complex than I had anticipated.

I wanted the communication between pycontrol board and bonsai-rx because I wanted to synchronize with visual stimuli presented on a display (via python script for psychopy or bonsai/bonvision). I am wondering if there is a more easier way to go about this than what I am trying to do.

[ThomasAkam]

If all you need to do is trigger stimulus presentation in Bonsai from pyControl, or trigger an event in pyControl from Bonsai, one option might be to just send TTL pulses from one to the other, rather than serial communcation. Bonsai can send and recieve digital inputs/outputs, e.g. using an arduino running Firmata as the physical interface to connect the wires to. You can use the BNC connectors on the pyControl breakout board as either digital inputs (to generate framework events on rising and/or falling edges of a digital signal) or digital outputs (to set the voltage on the BNC connector high or low from the pyControl task), see the hardware docs for more details. You could either program your pyControl task to set a digital output high to tell bonsai to show the next stimulus, or to wait for a digital input from Bonsai indicating that the stimulus is now on.

This approach would be more limited in functionallity that the serial communication based approach that @alustig3 was proposing, as you could only send pulses to trigger when something happens, rather than sending messages through the serial line, but might be simpler to implement.

[msoheib]

Thanks, Thomas. That seems to be the simplest setup for me at this point.

I was wondering if I could do this programmatically via pyboard module. I am able to check the status of the pin via mpremote in REPL mode. I was wondering if there would be a way to "listen" in for changes to the a specific port on the host computer and then run a visual stimulus based on that via psychopy.

[ThomasAkam]

I don't have any experience with psychopy i'm afraid so have no idea if you could make it interact with the pyboard directly over the USB serial.

If you did want to try and run a pyControl tasks on the pyboard, but interact with the pyboard over serial directly from psychopy (or any other python program), rather than using the pyControl GUI, the Pycboard class can be used as an API for uploading and running a task and reading the data coming back from the board.

[horsto]

@ThomasAkam, this looks interesting - could be something to try from within Bonsai (call a python script for start/end of acquisition). Do you have a small script example for how to start (upload of task, setting of subject, recording, ...ending the acquisition)?

[ThomasAkam]

The code below connects to a board, sets up a task, runs the task and saves the data to disk.

import time
import sys
sys.path.insert(0,'C:\\pyControl') # Add pyControl dir to path.

from com.pycboard import Pycboard
from com.data_logger import Data_logger

# Connect to board.
board = Pycboard(serial_port='COM4')

# Instantiate a Data_logger to record data from the board.
board.data_logger = Data_logger(print_func=print)

# Setup the 'blinker' task from the 'pyControl/tasks/examples' folder.
board.setup_state_machine('example/blinker')

# (Optional) Open a data file to save the data to.
board.data_logger.open_data_file(
    data_dir='',
    experiment_name='test_experiment', 
    setup_ID='test_setup',
    subject_ID='m1')

# Run framework for 10 seconds.
start_time = time.time()
board.start_framework()
while (time.time() - start_time) < 10:
    board.process_data() # Automatically passes data to board.data_logger
    time.sleep(0.1)
board.stop_framework()

# Close the data file.
board.data_logger.close_files()

[msoheib]

Thanks, everyone. I got it working well with the TTL signal.

Is there a possibility to do API calls for psychopy and my camera's API from run_experiment_tab.py?

[ThomasAkam]

Currently there is currently no support for external calls to other programs from the standard pyControl GUI, but some functionallity for this was developed by @neuromantic99 when he was working in Adam Packer's lab, though it never got integrated into the main pyControl GUI. His work on this is in the api_dev branch of his pyControl fork. The way it works in that the user subclasses an API class (located here), specifying methods that will be called when the task is initialised, when the run is started, and each time data is recieved from the board. The user can specify which API class to use with a given task by defining a v.api_class variable in their task file. There is an example included which implements a custom plot for a reversal learning task: API class code, Task code.

It should be reasonably straightforward to adapt this to work with the current version of pyControl and I would be happy to help with that, or you could work with the older version of the GUI.

[msoheib]

Thanks @ThomasAkam. I managed to start the camera API based on your suggestion and @neuromantic99's code into the current version.

[msoheib]

Although it seems it only works with the "run task" tab. Calling the camera API from the experiment tab (here) initializes the API (here) call but does not start recording. I am guessing it has something to do with the ThreadPoolExecutor? My camera call is using the treading module. Could that be a cause for the concurrency issue?

@ThomasAkam would appreciate your help in this.

[ThomasAkam]

Hi @msoheib. I think the issue is that in the run_experiment tab you are not calling the run_start method of the user API class when the task starts running. In the run_task tab this gets called at line 411. I think in your run_experiment tab code you could call the user API run_start method by addding a line to the Subjectbox.start_task method as:

self.run_exp_tab.APIs[self.setup_number].run_start()

[msoheib]

Great! Thanks!

I hard-coded in self.APIs[0].run_start(), and it seemed to somehow work. I will try this instead. Cheers!

If I could please ask another question about setting variables during a task run. Is it possible to set the task variable during the task run or intialization?

I want to randomize a list containing 8 orientation angles and pass them to my psychopy instance. Then use the TTL to flip through the randomized list, which should be the identical between pycontrol task (as states) and psychopy. I have been trying to do this by shuffling the list and then setting the task variable via the set_variable function in the api example.

But it runs without giving me an error but does not change the variable when I run the task.

The api/user_classes file is as follows.

from api.api import Api
import random

class orientation_listener(Api):

    def __init__(self):
        pass

    def run_task(self):
        self.orientations = ['degrees_135','degrees_180','degrees_225','degrees_270','degrees_315','degrees_0', 'degrees_45','degrees_90']
        random.shuffle(self.orientations)
        self.set_variable('orientations', self.orientations)

I am a bit stuck as to what I am not doing correctly. Thank you for your time!

[ThomasAkam]

I hard-coded in self.APIs[0].run_start(), and it seemed to somehow work. I will try this instead.

Using self.APIs[0].run_start() will work when you are running an experiment with a single setup, but not with multiple setups, as each setup has its own instance of the API class.

If I could please ask another question about setting variables during a task run. Is it possible to set the task variable during the task run or intialization?

I think this should be possible given that the API class has a set variable method. I supect that the reason it is failing silently is that if an error occurs in the GUI code, rather than causing the GUI to imediately close, the error is written to a file 'ErrorLog.txt' in the top level pyControl folder, and the GUI remains running. Take a look and see if you have an error message there indicating where the set_variable failed.