Merge pull request #298 from UHHRobotics22-23/docs/vision

Docs/vision

README.md

@@ -143,28 +143,33 @@ source devel/setup.bash
 Additionally to the UR5, the following devices have to be connected and configured before launching the project:
 1. Logitech StreamCam (packages marimbabot_vision and marimbabot_speech)
 2. Scarlett 2i2 USB Audio Interface (package marimbabot_audio)
+3. Wifi connection to the MalletHolder (package marimbabot_hardware)
 
 #### Logitech StreamCam (required for packages marimbabot_vision and marimbabot_speech):
-Change the parameter <i>device</i> of the node <i>audio_capture</i> in the launch file 
+Change the parameter <i>device</i> of the node <i>audio_capture</i> in the [launch file](marimbabot_speech/launch/command_recognition.launch) of the package marimbabot_speech:
 
 ```bash
 marimbabot_speech/launch/command_recognition.launch
 ```
 
-and modify the <i>device_id</i> parameter in the configuration file:
+and modify the <i>device_id</i> parameter in the [configuration file](marimbabot_vision/config/cv_camera.yaml) of the package marimbabot_vision:
 
 ```bash
 marimbabot_vision/config/cv_camera.yaml
 ```
 
 #### Scarlett 2i2 USB Audio Interface (required for package marimbabot_audio):
 
-Adjust the <i>device</i> parameter for the <i>note_audio_capture</i> node in the launch file:
+Adjust the <i>device</i> parameter for the <i>note_audio_capture</i> node in the [launch file](marimbabot_audio/launch/audio_feedback.launch) of the package marimbabot_audio:
 
 ```bash
 marimbabot_audio/launch/audio_feedback.launch
 ```
 
+#### Wifi connection to the MalletHolder
+You should be connected to its Wifi. Please see the [README](marimbabot_hardware/README.md) of the package marimbabot_hardware and [README](marimbabot_bringup/README.md) of the package marimbabot_bringup for further information.
+
+
 #### Launch the whole project
 In order to run the whole project on the real robot, one has to run two launch files. First, the launch file that sets up the robot and its hardware:
 

marimbabot_msgs/README.md

@@ -50,6 +50,7 @@ Have a look at [README](../marimbabot_speech/README.md#5-command-examples) for m
 
 ### [CQTStamped.msg](msg/CQTStamped.msg)
 This message contains information of the Constant-Q Transform(CQT). See [README](../marimbabot_audio/README.md#4-pipeline-of-music-note-detection) for more information.
+
 ### [HitSequence.msg](msg/HitSequence.msg)
 This message contains an array of HitSequenceElement messages.
 
@@ -58,7 +59,9 @@ This message contains the information single element of a HitSequence message. I
 
 ### [NoteOnset.msg](msg/NoteOnset.msg)
 Used for publish the detect music note.
+
 ### [SequenceMatchResult.msg](msg/SequenceMatchResult.msg)
 For published the final evaluation of the robot performance
+
 ### [Speech.msg](msg/Speech.msg)
 For published the transcribed test

marimbabot_vision/README.md

@@ -0,0 +1,13 @@
+# TAMS Master Project 2022/2023 - Vision
+
+## Scripts
+For more information on the usage of the scripts, please refer [README](../marimbabot_vision/scripts/README.md).
+
+## Src
+### [vision_node.py](src/vision_node.py)
+
+This ROS node is responsible for processing images from a camera source and recognizing notes in the images using a pre-trained model. It converts the image data into a textual representation of recognized musical notes and publishes them as ROS messages.
+
+
+### [visualization_node.py](src/visualization_node.py)
+The ROS node receives recognized notes from the vision_node and generates visual representations of the musical notations. It uses the LilyPond library to create musical staff notation and publishes the resulting images as ROS messages for visualization.

marimbabot_vision/scripts/README.md

@@ -11,7 +11,7 @@ This script generates a separate dataset including one sample folder for each ca
 Executes all data generating scripts (generate_data.py, generate_hw_data.py, generate_augmented_data.py) in order to generate a full dataset. The dataset is saved in the `data`, `data_augmented`, `data_hw` and `data_hw_augmented` folders.
 
 ### `generate_data.py`
-Generates a dataset of images of the random note sheets withing given a note-specific duration restriction (e.g. use a 1/16th note as a minimum duration). The dataset is saved in the `data` folder.
+Generates a dataset of images of the random note sheets withing given a note-specific duration restriction (e.g. use a 1/16th note as a minimum duration). The dataset is saved in the `data` folder. In the current configuration, also because of limited computational resources while training, the dataset is generated with 3 bars of music. 
 
 Arguments:
   -  num_samples: Amount of data to be generated.