Skip to content

Commit

Permalink
Merge branch 'docusaurus-version' into bha2
Browse files Browse the repository at this point in the history
  • Loading branch information
@limengdu
limengdu authored Jan 3, 2025
2 parents bb5d401 + 590d775 commit 97342e7
Show file tree
Hide file tree
Showing 16 changed files with 590 additions and 149 deletions.
2 changes: 1 addition & 1 deletion docs/Edge/M_2_Hat/m_2_hat_start_with_pi5.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ keywords:
- M.2 hat
- dual hat
- Raspberry Pi
image: https://files.seeedstudio.com/wiki/wiki-platform/S-tempor.png
image: https://files.seeedstudio.com/wiki/M.2_Hat/new/m.2-dual-hat-for-raspberry-pi-5.webp
slug: /raspberry_pi_5_uses_pcie_hat_dual_hat
last_update:
date: 12/24/2024
Expand Down
72 changes: 63 additions & 9 deletions docs/Edge/NVIDIA_Jetson/Application/Robotics/Projects/Lerobot_SO100Arm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ keywords:
- Huggingface
- Arm
- Robotics
image: https://files.seeedstudio.com/wiki/wiki-platform/S-tempor.png
image: https://files.seeedstudio.com/wiki/robotics/projects/lerobot/Arm_kit.webp
slug: /lerobot_so100m
last_update:
date: 12/24/2024
Expand Down Expand Up @@ -41,6 +41,8 @@ This wiki provides the assembly and debugging tutorial for the SO ARM100 and rea
4. **Open Source and Flexible Expansion**: It is an open-source platform that is easy to customize and expand, suitable for developers and researchers to conduct secondary development, and supports the integration of multiple sensors and tools.
5. **Multi-Scene Application**: It is applicable to fields such as education, scientific research, automated production, and robotics, helping users achieve efficient and precise robot operations in various complex tasks.

<iframe width="960" height="640" src="https://www.youtube.com/embed/JrF_ymUvrqc?si=vslu5NNI-ZIzVXLc" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

### Specification
| Specification | Arm Kit | Arm Kit Pro |
|--|--|--|
Expand Down Expand Up @@ -190,13 +192,18 @@ The calibration of the robotic arm should be carried out strictly in accordance

**Manual calibration of follower arm**

Contrarily to step 6 of the [assembly video](https://www.youtube.com/watch?v=FioA2oeFZ5I) which illustrates the auto calibration, we will actually do manual calibration of follower for now.
Firstly, you need to ensure that the serial port numbers of the robotic arm in the `\lerobot\lerobot\configs\robot\so100.yaml` file are consistent with yours, as shown in the following figure. You can view all serial port names according to `ls /dev/ttyACM*`.

<iframe width="960" height="640" src="https://www.youtube.com/embed/FioA2oeFZ5I?si=GjudmAovwF_X5m2f" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<div align="center">
<img width={800}
src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/so100yaml.png" />
</div>


You will need to move the follower arm to these positions sequentially:

<iframe width="960" height="640" src="https://www.youtube.com/embed/m1z9Dlk1gW8?si=Ln2PFmPnap2AeDZc" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

Make sure both arms are connected and run this script to launch manual calibration:
```bash
python lerobot/scripts/control_robot.py calibrate \
Expand Down Expand Up @@ -226,15 +233,49 @@ python lerobot/scripts/control_robot.py teleoperate \
--display-cameras 0
```

<iframe width="960" height="640" src="https://www.youtube.com/embed/hnRwfcyX1ZI?si=RuzYjP_FUTK16lfs" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

**Teleop with displaying cameras**
Follow [this guide to setup your cameras](https://github.com/huggingface/lerobot/blob/main/examples/7_get_started_with_real_robot.md#c-add-your-cameras-with-opencvcamera). Then you will be able to display the cameras on your computer while you are teleoperating by running the following code. This is useful to prepare your setup before recording your first dataset.
### Teleop with displaying cameras###
After inserting your two USB cameras, run the following script to check the port numbers of the cameras.
```bash
python lerobot/common/robot_devices/cameras/opencv.py \
--images-dir outputs/images_from_opencv_cameras
```

The terminal will print out the following information.
```markdown
Mac or Windows detected. Finding available camera indices through scanning all indices from 0 to 60
[...]
Camera found at index 2
Camera found at index 4
[...]
Connecting cameras
OpenCVCamera(2, fps=30.0, width=640, height=480, color_mode=rgb)
OpenCVCamera(4, fps=30.0, width=640, height=480, color_mode=rgb)
Saving images to outputs/images_from_opencv_cameras
Frame: 0000 Latency (ms): 39.52
[...]
Frame: 0046 Latency (ms): 40.07
Images have been saved to outputs/images_from_opencv_cameras
```

You can find the pictures taken by each camera in the `outputs/images_from_opencv_cameras` directory, and confirm the port index information corresponding to the cameras at different positions. Then complete the alignment of the camera parameters in the `\lerobot\lerobot\configs\robot\so100.yaml` file.


<div align="center">
<img width={800}
src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/so100camerayaml.png" />
</div>

Then you will be able to display the cameras on your computer while you are teleoperating by running the following code. This is useful to prepare your setup before recording your first dataset.
```bash
python lerobot/scripts/control_robot.py teleoperate \
--robot-path lerobot/configs/robot/so100.yaml
```

### Record a dataset
<iframe width="960" height="640" src="https://www.youtube.com/embed/EUcXlLlOjGE?si=6ncQ7o5ZFLR4PGTU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

### Record the dataset

Once you're familiar with teleoperation, you can record your first dataset with SO-100.

Expand All @@ -261,9 +302,22 @@ python lerobot/scripts/control_robot.py record \
--reset-time-s 10 \
--num-episodes 2 \
--push-to-hub 1
--single-task seeedstudio
```

### Visualize a dataset
```markdown
Parameter Explanations
- wormup-time-s: It refers to the initialization time.
- episode-time-s: It represents the time for collecting data each time.
- reset-time-s: It is the preparation time between each data collection.
- num-episodes: It indicates how many groups of data are expected to be collected.
- push-to-hub: It determines whether to upload the data to the HuggingFace Hub.
```

<iframe width="960" height="640" src="https://www.youtube.com/embed/wc-qh7UFkuQ?si=-eDB73KgUksyJXa-" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>


### Visualize the dataset

If you uploaded your dataset to the hub with `--push-to-hub 1`, you can [visualize your dataset online](https://huggingface.co/spaces/lerobot/visualize_dataset) by copy pasting your repo id given by:
```bash
Expand Down Expand Up @@ -303,7 +357,7 @@ python lerobot/scripts/train.py \
hydra.run.dir=outputs/train/act_so100_test \
hydra.job.name=act_so100_test \
device=cuda \
wandb.enable=true
wandb.enable=false
```

Let's explain it:
Expand Down Expand Up @@ -335,7 +389,7 @@ As you can see, it's almost the same command as previously used to record your t
1. There is an additional `-p` argument which indicates the path to your policy checkpoint with (e.g. `-p outputs/train/eval_so100_test/checkpoints/last/pretrained_model`). You can also use the model repository if you uploaded a model checkpoint to the hub (e.g. `-p ${HF_USER}/act_so100_test`).
2. The name of dataset begins by `eval` to reflect that you are running inference (e.g. `--repo-id ${HF_USER}/eval_act_so100_test`).


<iframe width="960" height="640" src="https://www.youtube.com/embed/wc-qh7UFkuQ?si=Y2SXU9T0DSmtz4ll" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

### Citation
TheRobotStudio Project: [SO-ARM100](https://github.com/TheRobotStudio/SO-ARM100)
Expand Down
10 changes: 5 additions & 5 deletions docs/Edge/NVIDIA_Jetson/Carrier_Boards/A603/A603_Flash_JetPack.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -458,10 +458,10 @@ wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.0/releas
Execute the following commands in order:

```bash
tar xf jetson_linux_r36.3.0_aarch64.tbz2
sudo tar xpf tegra_linux_sample-root-filesystem_r36.3.0_aarch64.tbz2 -C Linux_for_Tegra/rootfs/
unzip 603_jp60.zip # Unzip the driver package
sudo cp -r 603_jp60/Linux_for_Tegra/* Linux_for_Tegra/ # Replace all files in the Linux_for_Tegra directory with the files from the driver package
tar xf Jetson_Linux_R36.4.0_aarch64.tbz2
sudo tar xpf Tegra_Linux_Sample-Root-Filesystem_R36.4.0_aarch64.tbz2 -C Linux_for_Tegra/rootfs/
tar xpf 603_jetpack6.1.tar.gz # Unzip the driver package
sudo cp -r 603_jetpack6.1/Linux_for_Tegra/* Linux_for_Tegra/ # Replace all files in the Linux_for_Tegra directory with the files from the driver package
cd Linux_for_Tegra/ # Navigate to the Linux_for_Tegra path to run the flashing commands
sudo ./apply_binaries.sh
sudo ./tools/l4t_flash_prerequisites.sh
Expand All @@ -481,7 +481,7 @@ sudo ./tools/l4t_flash_prerequisites.sh
**Step 4:** Flash the Device.

```bash
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-orin-nano-devkit internal
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-orin-nano-devkit internal
```

**Tips:** Backup the System and Flash Using the Backup Image
Expand Down
83 changes: 83 additions & 0 deletions docs/Edge/NVIDIA_Jetson/Carrier_Boards/J401/reComputer_J4012_Flash_Jetpack.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -595,8 +595,91 @@ sudo ln -s /usr/src/linux-headers-5.15.136-tegra-ubuntu22.04_aarch64/3rdparty/ca
sudo apt install -y iwlwifi-modules
```
</TabItem>
<TabItem value="JP6.1" label="JP6.1">
Here we will use NVIDIA L4T 36.4 to install Jetpack 6.1 on the reComputer
**Step 1:** Download the system image to your Ubuntu PC corresponding to the Jetson module you are using:
<div class="table-center">
<table style={{textAlign: 'center'}}>
<thead>
<tr>
<th>Jetson Module</th>
<th>Download Link</th>
<th>JetPack Version</th>
<th>L4T Version</th>
</tr>
</thead>
<tbody>
<tr>
<td>Jetson Orin NX 16GB</td>
<td>
<a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EWCZOBNb9C9AoZe-mt23jLABZk942Lf0yopVGFJFTeL5DA?e=o7epES" target="_blank" rel="noopener noreferrer">Download
</a>
</td>
<td rowSpan={4}>6.1</td>
<td rowSpan={4}>36.4</td>
</tr>
<tr>
<td>Jetson Orin NX 8GB</td>
<td>
<a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EcvDRK7qgWhGty_H-P7yHZ8Bob3v9AEs_vFVd-zOC3WX5w?e=FmlfjD" target="_blank" rel="noopener noreferrer">Download
</a>
</td>
</tr>
<tr>
<td>Jetson Orin Nano 8GB</td>
<td>
<a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EeO8T8kNkpZGl9W2QfmnKYQBeXB8-M88aZWLMvPP7uARcA?e=fBuClA" target="_blank" rel="noopener noreferrer">Download
</a>
</td>
</tr>
<tr>
<td>Jetson Orin Nano 4GB</td>
<td>
<a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EY01pL5oq0FAmavoRY_p9DMBj4t-LHzRtEAh1UBQen3S3g?e=jUfauO" target="_blank" rel="noopener noreferrer">Download
</a>
</td>
</tr>
</tbody>
</table>
</div>
**Step 2:** Extract the downloaded image file:
```bash
sudo tar xpf mfi_xxxx.tar.gz
# For example: sudo tar xpf mfi_recomputer-orin-nx-16g-j401-6.1-36.4.0-2024-12-04.tar
```
**Step 3:** Navigate to the unzipped directory and execute the following command to flash jetpack system to the NVMe SSD:
```bash
cd mfi_xxxx
# For example: cd mfi_recomputer-orin-j401
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --massflash 1 --network usb0 --showlogs
```
You will see the following output if the flashing process is successful
<div align="center"><img width ="800" src="https://files.seeedstudio.com/wiki/reComputer-J4012/4.png"/></div>
:::note
The flash command may run for 2-10 minutes.
:::
**Step 4:** Connect the J401 to a display using the HDMI connector on the board and finish the initial configuration setup:
<div align="center"><img width ="800" src="https://files.seeedstudio.com/wiki/reComputer-Jetson/J401/jetpack6_configuration.png"/></div>
:::info
Please complete the **System Configuration** according to your needs.
:::
</TabItem>
</Tabs>
<!-- Code END -->
Expand Down
67 changes: 67 additions & 0 deletions ...er_Jetson_Series/reComputer_Industrial/reComputer_Industrial_Getting_Started.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -593,6 +593,73 @@ After that, the board will reboot and will be ready to use!

<div align="center"><img width ="800" src="https://files.seeedstudio.com/wiki/reComputer-Industrial/106.png"/></div>
</TabItem>

<TabItem value="Jetpack 6.1" label="Jetpack6.1">

- **Step 1:**Download the system image to your Ubuntu PC corresponding to the board you are using
<table>
<thead>
<tr>
<th>Device</th>
<th>Image Link</th>
<th>JetPack Version</th>
<th>L4T Version</th>
</tr>
</thead>
<tbody>
<tr>
<td>reComputer Industrial J4012</td>
<td><a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EUNMocOFBZNPqqC-W9uiIM4BgYUw-ZL0pk6juOVPFd_vqg?e=C8ldYX" target="_blank" rel="noopener noreferrer">Download</a></td>
<td rowSpan={6}>6.1</td>
<td rowSpan={6}>36.4</td>
</tr>
<tr>
<td>reComputer Industrial J4011</td>
<td><a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EfYrDXxD_oRCuk5nv8WmtqIBPY9FjkEHiAmsZDpvDj-sfQ?e=qxV5MG" target="_blank" rel="noopener noreferrer">Download</a></td>
</tr>
<tr>
<td>reComputer Industrial J3011</td>
<td><a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EeOq0sfMDhBEqZPX-ti_gJ0BzUdSGeFf5RrSdxFnQ70aNQ?e=GVr10a" target="_blank" rel="noopener noreferrer">Download</a></td>
</tr>
<tr>
<td>reComputer Industrial J3010</td>
<td><a href="https://seeedstudio88-my.sharepoint.com/:u:/g/personal/youjiang_yu_seeedstudio88_onmicrosoft_com/EXK1GomjByJKnmt3OdE5Vq8BSqzYZm1MY_yD18YjmRplLw?e=dIWPKA" target="_blank" rel="noopener noreferrer">Download</a></td>
</tr>
</tbody>
</table>

:::info
The source code for the above images can be found [here](https://github.com/Seeed-Studio/Linux_for_Tegra).
:::

- **Step 2:** Extract the generated file

```sh
sudo tar -xvf <file_name>.tar.gz
```

- **Step 3:** Navigate to the extracted file from before and execute the flash command as follows

```sh
cd mfi_xxxx
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --massflash 1 --network usb0 --showlogs
```

Now it will start to flash the system image to the board. If the flashing is successful, you will see the below output

<div align="center"><img width ="650" src="https://files.seeedstudio.com/wiki/reComputer-Industrial/99.png"/></div>


- **Step 4:** Connect the J401 to a display using the HDMI connector on the board and finish the initial configuration setup:

<div align="center"><img width ="800" src="https://files.seeedstudio.com/wiki/reComputer-Jetson/J401/jetpack6_configuration.png"/></div>

:::info
Please complete the **System Configuration** according to your needs.
:::

</TabItem>

</Tabs>

<!-- Code END -->
Expand Down
Loading

0 comments on commit 97342e7

Please sign in to comment.