I got a Nvidia-Agx-Orin (32GB) in April 2023. I want it to be the brain of autonomous car, a HexMan ros robotics platform with three Intel Realsense Camera.

However, there is no ready-to-go document for this. Most of them are out-of-date, the others use version-specific command like apt instal xxx=0.0.1. I have done many trails-and-errors, finally get a general instruction to make it work. The enviorments are as following:

The tag [optional] means you can skip this section if you don’t have enought materials. Ingore these steps will not lead to finally fail. However, completing these steps can greatly enhance your use experience.

1. Mount NVMe as home [optional]

Agx Orin have only 64GB eMMC memmory, which are stressful for putting down the whole system. So I bought a NVMe .m2 disk mounted as home. Note the .m2 slot in the board only support NVMe protocal (SATA disk can not be discovered).

• Check NVMe SSD available on your device: sudo fdisk -l | grep nvme

• Creat a partition talbe on NVMe SSD: sudo parted /dev/nvme0n1 mklabel gpt

• Creat a partition on NVMe SSD: sudo parted -a optimal /dev/nvme0n1 mkpart primary ext4 0% 100%

• Format to ext4: mkfs.ext4 /dev/nvme0n1p1

• Make a folder and mount on it: sudo mkdir /mnt/nvme | sudo mount /dev/nvme0n1p1 /mnt/nvme

• Get the UUID of disk: sudo blkid | grep nvme

• Edit the fstab to mount home on boot:

  # /etc/fstab
  /dev/root            /                     ext4           defaults                 0 1
  UUID=<NVMe-UUID>     /home                 ext4           defaults                 0 1
  

• Mount home sudo mount -a

2. Proxy [optional]

There are many Chinese-specific problem in installing the packages. Like Stucked rosdep, wstoll, vstool, or something throws time out, lost connection. Due to the poor condition of Chinese search engine, there are plenty of garbe blog to waste your time. So I highly recommend you to use proxy in following steps if you are in China.

The recipe is very simple: clash-premium for back-end service, and proxychains for command-prefix and shell enviroment.

2.1 Clash

wget https://github.com/Dreamacro/clash/releases/download/premium/clash-linux-arm64-2023.03.18.gz
gzip -d clash-linux-arm64-2023.03.18.gz
mv clash-linux-arm64-2023.03.18 clash
sudo chmod 755 clash
sudo chmod +x clash

Config tun mode, edit your yml configure file:

tun:
  enable: true
  stack: system # or gvisor
  auto-route: true # auto set global route
  auto-detect-interface: true # conflict with interface-name
# proxies seg here

optional, Move the files to /usr/local, and give a new alias to zshrc or bashrc

alias clash="nohup /usr/local/clash -f /usr/local/clash-config.yml &"

2.2 Proxychains

Install by apt

sudo apt install proxychains4

And edit the config file /etc/proxychains4.conf. Note the default http inbound port of clash is 7890, change it if you have different settings.

--- socks4  127.0.0.1 9050
+++ http    127.0.0.1 7890

2.3 Testing

Do not use ping, use curl instead.

curl ip.sb
# return the IP without proxy

proxychains4 curl ip.sb
# return the IP with proxy

If you get two different IP, cheers.

3. Realsense (Build from source)

Install Realsense from packages has limited supported L4T version up to 32.6.1 with jetpach 4.4.1. Now jetpack is update to version 5. So I decided to install from building the source code.

3.1 Install from source

[ optional ] Run clash and use proxychains to zsh (or bash). This will fix many problems if your are in China, or you need to use mirrors manually.

clash
proxychains4 zsh
# proxychains4 bash # if you are using bash

I select the key part of these install instructions: distribution_linux, installation_jetson and libuvc_installation.

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-get install git libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev

Posters like Jetsonhacks Realsense on Agx and Intel Realsese install on TX2 is out-fo-date (4-5 years ago). And there is no kernel patching for jetpack5. So I have to building using RSUSB Backend.

cd
cd librealsense/scripts
chmod +x libuvc_installation.sh
./libuvc_installation.sh -DBUILD_WITH_CUDA=true

3.2 Use realsense device in ROS

From realsense-ros, it’s simple to see the relationship between librealsense, realsense-ros and ros:

• step 1. Install ros (noetic-desktop is enough)
• step 2. Install realsense-camera (This can be done by installing librealsense)
• step 3. Install realsense-ros (Wrapper for librealsense in ros)

step 1 and step 2 has no chronological order, but they must be done before step 3.

4. VNC

When I want to debug by the tools like realsense and rviz, it’s straightforward to use desktop UI. However, I have no monitor switcher or input devices switcher.

Forward X11 message or get data from ros-node also help, but complex. The, VNC server is the best choice.

Login locally, unlock the user and enable auto-login, enable desktop-sharing

cd /usr/lib/systemd/user/graphical-session.target.wants
sudo ln -s ../vino-server.service ./.
gsettings set org.gnome.Vino require-encryption false 
gsettings set org.gnome.Vino authentication-methods "['vnc']"
gsettings set org.gnome.Vino vnc-password $(echo -n 'password'|base64)
gsettings set org.gnome.settings-daemon.plugins.sharing active true
sudo apt install xserver-xorg-video-dummy -y

Other advices like simple use xserver-xorg-video-dummy not works because there is a x-server runing in the background. Use other vncserver like x11vnc, tightvncserver and so on not works since the display not works. The only might-fix is the virtual screen in xorg.conf, I tried this and finally works. The key point is set Driver of Device to use x11-dummy other than nvidia.

Change following to /etc/X11/xorg.conf

Section "Device"
  Identifier "Configured Video Device"
  Driver "dummy"
  VideoRam 256000
EndSection
Section "Monitor"
    Identifier  "Configured Monitor"
    HorizSync 5.0 - 1000.0
    VertRefresh 5.0 - 200.0
    ModeLine "1920x1080" 148.50 1920 2448 2492 2640 1080 1084 1089 1125 +Hsync +Vsync
EndSection
Section "Screen"
    Identifier  "Default Screen"
    Monitor     "Configured Monitor"
    Device      "Configured Video Device"
    DefaultDepth 24
    SubSection "Display"
    Depth 24
    Modes "1920x1080"
    EndSubSection
EndSection

Reboot and connect vnc, see the desktop without DP connected.

Note: This configuration would cause no signal in DP output, make sure you have access to the board before setting dummy desktop. Any configuration, including latest jetpack-5.1.1, does not support headless VNC access (I have tried many configurations). If you really want VNC access with DP cable output, the DP dummy stick with default xorg.conf is your choice.

5. ROS (Build from source)

Install bootstrap tools like ` rosinstall_generator vcstool. Instal from debian packages is not recommended. Maybe out of date and unmated version. So try to install python3 and get them from pip`

sudo apt install python3 python3-dev python3-pip -y
sudo pip3 install -U rosinstall_generator vcstool

Then, go to zsh-under-proxy

proxychains4 zsh

Installation under proxy should not throw no network error.

Replace wstool to vcs

mkdir ~/ros_catkin_ws
cd ~/ros_catkin_ws
rosinstall_generator desktop --rosdistro noetic --deps --tar > noetic-desktop.rosinstall
mkdir ./src
vcs import src < noetic-desktop.rosinstall

# If wstool
# wstool init src noetic-desktop.rosinstall
# wstool update -t src
rosdep install --from-paths src --ignore-src -r -y --skip-keys "python gazebo"
./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release --merge

In one flash, the default python is python2.7, but I want python3.7 instead, so I tried below to change default python

update-alternatives --remove python /usr/bin/python2
update-alternatives --install /usr/bin/python python /usr/bin/python3 10

5.1 Realsense Wrapper in Ros

Install realsense-ros, official tutorial. Though ddynamic_reconfigure only up to kinetic branch, it succed in noetic.

cd ros_catkin_ws/src
git clone https://github.com/IntelRealSense/realsense-ros.git -b ros1-legacy
git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
cd ..
./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release --merge --pkg ddynamic_reconfigure realsense2_camera realsense2_description

It’s Ok to use rosinstall_generator instead of installing ddynamic_reconfigure from git. But realsense-ros has no index in ros-index, we need to git clone it manually.

Install other dependencies.

sudo pip3 install -U netifaces defusedxml

Other Command see in My Ros Common CMD

6. System fix [optional]

I have wrong cuda link in default installation, which makes /usr/local/cuda refer to /usr/local/cuda-10.0. Jetpack 5 has only cuda-11 installed. So revise the cuda link only if you have incosistent CUDA path.

sudo rm /usr/local/cuda
sudo ln -s /usr/local/cuda-11 /usr/local/cuda

Install jtop.

sudo pip3 install -U jetson-stats

Connect by COM

Recovery Mode