SpaceMouse controllers are one of the vital helpful pc enter units for an engineer. Whether or not one is designing a customized PCB or inspecting a 3D mannequin, the SpaceMouse makes it straightforward to navigate. And productiveness positive factors apart, these units are simply plain cool. Do you know that the primary SpaceMouse was used to govern a robotic arm on the Area Shuttle Columbia? That makes you need one in your desk much more, doesn’t it?
{Hardware} hacker Nikodem Bartnik has a SpaceMouse and customarily makes use of it for navigating 3D fashions. Nevertheless, realizing the historical past of this system, that appeared a bit too abnormal. So, Bartnik determined to place this SpaceMouse to higher use by hooking it as much as a robotic arm.
The NVIDIA Jetson Orin Nano (📷: Nikodem Bartnik)
The arm in query was the Seeed Studio SO-ARM101, a 3D-printed, low-cost robotic arm that sells for roughly $260 to $300 as soon as motors and electronics are included. Bartnik’s objective was to search out out if a 3Dconnexion SpaceMouse might exchange a second “chief” robotic arm in a standard leader-follower coaching setup. In that case, hobbyists might save each cash and bench house when amassing knowledge to coach machine studying fashions.
The SpaceMouse provides six levels of freedom, permitting translation and rotation in 3D house. The SO-ARM101, nevertheless, solely has 5 levels of freedom. That mismatch instantly brought about issues. Bartnik first tried to make use of inverse kinematics (IK) to map the SpaceMouse’s actions into 3D positions for the robotic’s gripper. However with one further diploma of freedom unaccounted for, the arm ceaselessly behaved unpredictably.
Reasonably than abandoning the experiment, he switched methods. As an alternative of utilizing IK, he immediately mapped particular SpaceMouse motions — tilting, twisting, and sliding — to particular person joints on the arm. This less complicated method proved much more steady and gave him usable teleoperation management.
On the software program aspect, Bartnik used the Lerobot library from Hugging Face however needed to fork it and write customized code to combine SpaceMouse assist. He initially ran the setup on an NVIDIA Jetson Orin Nano, however later migrated to a desktop Ubuntu PC as a consequence of restricted USB ports for twin digital camera feeds. Even then, he encountered lag throughout knowledge recording. The teleoperation loop ran at 60 Hz, however the recording script choked on the digital camera processing load. Optimizing the SpaceMouse knowledge queue finally resolved the responsiveness concern.
With the system working, Bartnik recorded 30 coaching episodes of a easy job: choosing up a LEGO brick and putting it right into a container. Not like a leader-follower arm — which requires nearly no follow — the SpaceMouse required 15 to half-hour of coaching earlier than clean management was potential. He then educated an ACT (Motion Chunking with Transformers) coverage on an NVIDIA RTX 4090, a course of that took about 45 minutes.
The educated mannequin might autonomously establish, grasp, and deposit the LEGO brick as supposed. Whereas the motions have been much less fluid than these educated with a second robotic arm, the experiment proved that “space-age” enter {hardware} can certainly prepare desk-scale robotics.