ME495 Final Project: HockeyBot

Authors:

• Ava Zahedi
• Marthinius Nel
• Ritika Ghosh
• Hanyin Yuan

Description

The HockeyBot package allows a Franka robot to play air hockey. We use a realsense camera and computer vision to detect where the puck is on the air hockey table. Our TrajCalc node predicts the trajectory of the puck and sends those positions to our SimpleMove API, which tells the Franka to move to meet the puck. All of these tasks are integrated in our Main node, which completes our workflow and allows the robot to play repeatedly.

HockeyBotDescription.mp4

Prerequisites

1. Make sure ROS packages are most recent and up-to-date

sudo apt update
sudo apt upgrade

2. Install moveit: sudo apt install ros-humble-moveit

3. Make a new ROS2 workspace and enter it

mkdir -p nuws/src
cd nuws

4. Install dependencies rosdep install --from-paths src -r -y

5. Add the numsr colcon mixin

colcon mixin add numsr_patches file://$(pwd)/src/numsr_patches/index.yaml
colcon mixin update numsr_patches

6. Build the workspace colcon build --mixin numsr

7. Install the udev rules

sudo cp src/librealsense/config/99-realsense-libusb.rules /etc/udev/rules.d

8. Source the environment source /opt/ros/humble/setup.bash

9. Git clone [email protected]:ME495-EmbeddedSystems/hw3group-HockeyBot.git into the /src directory of the workspace. This file will install the ROS dependencies required to run this project.

10. Additionally, users will need to install OpenCV-Python for Ubuntu by using sudo apt-get install python3-opencv

All code for this package was developed and tested in Python 3.10.

Hardware Requirements

This project requires the following hardware components:

• RealSense Camera: realsense-ros can be installed with apt install ros-humble-realsense2-camera
• Franka Emika Panda Robot (Panda Robot)
• Air-Hockey Table (including puck, paddle)
• To connect to the robot, plug in the ethernet from the robot's workstation PC into the ethernet port on the user's computer.
• The hardware must be set up by connecting the RealSense camera to the user's computer via USB cable.

Contents

Packages:

1. hockeybot
2. moveit_helper
3. moveit_interface

User Guide

1. Follow the instructions here to start the Franka.
2. Connect the RealSense camera (via USB cable) and the Franka Emika Panda arm (via Ethernet cable) to the user's computer.
3. Launch the hockeybot package using the command ros2 launch hockeybot main.launch.py robot:=true.
   • If you wish to only run the package in simulation, and not while connected to the Franka Emika Panda arm, use launch argument robot:=false instead.

Concepts and Overall System Architecture

Start-Up Sequence

hockeybot_startup.mp4

• Upon launch, the robot follows a start-up sequence to reach its home position. The robot follows a series of waypoints to reach the home x- and y-coordinates with an offset in the z. It then reaches down to grasp the adapter on the paddle and moves back up slightly. This slight increase in height allows the robot flexibility while moving, so that if it pushes down during movement, it will not apply a force into the table while still keeping the paddle level with the table.

Computer Vision

ComputerVision.mp4

• Intel RealSense D435i is used at 480x270x90 allowing the puck to be tracked at 90 fps. As soon as the streaming has been enabled, this node detects the center of the table in pixel coordinates. Then with the help of the depth camera, the deproject function is used to convert pixel coordinates into real world coordinates with respect to the camera. Since the distance between the air hockey table and the Franka robot is known, points from the camera's frame of reference can be transformed to the robot's frame of reference. Next, with the help of OpenCV's HughCircles function the center of the puck is tracked in real time. For the calculation of the trajectory, the puck is only tracked when going towards the robot and up to the center of the table. In order to get rid of noise, before publishing the puck's position it is checked whether the point is close (with a prefixed tolerance) to the best fit line of the previous positions obtained. Note: The output video shows the tracked puck encircled with a black border, regardless of whether all these points are published (in other words, the video shows the contour for every direction of movement of the puck).

Trajectory Calculations

Traj_rviz_hockeybot.mp4

• Calculates the predicted trajectory of the puck and the play waypoints for the robot by using two puck coordinates from computer vision. The node handles collisions by reflecting the impact angle about the normal line. The waypoints for the robot to hit the puck are constrained by the robot's workspace on the air hockey table. The most optimal waypoints are selected by considering all four sides of the robot's workspace. The robot will then move to the first waypoint that is on the predicted trajectory line of the puck and then move along the line to the second waypoint and hit the puck. A plot is dynamically generated and updated each time a new trajectory is calculated. The robot blocks if the trajectory is out of the workspace and unreachable.

Hitting the Puck

HittingaPuck.mp4

• After receiving waypoint and goal positions, the robot receives service calls to move to those points, thereby meeting the puck along its trajectory and hitting it. If there is an edge case where the robot cannot successfully meet the puck given its trajectory, the robot will block instead.

Return Home

• As the robot is moving to hit the puck, it also plans a path from where it hits the puck back to the home position. Once the robot detects that its end-effector has reached the goal, it begins executing the path back home. This process also resets all internal variables and restarts the loop so that the robot can continue playing.

Instructions for manually launching services:

1. To launch the Franka along with the simple_move node ros2 launch franka_moveit_config moveit.launch.py robot_ip:=dont-care use_fake_hardware:=true.
2. Run the simple_move node with ros2 run moveit_helper simple_move.
   • (Optional) Provide a starting configuration for planning with ros2 service call /initial_service moveit_interface/srv/Initial "{x: 0.5, y: 0.0, z: 0.0, roll: 1.0, pitch: 0.04, yaw: 0.0}".
3. Call the service to plan a path to a specific goal pose with ros2 service call /goal_service moveit_interface/srv/Goal "{x: 0.5, y: 0.0, z: 0.0, roll: 1.0, pitch: 0.04, yaw: 0.0}".
4. To execute the plan, use ros2 service call /execute_service moveit_interface/srv/Execute "exec_bool: True".
   • If you wish to cancel your plan without executing, pass exec_bool: False instead of True.
5. To add a box in the planning scene, use ros2 service call /add_obj moveit_interface/srv/Addobj "{id: 1, x: 0.3, y: 0.6, z: 0.5, dim_x: 0.2, dim_y: 0.2, dim_z: 0.2}".