Through the door - Open Door Using Toyota HSR

Push open the door

Pull open the door

ERL Smart City Competition (ERL Smart City) includes an episode in which the robot must open a hinged door with different configurations.​ The goal of this project is to open the doors using a robotic arm from the inside and outside of the C069 lab.​

Table of Contents

• Approach
• Requirements
• Usage
• Object Detection
• Description
• References

Approach:​

We present an overview of our approach, illustrated in the diagram below:

Requirements

Before using this code, ensure that the following prerequisites are met:

• ROS (Robot Operating System) is installed and configured.
• Python is installed (compatible with ROS).
• ROS packages such as std_msgs, geometry_msgs, sensor_msgs, tmc_control_msgs, tmc_manipulation_msgs, and tmc_msgs are installed.
• Python packages from requirements.txt file

Usage

To utilize this code for robot door-opening project, follow these steps:

Initial Setup:

Ensure all prerequisites are met as mentioned in the "Requirements" section.

Run get_force.py:

Execute the following command to run the get_force.py script:

python3 get_force.py

This script will initialize the force sensor capture and continuously monitor force and torque sensor data.

For Unlatching the Door:

To unlatch the door, run the door_open_with_feedback.py script. This script provides the necessary commands or actions to unlatch the door.

python3 door_open_with_feedback.py

For Push/Pull Action:

For push/pull action, execute the move_with_cmd.py script. This script performs the push or pull action.

python3 move_with_cmd.py

Object Detection

YOLOv5 Handle Detection Approach

2D Detection

• Initial approach:
  • Trained using the MiguelARD/DoorDetect-Dataset.
  • Faced challenges when detecting using Lucy's camera.
• What worked:
  • Trained on 500+ images of door handles taken from Lucy's head RGBD camera.
  • Result: Obtained bounding box coordinates for the door handle.

3D Detection

• Approach:
  • Depth point cloud superimposed on the 2D image.
  • Used depth value for the centroid of the handle based on the coordinates of the bounding box from the previous step.
• Result:
  • Obtained bounding box coordinates for 3D detection.

Description

Grasp the door handle

• After obtaining the 3D coordinates of the door handle using a detection algorithm

• Utilize motion planning to move the arm to the specified coordinates with fixed orientation

• But we encountered issues with motion planning did not work as expected, so we manually set and hardcode the joint angles to guide the arm to the door handle

• Once the robotic arm reaches the door handle, close the gripper to firmly grasp the handle

Unlatch door

• Once the door handle is grasped, continuously monitor force feedback

• If the torque in the x-component exceeds 0.5 Nm, initiate a clockwise rotation of the wrist

• If the torque does not exceed 0.5 Nm, go anti-clockwise instead

• After determining the rotation direction, begin rotating the wrist in the selected direction

• While rotating, gradually pull down on the door handle to completely unlatch it

Push/Pull

• After unlatching the door handle, the default action is to pull the door

• Calculate the directional force component resulting from the average components of y and z

• We verify if the force exceeds 45N, and if it does, we activate the door pull mechanism

• For pulling

  • If the directional force does not exceed 45N, continue pulling

  • Move backward with linear velocity until the directional force is less than –30degree

  • Once the force reaches -30degree, initiate lateral movement (left or right) based on the logic established during the door unlatching process

  • Continue moving left/right until the directional force reaches +15degree

  • After reaching +15degree, go back again until the force reaches -30degree, and repeat this loop

  • When the odom in the y-direction reaches a distance of 0.5 meters from the starting point, stop the movement and release the door handle

• For pushing

  • Begin by pushing the door slightly forward while keeping hold of the door handle

  • After pushing the door, release the door handle

  • Move the robot arm back to its home position

  • Initiate lateral movement (left or right) to create some distance from the wall

  • Finally, use the robot's body to apply force and push the door fully open, allowing the robot to pass through the doorway.

References

• ERL Smart City Rules