Could you share the evaluation code mentioned in the "4.2 Qualitative Evaluation" section?

[XiangdongHe]

Dear author, thank you for your outstanding open-source contribution.
You mentioned in the paper that you are using the evaluation method in Manipllm.
The qualitative evaluation in Manipllm's paper uses two-dimensional point positions and three-dimensional gripper directions.
But the points output in your paper are all three-dimensional and do not have the direction of the gripper. May I ask what algorithm you used for the evaluation and if the code can be made public?

[changhaonan]

Because we know the joint position and direction, so we can know the direction of action (You can image if we know the direction of USB hat, I know the action direction I need to apply). During our testing, we try two different directions: in and out.
As for open-sourcing the eval code. We are open to that. But for now, we are busy in CVPR. So we will try to clean the code after CVPR.

[XiangdongHe]

Thank you again for your contribution.😊😊
We look forward to you opening up the relevant code!!

[XiangdongHe]

Hello @changhaonan , do you have any plans to open up this part of the code in the near future?

[changhaonan]

Will do. Sorry about waiting. I am kind of busy recently as I am also looking for jobs. But I will try to find a time to clean and upload the eval code within two weeks.

[XiangdongHe]

Thank you for your reply. We look forward to you uploading the evaluation code.