Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hand Me That Task #546

Merged
merged 5 commits into from
Feb 21, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Rulebook.tex
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ \chapter{Tests in Stage II}
\newpage
\input{tasks/SetTable}

\newpage
\input{tasks/HandMeThat}

\newpage
\input{tests/Finals}

Expand Down
18 changes: 18 additions & 0 deletions scoresheets/HandMeThat.tex
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

The maximum time for this test is 15 minutes.
justinhart marked this conversation as resolved.
Show resolved Hide resolved
justinhart marked this conversation as resolved.
Show resolved Hide resolved

\begin{scorelist}
\scoreheading{Communicating}
justinhart marked this conversation as resolved.
Show resolved Hide resolved
\scoreitem[5]{500}{Correctly determine an item on the first try}
\penaltyitem[5]{-200}{Correctly determine an item on the second try}
\penaltyitem[5]{-400}{Correctly determine an item on the third or a subsequent try}
\penaltyitem[5]{-400}{Correctly determine an item after the referee has picked it up or touched it}

% No longer necessary, computes automatically
% \setTotalScore{1000}
\end{scorelist}


% Local Variables:
% TeX-master: "Rulebook"
% End:
60 changes: 60 additions & 0 deletions tasks/HandMeThat.tex
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
\section{Hand Me That [Party Host]}
A guest at the party speaks English, but with only a limited vocabulary. The robot will assist them in obtaining things that they gesture for.

%\subsection{Focus}
%Joint attention is a well-studied and important task in Human-Robot Interaction. The goal of this task is to really challenge the teams to perform a hard HRI task.

\subsection{Main Goal}
Five collections of items are set out. The referee will choose a random item from each collection to gesture to or describe to the robot. The robot must tell the referee its name, point to, or touch it. The robot can prompt the user with requests like \textit{point to it} or \textit{describe it to me}.
justinhart marked this conversation as resolved.
Show resolved Hide resolved

\noindent\textbf{Reward:} 2500pts\\


% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Setup
%
% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Setup}
\begin{enumerate}
\item \textbf{Object Collections:} The test area should collections of objects on various surfaces (2-5). Difficulty of understanding the point gesture will be regulated by how far apart the objects in each collection are ($100cm - 1m$). Difficulty of describing the object is difficult to quantify, and should be considered random.
justinhart marked this conversation as resolved.
Show resolved Hide resolved
justinhart marked this conversation as resolved.
Show resolved Hide resolved

\item \textbf{Random Object Cards:} The referee will carry a set of cards with random objects, arranged by difficulty. The job of the referee is to get the robot to say the name of, or touch, the object that they gesture to or describe, at the robot's discretion.
justinhart marked this conversation as resolved.
Show resolved Hide resolved

\end{enumerate}


% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Setup
justinhart marked this conversation as resolved.
Show resolved Hide resolved
%
% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Procedure}
\begin{enumerate}
\item \textbf{Pick an object} The robot will ask the referee what they want. The referee will walk near to the object that they want, and gesture to it. The robot can ask clarifying questions to help identify the object. The referee will not say the name of the object, or its color, but anything else is fair game.
justinhart marked this conversation as resolved.
Show resolved Hide resolved
justinhart marked this conversation as resolved.
Show resolved Hide resolved
\item \textbf{Repeat} Repeat up to 5 times for the maximum score.


\end{enumerate}


% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Additional Rules
%
% %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Additional rules and remarks}
\begin{enumerate}
\item \textbf{Keep going:} The robot should keep trying to determine the referred to object until they score or run out of time.
\item \textbf{Pass:} The robot may say pass to try the next object.
\item \textbf{Touch:} The robot may ask the referee may pick up or touch the object by saying \textit{pick up} or \textit{touch}. If the robot does this, identifying the object is worth only 100 points.
justinhart marked this conversation as resolved.
Show resolved Hide resolved
justinhart marked this conversation as resolved.
Show resolved Hide resolved
\item \textbf{Oops:} Incorrect guesses reduce the value of the correct guess by 200 points, each, but cannot make the value of the correct guess go below 100 points.
justinhart marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should not get points past the fourth guess. Teams could just have their robot literally guess all 30 possible objects and score 100 points.
Personally, as a referee I would not give points to a robot that is just rattling off guesses anyway but I still think it should be clear in the rules.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the rules tweak that I made fixes this. Let me know what you think.


justinhart marked this conversation as resolved.
Show resolved Hide resolved
\end{enumerate}
\newpage
\subsection{Score sheet}
\input{scoresheets/HandMeThat.tex}

% Local Variables:
% TeX-master: "Rulebook"
% End: