Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Robocup 2022: Receptionist #1176

Open
6 of 12 tasks
PetervDooren opened this issue May 7, 2022 · 1 comment
Open
6 of 12 tasks

Robocup 2022: Receptionist #1176

PetervDooren opened this issue May 7, 2022 · 1 comment
Assignees
@LarsJanssenTUe @lottemessing

Comments

@PetervDooren
Copy link
Contributor

PetervDooren commented May 7, 2022 •
edited by LarsJanssenTUe
Loading

Current state:
Tested challenge on real robot:

  • Tested speech recognition: heard name and drink correct, changed input sound to 62%.
  • Still needs to test People recognition: The guest was not found in the livingroom when it needed to be introduced to john.

Todo:

  • Make components easier to test
  • Introduce guest to every other guest (including John).
  • Speech recognition incorrect --> Work around is to check if that is correct
  • Bonus points: Describe first guest to second guest naming 4 characteristics.
  • Name of guest is None, not the actual name
  • Limit the number tries for incorrectly hearing the name and drink

Testing:

  • Test introduce guest to every other guest
  • Test if look at person taking/being described works.
  • Test checking if name and drink are correct

Issues by testing on real robot:

  • Can not find valid face
    image
  • Empty spot designator finds objects where there are none
  • Could not recognize Laura
    image
@MatthijsBurgh
Copy link
Member 

[INFO][/state_machine][1652206791.938021]: Using robot 'hero'.
[INFO][/state_machine][1652206791.947014]: State machine starting in initial state 'RESET' with userdata: 
	[]
efficient copy
[INFO][/state_machine][1652206791.992118]: State machine transitioning 'RESET':'done'-->'SET_INITIAL_POSE'
efficient copy
efficient copy
[INFO][/state_machine][1652206792.006455]: Set initial pose to 1.5, 0.5, 0.0
[INFO][/state_machine][1652206793.009048]: State machine transitioning 'SET_INITIAL_POSE':'done'-->'HANDLE_GUEST_1'
[INFO][/state_machine][1652206793.010495]: State machine starting in initial state 'LEARN_GUEST' with userdata: 
	[]
[INFO][/state_machine][1652206793.011564]: State machine starting in initial state 'GOTO_DOOR' with userdata: 
	[]
[INFO][/state_machine][1652206793.012713]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206793.013922]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206793.014992]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206793.146587]: 'I'm on my way!'
[INFO][/state_machine][1652206793.156103]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652206818.116112]: State machine terminating 'EXECUTE_PLAN':'succeeded':'arrived'
[INFO][/state_machine][1652206818.118016]: State machine transitioning 'NAVIGATE':'arrived'-->'STOP_ANALYSIS_SUCCEED'
[INFO][/state_machine][1652206818.120005]: State machine terminating 'STOP_ANALYSIS_SUCCEED':'done':'arrived'
[INFO][/state_machine][1652206818.121184]: State machine transitioning 'GOTO_DOOR':'arrived'-->'SAY_OPEN_DOOR'
efficient copy
[INFO][/state_machine][1652206818.122901]: 'Someone please open the door, I'm expecting guests'
[INFO][/state_machine][1652206821.935150]: State machine transitioning 'SAY_OPEN_DOOR':'spoken'-->'SAY_PLEASE_COME_IN'
efficient copy
[INFO][/state_machine][1652206821.937339]: 'Please come in, I'm waiting for someone to step in front of me'
[INFO][/state_machine][1652206825.947486]: State machine transitioning 'SAY_PLEASE_COME_IN':'spoken'-->'WAIT_FOR_GUEST'
efficient copy
[INFO][/state_machine][1652206826.004138]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206827.780566]: There are 1 people in front of me (1 is enough): ['d6cce35c487a58feab29032c2621f38d']
[INFO][/state_machine][1652206827.782425]: State machine transitioning 'WAIT_FOR_GUEST':'success'-->'SAY_HELLO'
efficient copy
[INFO][/state_machine][1652206827.784638]: 'Hi there, I'll learn your face now'
[INFO][/state_machine][1652206830.306535]: State machine transitioning 'SAY_HELLO':'spoken'-->'ASK_GUEST_NAME'
[INFO][/state_machine][1652206830.309630]: AskPersonName
[INFO][/state_machine][1652206830.313024]: 'What is your name?'
efficient copy
[INFO][/state_machine][1652206831.628530]: Question: Which option?, grammar: T['name': N] -> NAME[N]

NAME['anna'] -> anna
NAME['beth'] -> beth
NAME['ca..
[INFO][/state_machine][1652206831.631063]: Example: 'mary'
[INFO][/state_machine][1652206841.759588]: Preempt finished within specified preempt_timeout [1.00]
[INFO][/state_machine][1652206841.762929]: Robot did not hear you (timeout)
[INFO][/state_machine][1652206841.782680]: AskPersonName
[INFO][/state_machine][1652206841.785261]: 'What is your name?'
efficient copy
[INFO][/state_machine][1652206843.007674]: Question: Which option?, grammar: T['name': N] -> NAME[N]

NAME['anna'] -> anna
NAME['beth'] -> beth
NAME['ca..
[INFO][/state_machine][1652206843.011901]: Example: 'robert'
[INFO][/state_machine][1652206853.037417]: Preempt finished within specified preempt_timeout [1.00]
[INFO][/state_machine][1652206853.040658]: Robot did not hear you (timeout)
[WARN][/state_machine][1652206853.062002]: Speech recognition outcome was not successful. Using default name 'Operator'
[INFO][/state_machine][1652206853.065252]: State machine transitioning 'ASK_GUEST_NAME':'failed'-->'SAY_HELLO'
efficient copy
[INFO][/state_machine][1652206853.068241]: 'Hi there, I'll learn your face now'
[INFO][/state_machine][1652206855.587143]: State machine transitioning 'SAY_HELLO':'spoken'-->'ASK_GUEST_NAME'
[INFO][/state_machine][1652206855.590289]: AskPersonName
[INFO][/state_machine][1652206855.594520]: 'What is your name?'
efficient copy
[INFO][/state_machine][1652206856.914063]: Question: Which option?, grammar: T['name': N] -> NAME[N]

NAME['anna'] -> anna
NAME['beth'] -> beth
NAME['ca..
[INFO][/state_machine][1652206856.917378]: Example: 'mary'
[INFO][/state_machine][1652206861.009175]: Robot heard 'carmen' {'name': 'carmen'}
[INFO][/state_machine][1652206861.033155]: This person's name is: 'carmen'
[INFO][/state_machine][1652206861.037492]: Result received from speech recognition is 'carmen'
[INFO][/state_machine][1652206861.040327]: State machine transitioning 'ASK_GUEST_NAME':'succeeded'-->'LEARN_PERSON'
[INFO][/state_machine][1652206861.043359]: 'Please look at me while I admire your beauty'
efficient copy
[INFO][/state_machine][1652206861.474032]: [LearnPerson] Resolved name carmen
[INFO][/state_machine][1652206861.477649]: Creating subscriber
[INFO][/state_machine][1652206861.481099]: lazy subscribe to /hero/head_rgbd_sensor/rgb/image_raw
[INFO][/state_machine][1652206861.484342]: getting one image...
[INFO][/state_machine][1652206861.487710]: self._camera_last_image: None
[INFO][/state_machine][1652206861.492064]: len(self._camera_last_image): 921600
[INFO][/state_machine][1652206861.493515]: got 921600 bytes of image data
[INFO][/state_machine][1652206861.685381]: Found 1 face(s) in the image
[INFO][/state_machine][1652206861.688529]: Found 1 valid face(s)
[INFO][/state_machine][1652206861.690708]: annotating that face as carmen
efficient copy
[INFO][/state_machine][1652206861.962144]: State machine transitioning 'LEARN_PERSON':'succeeded'-->'SAY_GUEST_LEARNED'
[INFO][/state_machine][1652206861.966946]: 'Okidoki, now I know what you look like'
[INFO][/state_machine][1652206866.795181]: State machine transitioning 'SAY_GUEST_LEARNED':'spoken'-->'SAY_DRINK_QUESTION'
[INFO][/state_machine][1652206866.799360]: 'What's your favorite drink?'
[INFO][/state_machine][1652206868.619840]: State machine transitioning 'SAY_DRINK_QUESTION':'spoken'-->'HEAR_DRINK_ANSWER'
efficient copy
[INFO][/state_machine][1652206868.624130]: Question: Which option?, grammar: T['drink': O] -> OPTIONS[O]

OPTIONS['beer'] -> beer
OPTIONS['bifrutas'] ->..
[INFO][/state_machine][1652206868.626903]: Example: 'water'
[INFO][/state_machine][1652206874.980242]: Robot heard 'beer' {'drink': 'beer'}
[INFO][/state_machine][1652206875.002597]: State machine transitioning 'HEAR_DRINK_ANSWER':'heard'-->'RESET_1'
[INFO][/state_machine][1652206875.015519]: State machine terminating 'RESET_1':'done':'succeeded'
[INFO][/state_machine][1652206875.020223]: State machine transitioning 'LEARN_GUEST':'succeeded'-->'SAY_GOTO_OPERATOR'
[INFO][/state_machine][1652206875.023053]: Going to traverse path: ['drink'] into semantics: {'drink': 'beer'}
efficient copy
[INFO][/state_machine][1652206875.027076]: 'Okidoki, you are carmen and you like beer, lets go inside. Please follow me'
[INFO][/state_machine][1652206881.647777]: State machine transitioning 'SAY_GOTO_OPERATOR':'spoken'-->'GOTO_LIVINGROOM'
[INFO][/state_machine][1652206881.651945]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206881.655530]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206881.658886]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206881.799462]: 'Affirmative!'
[INFO][/state_machine][1652206881.811435]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652206896.065622]: State machine transitioning 'EXECUTE_PLAN':'blocked'-->'PLAN_BLOCKED'
[INFO][/state_machine][1652206896.070070]: Plan blocked
efficient copy
[ERROR][/hero/global_planner][1652206899.087241857]: There is no goal which meets the given constraints. Planning will always fail to this goal constraint.
[ERROR][/hero/global_planner][1652206899.087291496]: Received position constraint: frame: map
constraint: (x-10.600000)^2+(y-1.250000)^2 < 0.500000^2

[ERROR][/state_machine][1652206899.087731]: Global planner couldn't plan a path to the specified constraints. Are the constraints you specified valid?: [frame: "map"
constraint: "(x-10.600000)^2+(y-1.250000)^2 < 0.500000^2"]
[INFO][/state_machine][1652206899.091128]: State machine transitioning 'PLAN_BLOCKED':'blocked'-->'GET_PLAN'
efficient copy
efficient copy
efficient copy
[ERROR][/hero/global_planner][1652206899.215869842]: There is no goal which meets the given constraints. Planning will always fail to this goal constraint.
[ERROR][/hero/global_planner][1652206899.215903075]: Received position constraint: frame: map
constraint: (x-10.600000)^2+(y-1.250000)^2 < 0.500000^2

[ERROR][/state_machine][1652206899.216294]: Global planner couldn't plan a path to the specified constraints. Are the constraints you specified valid?: [frame: "map"
constraint: "(x-10.600000)^2+(y-1.250000)^2 < 0.500000^2"]
[INFO][/state_machine][1652206899.219383]: State machine terminating 'GET_PLAN':'unreachable':'unreachable'
[INFO][/state_machine][1652206899.223166]: State machine transitioning 'NAVIGATE':'unreachable'-->'STOP_ANALYSIS_UNREACHABLE'
[INFO][/state_machine][1652206899.226587]: State machine terminating 'STOP_ANALYSIS_UNREACHABLE':'done':'unreachable'
[INFO][/state_machine][1652206899.228845]: State machine transitioning 'GOTO_LIVINGROOM':'unreachable'-->'INTRODUCE_GUEST'
[INFO][/state_machine][1652206899.230246]: State machine starting in initial state 'SAY_INTRO' with userdata: 
	[]
[INFO][/state_machine][1652206899.231350]: 'Hi john, let me introduce you our new guest carmen. I'll show you in a bit'
[INFO][/state_machine][1652206899.248167]: State machine transitioning 'SAY_INTRO':'spoken'-->'FIND_OLD_GUESTS'
[INFO][/state_machine][1652206899.250902]: State machine starting in initial state 'DECIDE_NAVIGATE_STATE' with userdata: 
	[]
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206899.281625]: State machine transitioning 'DECIDE_NAVIGATE_STATE':'room'-->'NAVIGATE_TO_ROOM'
[INFO][/state_machine][1652206899.285132]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206899.288319]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206899.290556]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206899.508421]: 'Affirmative!'
[INFO][/state_machine][1652206899.532595]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652206899.634177]: State machine terminating 'EXECUTE_PLAN':'succeeded':'arrived'
[INFO][/state_machine][1652206899.637400]: State machine transitioning 'NAVIGATE':'arrived'-->'STOP_ANALYSIS_SUCCEED'
[INFO][/state_machine][1652206899.640965]: State machine terminating 'STOP_ANALYSIS_SUCCEED':'done':'arrived'
[INFO][/state_machine][1652206899.644122]: State machine transitioning 'NAVIGATE_TO_ROOM':'arrived'-->'FIND_PEOPLE'
[INFO][/state_machine][1652206899.647107]: 'please look at me while I am looking for you'
efficient copy
[INFO][/state_machine][1652206902.684939]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206903.306965]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206903.757948]: Found 0 people: []
[INFO][/state_machine][1652206903.759653]: 0 people remaining after None-check
[INFO][/state_machine][1652206903.761470]: 0 people remaining after distance < 10.0-check
efficient copy
efficient copy
[INFO][/state_machine][1652206903.777739]: 0 result_people remaining after 'in'-'living_room' check
[INFO][/state_machine][1652206903.781178]: 0 result_people remaining after 'in'-'living_room' check
[WARN][/state_machine][1652206903.783938]: Could not find people meeting the requirements
efficient copy
[INFO][/state_machine][1652206905.812706]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206906.244468]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206906.708973]: Found 0 people: []
[INFO][/state_machine][1652206906.712055]: 0 people remaining after None-check
[INFO][/state_machine][1652206906.716039]: 0 people remaining after distance < 10.0-check
efficient copy
efficient copy
[INFO][/state_machine][1652206906.740495]: 0 result_people remaining after 'in'-'living_room' check
[INFO][/state_machine][1652206906.741990]: 0 result_people remaining after 'in'-'living_room' check
[WARN][/state_machine][1652206906.743342]: Could not find people meeting the requirements
efficient copy
[INFO][/state_machine][1652206908.780076]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206909.193262]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206909.636033]: Found 0 people: []
[INFO][/state_machine][1652206909.638681]: 0 people remaining after None-check
[INFO][/state_machine][1652206909.641424]: 0 people remaining after distance < 10.0-check
efficient copy
efficient copy
[INFO][/state_machine][1652206909.660581]: 0 result_people remaining after 'in'-'living_room' check
[INFO][/state_machine][1652206909.661781]: 0 result_people remaining after 'in'-'living_room' check
[WARN][/state_machine][1652206909.665334]: Could not find people meeting the requirements
efficient copy
[INFO][/state_machine][1652206911.699642]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206912.243899]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206912.667710]: Found 0 people: []
[INFO][/state_machine][1652206912.671730]: 0 people remaining after None-check
[INFO][/state_machine][1652206912.675520]: 0 people remaining after distance < 10.0-check
efficient copy
efficient copy
[INFO][/state_machine][1652206912.699641]: 0 result_people remaining after 'in'-'living_room' check
[INFO][/state_machine][1652206912.703091]: 0 result_people remaining after 'in'-'living_room' check
[WARN][/state_machine][1652206912.706096]: Could not find people meeting the requirements
efficient copy
[INFO][/state_machine][1652206914.717738]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206915.393692]: Received rgb, depth, cam_info
efficient copy
efficient copy
/home/amigo/ros/noetic/system/src/ed_python/src/ed/entity.py:311: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  pp_dict = yaml.load(e.data)
efficient copy
efficient copy
[INFO][/state_machine][1652206916.122466]: Found 2 people: [Entity(uuid='486c8a807eeb4ba5b27ed53e5af9ad7e', etype='person', frame=FrameStamped(pos:(x=9.873345529068187, y=2.4742479005715072, z=1.5333434019628105), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=PersonProperties(age='21', gender='0', gender_confidence=0.287319, shirt_colors=['white'])), Entity(uuid='94a87f4c8c6e45ce1f813ae510713fed', etype='person', frame=FrameStamped(pos:(x=9.875447330690537, y=2.4728116244988927, z=1.5333438364878114), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=PersonProperties(age='21', gender='0', gender_confidence=0.121421, shirt_colors=['white']))]
[INFO][/state_machine][1652206916.129695]: 2 people remaining after None-check
[INFO][/state_machine][1652206916.135286]:  Position of found_people  is FrameStamped(pos:(x=9.873345529068187, y=2.4742479005715072, z=1.5333434019628105), rot:(r=0.0, p=-0.0, y=0.0) @ map)
[INFO][/state_machine][1652206916.139303]:  Position of found_people  is FrameStamped(pos:(x=9.875447330690537, y=2.4728116244988927, z=1.5333438364878114), rot:(r=0.0, p=-0.0, y=0.0) @ map)
[INFO][/state_machine][1652206916.142292]: 2 people remaining after distance < 10.0-check
efficient copy
efficient copy
[INFO][/state_machine][1652206916.161582]: 2 result_people remaining after 'in'-'living_room' check
[INFO][/state_machine][1652206916.165260]: State machine terminating 'FIND_PEOPLE':'found':'found'
[INFO][/state_machine][1652206916.170314]: State machine transitioning 'FIND_OLD_GUESTS':'found'-->'ITERATE_OLD_GUESTS'
[INFO][/state_machine][1652206916.173781]: Current elements: [Entity(uuid='486c8a807eeb4ba5b27ed53e5af9ad7e', etype='person', frame=FrameStamped(pos:(x=9.873345529068187, y=2.4742479005715072, z=1.5333434019628105), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=PersonProperties(age='21', gender='0', gender_confidence=0.287319, shirt_colors=['white'])), Entity(uuid='94a87f4c8c6e45ce1f813ae510713fed', etype='person', frame=FrameStamped(pos:(x=9.875447330690537, y=2.4728116244988927, z=1.5333438364878114), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=PersonProperties(age='21', gender='0', gender_confidence=0.121421, shirt_colors=['white']))]
[INFO][/state_machine][1652206916.177428]: State machine transitioning 'ITERATE_OLD_GUESTS':'next'-->'GOTO_OPERATOR'
[INFO][/state_machine][1652206916.180743]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206916.183744]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206916.187386]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
[INFO][/state_machine][1652206916.311037]: 'I'm coming!'
[INFO][/state_machine][1652206916.331164]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652206920.141237]: State machine terminating 'EXECUTE_PLAN':'succeeded':'arrived'
[INFO][/state_machine][1652206920.144258]: State machine transitioning 'NAVIGATE':'arrived'-->'STOP_ANALYSIS_SUCCEED'
[INFO][/state_machine][1652206920.147517]: State machine terminating 'STOP_ANALYSIS_SUCCEED':'done':'arrived'
[INFO][/state_machine][1652206920.149970]: State machine transitioning 'GOTO_OPERATOR':'arrived'-->'SAY_LOOK_AT_GUEST'
[INFO][/state_machine][1652206920.152661]: 'Hi john, let me show you our guest'
[INFO][/state_machine][1652206922.972044]: State machine transitioning 'SAY_LOOK_AT_GUEST':'spoken'-->'TURN_TO_GUEST'
[INFO][/state_machine][1652206925.974998]: State machine transitioning 'TURN_TO_GUEST':'done'-->'FIND_GUEST'
[INFO][/state_machine][1652206925.978354]: Trying to find carmen
efficient copy
[INFO][/state_machine][1652206925.980182]: 'carmen, please look at me while I am looking for you'
efficient copy
[INFO][/state_machine][1652206929.833128]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206930.313642]: Found 0 people: []
[INFO][/state_machine][1652206930.316549]: 0 people remaining after None-check
[WARN][/state_machine][1652206930.319349]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206932.348268]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206932.775829]: Found 0 people: []
[INFO][/state_machine][1652206932.778723]: 0 people remaining after None-check
[WARN][/state_machine][1652206932.781476]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206934.909424]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206935.346957]: Found 0 people: []
[INFO][/state_machine][1652206935.349560]: 0 people remaining after None-check
[WARN][/state_machine][1652206935.353031]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206937.376858]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206937.820982]: Found 0 people: []
[INFO][/state_machine][1652206937.824602]: 0 people remaining after None-check
[WARN][/state_machine][1652206937.827960]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206939.856585]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206940.280096]: Found 0 people: []
[INFO][/state_machine][1652206940.286991]: 0 people remaining after None-check
[WARN][/state_machine][1652206940.289760]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206942.336609]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206942.748462]: Found 0 people: []
[INFO][/state_machine][1652206942.751526]: 0 people remaining after None-check
[WARN][/state_machine][1652206942.756312]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206944.782875]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206945.209278]: Found 0 people: []
[INFO][/state_machine][1652206945.212227]: 0 people remaining after None-check
[WARN][/state_machine][1652206945.213490]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206947.254678]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206947.677682]: Found 0 people: []
[INFO][/state_machine][1652206947.680720]: 0 people remaining after None-check
[WARN][/state_machine][1652206947.683227]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206952.427181]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206952.847191]: Found 0 people: []
[INFO][/state_machine][1652206952.851286]: 0 people remaining after None-check
[WARN][/state_machine][1652206952.854167]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206955.002320]: Received rgb, depth, cam_info
[INFO][/state_machine][1652206955.419647]: Found 0 people: []
[INFO][/state_machine][1652206955.422268]: 0 people remaining after None-check
[WARN][/state_machine][1652206955.424581]: Could not find carmen
efficient copy
[INFO][/state_machine][1652206957.448006]: Received rgb, depth, cam_info
efficient copy
efficient copy
[INFO][/state_machine][1652206958.238127]: Found 1 people: [Entity(uuid='6dae73fe2ea694c47334862e127cfea6', etype='person', frame=FrameStamped(pos:(x=7.964356564076726, y=0.6744195586908716, z=1.380921326747448), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=PersonProperties(age='21', gender='0', gender_confidence=0.588355, shirt_colors=['black']))]
[INFO][/state_machine][1652206958.241692]: 1 people remaining after None-check
[WARN][/state_machine][1652206958.246300]: Could not find carmen
[INFO][/state_machine][1652206960.251593]: State machine transitioning 'FIND_GUEST':'failed'-->'INTRODUCE_GUEST_WITHOUT_POINTING'
[INFO][/state_machine][1652206960.256108]: Going to traverse path: ['drink'] into semantics: {'drink': 'beer'}
efficient copy
[INFO][/state_machine][1652206960.259643]: 'Our new guest is carmen who likes beer'
[INFO][/state_machine][1652206962.678447]: State machine transitioning 'INTRODUCE_GUEST_WITHOUT_POINTING':'spoken'-->'RESET_ARM'
[INFO][/state_machine][1652206962.694618]: State machine terminating 'RESET_ARM':'done':'succeeded'
[INFO][/state_machine][1652206962.697179]: State machine transitioning 'INTRODUCE_GUEST':'succeeded'-->'FIND_SEAT_FOR_GUEST'
[INFO][/state_machine][1652206962.699998]: State machine starting in initial state 'SAY_LETS_FIND_SEAT' with userdata: 
	[]
[INFO][/state_machine][1652206962.703267]: 'Let me find a place for carmen to sit. Please be patient while I check out where there's place to sit'
[INFO][/state_machine][1652206962.725278]: State machine transitioning 'SAY_LETS_FIND_SEAT':'spoken'-->'ITERATE_NEXT_SEAT'
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206962.784000]: Current elements: [Entity(uuid='couch_long', etype='', frame=FrameStamped(pos:(x=10.8, y=2.36, z=0.0), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=None), Entity(uuid='couch_short', etype='', frame=FrameStamped(pos:(x=10.8, y=0.0, z=0.0), rot:(r=0.0, p=-0.0, y=0.0) @ map), person_properties=None)]
[INFO][/state_machine][1652206962.787677]: State machine transitioning 'ITERATE_NEXT_SEAT':'next'-->'CHECK_SEAT_EMPTY'
[INFO][/state_machine][1652206962.790561]: State machine starting in initial state 'INSPECT' with userdata: 
	[]
[INFO][/state_machine][1652206962.793755]: State machine starting in initial state 'NAVIGATE_TO_INSPECT' with userdata: 
	[]
[INFO][/state_machine][1652206962.796621]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206962.799583]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206962.801047]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
[INFO][/state_machine][1652206962.927716]: 'Affirmative!'
[INFO][/state_machine][1652206962.950246]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652206970.368879]: State machine terminating 'EXECUTE_PLAN':'succeeded':'arrived'
[INFO][/state_machine][1652206970.372430]: State machine transitioning 'NAVIGATE':'arrived'-->'STOP_ANALYSIS_SUCCEED'
[INFO][/state_machine][1652206970.375167]: State machine terminating 'STOP_ANALYSIS_SUCCEED':'done':'arrived'
[INFO][/state_machine][1652206970.379374]: State machine transitioning 'NAVIGATE_TO_INSPECT':'arrived'-->'RISE'
[INFO][/state_machine][1652206970.383117]: State machine transitioning 'RISE':'succeeded'-->'SEGMENT'
efficient copy
Succesfully stored '/home/amigo/ed/kinect/2022-05-10/on_top_of_couch_long/2022-05-10-20-22-52_rgb.png'
Succesfully stored '/home/amigo/ed/kinect/2022-05-10/on_top_of_couch_long/2022-05-10-20-22-52_depth.png'
[INFO][/state_machine][1652206973.607201]: >> Segmented 3 objects!
2022-05-10 20:22:54.659457: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8400
2022-05-10 20:22:55.335920: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory
2022-05-10 20:22:55.443777: I tensorflow/stream_executor/cuda/cuda_driver.cc:739] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-05-10 20:22:55.444258: I tensorflow/stream_executor/cuda/cuda_driver.cc:739] failed to allocate 1.80G (1932735232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-05-10 20:22:55.515499: I tensorflow/stream_executor/cuda/cuda_driver.cc:739] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-05-10 20:22:55.515528: W tensorflow/core/common_runtime/bfc_allocator.cc:343] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
2022-05-10 20:22:56.024302: I tensorflow/stream_executor/cuda/cuda_driver.cc:739] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-05-10 20:22:56.024338: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 886.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
[INFO][/state_machine][1652206976.172793]:    - Object 0 is a 'towel' (ID: b543932b2c2b1f0823cbafac1281137d)
[INFO][/state_machine][1652206976.183882]:    - Object 1 is a 'bifrutas' (ID: 9c02f3e2f0d10e9226ed5897a18c4f9e)
[INFO][/state_machine][1652206976.188105]:    - Object 2 is a 'hairspray' (ID: ba0bdeddfbefa72de1a3a3b5531a3a8f)
[INFO][/state_machine][1652206976.191293]: State machine terminating 'SEGMENT':'done':'done'
[INFO][/state_machine][1652206976.194141]: State machine transitioning 'INSPECT':'done'-->'CHECK'
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206976.240238]: Occupied space is 0.0691314095063049, remaining space is 0.26206859049369513
[INFO][/state_machine][1652206976.243034]: State machine terminating 'CHECK':'partially_occupied':'partially_occupied'
[INFO][/state_machine][1652206976.244725]: State machine transitioning 'CHECK_SEAT_EMPTY':'partially_occupied'-->'POINT_AT_PARTIALLY_OCCUPIED_SEAT'
[INFO][/state_machine][1652206976.246010]: Rotate base by -4.308deg. At -18.000deg/s this takes 0.23931830150949226s
efficient copy
[INFO][/state_machine][1652206980.577459]: State machine transitioning 'POINT_AT_PARTIALLY_OCCUPIED_SEAT':'succeeded'-->'SAY_SEAT_PARTIALLY_OCCUPIED'
[INFO][/state_machine][1652206980.579101]: 'I think there's some space left here where you can sit carmen'
[INFO][/state_machine][1652206984.199765]: State machine transitioning 'SAY_SEAT_PARTIALLY_OCCUPIED':'spoken'-->'RESET_SUCCESS'
[INFO][/state_machine][1652206984.214403]: State machine terminating 'RESET_SUCCESS':'done':'succeeded'
[INFO][/state_machine][1652206984.217971]: State machine terminating 'FIND_SEAT_FOR_GUEST':'succeeded':'succeeded'
[INFO][/state_machine][1652206984.221585]: State machine transitioning 'HANDLE_GUEST_1':'succeeded'-->'HANDLE_GUEST_2'
[INFO][/state_machine][1652206984.224913]: State machine starting in initial state 'LEARN_GUEST' with userdata: 
	[]
[INFO][/state_machine][1652206984.227046]: State machine starting in initial state 'GOTO_DOOR' with userdata: 
	[]
[INFO][/state_machine][1652206984.232449]: State machine starting in initial state 'START_ANALYSIS' with userdata: 
	[]
[INFO][/state_machine][1652206984.235721]: State machine transitioning 'START_ANALYSIS':'done'-->'NAVIGATE'
[INFO][/state_machine][1652206984.238771]: State machine starting in initial state 'GET_PLAN' with userdata: 
	[]
efficient copy
efficient copy
efficient copy
[INFO][/state_machine][1652206984.387410]: 'I'm coming!'
[INFO][/state_machine][1652206984.405908]: State machine transitioning 'GET_PLAN':'goal_ok'-->'EXECUTE_PLAN'
[INFO][/state_machine][1652207004.915681]: State machine terminating 'EXECUTE_PLAN':'succeeded':'arrived'
[INFO][/state_machine][1652207004.920243]: State machine transitioning 'NAVIGATE':'arrived'-->'STOP_ANALYSIS_SUCCEED'
[INFO][/state_machine][1652207004.923825]: State machine terminating 'STOP_ANALYSIS_SUCCEED':'done':'arrived'
[INFO][/state_machine][1652207004.927007]: State machine transitioning 'GOTO_DOOR':'arrived'-->'SAY_OPEN_DOOR'
efficient copy
[INFO][/state_machine][1652207004.938330]: 'Someone please open the door, I'm expecting guests'
[INFO][/state_machine][1652207008.655185]: State machine transitioning 'SAY_OPEN_DOOR':'spoken'-->'SAY_PLEASE_COME_IN'
efficient copy
[INFO][/state_machine][1652207008.658701]: 'Please come in, I'm waiting for someone to step in front of me'
[INFO][/state_machine][1652207012.778395]: State machine transitioning 'SAY_PLEASE_COME_IN':'spoken'-->'WAIT_FOR_GUEST'
efficient copy
[INFO][/state_machine][1652207012.817728]: Received rgb, depth, cam_info
[INFO][/state_machine][1652207013.860264]: There are 1 people in front of me (1 is enough): ['48b179f75d6059d3a774b2906baa529a']
[INFO][/state_machine][1652207013.865093]: State machine transitioning 'WAIT_FOR_GUEST':'success'-->'SAY_HELLO'
efficient copy
[INFO][/state_machine][1652207013.871234]: 'Hi there, I'll learn your face now'
[INFO][/state_machine][1652207016.387772]: State machine transitioning 'SAY_HELLO':'spoken'-->'ASK_GUEST_NAME'
[INFO][/state_machine][1652207016.389830]: AskPersonName
[INFO][/state_machine][1652207016.393217]: 'What is your name?'
efficient copy
[INFO][/state_machine][1652207017.708724]: Question: Which option?, grammar: T['name': N] -> NAME[N]

NAME['anna'] -> anna
NAME['beth'] -> beth
NAME['ca..
[INFO][/state_machine][1652207017.712083]: Example: 'mary'
^C[state_machine-14] killing on exit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@LarsJanssenTUe LarsJanssenTUe

@lottemessing lottemessing

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MatthijsBurgh @PetervDooren @LarsJanssenTUe @lottemessing