Semantics aren't loading correctly (Hm3D, Instance Image Nav) #2090
Labels
bug
Something isn't working
solution proposed
A tested solution to the issue has been proposed in the comment thread.
Comments
|
Adding code here (same as the links given above): Data formation: """
Code used to create dataset for pixel goals. Uses latest habitat lab version.
"""
import os
import argparse
import numpy as np
import habitat
from habitat.tasks.nav.shortest_path_follower import ShortestPathFollower
import time
from pathlib import Path
import h5py
from habitat.utils.geometry_utils import quaternion_from_coeff, quaternion_to_list
from utils import *
seed_everything(42)
def main(args):
config_path = args.config_path
if not os.path.exists(config_path):
raise RuntimeError(f"{config_path} does not exist!")
habitat_config = create_habitat_config(config_path, args)
env = habitat.Env(habitat_config)
env.seed(42)
follower = ShortestPathFollower(env.sim, goal_radius=0.5, return_one_hot=False)
episode_counter = 0
time_before_loop = time.time()
hdf5_file_path = args.save_path
# create parent directories
directory_path = Path(hdf5_file_path).parent
directory_path.mkdir(parents=True, exist_ok=True)
# Creation loop
with h5py.File(hdf5_file_path, "w") as hdf:
while episode_counter < args.num_saved_episodes:
start_time = time.time()
try:
obs = env.reset()
except Exception as e:
print(f"An error occurred while resetting environment: {e}")
continue
rgb_data, depth_data, pose_data, action_data, normed_target_point_data = (
[],
[],
[],
[],
[],
)
timesteps = 0
(
goal_flag,
goal_image,
goal_mask,
goal_point,
height_uncorrected_goal_point,
) = random_pixel_goal(habitat_config, env, args.mask_shape)
if not goal_flag:
print(f"Rejected the current goal with goal-flag {goal_flag}")
continue
best_action = follower.get_next_action(goal_point)
if best_action == 0:
print(
f"Rejected the current goal with goal-flag {goal_flag} and best-action {best_action}"
)
continue
start_rgb_image = obs["rgb"]
start_semantic_image = np.squeeze(obs["semantic"])
print(np.unique(start_semantic_image))
start_depth_image = obs["depth"]
start_pose = np.concatenate(
[
np.array(env.sim.get_agent_state().position),
np.array(quaternion_to_list(env.sim.get_agent_state().rotation)),
]
)
last_best_action = None
while True:
assert best_action is not None and last_best_action != 0 # sanity check
action_data.append(best_action)
obs = env.step(best_action)
rgb_data.append(obs["rgb"])
depth_data.append(obs["depth"])
pose = np.concatenate(
[
np.array(env.sim.get_agent_state().position),
np.array(
quaternion_to_list(env.sim.get_agent_state().rotation)
),
]
)
semantic_image = np.squeeze(obs["semantic"])
print(np.unique(semantic_image))
pose_data.append(pose)
normed_target_point_data.append(
get_normalized_goal_point_location_in_current_obs(
habitat_config, env, height_uncorrected_goal_point
)
)
# gen_observations = env.sim.get_observations_at(
# position=pose[:3],
# rotation=quaternion_from_coeff(pose[3:]),
# )
# print(
# "Images are equal",
# np.array_equal(obs["rgb"], gen_observations["rgb"]),
# )
timesteps += 1
if not (env.episode_over or timesteps >= args.max_timesteps):
last_best_action = best_action
best_action = follower.get_next_action(goal_point)
else:
print(f"Goal reached in {timesteps} steps")
if timesteps < args.min_timesteps:
print(
f"{timesteps} less than min. timesteps of {args.min_timesteps}"
)
break
episode_group = hdf.create_group(f"episode_{episode_counter}")
episode_group.create_dataset(
"start_rgb_image", data=start_rgb_image
)
# episode_group.create_dataset("start_depth_image", data=start_depth_image)
episode_group.create_dataset("goal_mask", data=goal_mask)
episode_group.create_dataset("start_pose", data=start_pose)
episode_group.create_dataset("poses", data=np.array(pose_data))
episode_group.create_dataset("actions", data=np.array(action_data))
episode_group.create_dataset(
"normalized_target_points",
data=np.array(normed_target_point_data),
)
episode_group.create_dataset("rgb_images", data=np.array(rgb_data))
episode_group.create_dataset(
"height_uncorrected_goal_point",
data=np.array(height_uncorrected_goal_point),
)
episode_group.create_dataset(
"goal_point",
data=np.array(goal_point),
)
# episode_group.create_dataset("depth_images", data=np.array(depth_data))
episode_counter += 1
end_time = time.time()
time_taken = end_time - start_time
total_time_taken = end_time - time_before_loop
total_hours, rem = divmod(total_time_taken, 3600)
total_minutes, total_seconds = divmod(rem, 60)
print(
f"Done with episode {episode_counter} out of {args.num_saved_episodes} total"
)
print(f"Time taken for this episode: {time_taken:.2f} seconds")
print(
f"Total time taken: {int(total_hours)} hours, {int(total_minutes)} minutes, {int(total_seconds)} seconds"
)
print()
break
print(f"All episodes saved to {hdf5_file_path}")
env.close()
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Habitat Data Collection Script")
parser.add_argument(
"--stage", type=str, default="train", help="Stage (train/val/minival)"
)
parser.add_argument(
"--split", type=str, default="train", help="Split (train/val/valmini)"
)
parser.add_argument(
"--num_sampled_episodes",
"-nse",
type=int,
default=int(1e4),
help="Number of sampled episodes",
)
parser.add_argument(
"--max_timesteps", type=int, default=64, help="Maximum timesteps per episode"
)
parser.add_argument(
"--min_timesteps", type=int, default=5, help="Minimum timesteps per episode"
)
parser.add_argument(
"--mask_shape", type=int, default=3, help="Shape of the goal mask"
)
parser.add_argument(
"--config_path",
type=str,
default="hm3d_config_instance_image_nav_mod.yaml",
help="Path to Habitat config file",
)
parser.add_argument("--robot_height", type=float, default=0.88, help="Robot height")
parser.add_argument("--robot_radius", type=float, default=0.25, help="Robot radius")
parser.add_argument(
"--sensor_height", type=float, default=0.88, help="Sensor height"
)
parser.add_argument(
"--image_width", type=int, default=224, help="Image width"
) # don't change
parser.add_argument("--image_height", type=int, default=224, help="Image height")
parser.add_argument(
"--image_hfov", type=float, default=79, help="Image horizontal field of view"
)
parser.add_argument("--step_size", type=float, default=0.25, help="Step size")
parser.add_argument(
"--turn_angle", type=float, default=30, help="Turn angle in degrees"
)
parser.add_argument(
"--data_dir",
type=str,
default="/scratch/vineeth.bhat/sg_habitat/data/datasets/instance_imagenav/hm3d/v3",
help="Path to data directory",
)
parser.add_argument(
"--scene_dataset_dir",
type=str,
default="/scratch/vineeth.bhat/sg_habitat/data/scene_datasets/hm3d",
help="Path to scene dataset directory",
)
parser.add_argument(
"--scenes_dir",
type=str,
default="/scratch/vineeth.bhat/sg_habitat/data/scene_datasets",
help="Path to scenes directory",
)
parser.add_argument(
"--save_path",
"-s",
type=str,
default="/scratch/vineeth.bhat/pix_nav_point_based_data/training_100.h5",
help="Path to save data",
)
parser.add_argument(
"--num_saved_episodes",
"-n",
type=int,
default=int(1),
help="Number of saved episodes",
)
args = parser.parse_args()
main(args)The code used to create the config object: def create_habitat_config(config_path, args):
habitat_config = habitat.get_config(config_path)
data_path = f"{args.data_dir}/{args.stage}/{args.stage}.json.gz"
scene_dataset = (
f"{args.scene_dataset_dir}/hm3d_annotated_basis.scene_dataset_config.json"
)
print(f"Using data path: {data_path}")
print(f"Using scene dataset: {scene_dataset}")
if not os.path.exists(data_path):
raise RuntimeError(f"Data path path does not exist: {data_path}")
if not os.path.exists(scene_dataset):
raise RuntimeError(f"Scene dataset path does not exist: {scene_dataset}")
with read_write(habitat_config):
habitat_config.habitat.dataset.split = args.split
habitat_config.habitat.dataset.scenes_dir = args.scenes_dir
habitat_config.habitat.dataset.data_path = data_path
habitat_config.habitat.simulator.scene_dataset = scene_dataset
habitat_config.habitat.environment.iterator_options.num_episode_sample = (
args.num_sampled_episodes
)
habitat_config.habitat.simulator.agents.main_agent.height = args.robot_height
habitat_config.habitat.simulator.agents.main_agent.radius = args.robot_radius
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.rgb_sensor.height = (
args.image_height
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.rgb_sensor.width = (
args.image_width
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.rgb_sensor.hfov = (
args.image_hfov
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.rgb_sensor.position = [
0,
args.sensor_height,
0,
]
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.height = (
args.image_height
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.width = (
args.image_width
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.hfov = (
args.image_hfov
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.position = [
0,
args.sensor_height,
0,
]
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.max_depth = (
500.0
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.min_depth = (
0.0
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.depth_sensor.normalize_depth = (
False
)
habitat_config.habitat.simulator.forward_step_size = args.step_size
habitat_config.habitat.simulator.turn_angle = args.turn_angle
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.semantic_sensor.height = (
args.image_height
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.semantic_sensor.width = (
args.image_width
)
habitat_config.habitat.simulator.agents.main_agent.sim_sensors.semantic_sensor.position = [
0,
args.sensor_height,
0,
]
return habitat_configThe YAML file: defaults:
- /habitat: habitat_config_base
- /habitat/task: instance_imagenav
- /habitat/simulator/[email protected]_agent: rgbd_agent
- /habitat/dataset/instance_imagenav: hm3d_v2
- _self_
habitat:
environment:
max_episode_steps: 100
simulator:
turn_angle: 15
agents:
main_agent:
sim_sensors:
rgb_sensor:
width: 360
height: 640
hfov: 42
position: [0, 1.31, 0]
depth_sensor:
width: 360
height: 640
hfov: 58
min_depth: 0.05
max_depth: 50.0
position: [0, 1.31, 0]
semantic_sensor:
type: HabitatSimSemanticSensor
width: 360
height: 640
position: [0, 1.31, 0]
height: 1.41
radius: 0.17
habitat_sim_v0:
gpu_device_id: 0
allow_sliding: False |
|
I also meet the same problem home/zhoulei/dataset_total/habitat/scenes/hm3d_v0.2/val/00829-QaLdnwvtxbs/QaLdnwvtxbs.basis.scn exists but failed to load . Have you solved this? |
|
@zhouak47 Sorry, I haven't found a solution yet. |
|
@FlightVin I have found a solution . You can comment these codes out in .../habitat-lab/habitat-lab/habitat/core/env.py ,check your path of habitat.simulator.scene_dataset and try again: |
|
@zhouak47 That worked. Thank you so much! |
aclegg3
added
bug
|
Interesting, thanks for the report on this issue, folks. Logging it as a bug for someone to look into. |
sorry, i meet the same problem here. I use the habitat==0.2.5. And i have tried the solution in #2090 (comment). But it did not work for me. I got the error like this: Any suggestions, please. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Habitat-Lab and Habitat-Sim versions
Habitat-Lab: master
Habitat-Sim: master
Master branch contains 'bleeding edge' code and should be used at your own risk.
Docs and Tutorials
Did you read the docs? https://aihabitat.org/docs/habitat-lab/ -> Yes
Did you check out the tutorials? https://aihabitat.org/tutorial/2020/ -> Yes
Perhaps your question is answered there. If not, carry on!
Semantics aren't loading correctly
For the sake of brevity, consider the following image showing the data present in the
minivalset of HM3D:The relevant files that have been listed in README for semantics are present.
However, when I try to access the unique semantic labels here (permalink), I always get either 0 or 3. This is despite me setting the scene dataset correctly here.
I get the following warnings on InstanceImageNav-v1
Any help would be appreciated. Thanks!
The text was updated successfully, but these errors were encountered: