AI2-THOR (The House Of inteRactions) is a near photo-realistic interactable framework for AI agents.
-
(6/2020) We provide a mini-framework to simplify running AI2-THOR in Docker. It can be accssed at: https://github.com/allenai/ai2thor-docker.
-
(4/2020) Version 2.4.0 update of the framework is here. All sim objects that aren't explicitly part of the environmental structure are now moveable with physics interactions. New object types have been added, and many new actions have been added. Please see the full 2.4.0 release notes here
-
(2/2020) AI2-THOR now includes two frameworks: iTHOR and RoboTHOR. iTHOR includes interactive objects and scenes and RoboTHOR consists of simulated scenes and their corresponding real world counterparts.
-
(9/2019) Version 2.1.0 update of the framework has been added. New object types have been added. New Initialization actions have been added. Segmentation image generation has been improved in all scenes.
-
(6/2019) Version 2.0 update of the AI2-THOR framework is now live! We have over quadrupled our action and object states, adding new actions that allow visually distinct state changes such as broken screens on electronics, shattered windows, breakable dishware, liquid fillable containers, cleanable dishware, messy and made beds and more! Along with these new state changes, objects have more physical properties like Temperature, Mass, and Salient Materials that are all reported back in object metadata. To combine all of these new properties and actions, new context sensitive interactions can now automatically change object states. This includes interactions like placing a dirty bowl under running sink water to clean it, placing a mug in a coffee machine to automatically fill it with coffee, putting out a lit candle by placing it in water, or placing an object over an active stove burner or in the fridge to change its temperature. Please see the full 2.0 release notes here to view details on all the changes and new features.
- OS: Mac OS X 10.9+, Ubuntu 14.04+
- Graphics Card: DX9 (shader model 3.0) or DX11 with feature level 9.3 capabilities.
- CPU: SSE2 instruction set support.
- Python 2.7 or Python 3.5+
- Linux: X server with GLX module enabled
Please refer to the Documentation Page on the AI2-THOR website for information on Installation, API, Metadata, actions, object properties and other important framework information.
If you wish to make changes to the Unity scenes/assets you will need to install Unity Editor version 2019.4.20 LTS for OSX (Linux Editor is currently in Beta) from Unity Download Archive. After making your desired changes using the Unity Editor you will need to build. To do this you must first exit the editor, then run the following commands from the ai2thor base directory. Individual scenes (the 3D models) can be found beneath the unity/Assets/Scenes directory - scenes are named FloorPlan###.
pip install invoke
invoke local-buildThis will create a build beneath the directory 'unity/builds/thor-local-OSXIntel64.app'. To use this build in your code, make the following change:
controller = ai2thor.controller.Controller(
local_executable_path="<BASE_DIR>/unity/builds/thor-OSXIntel64-local/thor-OSXIntel64-local.app/Contents/MacOS/AI2-THOR"
)AI2-THOR uses Assembly definitions to declare dependencies between libraries within the project. To add a third-party package to the project, unpack the package to a newly created directory beneath the unity/Assets/ directory. Within the Unity Editor, find the new folder in the Project tab. Select the folder and right-click in the window that displays the contents of the folder and select Create -> Assembly Definition. This will create a new file in the folder - rename the Assembly definition to match the name of the plugin you are adding (e.g. 'Priority Queue' or 'iTween'). This allows the Assembly definition to be easily found during creation of the reference. One thing to be aware of is that if you don't rename the newly created file immediately the name in the Inspector within the Editor will not match the filename and you will have to manually update the definition name after renaming the file. To reference the plugin, a reference must be created in the Assembly Definition file located at unity/Assets/Scripts/AI2-THOR-Base. Locate the AI2-THOR-Base Assembly Definition file under unity/Assets/Scripts within the Project tab. Click on the file and locate the section titled "Assembly Definition References". Click the + sign to add a new entry, then click on the circle to the right of the newly created entry. That will bring up a menu with all the Assembly definitions in the project. Select the name of the new plugin/Assembly Definition that was just created. Scroll to the bottom and click the "Apply" button. The plugin should now be available to use. The AI2-THOR-Base Assembly definition and the new plugin folder will need to be commited. Additional information about Assembly definitions can be found at:
- https://learn.unity.com/tutorial/working-with-assembly-definitions?uv=2019.4#
- https://docs.unity3d.com/Manual/ScriptCompilationAssemblyDefinitionFiles.html
For building for browser, please refer to this page.
If you use iTHOR, please cite the original AI2-THOR paper:
@article{ai2thor,
Author = {Eric Kolve and Roozbeh Mottaghi
and Winson Han and Eli VanderBilt
and Luca Weihs and Alvaro Herrasti
and Daniel Gordon and Yuke Zhu
and Abhinav Gupta and Ali Farhadi},
Title = {{AI2-THOR: An Interactive 3D Environment for Visual AI}},
Journal = {arXiv},
Year = {2017}
}
If you use RoboTHOR, please cite the following paper:
@inproceedings{robothor,
Author = {Matt Deitke and Winson Han and Alvaro Herrasti and
Aniruddha Kembhavi and Eric Kolve and Roozbeh Mottaghi and
Jordi Salvador and Dustin Schwenk and Eli VanderBilt and
Matthew Wallingford and Luca Weihs and Mark Yatskar and
Ali Farhadi},
Title = {{RoboTHOR: An Open Simulation-to-Real Embodied AI Platform}},
Booktitle = {CVPR},
Year = {2020}
}
We have done our best to fix all bugs and issues. However, you might still encounter some bugs during navigation and interaction. We will be glad to fix the bugs. Please open issues for these and include the scene name as well as the event.metadata from the moment that the bug can be identified.
AI2-THOR is an open-source project backed by the Allen Institute for Artificial Intelligence (AI2). AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.