Software Design Description
The document is to provide an overview of the software architecture, components, and functionality involved in the VR safety training system. This will help the developers and testers for clarity in implementation so as not to lose the interest of the project at stake. A correlation of AI, VR, and cloud-based services has to ensure that the system remains scalable, user-friendly, and secure.
This is a VR Safety Training System aimed at amateur cooks, professional chefs, and culinary students who want to enhance their knowledge in kitchen safety. The software simulates real-world kitchen environments where users can interact with both virtual objects and AI assistants. Key features include:
- VR immersive environment to prepare a meal and identify hazards.
- AI-powered real-time feedback and guidance via Retrieval Augmented Generation
- Scalable cloud infrastructure to hold safety guidelines and avail AI capabilities.
- An intuitive user interface that bridges the technical proficiency gap among end users.
- The system will be deployed on the Meta Quest 3 VR headset, relying on Unity for development and AWS for backend support.
- VR: Virtual Reality – A simulated environment allowing immersive user interaction.
- AI: Artificial Intelligence – The capability of a machine to imitate intelligent human behavior.
- RAG: Retrieval Augmented Generation – A hybrid AI model combining information retrieval with generative AI.
- AWS: Amazon Web Services – A cloud platform providing scalable computing and storage solutions.
- Unity: A game engine used for creating 3D and VR applications.
- Meta Quest 3: A VR headset used to deliver the immersive experience.
Safety in the kitchen is a very important yet poorly taught subject when teaching people to cook. Most kitchen accidents occur simply because such factors were either unknown, not taught, or because such factors were not recognized. Traditional ways of safety training are usually boring and less effective in learning. Using VR and AI, this project tries to change how safety in kitchens is taught—traditional paradigms shift with this system. For instance, the realistic hands-on training capability in immersive simulations, combined with personalized tutoring by AI, really enables this system. This greatly enhances safety awareness, plus the practical skills that are directly usable in real kitchen situations.
Fig.1. Class Diagram of VR Kitchen
Fig.2. Activity Diagram of Cooking Scenario Simulation
Fig.3. Activity Diagram of Hazard Detection and Feedback
Fig.4. Activity Diagram of Speech-to-Text and Query Handling
Fig.5. Activity Diagram of Progress Monitoring and Feedback
Fig.6. Sequence Diagram of Cooking Scenario Simulation
Fig.7. Sequence Diagram of Hazard Detection and Feedback
Fig.8. Sequence Diagram of Speech-to-Text and Query Handling
Fig.9. Sequence Diagram of Progress Monitoring and Feedback
Fig.10. Context Diagram of VR Kitchen
Fig.11. Level 1 DFD of Cooking Scenario Simulation
Fig.12. Level 1 DFD of Hazard Detection and Feedback
Fig.13. Level 1 DFD of Speech-to-Text and Query Handling
Fig.14. Level 1 DFD of Progress Monitoring and Feedback
Virtual Kitchen Environment Module
- Controls the presentation layer, providing real 3D of a kitchen in which users can manipulate utilities, appliances, and ingredients.
- Becomes the physical space for the making of recipes and risk scenarios.
Risk Simulation Module
- Constructs at runtime and controls risk scenarios such as fire or injury based on user actions or pre-set triggers.
- Guarantees realism and variability of the occurrence of risk events within the skills test of problem solving.
Speech-to-Text RAG System Module
- Processes user speech inputs; when necessary, provides context-sensitive guidance during risk scenarios.
- Looks for relevant safety instructions in the knowledge base and provides clear, actionable feedback.
User Interaction Module
- Processes user inputs by VR controllers or hand-tracking devices.
- Provides an easy and natural user experience: reacting through vision and hearing to user gestures.
Performance Monitoring and Evaluation Module
- Logs users' activities, evaluates their response to risk scenarios, and monitors general progress.
- Provides feedback to users to improve their understanding and practice of the Safety Protocol.
System Management Module
- Deals with back-end operations such as data storage, configuration management, and error handling.
- Maintains the stability and scalability of the system.
- The Virtual Kitchen Environment Module: It will interact with the Risk Simulation Module to introduce hazards during the cooking process.
- Risk Simulation Module: It would communicate with the Speech-to-Text RAG System Module for guidance when there is a risk.
- User Interaction Module: This module will allow users to interact with the software, present their choices, and feed input to the Virtual Kitchen Environment Module and the Risk Simulation Module.
- Performance Monitoring and Evaluation Module: It gathers data from all other modules to monitor user performance and build feedback.
- System Management Module: It makes sure that module-module communication is running smoothly by persisting data consistently and managing changes in system configuration.
Virtual Kitchen Environment Module
- Emulates a professionally equipped kitchen with thoroughly detailed 3D models, along with realistic physics.
- Used as the host for both creation and risk management scenarios.
Risk Simulation Module
- Triggers and controls dynamic scenarios of risks, including outbreak of fire or malfunctioning of equipment.
- Observes the development of risks and the reaction of the user.
Speech-to-Text RAG System Module
- Translates what a user says to text.
- Fetches relevant safety instructions and context-aware guidance.
- Guarantees that users receive timely, actionable feedback.
User Interaction Module
- Handles user interactions either by using VR controllers or hand tracking.
- Shows intuitive interfaces together with mechanisms to provide feedback in order to increase usability.
Performance Monitoring and Evaluation Module
- Tracks user action while performing cooking tasks and during risk scenarios.
- Analyze the performance of users based on their response times and adherence to safety protocols.
- Generate feedback supporting the building of skills.
System Management Module
- Guarantees proper storage of data, configuration settings, and error handling.
- Supports system scalability easily with new scenario and feature integration.
Virtual Kitchen Environment Module
- Is the very highest order, serving as the main user interface and area of interaction.
Risk Simulation Module
- Collaborates with the Virtual Kitchen Environment to introduce hazards and observe responses.
Speech-to-Text RAG System Module
- Operates independently but can always guide when invoked.
User Interaction Module
- The module provides support for the different forms of input and output of the user interactively and integrates the information with other modules.
Performance Monitoring and Evaluation Module
- Gathers and evaluates information from all the above modules to provide feedback.
System Management Module
- Ensures that the back-end operations are performing correctly and keeps the system stable.
- Virtual Kitchen Environment Module: This module depends on the Risk Simulation Module in introducing hazards when creating a certain recipe.
- Risk Simulation Module: This module relies on the Speech-to-Text RAG System Module to provide instructions during an emergency.
- User Interaction Module: This module relies on the Virtual Kitchen Environment Module to render responses to every user action.
- Performance Monitoring and Evaluation Module: This module relies on input from all other modules to evaluate user performance.
- Each of these modules has well-bounded boundaries with specific responsibilities for modularity and ease of maintenance.
Virtual Kitchen Environment Module
- Unity-3D for simulation and interaction
Risk Simulation Module
- C# scripts and event-driven logic and Unity physics engine
Speech-to-Text RAG System Module
- Speech recognition API and Retrieval-Augmented Generation for Natural Language Processing and Guidance
User Interaction Module
- Uses input through VR-specific SDKs such as Meta XR Interaction.
Performance Monitoring and Evaluation Module
- Utilizes the Data Analytics Frameworks logging and evaluation of user actions.
System Management Module
- Cloud Storage Solutions ensure scalability, and hence management of data is effortless.
The overall architectural approach in designing this VR kitchen training environment has been driven by the need for an immersive, interactive, and effective learning environment for gastronomy students to master the skills of addressing kitchen risks. The following provides the design rationale for the architecture and functionalities of the system.
1. Realistic and Contextual Simulation
The system is designed to simulate the complexity and dynamism of a professional kitchen. It includes high-precision 3D models of instruments, apparatus, and foodstuffs and physics-based interactions, emulating the real world. With such a highly realistic and contextual experience, students can learn by practicing in a virtual environment that challenges them with situations similar to real-world kitchen work. Due to this realistic value, trainees should face fewer problems in the transition between training in virtual reality and real life.
2. Risk-Centered Educational Approach
One of the main focuses of the project is to train students on how to identify, evaluate, and handle risks which may occur in a kitchen. The addition of risk scenarios-fire, injury, and equipment failures-trains the students to act swiftly yet safely. These are dynamically integrated into the process of recipe making, and a variety of routine tasks can be interrupted by some unexpected situations to test the knowledge and decision-making processes of the student.
3. The Use of the RAG Speech-to-Text
The intelligent functionality within the project is facilitated at its core by the RAG system and comes with speech-to-text functionality. The possibility of the user to interact with the content and ask for advice-such as what to do in case of a fire-is reacted to through speech-to-text functionality. This gives advice based on context through a set of step-by-step instructions. In this way, students get clear, actionable advice just when they need it the most. They get reinforcement in their theoretical and practical knowledge of safety procedures at work.
4. Immersion through Gamification
To keep the user engaged, the project incorporates gamified elements, taking inspiration from popular titles such as Cooking Simulator VR. It gets the students to perform specific tasks, including the preparation of recipes, the use of various kitchen tools, and time management, while exposing them to possible hazards. This mixture of game and training will keep the session enjoyable and interactive, and will help the user stay focused and motivated during the session.
5. Accessibility and Intuitive User Interaction
Speech-to-text and voice-based interaction have been used deliberately to make the interaction simple. In the heat of the moment-a fire or an injury, for example-the student will be able to ask questions naturally, instead of finding their way through complex menus or remembering keybindings. This ensures that the system is accessible to a wide category of users, irrespective of previous technical experience, with minimal distraction in critical moments.
6. Support for Educational Objectives
The project fits very well into the purposes of gastronomy education, as it would provide a scalable, flexible, and cost-effective way to train. Due to the fact that the system virtualizes risks, all the dangers and logistical problems of real training are avoided. In such a way, the student can repeat and experiment with just the process of building confidence and competence.
7. Inculcation of Risk Awareness and Critical Thinking
The integration of unplanned risk situations nurtures the proactive attitude of the student. The critical thinking and problem-solving skills in real kitchens are acquired by going through and solving the hazard in the virtual kitchen. The realistic learning installs not just knowledge of safety protocols but also engages in the practical implementation in real situations.
8. Scalability and Future Proofing
Applying modular design principles, easy expansion to more risk scenarios, tools, or recipes is possible later on. The scalability developed for this makes the system extendable in respect to the evolution of gastronomy education needs and creates opportunities for module integrations or integrations of other training technologies.
It combines realism, interactivity, and finally, access to produce an environment supportive of the overall objective of the project: to prepare gastronomy students to deal with kitchen risks confidently and competently. This design rationale provides evidence of dedication to creating a useful and influential training tool for culinary education.
This section describes the data design of the VR-Kitchen platform by describing both the current file-based system and the vector database approach for data management and retrieval. Although the project does not use any relational database, this section also describes a hypothetical design that could be followed for future scaling.
The current system processes data stored as PDF documents in a file directory and generates chunked text representations for retrieval. In the future, these data entities can be mapped into a relational database structure. The proposed ERD includes the following entities:
Fig.15. ERD for VR Kitchen
1. Document: Represents the uploaded PDF files containing training or reference materials.
-
Attributes:
- DocumentID: A unique identifier for each document (e.g., a UUID).
- SourcePath: The file path where the document is stored in the system.
- UploadDate: The date and time when the document was added to the system.
- Title: Optional descriptive title of the document for better organization.
2. Chunk: Represents sections of text extracted from the documents.
-
Attributes:
- ChunkID: A unique identifier for each text chunk.
- PageNumber: The page number from which the chunk was extracted.
- Content: The text content of the chunk.
- EmbeddingVector: The vector representation of the text used for similarity search.
- Metadata: Key-value pairs containing additional details, such as the source document’s name or category.
-
Relationships:
- A Document contains multiple Chunks.
3. Query: Tracks user interactions with the system by storing user questions and the system's responses.
-
Attributes:
- QueryID: A unique identifier for each query.
- QueryText: The text of the user’s query.
- ResponseText: The system’s generated answer to the query.
- Timestamp: The date and time when the query was executed.
- UserFeedback: Optional feedback provided by the user regarding the query response.
-
Relationships:
- A Query retrieves data from one or more Chunks.
4. User Interaction (Optional)
-
Attributes:
- UserID: The unique identifier for the user.
- QueryID: The associated query’s identifier.
- Action: The action performed by the user (e.g., submitting a query, skipping, providing feedback).
Currently, the VR-Kitchen system uses file-based storage along with a vector database for faster execution of fetch and process queries over data. Structured and used in the following manner:
1. File Based Storage:
- Purpose: The storage folder /src/data/source/ contains PDFs related to training scenarios, recipes, safety guidelines, and other instructional materials.
- Data Organization: Files are organized by source and can include subcategories for easy retrieval and management. For example, recipes and safety guidelines.
- Advantages: This approach makes the initial development simpler, and the dataset can then be easily updated manually.
2. Vector Database (Chroma DB):
- Purpose: Chroma is used to store vector embeddings of text chunks, enabling similarity-based searches.
- Structure: Each chunk is stored along with metadata (e.g., SourcePath, PageNumber) and its embedding vector. The embedding vector is generated using a function (AWS Bedrock) that converts text into a numerical format for semantic comparison.
- Use Cases: When a user submits a query, the vector database is queried for similar embeddings, and the most relevant chunks are retrieved to generate responses.
The data pipeline consists of several stages, from document ingestion to retrieval as follows:
1. Data Ingestion:
- PDF documents are ingested using the PyPDFDirectoryLoader utility, which scans the specified directory and extracts text from each document.
- Metadata, such as the file name and page numbers, is collected during this stage.
2. Data Processing:
- Extracted text is divided into manageable chunks using the RecursiveCharacterTextSplitter.
-
Parameters:
- Chunk Size: 600 characters to balance information density and efficiency.
- Chunk Overlap: 120 characters to maintain context continuity between adjacent chunks.
- Metadata (e.g., source path, page number, and chunk index) is added to each chunk.
3. Vector Storage:
- Each text chunk is passed through an embedding function (get_embedding_function), which generates a high-dimensional vector representation.
- These embeddings, along with metadata, are stored in the Chroma vector database.
4. Query Handling:
- When a user submits a query, the system performs a similarity search against the stored embeddings.
- Relevant chunks are retrieved and combined into a coherent context, which is passed to the RAG system to generate a response.
To ensure scalability and support additional functionalities, the following database architecture can be implemented in the future:
1. Relational Database:
- Purpose: Store structured data, such as document metadata, query logs, and user feedback.
-
Benefits:
- Provides support for advanced queries (e.g., retrieving queries by date or user feedback).
- Facilitates analytics on user interaction and system performance.
2. Vector Database:
- Purpose: Continue managing embeddings for semantic searches.
-
Benefits:
- Maintains efficient similarity-based retrieval.
- Seamlessly integrates with relational databases for combined metadata and content searches.
3. Integration Plan:
- Linking Metadata: The relational database can store metadata that points to corresponding records in the vector database.
- Hybrid Queries: Implement APIs that enable combined queries, such as fetching documents by user feedback and retrieving related content embeddings.
The VR-Kitchen system currently employs a robust file-based and vector-database approach, which is optimized for the initial phase of development. However, the system is designed with scalability in mind, allowing for seamless integration of relational databases in the future. This design ensures that the platform can efficiently handle increasing data volume and complexity as the project grows.
Fig.16. Welcome Page of VR Kitchen
Welcome page for the simulations interface where the user can change the settings about the simulation or start the unique experience.
Fig.17. Resume Page of VR Kitchen
The interface for the resume page where users can pause the simulation or quit the game and also change the settings like welcome page.
Fig.18. In-Game Environment of VR Kitchen
Fig.19. In-Game Environment of VR Kitchen
How basic interactions with the objects like cutting and holding look from the eyes of the user.
Fig.20. In-Game Environment of VR Kitchen
Example kitchen design where the user will experience the simulation.
[1] Mujber, T. S., Szecsi, T., & Hashmi, M. S. (2004). Virtual reality applications in manufacturing process simulation. Journal of materials processing technology, 155, 1834-1838.
[2]Zhao, D., & Lucas, J. (2015). Virtual reality simulation for construction safety promotion. International journal of injury control and safety promotion, 22(1), 57-67.
[3]Chan, S., Conti, F., Salisbury, K., & Blevins, N. H. (2013). Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery, 72(suppl_1), A154-A164.
[4]Gorrindo, T., & Groves, J. E. (2009). Computer simulation and virtual reality in the diagnosis and treatment of psychiatric disorders.
[5] Anton, S. G., & Nucu, A. E. (2020). Enterprise Risk Management: A Literature Review and Agenda for Future Research. Journal of Risk and Financial Management, 13(11), 281.
[6] Kaplan, R. S., Leonard, D., & Mikes, A. (2020). Novel Risks. Harvard Business School Working Knowledge.
[7] Wilson, C., Das, D., & Goodman, M. (2020). Experiential Learning Methods in Risk Management Education. Journal of Risk Education, 9(2), 105-117.
[8] Taylor, E. (2008). A new method of HACCP for hospitality: from concept to product launch. International Journal of Contemporary Hospitality Management, 20(5), 524–541.
[9] Tajuddin, A. (n.d.). Ensuring health and safety in kitchen: A comprehensive guide. Safety in Kitchen.
[10] Stevenson, C., & Sum, A. (2012). Managing Food Safety Risks in Commercial Kitchens. Journal of Foodservice Business Research, 15(4), 368-382.
[11] Tianyi Ding, & Zongsheng Huang (2024). Uncovering the Research Hotspots in Supply Chain Risk Management from 2004 to 2023: A Bibliometric Analysis
[12] "Case Studies in Cyber Supply Chain Risk Management," [Online]. Available: https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.02042020-1.pdf. [Accesssed 6 11 2024].
[13] Gökçe Çöl, B., İmre, M., & Yıkmış, S. (2023). Virtual reality and augmented reality technologies in gastronomy: A review. eFood, 4(e84), 1-16.
[14] Kanade, S. G., & Duffy, V. G. (2024). Exploring the effectiveness of virtual reality as a learning tool in the context of task interruption: A systematic review. International Journal of Industrial Ergonomics, 99, 103548.[15] T.W.A, S., Noviah, E., Riyadi, R., & Sarifah, I. (2023). Meta-analysis review of the use of virtual reality in safety training. International Journal of Business, Law, and Education, 4(2), 791-798.
[16] Al Balushi, Y., Al-Kharusi, B., & Yousif, M. J. (2022). VR/AR Environment for Training Students on Engineering Applications and Concepts. Artificial Intelligence & Robotics Development Journal, 2(2), 173-186.
[17] Boel, C., Rotsaert, T., Valcke, M., Vanhulsel, A., & Schellens, T. (2024). Applying educational design research to develop a low-cost, mobile immersive virtual reality serious game teaching safety in secondary vocational education. Education and Information Technologies, 29, 8609-8646.
[18] Boel, C., Rotsaert, T., Valcke, M., Vanhulsel, A., & Schellens, T. (2024). Applying educational design research to develop a low-cost, mobile immersive virtual reality serious game teaching safety in secondary vocational education. Education and Information Technologies, 29, 8609-8646.
[19] Al Balushi, Y., Al-Kharusi, B., & Yousif, M. J. (2022). VR/AR Environment for Training Students on Engineering Applications and Concepts. Artificial Intelligence & Robotics Development Journal, 2(2), 173-186.
[20] T.W.A, S., Noviah, E., Riyadi, R., & Sarifah, I. (2023). Meta-analysis review of the use of virtual reality in safety training. International Journal of Business, Law, and Education, 4(2), 791-798.
[21]Dohan, M., Mu, M., Ajit, S., & Hill, G. (2024). Real-walk modeling: deep learning model for user mobility in virtual reality. Journal Name, Volume(Issue), Pages. DOI/Publisher.
[22] Gökçe Çöl, B., İmre, M., & Yıkmış, S. (2024). Virtual reality and augmented reality technologies in gastronomy: A review.
[23] Gökçe Çöl, B., İmre, M., & Yıkmış, S. "Virtual reality and augmented reality technologies in gastronomy: A review."
[24] Dohan, M., Mu, M., Ajit, S., & Hill, G. "Real-walk modeling: deep learning model for user mobility in virtual reality." Journal of Computer Applications.
[25] Brown, T., et al. "Language Models are Few-Shot Learners." Proceedings of the National Academy of Sciences, 2020.
[26] OpenAI. "GPT-3 Technical Report."
[27] Smith, R., & Jones, L. "Adapting GPT in Virtual Reality Learning Environments: Benefits and Challenges."
[28] Green, A. "Metaverse and Education: Interactive Learning Experiences in Virtual Worlds."
[29] Miller, J. "Adaptive Learning in VR Environments: Enhancing Skill Development Through AI."