Types of Prototypes
Definition
- There are many reasons why prototyping is a valuable way of evaluating concepts during development.
- Different prototyping techniques enable gathering user feedback and evaluating the user experience of autonomous driving systems.
- Using different prototypes, you can simulate and evaluate scenarios that may be challenging or dangerous in real-world testing. This iterative approach allows for identifying and rectifying issues before deploying the technology on actual roads.
- Utilising prototypes such as wizard-of-oz simulations, driving simulators, and virtual reality environments can significantly reduce costs and development time compared to physical testing with real vehicles. Prototyping allows for rapid iterations, quick modifications, and evaluation of different design choices without the need for expensive hardware or the risks associated with real-world testing.
- The use of different prototypes allows for an iterative development and validation process. Researchers and developers can start with simpler prototypes like wizard-of-oz, gradually progress to driving simulators, and ultimately transition to real-world testing. Each iteration helps refine the system, validate its capabilities, and uncover potential issues or areas for improvement.
Physical Driving Simulator
Simulators offer a controlled environment to simulate various driving scenarios and conditions. They provide a realistic representation of the driving experience, including vehicle dynamics, traffic, and road conditions. Using a driving simulator allows researchers and developers to test autonomous driving algorithms, assess their performance, and fine-tune the system in a safe and reproducible manner.
Physical driving simulator used in Schneider, Tobias, et al. "Explain yourself! transparency for positive ux in autonomous driving." Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.
| Advantage | Disadvantage |
|---|---|
| Safe Testing Environment: Driving simulators provide a controlled and safe environment to test autonomous driving algorithms and systems without the risks associated with real-world testing. Developers can simulate various driving scenarios, road conditions, and traffic situations to evaluate the system's performance. | Unrealistic Driving Experience: Despite efforts to create realistic driving scenarios, simulators may not fully replicate the physical sensations and cognitive load experienced during actual driving. This can affect user feedback and the system's performance evaluation, as the human operator's behavior may differ from real-world driving. |
| Reproducibility and Standardisation: Simulators offer a high level of reproducibility, allowing researchers and developers to conduct tests under consistent conditions. This enables the comparison of different algorithms, configurations, and improvements in a standardised manner. | User Acceptance and Adaptation: The experience of driving in a simulator may differ significantly from driving a real vehicle. Users, particularly those unfamiliar with simulators, may take time to adapt to the interface, controls, and overall experience, potentially affecting their feedback and evaluation of the autonomous driving system. |
Virtual Reality Driving Simulator
VR technologies can create immersive and realistic driving scenarios, allowing users to experience autonomous driving in a very immersive environment. VR provides a cost-effective alternative to physical testing, enabling developers to simulate various road conditions, traffic situations, and potential hazards. It also allows for rapid iterations and testing of user interfaces and human-machine interactions.
VR prototype used in Schneider, Tobias, et al. "Increasing the User Experience in Autonomous Driving through different Feedback Modalities." 26th International Conference on Intelligent User Interfaces. 2021.
| Advantage | Disadvantage |
|---|---|
| Immersive and Realistic Experience: VR driving simulators provide a highly immersive and realistic driving experience by creating a virtual environment. Users can experience the perception of driving in a simulated autonomous vehicle, including visual and audio feedback, which closely resembles real-world driving scenarios. | Cost and Setup Complexity: VR driving simulators can require significant investment in hardware, software, and infrastructure. The setup and calibration of VR systems, including headsets, motion trackers, and computing resources, can be complex and time-consuming. |
| Safe and Controlled Testing: Similar to traditional driving simulators, VR driving simulators provide a safe and controlled environment for testing autonomous driving systems. Developers can assess the system's performance, evaluate different algorithms, and test various scenarios without the risks associated with real-world testing. | Simulator Sickness: Some users may experience discomfort or simulator sickness, commonly known as motion sickness, when using VR simulators. The sensory disconnect between virtual motion and physical motion can lead to discomfort, which may impact the validity and reliability of user feedback and system evaluation. |
| Reproducibility and Standardization: VR simulators offer a high level of reproducibility, allowing tests to be conducted under consistent conditions. This enables developers to compare different algorithms, configurations, and improvements in a standardized manner and track the progress of the autonomous driving system. | Learning Curve and Adaptation: Users may require time to adapt to VR environments and become familiar with the controls and interactions within the simulation. The initial learning curve and adjustment period can impact the user's feedback and potentially introduce bias or limitations in the evaluation process. |
Wizard-of-Oz with a Real Vehicle
The Wizard-of-Oz technique allows for the simulation of autonomous driving by having a human "wizard" control the vehicle behind the scenes while giving the illusion of an automated system. This approach is helpful as it provides a low-cost and flexible way to gather user feedback and test the overall user experience. Furthermore, it allows participants to experience a real-world drive which can lead to higher immersion.
Passenger shielded from driver, setup used in Schneider, Tobias, et al. "Don’t fail me! The Level 5 Autonomous Driving Information Dilemma regarding Transparency and User Experience." Proceedings of the 28th International Conference on Intelligent User Interfaces. 2023.
| Advantage | Disadvantage |
|---|---|
| High Immersion: Participants feel the vehicle with all driving forces and do not have to imagine that they are driving around. Driving situations might have a higher impact on participants as they are fully present in the environment and not solely placed in front of a screen. | Scalability Challenges: Wizard-of-Oz prototypes may not scale well when testing with a large number of users. The need for human operators and individualized interactions can make it time-consuming and impractical to gather feedback from a diverse and extensive user base. |
| Cost-Effective: Wizard-of-Oz prototypes require minimal hardware and development resources compared to building a fully functional autonomous driving system. This makes it a cost-effective option for early-stage testing and gathering user feedback. | Operator Bias and Reproducibility: The behaviour and decisions of the human operator (wizard) may introduce unintentional biases and inconsistencies in the simulation, potentially affecting the validity and reliability of the collected data and user feedback. |
Sources
[1] - Nielsen, Jakob. "Usability 101: Introduction to usability (2012)." URL: http://www.nngroup.com/articles/usability-101-introduction-to-usability/[Accessed September 2022] 9 (2012): 35.