In an assistive driving system, identifying a potential hazard is only part of the job. A driver also needs to understand why the system is raising an alert and what the warning means—often with little time to respond. Explainable AI driving systems address this human-machine interface (HMI) challenge by connecting a model’s detection to a clear, localized indication of risk, rather than presenting a prediction without context.
A prototype explored this approach for road scenes involving vulnerable road users (VRUs), including pedestrians using mobile phones and cyclists. Its central design principle is relevant beyond vehicles: when an AI system informs a safety-related decision, its output should be understandable and usable in the time available to the human operator.
Why explainability matters in assistive driving
Many AI models are treated as black boxes: they receive input and produce a prediction, but do not readily show what factors shaped that result. A statistical prediction may identify a pattern without explaining the wider cause or circumstances in a way a person finds meaningful. That gap can make it difficult for an operator to judge whether to act, and can weaken confidence in the system.
In a driving scenario, a lengthy explanation is not necessarily helpful. The driver’s attention is already focused on the road, and the decision window may be short. The system therefore needs to communicate enough information to support a quick assessment without competing with the driving task or obscuring the view.
The prototype was designed as an assistant, not a replacement for the human driver. It detects and assesses potential risks, then presents information intended to help the driver respond to challenging situations. This distinction matters: the AI output supports a human decision rather than claiming to make the entire driving decision independently.
Separating object detection from risk assessment
The system divides its work between a convolutional neural network (CNN) and additional modules that analyze risk and communicate warnings. The CNN’s role is to identify and locate specific objects in the video stream, such as pedestrians and cyclists. It does not, by itself, determine the complete collision risk.
The prototype used a Single Shot MultiBox Detector (SSD) architecture with VGG16 as its feature-extraction backbone. SSD processes an image in one pass and proposes bounding boxes around detected objects. Those locations give later stages of the system a concrete basis for tracking and assessment: the warning can point to an object in the scene, rather than simply report an unexplained classification.
The model was trained on a manually collected, relatively small dataset covering people using phones, general pedestrians and cyclists. Transfer learning and hyperparameter tuning produced an accuracy of about 73% in the reported prototype. The results were treated as sufficient for an initial investigation, not as evidence that the model was ready for deployment. Limited training data remained a significant constraint.
From video stream to driver warning
The system operates as a continuous sequence. A camera supplies images, an object tracker follows detections, and the risk module filters and updates potential hazards. The actions module then determines the warning to display, while the system can store identified risks and actions in a database.
Risk assessment considered several factors: the combined speed of the subject and vehicle, a risk factor based on the subject’s location, and estimated distance to the subject. Distance was calculated using the similar-triangles relationship between the real object’s height, its height in the image and the camera’s focal length. These inputs helped the system move beyond object recognition toward a risk estimate.
Risk categories combined the likelihood and consequence of a collision, ranging from low to extreme. To present the result, the interface used colour-coded markers around the localized object, with concise supporting information:
- A warning icon representing the identified risk factor.
- Estimated distance to the object in metres.
- A likelihood-of-collision value.
This design makes the detected object and the reason for attention visible together. It also reflects a broader HMI principle: present decision-relevant context close to the event that prompted the alert.
What the evaluation revealed
Evaluation considered both technical performance and the user experience. Quantitative tests assessed model accuracy and how close the video output came to real time. The reported average frame rates ranged from 14–18 FPS on a desktop system and 18–22 FPS on a laptop system. The study noted that performance varied with the hardware and video input; a mobile-phone stream over Wi-Fi showed a significant frame-rate drop, with poor Wi-Fi identified as a possible cause.
Detection quality was less consistent. The system sometimes missed pedestrians or cyclists and sometimes classified ordinary pedestrians as potential risks. In lower-visibility scenes, identification was also more difficult. These outcomes underline why an AI-generated alert should not be interpreted as a definitive account of the scene, particularly when the underlying model has limited training data.
User feedback focused on the presentation of risk. Participants preferred brief, high-quality visual indicators over lengthy text, especially in edge cases requiring quick decisions. They also warned that too much on-screen information could obscure objects and reduce the driver’s view. An explanation must therefore be not only informative but also proportionate to the available attention and screen space.
Practical implications for safety-focused AI interfaces
For teams developing AI-enabled systems, the prototype points to several practical considerations:
- Separate detection from interpretation. Make clear which module identifies an object and which evaluates its risk.
- Connect alerts to evidence. Localizing the object gives users immediate context for a warning.
- Keep explanations concise. Short visual cues can be more usable than paragraphs when response time is limited.
- Test failure modes. False positives, missed detections and difficult visibility conditions all affect how an operator should interpret an alert.
- Assess the full interaction. Model metrics and processing speed do not, on their own, establish that a warning is understandable or safe to use.
These principles apply to industrial automation as well as driving. In an operator-facing system, an AI alarm or risk classification is more useful when it identifies the relevant asset or event, communicates the basis for attention in an accessible way, and avoids adding unnecessary visual or cognitive load.
Conclusion
Explainable AI in assistive driving is not simply a matter of describing a model’s internal calculations. It also depends on how detections become risk assessments and how those assessments reach a human in a usable form. The prototype demonstrates a layered approach—object detection, risk analysis and localized visual warnings—while its missed detections, false positives and dataset limitations show why performance and interface design must be evaluated together. For safety-critical applications, concise explanations can support faster human judgment, but they must remain grounded in the system’s actual capabilities and presented without obscuring the task at hand.
Access Official Technical PDF