Developers of automotive and other safety-critical software face the challenge of accelerating software development to deliver reliable systems faster and at a lower cost while still satisfying important functional-safety standards like ISO 26262.
At the same time, the inherent complexity of multicore processor architectures introduces design issues, which makes validating and verifying code more difficult for automotive systems. For example, hidden timing coupling interference between cores can significantly impact worst-case execution time (WCET) and increase the risk of systems failing to meet essential real-time deadlines.
Many OEMs are turning to artificial intelligence (AI) to accelerate software development. For safety-critical systems that must comply with industry safety standards, how AI is used across the software-development lifecycle affects the ease or difficulty of certification.
In this two-part series, we’ll explore human-in-the-loop agentic AI workflows as a practical approach to automate and accelerate software verification in a manner that aligns with an automotive OEM’s need to certify systems. We’ll also cover the concept of a trusted process framework and the role of WCET analysis in certifying safety-critical systems.
Human-in-the-Loop
Developers need to be able to verify and validate that functionally safe systems perform the tasks they’re designed to perform and that they perform them as expected. To achieve this, developers utilize requirements-based testing, which includes assuring that the system meets all design requirements as well as relevant safety and security requirements. Thus, requirements traceability is critical to the implementation of software code, the creation of tests, and the overall testing process.
Using AI to write code or create tests, however, is like working with a black box whose internal mechanisms can’t be monitored. Consider the following prompt to an AI: Optimize this code to execute faster. Such a prompt, even with more detailed constraints, runs the risk of doing exactly what was asked and still completely missing the intended objective.
For example, the AI could delete compute-intensive parts of an algorithm to achieve the objective of running faster. As a result, when using code created by an unqualified AI for a safety process, a person must comprehensively review the output of the code to assure it executes exactly as intended.
The same applies when AI is used to write tests to verify and validate code. The AI-generated test itself must be verified and validated to confirm that the test is relevant, provides sufficient coverage, fully exercises code, and correctly implements the requirements being verified. Again, there’s a need for a human-in-the-loop.
The challenge with AI is that without a process framework and guardrails in place, developers run the risk of spending more time reviewing, verifying, and correcting what the AI has created than it would be to do the work themselves.
In some cases, using AI can reduce rather than improve overall efficiency as well as introduce additional risk. This can erode an OEM’s confidence in the system. In addition, developers may run into difficulties when trying to certify systems utilizing AI-generated content to strict industry standards.
For AI to accelerate software development in a certifiable manner, it must be used in a way that developers can trust as well as verify that trust.
Building a Trusted Process Framework
One approach to using AI in a way that’s certifiable is to avoid having to trust the AI. This can be achieved by working within a trusted process framework that puts guardrails around AI-based activities. Rather than take an approach where AI replaces developers, the strategy is to use AI to amplify a developer’s capabilities. Specifically, this is done by having AI perform clearly defined tasks in a manner that’s transparent, traceable, and verifiable.
For safety-critical applications, AI must be limited to assisting with tasks within a trusted process framework, where the scope of AI’s impact is well understood. The process framework can be trusted when:
- The tasks and processes in which AI is utilized are clearly defined in terms of what the AI is expected and allowed to do.
- The AI is prevented from extending beyond the scope of these tasks by using guardrails.
- Ultimately, a human must be responsible for the design of the system. To achieve this, a human needs to evaluate and take ownership of the AI’s contribution.
Without question, AI can play a substantial role in accelerating the development of safety-critical systems. However, to maintain reliability and certifiability, AI needs to be used in ways that are subject to human oversight and discernment.
For example, AI can be used to generate test cases within the trusted process framework. When testing complex multicore systems, it’s impossible to test every combination of possibilities when evaluating timing coupling. Therefore, developers must select a subset of combinations to test that provide a comprehensive representation of the overall possibilities.
This is one area where AI can shine as AI agents may be able to suggest test cases that were overlooked or not considered. It’s important to note that AI isn’t being relied on to replace the developer and provide all of the suggested test combinations. Rather, the AI is used to catch possible blind spots. The developer then reviews the tests the AI has generated and adds relevant tests to the tests already selected by the developer. In other words, the AI builds on the developer’s expertise to create a selection of tests that’s broader than the developer might make on their own.
Another key area where AI can be used within a trusted process framework is automation of complex manual process tasks. In this sense, AI is less about having the AI produce outputs and more about defining processes that can accurately, reliably, and consistently managed by the AI agent. This approach fits neatly into certification workflows, especially if the AI can ask for help from a human when an unexpected result occurs.
AI also accelerates testing by being allowed to resolve well-defined problems. For example, if a particular combination results in a compilation error, an agentic AI can assess the problem, mitigate it to resume testing, and log the incident to enable human oversight. This has the advantage of preventing simple errors from disrupting and stalling an overnight testing campaign.
What is Worst-Case Execution Time Analysis?
WCET analysis is an implicit and necessary part of ISO 26262 compliance. In the ISO 26262 framework, the obligation to demonstrate that safety-critical software components don’t interfere with each other is expressed as Freedom from Interference (FFI). Coupling coverage analysis is a primary means by which FFI is evidenced at the software level. It’s achieved by showing that all identified coupling pathways (data, control, and timing) have been exercised and the observed behaviors are consistent with the system’s safety requirements.
The WCET of a task is defined as the maximum time that task could take to execute across all possible inputs, all possible system states, and all possible patterns of interference from concurrently executing tasks. WCET plays an important role in verifying that a system is deterministic and can meet all of its critical real-time deadlines.
It’s critical to note that WCET isn’t the longest time observed in testing. The actual WCET may be longer based on circumstances that weren’t explicitly tested. The difference between the maximum observed execution time and the actual WCET is where undetected deadline violations can occur and represents the risk in the safety-critical system (see figure).
Worst-case execution timeWorst-case execution time (WCET), defined as the maximum time that a task could take to execute, is an implicit and necessary part of ISO 26262 compliance. The difference between the maximum observed execution time and the actual WCET is where undetected deadline violations can occur and represents the risk in the safety-critical system.
In Part 2, we’ll explore how to accurately assess WCET, using agentic AI to automate and accelerate iterative testing, and how all of these concepts together can simplify and accelerate the certification of complex automotive systems that must meet strict safety standards.
Source: Electronic Design




