Bebawy, Yosab

Safety-critical real-time systems that rely on AI-based object classification and detection for critical decision-making require strict guarantees of reliability and timing correctness, as even minor deviations in behavior can lead to catastrophic consequences. Integrating AI hardware accelerators into such real-time environments therefore demands not only high computational throughput but also predictable and analyzable execution behavior. Conventional accelerators, such as the Versatile Tensor Accelerator (VTA), meet performance requirements but fall short in predictability due to their event-driven execution model and limited timing control. This variability introduces significant risks in scenarios where timely and deterministic inference is essential. To address these limitations, this work adopts a time-triggered execution paradigm for AI acceleration. It first introduces the Time-Triggered VTA (TT-VTA), which replaces the non-deterministic behavior of the original VTA with a deterministic counterpart based on statically defined, pattern-based timing schedules. These schedules are generated offline at design time using a scheduler with a built-in simulator for the VTA architecture. The scheduler further integrates additional simulation frameworks: DRAMSim2 is used to model memory-related instructions, while a cycle-accurate simulator captures compute-bound operations. Together, these components enable precise timing analysis, deterministic resource utilization, and predictable execution behavior. Evaluation with a CNN classification model shows that the TT-VTA preserves classification accuracy while ensuring predictable execution, however with a slight increase in inference latency (53 ms compared to 51 ms for conventional VTA), indicating room for further optimization. Building on this foundation, the Superscalar Time-Triggered VTA (STT-VTA) is proposed to improve performance without sacrificing determinism. STT-VTA extends the architecture by exploiting parallelism through multiple coordinated accelerator functional-units, whose configuration is determined at compile time via an enhanced architecture management and scheduling framework. This framework analyzes the computational structure needed for the target AI model and generates optimized schedules that maintain strict timing guarantees while increasing throughput. Experimental results demonstrate that the STT-VTA achieves the same classification accuracy as both TT-VTA and conventional VTA, while reducing inference latency by 17-41% compared to the TT-VTA. With the STT-VTA, another challenge arose in managing memory-intensive workloads, which introduce performance bottlenecks. To address this, the work further incorporates deterministic caching and pre-fetching strategies, leading to the Superscalar Time-Triggered VTA with Caching and Pre-fetching (STA-CP). Unlike conventional dynamic approaches, these mechanisms are integrated directly into the static scheduling process. At compile time, all tensor operations and memory transactions are fully specified, enabling predictable data movement and efficient overlap between computation and memory communication. By eliminating redundant memory accesses and hiding memory latency within a fully analyzable execution model, STA-CP achieves improved throughput while preserving strict temporal determinism. Experimental evaluation on different CNN classification models demonstrates up to a 30% reduction in deterministic inference latency compared to TT-VTA. Overall, the proposed STA-CP architecture demonstrates that the integration of time-triggered execution, controlled parallelism, and deterministic memory-management techniques can simultaneously achieve high performance and predictable execution behavior. These characteristics are fundamental for deploying AI accelerators in safety-critical real-time systems, where both computational correctness and strict timing guarantees are essential.

Bebawy, Yosab

Filters

Settings

Sort By

Results per page