Research indexJournal article
Efficient and Interpretable Code Analysis with Transformers
Mathur, Sandilya, Singh and 1 other
International Conference on Contemporary Computing7 Aug 2025
NobleIDark:/48914/w1/20260525/C3F647D1DOI10.1109/ic366947.2025.11290129
Efficient and Interpretable Code Analysis with Transformers
Mathur, Sandilya, Singh and 1 other · International Conference on Contemporary Computing · 2025 · Subscription
NobleIDark:/48914/w1/20260525/C3F647D1DOI10.1109/ic366947.2025.11290129
The full text is at the publisher
You can read the abstract here, or open it at the publisher.
Abstract
Modern software development benefits greatly from automated code analysis tools that can detect bugs and suggest improvements. In this work, we present a transformer-based framework for code analysis that emphasizes both efficiency and interpretability. Our approach leverages pre-trained models to understand the syntax and semantics of the code and includes mechanisms to visualize the model’s attention for transparent error explanations. The system focuses on Python code, integrating pre-trained models like CodeBERT with modules for attention visualization, semantic analysis, and code parsing to address four key challenges: handling long code sequences via sparse attention mechanisms, mitigating biases in transformer predictions through adversarial training, incorporating code structure using Abstract Syntax Trees, and optimizing energy efficiency via model quantization and pruning. By prioritizing transparency through attention visualization and scalability through fragmentation strategies, this work provides a foundation for lightweight, developer-friendly code analysis tools. The results highlight the viability of transformer-based approaches for practical code optimization tasks while underscoring the need for continued research on energy-efficient model deployment and cross-language generalization.