NobleBlocks

Research indexJournal article

Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot

Siddiq, Samee, Azgor and 3 others

2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE)1 May 2023

NobleIDark:/48914/w1/20260525/6D5678A3DOI10.1109/nlbse59153.2023.00018

Paywalled

The full text is at the publisher

You can read the abstract here, or open it at the publisher.

Read at the publisher

Abstract

Code generation models are gaining popularity because they can produce correct code from a prompt, speeding up the software development process. GitHub Copilot is currently one of the most commonly used tools for code generation. This tool is based on GPT3, a Large Language Model (LLM), and can perform zero-shot prompting tasks i.e., tasks for which the model is not specifically trained. In this paper, we describe a preliminary study that investigates whether GitHub Copilot can predict the runtime complexity of a given program using zero- shot prompting. In our study, we found that GitHub Copilot can correctly predict the runtime complexity 45.44% times in the first suggestion and 56.38 % times considering all suggestions. We also compared Copilot to other machine learning, neural network, and transformer-based approaches for code complexity prediction. We observed that Copilot outperformed other approaches for predicting code with linear complexity $\mathbf{O}(n)$ .

Discussion

Citations

Topics

More

Suggest a correction or report