According to a recent study, Software engineers who use AI systems to generate code are more likely to cause security vulnerabilities in the applications they develop. The paper, co-authored by a team of Stanford-affiliated researchers, highlights the potential pitfalls of code generation systems as vendors like GitHub begin to commercialize them in earnest.
«Currently, code generation systems are not a substitute for human developers»Neil Perry, PhD student at Stanford and co-lead author of the study, explains in an email interview. “Developers using them to complete tasks outside of their own areas of expertise should be concerned, and those using them to speed up tasks they are already adept at should carefully check the results and the context in which they are used in the game. general project.»
The Stanford study focused specifically on Codex, the AI code generation system developed by San Francisco-based research lab OpenAI. The researchers recruited 47 developers—ranging from college students to industry professionals with decades of programming experience—to use Codex to solve security-related problems in programming languages like Python, JavaScript, and C.
Codex is trained on billions of lines of public code to suggest additional lines of code and features based on the context of existing code. The system proposes a programming approach or solution in response to a description of what a programmer wants to achieve (for example, "Say hello world"), based on both the programmer's knowledge base and the current context.
According to the researchers, study participants who had access to Codex were more likely to write incorrect and "unsafe" (in the cybersecurity sense) solutions to programming problems compared to a control group. And what is even more worrying, they were more likely to say that their unsafe answers were safe compared to people in the control group.
megha srivastava, a graduate student at Stanford and the second co-author of the study, stresses that the results are not a complete condemnation of Codex and other code generation systems. For example, the study participants did not have security knowledge that would have enabled them to better detect vulnerabilities in the code. Other than that, Srivastava believes that code generation systems are useful for non-high-risk tasks, such as exploratory research code, and that they could improve their coding suggestions with fine tuning.
"Companies developing their own [systems], perhaps with more training on their internal source code, might fare better, as the model could be encouraged to produce results more in line with their coding and security practices"says Srivastava.
So how could providers like GitHub prevent developers from introducing security flaws using their code-generating AI systems? The co-authors have a few ideas, including a mechanism to "refine" user instructions to make them more secure, much like a supervisor going through and reviewing drafts of code. They also suggest that developers of cryptographic libraries ensure that their default configurations are secure, as code generation systems tend to stick to defaults that are not always exploit-free.
“AI assistant code generation tools are a really exciting development and understandably so many people are eager to use them. However, there are issues with these tools that need to be considered going forward… Our goal is to make a broader statement about the use of code generation models.”says Perry. "We must continue to explore these problems and develop techniques to solve them."
In Perry's opinion, the introduction of security vulnerabilities is not the only flaw in code-generating AI systems. At least some of the code Codex was trained on is under a restrictive license; Users have been able to ask Copilot to generate Quake code, code snippets from personal codebases, and sample code from books like "Mastering JavaScript" and "Think JavaScript." Some legal experts have argued that Copilot could endanger companies and developers if they inadvertently incorporated copyrighted hints of the tool into their production software.
GitHub's attempt to rectify this is a filter, first introduced on the Copilot platform in June, that checks code hints against their surrounding code of about 150 characters against GitHub's public code and hides the hints if there is a match. or "near coincidence." But it is an imperfect measure. Tim Davis, a computer science professor at Texas A&M University, found that turning on the filter caused Copilot to output large chunks of its copyrighted code, including all license and attribution text.
"[For these reasons,] we strongly caution against using these tools as a substitute for educating beginning developers on sound coding practices," Srivastava added.
