Amazon announced a preview of an automated coding assistance tool called CodeWhisperer at its re:Mars conference in Las Vegas on Thursday.
CodeWhisperer is available to those who received an invite this week through the AWS IDE Toolkit, a plugin for code editors to help write AWS applications.
In a blog post, Jeff Barr, chief evangelist for AWS, said CodeWhisperer’s goal is to make software developers more productive.
“CodeWhisperer will continually examine your code and comments and provide you with syntactically correct recommendations,” Barr said. “Recommendations are synthesized based on your coding style and variable names, not just snippets.”
The service is free during the preview period, as is Microsoft’s GitHub Copilot, which now costs $10 a month.
According to Barr, CodeWhisperer analyzes various signals to suggest appropriate bits of code, including cursor position, previous code, comments, and code in other project files. The software, he said, is trained on billions of lines of code in open-source repositories, internal Amazon repos, API docs, and online forums.
Therefore, the same issues as Copilot can arise in terms of code quality and potential licensing concerns. The academic community has already started looking into the performance of AI coding tools like Copilot in terms of code quality.
So far the answer is not that good. For example, a research paper from 2021 [PDF] by NYU computer scientists found that 40 percent of 1,689 programs built using Copilot suggestions produced code with known weaknesses. But continued attention and iteration for these systems should help them eventually improve.
Another recent paper entitled “Is GitHub Copilot a Substitute for Human Pair-programming? An Empirical Study” concludes that Copilot accelerates the production of poor quality code.
The answer might be more AI, at least for those who have farmed on machine intelligence. Amazon already has a service called CodeGuru that has been trained through machine learning to find vulnerabilities and automate the code review process. Once you’re in the business of fixing bad code, why not provide an adequate supply?
Amazon has attempted to address concerns that proposed code violates someone’s copyright or license terms by showing this information.
“CodeWhisperer’s reference tracker detects whether a code recommendation might be similar to certain CodeWhisperer training data and can provide you with those references,” explains the service’s FAQs. “This way you can easily find and review this reference code and how it is being used in the context of another project.”
Amazon promises not to use developer-written code to train its machine learning models — during the preview period — though it does measure developer metrics like acceptance and rejection of code suggestions to improve the service’s performance. After the preview is complete, who knows, but the specificity of Amazon’s exclusion suggests that the company will begin scanning customer code to improve its ML model.
Barr provides some examples of CodeWhisperer’s capabilities. He describes how he typed the Python comment line
# See if a number is prime and CodeWhisperer responded with a function to do that calculation.
CodeWhisperer also understands how AWS services work and is therefore able to generate potentially useful suggestions for those building on the AWS platform. Presented with the comment
# create S3 bucketCodeWhisperer generates the appropriate steps.
https://www.theregister.com/2022/06/23/amazon_codewhisperer/ Amazon debuts CodeWhisperer, an AI programming assistant • The Register