Github Copilot – an AI coding assistant – is trained on publicly accessible open source licensed code, which has led to concerns that it infringes on copyright or will be the ‘death of open source‘ by disincentivising open source development. This paper looks at how CoPilot is used and what problems it is good at solving. The paper mines Github issues and StackOverflow posts and finds that
- ‘Usage Issues’ (e.g. installation problems) and ‘Compatibility Issues’ (e.g. Editor/IDE Compatibility) are the most common problems faced by users
- ‘Copilot Internal Issues’, ‘Network Connection Issues’, and ‘Editor/IDE Compatibility Issues’ are the most common causes of issues
- ‘Bug Fixes’, modifications to ‘Configurations/Settings’ and ‘Use Suitable’ (e.g. setup/operation issues) are the most common solutions to issues.
The paper suggests that there remain some compatibility problems across various IDEs and editors and that there is still a lot of scope to improve the quality of the code CoPilot generates.