GitHub Copilot

GitHub Copilot is a game-changer when it comes to increasing the speed and efficiency of development. It autocompletes code (functions, tests, comments, etc.) and reduces the number of user-inputs. Copilot is also great for developing POCs and prototyping to rapidly test whether an idea is feasible and suits business needs, making it ideal for early product iterations. Additionally, Copilot offers good context awareness through understanding project content and providing suggestions intelligently. It also works across several programming languages, including Python and JavaScript, which are used heavily by my team. Copilot also integrates seamlessly with several IDEs. My team uses VS Code and Copilot integrates easily with VS Code's UI.

I like how it can make tedious work way faster, and while inconsistent it can be useful sometimes to explain some things in the code base but you still may want to cross check with your team or have some context yourself and always take the advice with some skepticism.

Easy to set up, easy to get into. Like most Saas companies, the tech stack is quite common: MVC framework in Rails/Django/Spring/Node. Co-Pilot does well at setting up the boilerplate structure to get from 0 to 1 and does a decent job of helping one understand code paths as well.

GitHub Copilot excels at suggesting solutions to simple, pattern-based code, but struggles with complex, algorithms or system architecture. So, while it is not suitable for all use cases, it does streamline the process of generating repeatable code. Code generated by Copilot requires human review as its suggestions can be inconsistent and non-performant at times. Use of Copilot is great for advanced engineers with the ability to evaluate whether Copilot's output is valid, but may increase the risk of faulty code when used by junior developers. Finally, Copilot can struggle with context-awareness in large, multi-file projects. It is great at understanding context in simple projects, but can struggle with larger projects.

There is a lot of inconsistency when it comes to the willingness of the AI agent to process a request, when asked the same prompt even if it did it before. It might refuse to do it after or give an entirely different answer. The auto complete feature is intrusive and a lot of times the suggestions do not make sense at all, and it can be super useful for straightfoward tedious work but for everything else we found more productivity in just turning that feature off. Additionally, sometimes advice doesn't really answer the question if it's an elaborate one. Lastly, it can break the compilers in certain IDE's when it's enabled, and the only way to really fix this is to disable, restart the IDE and enable copilot again.

unlike other tools out there, co-pilot did not do a great job at detecting potential security issues and exploits. For example, it failed to call our a SQL injection attack vector when we allowed arbitrary search parameters for a search REST endpoint. It also didn't know about supply chain attacks and known CVE threats with some experimental open source libraries we were testing. although it's good at boilerplate suggestions, we dont have great confidence that it can actually tell the difference between good code and common code. We noticed in our monolithic repo, it was repeating anti-patterns in the way we set up large unit tests rather than suggesting more maintainable, DRY-er ways to approach it altogether.