AI Code Review

Following GPT-4’s introduction in March 2023, we began incorporating AI-powered review into our coding exercise interface from that Autumn. The reasoning behind this has been severalfold:

• Firstly, offering greater flexibility to our learners – allowing them to make timely progress when mentors aren’t around, and quickly resolving minor issues. This was seen as a priority for many of our learners, especially those who – for personal or practical reasons – do their learning outside of typical hours (e.g. working night shifts, caring responsibilities, etc.)
• Secondly, it expands the range of useful feedback which we can provide to learners beyond pass/fail – highlighting best practice, or pointing out minor issues with e.g. code formatting to help with reading, misplaced brackets (a pain point for new programmers)

During the initial build of our guided editor we ran into a significant challenge: “How do we test that our learners are using the precise language syntax that the exercise requires?” For example, in an early JavaScript exercise we expect our learners to write comments and declare variables using the correct keywords (let and const). Pre-GPT, in phase one, our solution for this was necessarily complex: converting the code into an ‘abstract syntax tree’ (AST) then querying its structure.

Switching to GPT-4 has allowed us to lean on OpenAI’s inbuilt understanding of code, and build our exercises using natural language rather than querying the AST. It has also allowed us to think more creatively about how we guide the learner towards improving their submitted code. The current Code Review iteration uses a multi-step “chain of thought” prompt which reasons through the process of alternatively solving, comparing, and providing guided feedback on a learner’s exercise submission.

Walkthrough

Live Collaboration

Participants’ own exercise submissions are live and interactive on a unique URL, shareable through chat (WhatsApp, Slack, etc.) or wherever group discussion takes place.

Screen Recording 2025-01-06 at 10.46.47.mov

New features currently in progress (early 2025):

• In-app chat and shared profiles, and a message hub for accessing previous conversations/profiles.
• Live ‘match-making’ to bring learners and mentors together to tackle the same exercise.
• Mini-projects (on a shareable ‘showcase URL’)