Gavin Reid

2025– · Independent project

IELTS Writing Task 1

AI-graded IELTS Academic Task 1 preparation, organised into hand-built lessons.

Architecture of IELTS Writing Task 1AI-graded IELTS Academic Task 1 preparation, organised into hand-built lessons. chart item bank 15 chart items chart data observable facts language banks learner writes 4 sentences tier classify target language essay rubric grading structured feedback GPT-4o · 4 calls rubric in Postgres Zod validate · soften

What it is

IELTS Academic Task 1 asks a candidate to describe a chart in at least a hundred and fifty words. This is a preparation product for that one task, and it is built as a six-step lesson rather than an essay-marking box.

A learner writes four sentences unaided, is placed into an internal tier from what they wrote, is given a bank of hand-authored target language for that tier, rewrites the sentences using it, then writes the full response and is graded against the four IELTS criteria. A closing summary compares the first attempt with the last.

The problem

A grader that only returns a band score teaches nothing. The learner already knew they were not good enough; what they need is the specific language that would have moved them, in their own sentence, on the chart in front of them.

The harder problem is honesty. Nothing here is calibrated against real examiners, so a declared band score would be a claim the system cannot support.

How it works

Lesson content is authored by hand as language functions, each with target-language phrases written separately for three tiers. The phrases are never generated. Chart items are imported separately and linked to lessons, so one lesson can be run against several charts.

Four model calls sit inside a lesson: tier placement, feedback on the rewrite, the graded response, and the summary. All four run in JSON mode against Zod schemas validated in code. Every system prompt lives in a database table rather than in the codebase, one active version per slug, so a prompt is versioned content rather than a code change. In practice each new version has still shipped as a migration, because nothing in the product edits prompts.

Two rules matter more than the model choice. The grader may never state a band outright: an architectural decision record fixes this, and a post-processing check detects any declarative band and rewrites it into directional language. And a chart-fact tolerance rule, written into the prompts, stops a learner being marked wrong for reading forty off a chart whose underlying value is thirty-nine, because the gridlines do not permit better.

Billing is webhook-driven with idempotent event handling, so a replayed Stripe event cannot double-credit an account.

What I’d change

There is no rater-reliability study. No agreement analysis against human examiners, no inter-rater figures, no confidence estimation. The scoring is plausible and unvalidated, and for an assessment product that is the most important thing missing.

The band rule is enforced where it is cheap and asserted where it is not: the declarative-band check is real code with tests behind it, while the chart tolerance is instruction text a model can simply ignore, with nothing measuring whether it does.

Monitoring is thin as well: console logging, no token-cost dashboard, no per-step error view.