Elsie Item-Writing Academy
About 45 minutes · Academy module: Flaw Clinic: Step 1 — Flaws That Corrupt the Vignette Item
Unofficial faculty-development resource. Not affiliated with, endorsed by, or representing the AAMC, NBME, USMLE, LCME, or NRMP. Supports faculty-development documentation.
At the Step 1 level, vignette items dominate, and the flaw taxonomy shifts accordingly: the flaws that matter most are the ones that interact with clinical reasoning. A Step 1 item with five options has more room for error — and more places for a flaw to hide. This module covers the full taxonomy with Step 1 emphasis.
Unfocused stems. The classic Step 1 failure is the stem that ends with "Which of the following is true?" after a 150-word vignette. The student cannot form a hypothesis before reading the options, so the item tests recognition rather than reasoning. A focused stem converts the vignette into a question with one task: "Kidney biopsy is most likely to show which finding?" Lead with the question; let the vignette serve it.
Two defensible answers. This is the most expensive flaw at the licensure level, because it survives review and reaches scoring. Overlapping syndromes (e.g., two causes of primary hypothyroidism that share most features) create items where both the key and a distractor can be defended from the stem. The fix is discrimination: add or sharpen the one finding that separates them — a lab value, a time course, a physical sign — until only one option survives scrutiny. Every model rewrite in this track passes two independent cold solves before it ships.
Negative stems without emphasis. "Which of the following is not a feature of…" buried in lowercase is a reading-comprehension trap, not a basic-science question. Reserve negative stems for the rare case where ruling out is the clinical task, and then emphasize the negation: EXCEPT, NOT, LEAST in capitals. Better still, rewrite as a positive: instead of "all of the following are seen except," ask "which finding best supports the diagnosis."
Absolute terms. "Always" and "never" have no place in biology or medicine. They invite the test-wise to eliminate and the knowledgeable to object — either way, the item measures something other than the intended objective.
"All of the above" / "none of the above." In a five-option Step 1 item, these options degrade both the item and its statistics: a "none of the above" key makes every distractor analysis uninterpretable, because you cannot tell which misconception a student held — only that they rejected all five. Write five genuine options.
Window dressing. The vignette should contain exactly the findings needed to reason to the answer: no more, no less. Extraneous social history, irrelevant vitals, and decorative comorbidities slow strong students and give weak students random cues to latch onto. If a detail does not discriminate between the key and at least one distractor, cut it.
Clueing between stem and options. Watch for the stem's phrasing reappearing in the key ("Kimmelstiel–Wilson nodules" in both stem and option) while distractors use different vocabulary. Match the register of all five options to the stem.
Grammatical cueing and implausible distractors. Read every option into the stem to catch article and agreement giveaways, and retire any distractor that no student with a real misconception would select — a throwaway option in a five-option item silently converts it to a four-option item.
---
Flawed item (intended key: E)
A 54-year-old man with long-standing type 2 diabetes is found to have a serum creatinine of 2.1 mg/dL (baseline 0.9 mg/dL) and 3+ proteinuria on urinalysis. Fundoscopic examination shows background diabetic retinopathy. Which of the following is true?
A) Immune complex deposition in the mesangium B) Segmental glomerulosclerosis with tip lesions C) Linear IgG deposition along the glomerular basement membrane D) Wire-loop lesions on immunofluorescence E) Mesangial expansion with Kimmelstiel–Wilson nodules
Flaw taxonomy label: Unfocused stem.
Correction: "Which of the following is true?" is not a question — it is an instruction to go read the options and work backward. The student cannot form a diagnostic hypothesis from the stem alone, so the item rewards recognition over reasoning. The vignette already converges on one task (predict the biopsy); the rewrite simply asks it, turning the same five options into a test of pathophysiologic reasoning instead of a matching exercise.
Model rewrite — cold-solve verified: 2/2 (solver 1: E, solver 2: E)
A 54-year-old man with long-standing type 2 diabetes is found to have a serum creatinine of 2.1 mg/dL (baseline 0.9 mg/dL) and 3+ proteinuria on urinalysis. Fundoscopic examination shows background diabetic retinopathy. Kidney biopsy is most likely to show which of the following findings?
A) Immune complex deposition in the mesangium B) Segmental glomerulosclerosis with tip lesions C) Linear IgG deposition along the glomerular basement membrane D) Wire-loop lesions on immunofluorescence E) Mesangial expansion with Kimmelstiel–Wilson nodules
Key: E
Flawed item (intended key: A)
A 19-year-old woman presents with several months of fatigue, weight gain, cold intolerance, and constipation. Physical examination reveals dry skin and delayed relaxation of deep tendon reflexes. Laboratory studies show TSH 18.2 mIU/L (elevated) and free T4 0.5 ng/dL (low). Which of the following is the most likely diagnosis?
A) Hashimoto thyroiditis B) Iodine deficiency C) Subacute thyroiditis D) Graves disease E) Toxic multinodular goiter
Flaw taxonomy label: Two defensible answers.
Correction: The stem establishes primary hypothyroidism but contains nothing that separates Hashimoto thyroiditis from iodine deficiency — both produce exactly this picture, and a content expert could defend either. An item with two defensible answers has no key; it has a dispute that will survive into scoring. The fix is discrimination, not rewording: add the one finding that separates the overlapping syndromes. A diffusely enlarged firm gland plus strongly positive thyroid peroxidase antibodies makes Hashimoto definitive and excludes iodine deficiency, so only one option survives scrutiny.
Model rewrite — cold-solve verified: 2/2 (solver 1: B, solver 2: B)
A 19-year-old woman presents with several months of fatigue, weight gain, cold intolerance, and constipation. Physical examination reveals dry skin, a diffusely enlarged firm thyroid gland, and delayed relaxation of deep tendon reflexes. Laboratory studies show TSH 18.2 mIU/L (elevated), free T4 0.5 ng/dL (low), and strongly positive thyroid peroxidase antibodies. Which of the following is the most likely diagnosis?
A) Subacute thyroiditis B) Hashimoto thyroiditis C) Graves disease D) Toxic multinodular goiter E) Iodine deficiency
Key: B
Flawed item (intended key: C)
A 6-year-old boy develops periorbital edema, massive proteinuria, and hypoalbuminemia two weeks after an upper respiratory infection. Which of the following is not expected in this patient?
A) Hyperlipidemia B) Lipiduria C) Hematuria with red-cell casts D) Proteinuria exceeding 3.5 g/day E) Hypercoagulability
Flaw taxonomy label: Negative stem without emphasis.
Correction: The negation ("not") is buried in lowercase mid-sentence, so the item punishes the student who reads quickly rather than the student who misunderstands nephrotic syndrome. The content is sound — minimal change disease is the classic pediatric nephrotic syndrome, and red-cell casts point toward a nephritic process — but the ask must be unmissable. Capitalizing EXCEPT is the minimum repair; the stronger move, shown in the rewrite, keeps the emphasized negation because ruling out is the actual clinical task here (distinguishing nephrotic from nephritic features).
Model rewrite — cold-solve verified: 2/2 (solver 1: C, solver 2: C)
A 6-year-old boy develops periorbital edema, massive proteinuria, and hypoalbuminemia two weeks after an upper respiratory infection. All of the following are expected findings in this patient's condition EXCEPT:
A) Hyperlipidemia B) Lipiduria C) Hematuria with red-cell casts D) Proteinuria exceeding 3.5 g/day E) Hypercoagulability
Key: C
---
Recording it adds the module to your Academy completion record on this device — module, track, level, and date, ready to download from your account page for faculty-development documentation.