A 10-Year-Old CNN Just Beat GPT-6 and Claude at Diagnosing Skin Cancer
Frontier models are losing to a decade-old architecture fine-tuned for twenty minutes on a single GPU.
Models · Source: Hacker News
What happened
A dermatologist tested two top-tier AI models against a decade-old neural network. Dr. Magnus Lynch pitted GPT-6 Astra and Claude Fable 5.1 against Inception v3. The task was diagnosing skin cancer from 200 biopsy-proven lesion images. These images came from the MILK10k dataset. The old model won by a landslide.
The CNN identified the correct diagnosis out of six possibilities 62 percent of the time. The frontier models tied at a dismal 42 percent. The CNN was fine-tuned for just twenty minutes on a single H100 GPU using 68,011 public dermatology images. It is small enough to run entirely in a phone browser using WebGPU. No patient data ever needs to be uploaded.
The LLMs failed in highly predictable ways. GPT-6 Astra over-indexed on basal cell carcinoma, guessing it 75 times when only 25 existed. Claude Fable 5.1 defaulted heavily to actinic keratosis. When prompted as a patient asking if they should see a doctor, both LLMs simply said yes to almost everything. They told patients to see a doctor for harmless lesions just as often as they did for melanomas.
Key facts
- 62% — Accuracy of the 10-year-old CNN on 200 lesions
- 42% — Accuracy of both GPT-6 Astra and Claude Fable 5.1
- 20 minutes — Time it took to fine-tune the CNN on a single H100 GPU
- 68,011 — Number of training images used for the CNN
Why it matters
Generalist models are not magic bullets for specialized medical tasks. Builders keep trying to shove massive LLMs into every product. This test proves that a targeted, narrow architecture still dominates specific visual classification problems. You do not need a trillion parameters to solve a well-defined problem. The CNN used here is deliberately not even state of the art. Modern specialized skin cancer AI uses ensembles of image transformers and performs even better.
Healthcare AI will likely bifurcate based on these realities. Patient-facing chatbots will handle triage and communication using LLMs, but the actual diagnostic heavy lifting will remain with specialized, smaller models. Startups building medical AI can stop burning cash on frontier model API calls for image classification. They should just train a CNN. The cost savings and performance gains are too large to ignore.
For builders
Stop defaulting to frontier LLMs
Do not use expensive API calls for narrow classification tasks. A small model fine-tuned on clean data beats a massive generalist every single time. Founders save compute costs and get better accuracy by using the right architecture.
WebGPU enables local medical AI
The winning CNN runs entirely in a phone browser without uploading anything. This means zero data leaves the user device. Startups can build privacy-first diagnostic tools without dealing with complex compliance for cloud storage.
LLM safety filters ruin utility
The frontier models advised seeing a doctor for almost every lesion to avoid liability. If your product relies on an LLM for triage, you will flood doctors with false positives. Patients pay the price in anxiety, and clinics lose valuable time.
My take
I see founders burning millions trying to prompt-engineer generalist models into doing specialized work. It is lazy engineering and terrible business. If a ten-year-old CNN fine-tuned in twenty minutes beats the most expensive models on earth, you need to rethink your tech stack. Stop treating LLMs like a hammer for every nail. Build the right tool for the job.
Original reporting: Hacker News. This is my rewrite and opinion.