Loading…
17
The expert who swore old training data would ruin my fine-tune
A guy on a Discord server with like 40k karma told me last fall that fine-tuning on my own niche dataset was a waste because the base model already knew enough. I listened, spent 3 weeks on prompt engineering instead, and got nowhere with my niche legal document summaries. Finally I tried his exact counter-advice anyway, using just 200 docs I annotated myself over two weekends. The fine-tuned model beat my best prompt setup by a ridiculous margin, cutting errors from 18% to 6% on my test set. He actually came back and said his claim was based on a paper from 2022 that the field has since moved past. Has anyone else had a big-name community figure give advice that was outdated by the time it reached you?
1 comments
Log in to join the discussion
Log In1 Comment
ivan_perez6d ago
Friend got burned by the same outdated paper trick.
1