Question about whether AI training data really needs constant purging

I keep seeing people argue that we should delete old AI training data to fix bias and accuracy problems, but that feels backwards to me. Last month I ran tests on a local model that kept 2019-2022 web content, and it actually handled new slang and tech terms better than a version where I filtered everything older than 2023. Maybe it's just me, but throwing away decades of context for the sake of freshness seems like we're losing more than we gain. Has anyone else tried keeping full historical datasets and found real downsides, or is this purge trend mostly hype from vendors selling storage?

1 comments

Log in to join the discussion

Log In

1 Comment

patel.phoenix2mo ago

Man, that's a really interesting point! I was just reading some research paper last week about how current training data gets stale way faster than people expect. The guy was showing how models trained on the last few years alone start making dumb mistakes about basic stuff, like mixing up historical events or using words wrong because they never saw older usage patterns. Honestly, your test matches what I've seen in some open-source communities too, where people refuse to cut old data and their models just feel more grounded. The whole purge trend does smell like a marketing trick to me, especially since big vendors make money off telling you your data is outdated garbage. Maybe the real fix is smarter weighting of old vs new info, not throwing the old stuff in the dumpster. Great experiment, keep messing with it.

3