I was talking with a friend yesterday as we transported a few computers to his car. Some old systems we were going to put together for a cluster of sorts - though quite a heterogenous system at that. The one system - and early Mac system with a 68000 processor - was being transported and he was debating getting rid of it. The system, however, will still run Linux quite well.
We mused however, as the Linux folks will always - or nearly always at least - be able to cry "But it'll still run Linux!" as even the latest kernel series (2.6) will still run a ton of old hardware. Yes, you may have to specially build it, but you can usually find a distribution for some of them, and then build to the latest if you desire. Regardless, the memory requirements of the kernel will still do very well for older processors and computers. No, it doesn't necessarily mean that one would be able to run the latest and greatest distro that comes out - likely not. But with some crafting, most software - even X Windows - is still fully usable on older hardware.
So, when will it ever be said "But it'll still run Windows?" I doubt one will ever hear that said and still be able to run what Microsoft is just releasing.
So if you have some old hardware to play with, then check out the competition. Learn a bit more (or a lot more) about computers and their uses. Linux is coming around. It'll extend the life of older computers for quite a long time. (My Sun IPC is still supported by the 2.6 series. It was built in 1991.) And perhaps you'll find that you like it well enough to run it on your main computer.
But, if nothing else, it'll give you or your kid something fun to do, and teach the better side of computers and show you just how safe a computer can really be.