If you read the RPI site they'll tell you that if you put even
slightly worn tappets with a new cam, or vice versa, that they will return a very poor service life, but then again, it's not like they're trying to sell you anything

.
I put a second-hand cam in mine that had maybe 1200 miles on it, but it had most of the factory paint left on it even on the tips of the lobes, and no scoring or anything. I fitted it with new tappets and I expect it to last the same amount of time as any other cam.
Also check your rocker assemblys when you dismantle it. check for play on the rocker shaft. The insides of the rocker arm (where it slots over the shaft) should be an even colour, not worn. It's worth sliding them down the shaft (against the springs) to see if they've worn grooves in the shaft/inner bearing of the rocker.
If one or more rockers or the shafts are dodgy, it's probably best to change them all. If they DO look dodgy, and it's not a very high mileage engine, I'd get the oil pressure tested. If the rocker assemblys are starved, your tappets are starved too, as are the lobes on the cam.
Oh, and if you're doing the camshaft, do the chain as well. Duplec chains are best, but if they seem a little pricey (as they did to me) I got a 'Clevite' HD steel gears and chain - much better than the crappy standard resin ones.
One final thing - is the cam standard or performance? If you changed the lifters on a non-standard cam without resetting the preload, your push-rods may have some play, so they will rattle around in their seats when the valve is closed.
A leaking head gasket or exhaust gasket can cause a kind of chuffing/rattling noise. Mine did!
That's about the extent of my knowlege!!