There's no definitive answer as to how long it takes, but it does take years. If you find a fresh cigar with "plume" on it, you need to take a harder look. Every couple months a thread pops up with people asking the plume vs mold question... if you have to question, it's mold more than likely or possibly something in the humidor and/or active humidifcation system landing on the cigar.
And it doesn't really affect the taste, but it sure can make the cigar more enjoyable when you dig in your humi, pull out a box that's been sitting for awhile only to see some fine crystal jewels on your cigar! LOL