If you watch any of the home improvement shows, anytime they find mould they remove it and any wood to which it is attached.
The reason being, that the mould spores become airbourne when dry. It is these airbourne spores that can be dangerous if inhaled.