From the investors point of view, Madof had been around for a couple decades, so people must have thought he was legit.
Our society always likes to blame the victim, allot of these people were regular joes who worked all their lives and knew nothing about investing or that they should spread their money around, it's easy for savy investors to say they should have spread it around.