It takes huge powers of memory and concentration to deal with all the math you need to learn as a base, and also you need to dive very deeply into one topic to learn enough to contribute something new. And be gifted enough to develop a new insight.
Those abilities decline naturally, but also take a big hit if you're not living like a monk. People still get insights as they mature, and likely have a better ability to see the big picture, but their mental strengths are changed.
How many people with wife, kids, etc in their forties and fifties think they have the same ability to focus as a twenty year old? Where are those damn car keys anyway?
There was a part in Cryptonomicon, a novel partly about a code breaking mathematician, where he concluded that he had better get married to a woman he found distracting, and get himself properly laid, or else the war effort might suffer. (Meant to be funny, but a bone of truth.)