Depending on the server, an e-mail account can be purged if it's inactive for a specific length of time.
@barnacler: If your account was purged because of continuous inactivity, the e-mail name became available to anyone signing up for a new account. The only connection to you is if someone sent an e-mail to that address, in the belief that you would be the recipient. Any saved e-mails and contacts would have been deleted during the account purge.
The only ways that someone else could access your saved e-mails and contact list is if they hacked your password, or if you last used that account on a public computer, and didn't sign out of the account when you left. That happened to me with a Yahoo mail account about ten years ago. From time to time, I get a junk e-mail from "Me", with .ccs for some or all of the people who were on my contacts list for that account. It was an account I used only for messaging with my students, so it was no big deal to me.