Hey SkyRider,
The $100.00 transfer fee is quite common with the banks, just another service charge in their long list.
What Bank B told you is correct, any withdrawal from a TFSA cannot be re-invested until the following year. TFSA's were introduced in 2009. Maximum annual contributions per year is $5,000.00 so if someone has never made a contribution to their TFSA to date, they could invest $15,000.00 ( 5 grand for '09,'10,'11)today in one lump sum or any portion of. If one doesn't make their maximum or only makes a partial contribution in any year, that unused contribution room carrys-forward indefinately until used.
It sounds like you have only contributed $5,000.00 to your TFSA so far. If that's the case you could cash out you TFSA, receive the cash or better yet just get them to deposit it into your chequing account, it's less hassle when re-investing at the other bank to write a cheque as they like to see the source of the funds, and re-invest it as a new contribution based on your unused carry-forward room.
If you are not sure what you have contributedin the past, you can refer to your notice of assessment from CRA as they are tracking this info.