I'd skip the car, unless there is something specific in the suburbs you want to see for a day.
I've driven in lots of big cities, Chicago, LA, Houston etc... too, and you'd be fine managing the drive, but the Toronto traffic is horrendous at times. Particularly if you decide to stay downtown...