From what I remember Horseshoe Meadows is 29 miles up the Horseshoes Meadows road. Since you drive up from about 5000 feet to 10000 feet there are many switchbacks and you can't really drive fast.
http://outdoors.webshots.com/photo/290536282/1290555993061435028VZUOMW From the Horseshoe Meadows road/ Whitney portal road intersection to the portal is around 9 miles.
So the total drive from Horseshoe Meadows to the Portal is about 38 miles, much on steep and windy road. I am positive the drive is more then 45 minutes and a little less 60 minutes.
I used to stay there often, but the long drive started to wear on me. In spite of the elevation advantage, nothing can beat the convenience of the Whitney Portal Campground. And I didn't like getting up at 2:30 am to get started at 4:00 am.