I have done that hike in a day its a easy hike. One thing you could do is hike down in one day and use the other 2 or 3 days to get back.
Or park in the valley and catch a ride up to TM. I have hitched quite a bit in Yosemite and have never had more than a few cars go by before getting a ride. I prefer to catch the ride at the begining so I come back to my truck when I finish, it makes going home easier.
I may be up there that week, I am in the north bay.