It appears that you have never driven in Boston, New York, LA, or San Francisco.
That said we have rented for 3 out of our 5 trips, either way works.
The taxi's have never been an issue. The Buses run regular, are safe, and clean.
Rented once from Avis, once from Hertz, and in very shortly our second time from Thrifty. Never had an issue.
Yes, most things are close, but the convince of having a car on hands outweighs the wait for taxi's or buses. At an average of less than $30 a day IMHO it is the way to go for me.

Have Fun, Pete