Most of the buildings I mentioned above are in Oranjestad and within walking distance. However, keep in mind, that it is very hot in Aruba. Having a rental car with air-condition will be so much more convenient. Going on a little sight-seeing tour by car might be fun.
Here is a google map of Oranjestad to give you an idea. There is lots to see. The old Parliament building for instance, which across from the Yacht haven. And the Yacht haven itself. There is the Seaport Marketplace Mall with little restaurants and cafes and much more:
"The California Lighthouse" ~ It's still being renovated, not sure, if the work will be finished by the time you arrive. But, IMO, it's definitely worth to visit, the view is spectacular! There is as well a nice restaurant next to the lighthouse: