What are some must-sees in Atlanta?

So I'll be in Atlanta next week, with a bit of time free to explore the area, and I would love peoples' recommendations of things to see. I am fond of nature scenery and museums and quirky little offbeat places, and less so of anything swarming with tourists. (For instance, if an out of town visitor were coming to SF, I'd take them exploring along the coastline and to the ScharffenBerger chocolate factory, and I'd steer them well clear of Fisherman's Wharf and Ghirardelli Square and that sort of thing). Recommendations of indoor activities would be a plus, as I hear the forecast is for rain.

Likewise, any must-try local foods (keeping in mind I'm vegetarian)? (Again, as an example, I'd tell someone visiting NYC that they needed to try some NY pizza and a bagel and a pushcart knish and some of those little sweet steaming praline things).

Thanks!