I'm not that far South, but in West Virginia, there are many historical sites to visit....too many to list! If you're coming in the fall, it'll be that much better with the leaves changing! As far as good food....plenty of places to find authentic regional food!
Charleston, South Carolina. I wrote a long answer, but it got tagged for bad language for no reason.