Yes they can, but the banana's may stick to their teeth and cause tooth decay. They also like fresh strawberries, pears, apples, lettuce, and many more vegetables, as well. These should only be given as an occasional treat though as they are by nature grain, grass, and hay waters.
Yes ponies can eat bananas. Yes you can also give them carrots, apples,light oats, whole grain. But do remember not to feed them sugary, sweet cereals etc.