What does it actually mean to 'upvote' or 'downvote', with the red and green squares beneath the avatars? Does it effect the 'thumbs-up' count? Does the person who replied, notice the vote?

The upvote means you liked the answer and gave it a vote. The red means you didn't care for it. Actually, when you " Thank writer ", that's what actually shows to the person who answers. Not how you voted or if you did. They kind of go hand in hand with each other.