How Do I Make My Husband Feel Secure In Our Marriage?

He has some inferiority complex. It will stay with him. You have to support him in every corner of life. Don't let him feel alone. He needs as much care as a child because he is older than you and may be you are very pretty. It is in his mind that you will leave him for some beautiful person.

Read his mind and act according to that. What things make him feel insecure? Do not do things which he dislikes.

Wow, what an interesting question.  Let me ask you one back.  Do you find yourselves discussing your previous marriage often? If so, does he seem fixated on what your ex did wrong before you left?  

I am asking, simply because perhaps he fears he is making similar mistakes.  Or he cannot help but compare his strengths and weaknesses against your ex, or most likely a combination of the two.

If you have kids with your ex, then what I suggest is a little more difficult (regardless, you may want to suggest to him that you head off any problems by seeing a counselor... Be sure you let him know that it is so you can grow as a couple, not to prepare for the END).
Now, my suggestion, other than the counseling is that you set up boundaries (much easier if no kids from previous marriage).  When you find yourselves turning to the subject of your ex, you simply say I refuse to compare you two, you are two completely different individuals, and I am with the man I want to be with... Then turn the tables and steer the conversation to your relationship such as:

"Tom" I don't want to talk about "Mitch" or compare you two.  You both are very different people, and I have already made my choice between you...  What can we do right now, to bring us closer together?  
Or
That part of my life is past, and I am quite enjoying the life and man I have right here.  And this what I want o focus on... You and I.  
I hope this helps.