In anguish - please help!!?

I've been married to my husband for 4 years. We were high school sweethearts who reconnected 20 years later. It was a mistake - I jumped into the marriage too quick and now regret it. The thing is, he is very sweet and good to me and loves me - but I am not in love with him. Our personalities are too different and I am constantly frustrated. I'm not physically attracted to him and thought I would get over that - but have not. He's kind of a "numbskull" and loses everything and seems confused at times. He's a slob - but is not a lazy person. He can't make a decision on his own and has to ask me about EVERYTHING which to me, is a real turn off in a man. I don't respect him like I wish I did. I don't know what I was thinking when I married him. I want to tell him that I want a divorce but can't bring myself to hurt him. He's a good man and deserves a wife who loves him completely and feels close to him. I am so afraid to hurt him. He will hate me for this.