Hache,
I was in a similar-sounding situation about 8 years ago when I decided to divorce my wife. For a while there was lots of guilt, self-doubt and confusion but the more time passed the more I realized it was the right move.
In my case she just wouldn't let me live my life. Nag, nag, nag, and a big guilt trip if I ever did something which was for me only like go out with friends, gamble, etc. As it was I went out only 1 or 2 nights a week and even that was too much. They were making fun of me at work because of how henpecked I was. That started me thinking that there may be another way to do things.
Then there was about a 3 year period of depression followed by a mid-life crisis followed by a womanizing period (at one point I had 4 girlfriends at the same time) and then a search for the right one with the idea of settling down. I think I've finally found the right one now and she's 3 months pregnant so things are looking up at the moment. But the future always holds many surprises (thank God, otherwise life would be boring) so I can't say for sure whether this one's forever either but I can say that I'm hopeful.
What I learned from all this is that you really have to know yourself well to know who you will be happy with.
I think if your only issues are that you find yourself attracted to other women, but otherwise there's no huge strain, then I suggest having a lover on the side. I think all relationships lose their spark after a while so the same thing will happen in the next one, and the one after that, etc. with no end to the cycle. If you have a lover OTOH, your need for excitement will be satisfied and you will keep the security of a stable relationship without having to hurt someone you've already grown close to. The down side is that it requires a lot of time and energy to live like this and you need to be willing to make the investment.
If the strain is too great, though, from the nagging etc. then you're probably better off getting out of it. I never regretted it in my case because I know I'd rather be a hermit than get nagged all the time. And finding someone who doesn't nag, well, that's even better.
As for being in love, I'm not really sure what love is. I could say I've been in love lots of times and give good arguments for that, or I could say I've never been in love and give good arguments for that, too. Howard Jones wrote a song about that in the 80's called "What is Love?" Pretty good song and the lyrics are thoughtful also.
Anyway, those are my thoughts...hope they are of some interest.