He said that his wife would be haraam to him if he took a second wife and now he wants to marry again

Firstly: 

Divorce has not taken place in this case, but he must offer kafaarat yameen (expiation for breaking a vow) if he takes a second wife, because by swearing that oath he meant to stop himself (from marrying another woman), not to divorce his wife. 

Ibn ‘Uthaymeen, Fataawa Manaar al-Islam, 2/584. 

The questioner’s words, “meaning to close the door to a second marriage more than to divorce his wife if he took a second wife” appear to mean divorce and stopping himself at the same time, but the idea of stopping himself is stronger. The ruling does not change because the emphasis is more on stopping himself. 

Shaykh Khaalid al-Mushayqih  

Secondly: 

If what he intended when he swore the oath was divorce, then divorce has taken place. If he committed that sin, the divorce has indeed taken place. But if what he intended was to stop himself comitting this sin – which is usually the case when such words are spoken – then he must offer kafaarat al-yameen. 

Fataawa al-Talaaq by Shaykh Ibn Baaz, 1/141 

The Muslim should not play about with the word of divorce (talaaq), and swear to divorce his wife as a means of preventing himself from doing something, because many of the scholars think that divorce actually takes place in such cases, even if he did not intend that it should take place. 

Thirdly: 

Our advice to the questioner is that he should repent sincerely from the zina to which he has confessed, for this is one of the most repugnant of sins, which generates darkness in the heart and on the face of the one who does it. He must strive to follow the means that will prevent him from committing this sin. The scholars (may Allaah have mercy on them) have stated that marriage is obligatory if a man fears that he may fall into sin if he does not marry. 

Al-Mughni, 9/341 

So you must strive to bring your wife to live with you in the same city, because this is a way of treating her in a kind manner. You should realize that she is suffering just as you are. You should not be concerned about protecting yourself from falling into sin, then leave your wife to suffer, for that is contrary to the kind treatment enjoined by Allaah and His Messenger. If you cannot manage to do that, then you should take a second wife, but that does not mean that your first wife will be divorced, as stated above; rather you will have to offer kafaarat yameen. 

Kafaarat yameen is mentioned in the aayah (interpretation of the meaning): 

“Allaah will not punish you for what is unintentional in your oaths, but He will punish you for your deliberate oaths; for its expiation (a deliberate oath) feed ten Masaakeen (poor persons), on a scale of the average of that with which you feed your own families, or clothe them or manumit a slave. But whosoever cannot afford (that), then he should fast for three days. That is the expiation for the oaths when you have sworn”

[al-Maa’idah 5:89]

May Allaah send blessings and peace upon our Prophet Muhammad, and upon all his family and companions.