He goes out to call people to Allah and leaves his family despite their need for him

Firstly: 

There is no doubt that those who are most deserving of a
person’s care, protection, concern, upbringing and call are his family, the
members of his own household. 

Allah, may He be exalted, says (interpretation of the
meaning):

“O you who believe! Ward off yourselves and your families
against a Fire (Hell) …”

[al-Tahreem 66:6]. 

And the Messenger of Allah (blessings and peace of Allah be
upon him) said: “Each of you is a shepherd and each of you is responsible
for his flock. The ruler of the people is a shepherd and is responsible for
his flock. A man is the shepherd of his household and is responsible for his
flock. A woman is the shepherd of her husband’s house and children and is
responsible for her flock. The slave is the shepherd of his master’s wealth
and is responsible for it.”

Narrated by al-Bukhari (893) and Muslim (1829). 

If a man has shar‘i obligations having to do with the rights
of Allah and with the rights of His creation, and he himself wants to strive
in some good causes and do a lot of supererogatory deeds, then he is
enjoined to strike a balance between all these rights, and he should give
precedence to those which are obligatory over those which are recommended
and encouraged. If he has too many duties, then he should strive to combine
all these interests, whenever it is possible to do so. 

If there is a conflict between some of them and others, or he
is not able to combine them, then he must give priority to the more
important of them over others, and he should start with the most binding of
obligations upon him, then move to those that are less important. 

It is proven in Saheeh al-Bukhari (1968) from ‘Awn
ibn Abi Juhayfah, that his father said:

The Prophet (blessings and peace of Allah be upon him)
established the bond of brotherhood between Salman and Abu ad-Darda.
Salman visited Abu ad-Darda and saw Umm ad-Darda looking umkempt. He said
to her: What is the matter with you? She said: Your brother Abu ad-Darda has
no need of worldly matters. Abu ad-Darda came and made some food for him.
Salman said: Eat (with me). Abu ad-Darda said: I am fasting. Salman said:
I will not eat until you eat – so Abu ad-Darda ate. When night came,
Abu ad-Darda wentto get up (to pray) and Salman said: Sleep, so he slept.
Then he went to get up again and Salman said: Sleep. Then towards the end
of the night, Salman said: Now get up – and they prayed. 

Salman said to him: Verily your Lord has a right over you,
your own self has a right over you, and your wife has a right over you, so
give each one who has a right, his right. 

Then he came to the Prophet (blessings and peace of Allah be
upon him) and told him about that, and the Prophet (blessings and peace of
Allah be upon him) said: “Salman is right.” 

In this hadith we see that one should strike a balance
between all duties, and not pay excessive attention to some at the expense
of others. 

Secondly: 

A man’s care and protection of his family includes taking
care of whatever they need in their lives of maintenance and accommodation,
as is obligatory. That also includes protecting them from anything that may
pose a danger to them, whether that has to do with their lives, their honour
or their wealth. He is enjoined to take care of them and protect them in all
those ways. 

This is something that varies from one time and place to
another. If a man is in a place where his family will be safe if he leaves
them, either because they are surrounded by other family members and they
have someone to take care of their affairs, or because the place where they
reside is generally safe, and there are few causes of fitnah there, then in
that case it is permissible for him to travel and leave them, or to go out
to take care of his own interests, to an extent that will not cause them any
difficulty or harm. 

But if his family are in a place where their lives, wealth or
honour are not safe, or they fear fitnah in that place, then what he must do
is stay with them and not go out and leave them, except in the case of
absolute necessity, in which case he must pay attention to providing means
of care and protection for them, or appointing someone to look after them,
or moving to another place that is more safe. 

It was narrated that ‘Abdullah ibn ‘Amr ibn al-‘As said: A
man came to the Prophet of Allah (blessings and peace of Allah be upon him)
and said: I swear my allegiance to you, that I will migrate and engage in
jihad, seeking reward from Allah. He said: “Are either of your parents
alive?’ He said: Yes, both of them. He said: “Are you seeking reward from
Allah?” He said: Yes. He said: “Then go back to your parents and be a good
companion to them.”

Narrated by al-Bukhari (1671) and Muslim (2549) 

Shaykh Muhammad ibn ‘Allan as-Siddeeqi (may Allah have mercy
on him) said:

The Lawgiver waived the obligation of migration in his case,
giving precedence to the rights of his parents. Even though migration was
obligatory for him, it conflicted with something that was more obligatory
than it, namely the rights of parents. If it was not obligatory, then that
which is obligatory would take precedence, but this is only applicable in
the case of one whose religious commitment is safe in the place where they
(his parents) are. But if he fears for his religious commitment, then he
must flee for the sake of protecting it, and leave behind his parents and
children, as the Muhajiroon did, who are Allah’s chosen elite among His
slaves.

End quote from Daleel al-Faliheen li Turuq Riyadh as-Saliheen
(2/463) 

This has been discussed previously in the answer to question
no. 177195 

In fact some of the scholars stated that in such a case,
where a person fears that if he leaves his family alone in the place where
they are they may be exposed to harm or danger, the obligation of offering
prayer in congregation is waived for him, despite the fact that it quickly
ends and requires little time. 

It says in Kashshaf al-Qina‘ (1/456):

Yes, if his going to the mosque will lead to leaving his
family on their own, then he must establish the prayer in his house, so as
to fulfil the obligation. End quote. 

In fact they also stated that one of the aims of sharee‘ah,
to which the man should pay attention, is keenness to give company to his
family,  even if that is only for a short time. 

It says in Tuhfat al-Muhtaj (2/471): He has the
choice when coming back (from Friday prayer) between riding and walking. 

Ash-Sharwani said in his commentary thereon: 

This is to be understood as meaning: unless there is
something good he wants to do (in which case he should come back quickly),
such as giving company to his family, or doing some religious duties
concerning them, and so on, or he wants to protect his faculties from
falling into sin that may be expected when one is outside the home. 

As that is the case: 

What is prescribed for the husband, if you are living in a
place where you and your family are not safe, is for him to stay with the
members of his household, and he can still do all the things that he wants
to do of da‘wah,  good deeds, prayer, and i‘tikaf. He can still do those
things in the same area or city in which you are living. With regard to
fulfilling these aims of sharee‘ah, it is not stipulated that they should be
done in one place and not another, and they are not rituals in which it is
stipulated that one should be in Makkah or Madinah. Rather they are kinds of
good deeds which a person may do in the place where he is, in his own
neighbourhood and city. Thus he may combine doing all kinds of good deeds
with giving each one who has a right his right, and he will not be like the
one who builds a castle but destroys the city. 

For more information, please see the answers to questions no.
177195, 155577 and
3043. 

Thirdly: 

With regard to the husband quoting the story of the Prophet
of Allah Ibraheem (peace be upon him), which was narrated by al-Bukhari
(3364) from Ibn ‘Abbas (may Allah be pleased with him), in which it says: 

“Then Ibraheem brought her and her (infant) son Isma’eel,
whom she was nursing, and left them beside the House (the Ka‘bah) near a
tree, above Zamzam in the upper part of the mosque. At that time there was
no one in Makkah, and there was no water there. He left them there, and he
left with them a sack in which there were some dates, and a waterskin in
which there was some water. Then Ibraheem turned to leave, and the mother
of Isma‘eel followed him, saying: O Ibraheem, where are you going and
leaving us in this valley, in which there is no one and nothing? She said
this to him several times, but he did not pay any attention to her. Then she
said to him: Is it Allah Who has commanded you to do this? He said: Yes. She
said: Then He will not forsake us. Then she went back, and Ibraheem kept
walking until, when he reached the mountain pass where they could not see
him, he turned to face the Ka‘bah, then he offered supplication with these
words, raising his hands and saying: “O Lord, ‘I have made some of my
offspring to dwell in an uncultivable valley by Your Sacred House (the
Ka‘bah at Makkah); in order, O our Lord, that they may perform As-Salat (Iqamat-as-Salat),
so fill some hearts among men with love towards them, and (O Allah) provide
them with fruits so that they may give thanks’ [Ibraheem 14:37]” –

his quoting of this story as evidence is entirely
inappropriate, and has nothing to do with your story or your situation at
all. 

Ibraheem (peace be upon him) only left his wife and son in
that place, in the sacred land of Allah, beside the Sacred House of Allah,
for a legitimate shar‘i purpose, based on the wisdom of Allah, that would be
served by them residing in that place. 

The mother of Isma‘eel was not aware of the wisdom of Allah
behind her staying in that place, in which there was no cultivation or
water, and no people to keep her company, but she asked the Prophet of Allah
Ibraheem whether he was doing that to them on the command of Allah, and he
replied: Yes. So she submitted to the command of Allah and said: Then Allah
will not forsake us. 

The one Who revealed to Ibraheem (peace be upon him) that he
should do this action, which appeared to be contrary to proper conduct and
what people usually do, was Allah. 

It was not an idea that Ibraheem (peace be upon him) came up
with by himself; rather it was a revelation from Allah. 

Has Allah revealed to us or commanded us that one of us
should fall short in protecting his family and the members of his household,
then say that he was entrusting them to the care of Allah? 

So why did the Prophet (blessings and peace of Allah be upon
him) instruct that Sahabi to go back to his parents and be kind and keep
them company? Allah, may He be exalted, instructed His Close Friend
Ibraheem to sacrifice his son Isma‘eel (peace be upon them both), but can
it be valid – whether according to reason or religious teaching – for a man
to neglect his family and children, let alone expose them to harm and
danger, let alone go and sacrifice one of them, giving as his reason the
fact that Ibraheem (peace be upon him) did that? 

Can it be valid for a man to leave his family members in the
desert, in a land where there is no cultivation and no water, and leave
them, then say that Ibraheem (peace be upon him) did that? 

But after all that, and before all that, we advise you to try
to reach an understanding with your husband about this problem, in a calm
manner and avoiding trying to prove him wrong. Rather you should try to
avoid angering him, as much as you are able, and try to explain to him that
you understand and respect his convictions, but all you are asking him to do
is to pay attention to your needs and your circumstances in the place where
you and your family are living, and that he should do whatever he can to
have a moderate, balanced approach, and make some effort so that you do not
reach the stage of tension in the relationship or exacerbation of the
problems between you. 

We ask Allah to make things easy for you, and to make your
husband treat you well and make you treat him well, and to keep you together
in the best way. 

And Allah knows best.