What is the ruling if the worshippers turn about 40 degrees from the imam?

Firstly: 

What is required of the one who is far away from the Ka’bah is to turn to face its direction, according to the majority of scholars.  

Ibn Qudaamah (may Allaah have mercy on him) said in al-Mughni (1/262): Turning to face the qiblah is a condition of prayer being valid, and there is no difference between the obligatory and naafil prayer, because it is a condition of prayer, so it applies equally to obligatory and naafil prayers, like purification. And because the words of Allaah (interpretation of the meaning): “And wheresoever you people are, turn your faces (in prayer) in that direction” [al-Baqarah 2:144] are general in meaning and apply to both (obligatory and naafil). Moreover, he can see the Ka’bah, then it is obligatory to pray facing it directly. We do not know of any difference of opinion concerning that. Ibn ‘Aqeel said: If part of his body does not face directly towards the Ka’bah, then his prayer is not valid.  

What everyone who is far from Makkah must do is to look for the direction of the Ka’bah, without having to face it directly. Ahmad said: Whatever is between the east and the west is qiblah, and a small deviation from the qiblah does not matter, but he has to try to get it right as much as possible. This was also the view of Abu Haneefah. This view is based on the hadeeth in which the Prophet (peace and blessings of Allaah be upon him) said: “Whatever is between the east and the west is qiblah.” Narrated by al-Tirmidhi, who said it is a saheeh hasan hadeeth. End quote. 

In al-Muwatta’ of Imam Maalik (4600 it is narrated that ‘Umar ibn al-Khattaab (may Allaah be pleased with him) said: “Whatever is between the east and the west is qiblah if you face the direction of the House.” 

Shaykh al-Islam Ibn Taymiyah (may Allaah have mercy on him) said in Sharh al-‘Umdah (2/539): Al-Athram narrated that ‘Umar, ‘Ali and Ibn ‘Abbaas said: “Whatever is between the east and west is qiblah.” And it was narrated from ‘Uthmaan that he said: “How can a man make a mistake with regard to his prayer when whatever is between the east and the west is qiblah, so long as he does not face the east deliberately?” End quote. 

Secondly: 

The imam and worshippers must face the direction of the Ka’bah and try hard to do that. It is not sufficient for the imam only to face that direction. 

The scholars of the Standing Committee for Issuing Fatwas were asked: In our country there are several mosques in which the mihraab is off-course and faces to the right. The reason for that is that some people think that the words of the Messenger (peace and blessings of Allaah be upon him), “Whatever is between the east and the west is qiblah.” Based on that, is it sufficient for the imam only to turn to face the qiblah, and not the members of the congregation?  

They replied:

What is required of both the imam and the person praying behind him is to face the direction of the Ka’bah, because Allaah says (interpretation of the meaning): “And wheresoever you people are, turn your faces (in prayer) in that direction” [al-Baqarah 2:144], and because the Prophet (peace and blessings of Allaah be upon him) said: “Whatever is between the east and west is qiblah” (narrated by al-Tirmidhi, who said this is a saheeh hasan hadeeth). This was addressed to the people of Madeenah and others like them who were to the north or south of the Ka’bah. The apparent meaning is that anything between these two points is qiblah. 

But for one who is to the west or east of the Ka’bah, the qiblah in his case is between the south and the north, because if we say that everyone who is far away from the Ka’bah must face its exact direction, the prayer of the people in a longs straight line would not be valid, and neither would the prayer of two who are far away and face one qiblah, because it is not possible for a long line of people all to be facing the Ka’bah directly. End quote. 

Fataawa al-Lajnah al-Daa’imah (6/316). 

But if they all turn to face the qiblah and the congregation turns away from the imam, to the right or left, whilst still facing the right direction, then his leading them in prayer is valid. 

Ibn Qudaamah said in al-Mughni (1/267): If one of them turns towards the right and the other turns towards the left, but they still face the same direction, then there is no difference of opinion in our madhhab that one of them may follow the other in prayer, because what is required is to face the qiblah and they have both done that. End quote. 

Turning 40 degrees away from the qiblah does not affect their facing that direction, so the prayer is valid even with this turning away. But what the people of each mosque should do is to work out the direction of the qiblah precisely, and all the worshippers (the imam and the members of the congregation) should turn to face that direction, and try to work it out, as it says in the words of Ibn Qudaamah (may Allaah have mercy on him). 

And Allaah knows best.