Isti`adhah in prayer
Seeking refuge with Allah before starting to recite is Sunnah according to the correct scholarly view. The correct view is that the time for that is in the first rak`ah.
Reciting the Basmalah before Al-Fatihah in prayer
Saying the Basmalah before reciting Al-Fatihah is one of the Sunnahs of prayer. Based on that, if someone omits it deliberately or because he forgot , he does not have to do the prostration of forgetfulness, although it is recommended for him to do it if he omitted it by mistake, and he usually says it. Otherwise, it is not prescribed to do that.
But if he omits it deliberately, he should not do the prostration of forgetfulness, because the prostration of forgetfulness is only for one who omits any of the actions or words of the prayer by mistake, because the Prophet (blessings and peace of Allah be upon him) said: “If one of you forgets, let him prostrate twice.” (Narrated by Muslim (572)
Reciting the Basmalah according to Abu Hanifah
With regard to the Basmalah according to the madhhab of Abu Hanifah (may Allah have mercy on him), it is also Sunnah according to their view.
It says in Al-Mawsu‘ah al-Fiqhiyyah (8/87):
“To sum up the view of the Hanafis regarding that, it is Sunnah to recite the Basmalah quietly for the imam and the one who is praying on his own before reciting Al-Fatihah in every rak`ah, and it is not Sunnah to recite it between Al-Fatihah and the following surah at all, according to Abu Hanifah and Abu Yusuf, because the Basmalah is not part of Al-Fatihah, and it is only mentioned in the beginning for the purpose of seeking blessing (barakah)…”
And Allah knows best.