For Eid, should he travel to join his father or to join his wife and children?

Honouring one’s parents and treating them kindly is one of
the most important acts of obedience, which Allah mentions alongside
worship of Him in several places in His Book. Allah, may He be exalted,
says (interpretation of the meaning): 

“Worship Allaah and join none with Him (in worship); and
do good to parents…”

[al-Nisa’ 4:36]

“And your Lord has decreed that you worship none but Him.
And that you be dutiful to your parents”

[al-Isra’ 17:23]. 

Treating one’s wife kindly, taking care of one’s children,
visiting them and checking on them are also acts of obedience. 

If your responding to your father’s request will make your
wife and children sad, and going to them will make your father sad, and this
situation is something that happens repeatedly with you, then it is better
for you to alternate your visits: this Eid with your father and the next Eid
with your children. This does not mean that you cannot visit your children
at other times, as you mentioned. 

To sum up, one should pay attention to what is wise and
strive to strike a balance between various interests so long as it is
possible. If it is not possible to combine various interests, then in that
case you should look at which are most important and most obligatory, and
this varies from one case or person to another. 

And Allah knows best.