Can customs duty be taken from non-Muslims?

Yes, there is nothing wrong
with taking customs duty from non-Muslims for the goods that they bring to
sell to the Muslims. 

Al-Bayhaqi (18543) narrated
from Anas (may Allaah be pleased with him) that ‘Umar ibn al-Khattaab (may
Allaah be pleased with him) ordered that one half of one-tenth be taken from
ahl al-dhimmah (non-Muslims living under Muslim rule and protection) and
that one-tenth be taken from those who were not living under Muslim
protection. 

It says in al-Mawsoo’ah
al-Fiqhiyyah (30/102, 103): 

One-tenth is to be taken
from non-Muslim merchants when they enter the Muslim land, and that is in
general terms. … The fuqaha’ cited as proof that it is prescribed to take
one-tenth from non-Muslims the Sunnah, scholarly consensus and common sense.
As for the Sunnah, it is the hadeeth: “The ‘ushoor (one-tenth) is to be
taken from the Jews and Christians, and there is no ‘ushoor for the
Muslims.” [The hadeeth is da’eef, and was classed as such by al-Albaani in
Da’eef Abi Dawood (3046).] 

This hadeeth indicates that
no wealth is to be taken from the Muslims apart from the zakaah, but
one-tenth of trade goods is to be taken from the Jews and Christians as the
jizyah is taken from them. As for scholarly consensus: ‘Umar ibn al-Khattaab
(may Allaah be pleased with him) sent the ‘ushshaar (collector) to collect
the ‘ushoor (one-tenth), and that command was issued in the presence of the
Sahaabah (may Allaah be pleased with them), and no one disagreed with him
concerning that, so their silence represented consensus. 

As for common sense: the
merchant who brings goods from one country to another needs security and
protection against thieves and bandits. The Islamic state guarantees to
secure its roads and trade routes, and the ‘ushoor that is taken from the
merchant is in return for that protection and in return for using the public
facilities of the Islamic state. End quote.

With regard to taking
customs duty from Muslims, this has been discussed in the answer to
questions no. 39461,
42563 and
93088. It is haraam and is a
major sin.