Can he transport people in his own car for a fee?

Yes, it is permissible for a person to transport someone else on his mount or in his car to a specific place in return for a fee that they agree upon when making this deal. 

It says in Fath al-Qadeer (9/83): It is permissible to hire animals for riding and carrying goods, because it is a benefit that is clearly defined and it is something common. End quote. 

In al-Mawsoo’ah al-Fiqhiyyah (1/302) it says: Renting modern means of transportation. The earlier fuqaha’ did not explain the ruling on hiring modern means of transportation such as cars, planes and large ships, rather they discussed the hiring of animals, people and small boats. But the rulings on hiring these things may be applied to modern means of transportation. If there are debates in some cases, such as regarding the situation of the passenger, these may be referred to what is customary. There is no difference between one person and another with regard to hiring cars or planes, unlike animals, because that may be affected by the size of the passengers, whether they are heavy or not. With regard to the luggage that a person brings with him, reference should be made to the conditions; if there are no conditions then the reference point is what is customary. As for the fee, whether that is for transporting people or goods, the reference point is also the conditions, otherwise it is what is customary. End quote. 

The fact that a person does not own a taxi licence does not mean that it is not permissible for him to transport someone else in return for payment, as stated above, because he is the owner of the vehicle and he may make use of it by selling it, hiring it out and so on. 

And Allaah knows best.