Why is Orlando reluctant to leave the court?

Orlando is thrown by Adam's news about Oliver, surprised that his brother's hate for him goes so deep. However, he has no money and nowhere to go as Oliver has his inheritance and he has lived his whole life in the city.