Why does it seem the priests cannot enter Japan?

Valignano refuses to allow the three priests to enter Japan. Since he is in control of missionaries entering the country, there seems that little can be done to salvage the expedition.