wouldn't firing somebody because you cant understand their accent be tantamount to racism
it would be if it was an Asian or an Irish person they were firing
Aside from the fact that it *wouldn't* be racism to fire someone paid to speak for being unintelligible, there are different rules which govern casting in the entertainment industry.
In short, you're allowed to cast anyone you like who fits the bill, and you're able to refuse to see someone who doesn't - irrespective of the criteria. Sometimes people object - on Broadway a couple of years back, Jonathan Pryce was asked to reprise his role in Miss Saigon, made up to look Asian. This angered some Asian actors in the States, who said that it was inappropriate for a white male to be cast over someone of the correct ethnicity.
The objections were ultimately ignored - as its an entertainment show, the right person for the job is whoever the director and producer thinks is the right person for the job.
While lots of shows have casting changes its VERY uncommon for for it to be done in the public domain, especially when it happens on Cowell shows every year.
The only reason that Cowell does it is to gain free publicity - and even raising the question you did, Steve, means you're spending time discussing his production.
I would say "shrewd" on Simon's part, but now its obvious and tiresome.