Simon Cowell is having sleepless nights as he tries to pick the judging line-up for the Canadian and U.S. version of 'The X Factor'.

The music mogul has so far only announced that he will be joined on the show by record company chief L.A. Reid, and he admits the decision is being delayed because he and his production team keep changing their minds about the panel.

He said: "The truth is that the reason we haven't announced the panel yet is that nobody can agree. New people keep coming along every week who we want to see.

"What you do is you make a decision, then you get nervous about it, then you wake up in the middle of the night saying, 'I've made a terrible mistake.' You call everybody the following day to say you've changed your mind and then 24 hours later you go through the same process again.

"But this happens on all of our shows. I've been on shows where the day before we're due to film, we haven't decided who's going to be on the show."

Simon quit his role as a judge on 'American Idol' to launch 'The X Factor' in America and he admits he sometimes forgets he is no longer a part of the show, and still forms judgments on its contestants.

He added in an interview with the Hollywood Reporter: "I watched it a couple of weeks ago. And it was interesting. I was watching it with some friends. And for a moment, it's like you're on the show and somebody's singing and you want to say something, and then suddenly you realise you're not on the show and you're listening to somebody else. It's a really odd experience. But they genuinely seem to be happy and the audience seemed to like it so I guess everyone's happy."

Cheryl Cole, Jessica Simpson and Katy Perry are believed to be among the stars in line to judge the pop hopefuls.