Is X-factor going down the pan?

Most of the papers seem to think that X-factor is going to head down hill pretty fast without Simon, Cheryl and Dannii on the panel. May be that's of Cowell's master plan, run British X-factor into the ground and then ITV will start to show US X-factor in it's place so he can get paid twice for doing the same show. That eeally would be pure genius if he pulled it off.