Quote Originally Posted by Eduardo- View Post
That way is exactly how people are deceived. 99 things true, 1 false.
True but this assumes motive to deceive, something we are assuming our hypothetical super AI doesn't have.

Quote Originally Posted by Eduardo- View Post
But what if you have a 1/10 failure rate?
And what if it has 1/100?
You would be playing a Russian roulette.
Yea, this is a very good point. Just because we haven't seen it fail before doesn't mean it can't fail in the future and there is no way to know that it can't fail in the future. I think what we're actually talking about here is the halting problem or a variation of it.