do you just need to check if both conditions are met? I'm not sure I understand what you're asking for.

Code:
if(checkedControls.length > 0 && selected == 0){

  // do something

}else{
  alert("You must selected at least one checkbox");
}