The nice things about the ProveIt exams is that you simply tell them to "test the applicant's VB knowledge" ... and it will pull questions from their extensive library. I think you can also vary the difficulty of the exam - entry level vs sr. developer - the results it gives shows how the applicant did (got 6 out of 10 right) and does a comparison against others that have taken the test (did better than 80% of other examinees...) At least that's the impression I got. The ones I took (I took a couple of VB.NET, as well as VB-in-general, and SQL ones ) and did well enough. The length ran anywhere from 30 to 40 questions and ranged from simple things (adding a reference) to slightly more advanced topics (like reading through 4 options of code and selecting which code segment will produce the desired outputs). Sometimes the questions involved reading some code and determining which answer is the correct result.

If you're looking for something more on the fly as you put it... just think about some of the things you do in your shop from day to day... at a previous job we had out candidates white board a database design with a given senario. And then white board the classes and UI design too. shows us if they can think on their feet, do they ask questions (generally speaking, what we give them is vague intentionally... no one ever gets perfect requirements from the user, right?) and it tests their ability to reason things out logically.

One of my supervisor's question was to ask the candidate: If you were tasked with counting all of the gas stations in the world... how would you go about doing so?

There was never a wrong answer or a right one either. But the look we'd get and the answers were interesting.

One of my favorite questions to ask - this tests someone's reasoning skills - and there actually is a right answer on this one is: Why are manholes round?

I'll leave you to figure out the answer.

-tg