What am I evaluating when I have a candidate do a pairing exercise with me during an interview?
Communication skills: Communication is one of the most important skills a programmer has. Of course, we all need to be able to solve a problem, but in order to identify that problem in the first place, we need to be able to communicate with our customers to understand it.
Most of us are not the only programmer in-house, so we need to be able to talk to other programmers about the code we are working on or the problem we are trying to solve. We need the ability to clearly communicate our intentions through our code as well.
By doing a pairing exercise with the candidate I can get a pretty good feel of many aspects of their communication skills. I get to see the code that they write to see if they are clearly communicating their intentions through the code (good names, good method decomposition, etc). I also get to play customer and clarify the requirements for them. This allows me to evaluate how well they can understand requirements and ask clarifying questions.
Lastly, I get to see how well they can talk about the code they are writing as a technical peer. Are they able to articulate what they are trying to accomplish in the code? Are they able to articulate why the approach they are taking is giving them troubles, or how they intend to break the problem down into manageable chunks.
Code quality: This is the no-brainer answer. Yes, I am interested in seeing how you code. I want to see if you can produce quality, readable, maintainable code.
I am looking at things like naming (for variables, methods and classes), at method decomposition, simple formatting. I am looking to see if the candidate will start with tests. Does he/she at least write testable code?
What about comments? If he/she comments, are they useful comments, or is it simply reiterating what the code does? Is there any dead code remaining? Does he/she refactor the code?
Algorithmic thinking: By giving the candidate a problem that does not have a readily apparent solution, I get to see how the candidate thinks through a tricky problem.
Does the candidate just freeze up? Or is he/she able to break the problem down into smaller chunks? If so, do these chunks seem like a logical way to break down the problem?
Ability to take feedback: While doing a coding exercise, I will typically either ask them to change how they are approaching a solution, or help advise them on a way to achieve their goals. This is typically intended to allow me to see how well the candidate will incorporate feedback. (I have seen candidates who refused to take any feedback given to them, even though they were struggling with achieving their goals).
Whether I am working in a pairing environment or not, I believe that multiple sets of eyes should see the code. I encourage code reviews, and I expect each person to incorporate some, if not all, of the feedback given in such reviews (at the very least, the feedback needs to spark a conversation).