Professional-Lurker: Lessons learned from AoIR abstract reviewing
LS writes this nice post about being a reviewer. I would have commented there, but alas, she appears to have killed comments.
I recently was a reviewer for a conference and it was a very interesting experience. I can't agree more with her points. Somethings to keep in the back of your mind for your next submission.