I can only answer from the perspective of the amateur piano competitions I've attended:
ShiroKuro At piano competitions, do judges usually have the scores out for all of the pieces being performed?
Not all competitions require that competitors submit scores for the pieces they are playing (most seem to be unclear about this). Boston lists explicitly "If a contestant will perform a work by a lesser-known composer, and in particular, a contemporary work, they must provide 5 copies of the score for the jury", which of course still leaves some leeway for interpretation. I hear San Diego required people to send in PDFs of their scores. Paris, WIPAC, and Oberlin did not require scores.
ShiroKuro And if so, are they actually reading along?
I imagine this depends on the individual jury members. In Boston some of them definitely read the score — when I got feedback from them, some had noted specific measure numbers.