At th beginning of the underwear scene when Joel slides in, you see that his collar is down. Then when he is at the fire place his collar is up. Then when he gets off the couch he pulls it up again as if it were down before!