I used to think a little grease didn't matter until I tried to swap out a light switch cover. The screwdriver just spun in my fingers and left a permanent smudge on the wall. Now I wash my hands before any small job.
Oh man, that's the worst! Trying to change a phone case with chip grease on your fingers is just as bad. Every single surface ends up with a weird shiny smear, and you can't get a good grip on anything. The tiny little buttons or screws become impossible to handle. Feels like you're trying to build a ship in a bottle but everything's coated in slime.