lol I'll be honest I'm not even sure what a roly poly is but whatever that may be no more for him until he's a bit bigger.
how long out of the tank ? you have longer than you think; people have come home to find a fish on the carpet after a day at work, and they have lived, but ideally get him in and out within 5, 6 minutes or so; its very stressful. Keep a firm hold on him with one hand, and tweezers with the other, don't put him down on e.g. a towel or something that will rub off onto his slime coat. A spare pair of hands would really help. As you are only going to go in for him if he looks like he's about to shuffle off his mortal coil, he should be an easy capture, but don't be surprised if he bolts even if he looks very out of action; have two nets, get one in behind him and one from the front.
When you put him back in, turn off the tank lights and let him get over it; I would put a bit of melafix or perhaps some salt, to quickly regenerate any slime coat that gets damaged.
Again, hopefully it will not be necessary. How is he doing right now ?