I have shot my .416 Rigby single-shot twice now, using the PAST pad, and have not found recoil to be objectionable or painful in the least.
Just a big boom and a shove.

I will eventually install a pachmayr decelerator on the rifle.
Until then, how much more will the rifle "hurt" if I try shooting it without the bulky PAST shoulder pad?
Say, wearing a t-shirt and a long-sleeved shirt?

Dude, try it and find out!! Just don't benchrest it yet. Wear a short sleeve sweatshirt for just a little more material than a t-shirt and shoot it off hand. Since the Ruger No.1 is a bit lighter than a 77 it will boom a bit. What are you shooting, factory loads or reloads?

On any of the big ones, shoot them often, not a lot, and it won't be bad. About 7-8 shots per outing is enough for me, no desire to cultivate a flinch. I was shooting with my brother handed it to him, he didn't even look at it, knew if I had a rifle that heavy, it was a big bore, wouldn't touch it. They won't kill you, but the cumulative effect of a bunch of rounds will cause problems it will take months to overcome if you develop a flinch.

No Shit Sherlock! Reminds me of some guy yacking up his 30 cal sooper pooper boomer when I showed up with my converted 375 Ultra and basically told me that I had to have a big bore because of my inadequacies. I only load my 375 Ultra to tighten up the long range shots and it works. I shot right next to him at 100 yds and shot 1.7" 5-shot group first around and he says that he can do a bit better and I let him. The first round basically killed him as he refused to use my upright benchrest position and went into the low varmint benchrest position and rocked almost off the bench. He almost dropped my 77 into the dirt if I hadn't caught him and my toy at the same time. When he handed it back to me, I was so relaxed that I shot 1" groups for the rest of day. Right after that, he quietly snuck away without a word of thanks for the education.