Output / Input Voltage Question

I really do hope you're getting all this.  
https://youtu.be/pkOYrQ3QhRI?t=605

But as usual the self proclaimed Guru of high end audio likes to show his wit with a ridiculous answer to the problem that the OP is having! 

For the record, it is you - and others! - who have proclaimed me guru. I have never done so myself. Don't really consider myself a guru. Maybe I should, since so many others do. Be that as it may, all I did was help the OP with overwhelming complexity by showing how to make it all very simple. With my clumsy words. 

Now learn from a real guru. Seriously. Listen and learn.

https://youtu.be/pkOYrQ3QhRI?t=70

Beware audiophile tech talk, they will screw you up every time! Basically what happens, they will have you going round and round with a whole lot of stuff that is pretty much irrelevant. Not totally, and that is where they get you. Because every once in a while some of the stuff they talk about actually does matter. Problem is it takes you a good 10, 20 years of wasted time and effort to finally understand the vast majority of the time this stuff they go on and on about is well and truly irrelevant. Most of the time.

Like now. All that you have is an amp that does not require very much input voltage to drive it to full power. That is it. Period. Full stop. End of story. Nothing in audio matters less than where your volume knob is pointed. You get that now.

If you want you can take the time to study this stuff and when you understand it well enough then you will know ahead of time to expect the knob to be where it is. You will even be able to look at your source, be it CD or phono stage or whatever, and have a pretty good idea where the volume knob will be. It is actually pretty simple stuff once you get the hang of it.

Big picture though, keep it simple. Pay attention to speaker sensitivity. As long as that is 92dB or more you know you will always be able to find an amp to drive it nice and loud real easy without needing hundreds of watts. With turntables you want cartridge output of around .4mV or more in order to do the same, make it easy to find a phono stage. This is how to use specifications intelligently, to make your life easier not harder.
Do those two things and pretty much all the rest you can tell yourself, don’t sweat it. Eventually you may learn enough to understand why but in the meantime just tell yourself don’t sweat it.

Can't go past 40% volume without being way too loud.

Why didn't you say so? All this voltage stuff is irrelevant. Your problem is you think that where the volume knob is pointed means anything at all. Relax. Where the volume knob is pointed means nothing at all. Or if you want you can tell people your amp is so incredibly powerful look how loud it is and the volume is barely cracked open. But seriously, where the knob is pointed is utterly irrelevant.   

This is in stark contrast to the numbers on the dial, which are incredibly important.  https://www.youtube.com/watch?v=4xgx4k83zzc  

That's a joke, son. (I have to say that because audiophiles have no sense of humor.)