High power amp, does it matter at low volumes?

I used to own N803. Progression with the amps was as follows - McCornack DNA125, McCormack DNA225, Pass X250.5. 
Each step up in the amplifier quality increased the performance at lower levels. I lived in an apartment back then and we had a newborn. And neighbors. Late at night at low volume you get excellent sonics from B&Ws but obviously you don’t get the woofers moving air the same way they do when you crank it up. Go for the best possible amp you can afford that produces high quality high current power.

Look into Coda amps. Killer performance for the money. Pass X260.8 mono blocks, or X350.8 stereo with a good preamp would be a nice match. You need lots of effortlessly delivered current for the B&Ws. 

One other important bit with the Pass Labs X260.8 mono amps you get 34 watts of Class A power into 8ohms. This means, if I am not mistaken, 17 watts of Class A into 4ohm speakers. This is enough high current to never exit Class A with your B&Ws at lower volumes. Depending on a room size and what you define as low level listening but it will most likely stay in Class A the whole time and will start moving the needle into A/B at around 80-85db if not higher. 

Per Lanx0003 my amp seems to have a rated THD+N of .05 or -66 dB. A stereophile review showed it was actually .0019 or -94 dB Cold and .006 or -84 dB Hot.  Crosstalk is over 100 dB below 1 kHz and between 81 and 85 dB R-L and L-R.   Not the greatest specs..

I downloaded a sound meter app, I understand it runs on an Android, so accuracy is really an indication only. At normal listening levels I didn't exceed 65 dB at my listening position. Okay, some times I crank it up, but rarely. 

My speakers are rated to be 90 dB efficient, and I sit 10 ft away from them.  You are supposed to need a minimum of 50 watts upto 500 watts.

For solid state amps... of the same sonic quality it matters. The doubling of wattage is showing the current the amp has... which is what you care about and will make the speakers sound better more dynamic even at low values.  I have run 250wpc to 350wpc high current amps for decades until I discovered tube amps. 

This is not nearly as pronounced in tube amps and not as high a wattage is required on tube amps. Tube amps tend to sound different than solid state, although there are a number of solid state designed to sound warm and natural like tube amps generally do... like Pass XA series, the new Audio Research - series. 

The first watt matters.  A better low power will give you a better SQ than more power.  The Gaku-on from Audio Note is/was probably the finest sounding amp in existence and is 45 wpc - and was about $200K 15 years ago.

For much of the music I listen to my SE 2A3 amp is gorgeous (a few wpc).  I am currently in an orchestral frame of mind and have put the 50 watt into 4 ohm XA 25 back in, it is cleaner on crescendos, but a hair of the magic is missing on pp voices.

Class A sounds better than AB.

If you are listening at low-medium volumes you are probably not going above that first watt.

Download/stream a test tone (AudioCheck.net and Spotify e.g.) adjust the volume so that it is a loud as you want at your listening seat and put a voltmeter across the speaker terminals.  It will probably measure 2 volts or so.  ( P = V^2/R) => 1/2 watt into 8 ohms.  Your speakers are nominally 90dB @ 2.83 volts so that is 87dB @2v.

I measured a 3 or 4 dB drop from 1 meter to my listening position so the 1/2 watt gives about 83 dB.  Enough to damage hearing after 4 hours.