Like you, I had a pair of B&W Silver Signatures for nine years. I decided to replace mine, not because I wasn't happy with them, simply because I wanted something new.
This was at the same time that B&W came out with their first diamond series of speakers. I listened to a pair of 803D speakers three different times. I would never have considered replacing my Silver Signatures with the 803Ds, the music coming out of those speakers just wasn't anywhere close to the Silver Signatures.
The Silver Signatures were the original 805s, but custom made to commemorate John Bowers. I can see the 805Di speakers outperforming your old N804 speakers.
I'd also suggest that you listen to some Dynaudio speakers, especially the C1. The Magnepan 1.7 and 3.7 speakers are also worth a listen, but you need a big room for them. The Quad 2805 and 2905 are also worth a listen. You also might try a Thiel 2.4 or 2.7 if you have a large room like you'd need for the Magnepans.