What should I upgrade next?

I am wondering what others think is my "weakest link" in my system.... Here's what I have:
Ayre C5xe CD Player
Pass Labs X-1 preamp
Pass Labs X350.5 amp
B&W 802N (original generation)
transparent ultra XL xlr from CD player to preamp
transparent reference w/XL xlr from preamp to amp
transparent ultra bi-wire speaker cables.
Wow...difficult question to answer...your system is extremely well balanced.

Lets try this another way...is there anything you wish had more of, or better in the sound you hear? (I wonder if your room might actually be something to consider.)
Replace the Speakers, go for something bearable, Verity Audio for example.
Nice system. Get a computer based music server and download hi-rez files. You'll a separate D/A converter so budget at least $2.5k. It really would be nice if the Ayre had a digital input.
I am not a huge fan of B and W so now you know what I would change. Realistically speakers are so subjective that it might the least sense to you. You could always do the smart thing and save your money nfor a future need/desire.
Wow, nice system. Maybe you should spend some time/money on tweaking the room...

I don't like those speakers either so I would change those out. Plus they do seem the most dated of everything you have. If you love B&W then maybe the new diamond version which I have not heard.
I heard the 802ds in a system where they replaced the 802s and I thought it was a significant upgrade (in the high end, of course). Otherwise, have not heard any of the other components. I AM a fan of computer audio, so you might explore in that direction.
What do you think your weakest component is? If not that, then what do you want to improve? Personally, I don't like your speakers. It doesn't really matter what I think, though. You're the one who has to listen to them.

I will say that your speakers and electronics are pretty well matched. It would be a good idea to focus on specifically what you want to improve. If you upgrade something without being sure, you can easily make a mistake.
I agree with Rlwainwright, take a look at tweaking the room.
I would also want dedicated AC to your outlets, and speaking of outlets what type are you using?
Should your room be tweaked to your satisfaction than the speakers would be my next concern. When you begin to hear what your don't hear so to speak, it may be time to upgrade your speakers. Beyond that it may be that expanding your system into a computer based or media server component will bring more joy and more use of a already fine system.
At the time B&W wanted to shift (rise) the price of the big 800 series and they managed to justify by building a new tweeter. (this gave a +50% price rise).

I would keep the speakers, the diamond tweeter does not worth the difference.

on the other hand i would do something with the sources. I am a TT fan but since you prefer digital i would buy a DAC and rip my cd's in flac format. try downloading a couple of hi res tracks and you will hear the difference.

make that a DSD capable dac, plenty starting at relatively low prices
I recently spotted an ad for Audiopoints that fit B&W here on Audiogon. We had vintage 801 Matrix III’s in the studio for some time and found that adding these devices led to a much more musical outcome. These brass cones worked beyond our expectations so we decided to hold on to the monitors. If you are happy with the 802’s then in my opinion I would upgrade them. The increase in speaker performance is well worth the price of the points.

I would also follow up with LAK’s advice and work on cleaner power, isolation transformers, AC distribution or better racking prior to room treatments.
What specifically is missing from the sound of your system? It appears that your system is very nice indeed. I agree that room tweaking is very important. Also, you may want to borrow a nice DAC to see if that makes an appreciable difference in sound (it definitely will) before tweaking too much or buying anything.

Always demo in your system before purchasing.

I agree with those suggesting tweaking the room acoustics to include a reevaluation of speaker placement, listening position, and equipment placement.

I also agree that computer audio would be a nice fit.