Realism in instruments how to achieve it?

Well, the answer to your question is, "all the above."  Or better yet, a combination of all the above.  You know what kind of sound you like, now the challenge for you is to conduct due diligence through experimentation until you achieve that sound.  Heck, it could be nothing more than your cabling that’s holding you back.  Happy listening.      

Refine your point of reference for how instruments sound.

Is there a symphonic hall, performing arts, etc venues near where you live?

Go and sit close to an orchestra in one of them. Or sit up front when a wind ensemble’s playing, let that clarinet player or sax player get close.

If i went to town on my Yamaha grand with you sitting 4 feet away, i guarantee you won’t be sitting too close if you are the Diana Krall @60db type of listener.

Eitherway, when you come back home, you will realize that your small closet sized room and these type of speakers you are thinking of are just not going to cut it.

If you have the room and the cash, some of the bigger Avantgarde systems (with the big subs) or a Daniel Hertz M1 (from Levinson), etc, etc might get closer to doing it some justice. But, you’d need the cash and some room for such systems.

In fact, I can't think of anything that will work in a small room.

For your small room, just lower your expectations and learn to live with it.

Is it the speakers that gives that realism to instruments?

I cycle between Totem Arro and Sonus Faber Grand Piano Home speakers. Also, have a JBL 530 in storage but needs a speaker stand.

Another upgrade I considered is going for British bookshelf speakers like Spendor, Harbeth, Graham Audio.

I always recommend going to a symphony... sitting as close to 7th row center as you can and listening as if it was an audio system. It is very eye opening. 

+1 @ghdprentice !  I just saw Steve Hackett do lots of old Genesis stuff from the 10th row dead center at the Warner Theatre in Washington DC.  Amazing sound, amazing show.  All the goodies from Lamb Lies Down plus lots of other great stuff!

It is so valuable to hear how real bands image...and while not a true or full symphony, the memory of artists center, left, right, front and rear is easily remembered when listening at home for comparison.  And yes, lots up to the person at the sound board!

@mapman So this gonna sound stupid but I have a sub and Roon! not one of the premium sub like Rel but Elac 1010, I was using it with the Rotel amp before I got the BAT preamp which have 2 pre out xlr/rca. I use the xlr for the amp but never thought to try the other rca out for the sub. I tried it at the moment and glad I did! 

Still early and it's not "realism" yet but with crossover at 80 Hz I hear more impact from instruments even on Jazz album like Dexter Gordon One Flight Up, Clarity somehow on a few songs from Led Zeppelin, The Doors, Cream than what I'm used to. I don't know why I neglected the sub but I was afraid to insert it in a tube preamp with no sub preout. I'll do more listening in the next few days but sub is a new improvement.

Regarding Roon, I used the Parameter EQ (I don't see a specific DSP toggle in the Muse menu, unless DSP is EQ) few weeks ago to smooth out treble and bass response but didn't notice much difference and the room response was not that bad just a small spike in mid bass when I measured it using a measurement app so I turned it off after a week, that was with the Totem the Sonus Faber in this room definitely needs EQ. Thanks for the help!