Perhaps Oppo HA-2 for about the same money might be to your liking? I have it and the sound is outstanding.
The HA-2 is an excellent accessory for personal listening. Makes my Shure 415 sing so sweetly. But,
strapping it to a phone makes it a bit unwieldy if you want to listen while you're doing something like mowing the lawn or even just walking. You need big cargo pockets. I keep looking for that "Tunebelt" I bought from LL Bean back in the 80s. Like a foam rubber belt type fanny pack but sized for an old school Sony Walkman with a velcro cover flap.
As much as I love the HA-2 for desk, sofa, or even flightseat listening, the idea of a simple small dedicated player that fits in my shirt flap pocket appeals to me more and more.
Choosing between the Fiio X3 or X5, I would definitely consider what you'll be doing while using it. If you'll be active then the possibility of damage makes going for the less expensive version a smarter choice. A bit better battery life wont be needed and a marginal potential sound quality difference may not be appreciable. In those environments.
On the other hand, if you're looking for a listening device for the car/plane on those long trips or desktop listening while drafting your novel when the babe's asleep, go for the X5.