Is Budget Audiophile an oxymoron?

It depends, depending on the type of audiophile.  For the type of audiophile only focuses on SQ, budget audiophile is NOT an exymoron.  For the type of audiophile purports price defines hi-fi or plays music, then yes budget audiophile is an oxymoron.  So, to recap

1. For SQ-centric audiophile - budget audiophile IS NOT an exymoron.

2. For PRICE-centric audiophile - budget audiophile IS an exymoron.

Got it?  What type of audiophile are you are or have you been?

No! The issue with many frugal Audiophiles is the constant need to devalue audio components that have more potential. "Law of Diminishing Returns" or "Rabbit Hole" are common negatively biased catch phrases that only serve personal validation. Good enough will satisfy some Audiophiles but not all!

Scan your local Facebook Marketplace. There are bargains galore to be found. 
See a pair of ADS L520s or L570s in good shape for $300 a pair?  Pounce. 
A guy is selling a pair of Boston VRM 50s near me for $80!  Those are fantastic compact speakers. 

Audiophile Definition: An audiophile is a person who is deeply enthusiastic about high-fidelity sound reproduction. Their primary goal is to achieve the highest possible sound quality, listening to music or recordings in a way that is as clear, accurate, and faithful to the original source as possible.

So simply obtaining gear, free or otherwise, for a sonic improvement would qualify.

Many of us strive for Sonic’s noticeably higher than what is offered in box stores.But better parts, better engineering, in a small niche market drive up costs and prices aka better audiophile components are significantly costlier.  

As with anything technological, there are levels of performance. However, enjoyment of music with electronics can happen at any level of sound quality. 

If you long for better sound, consider budgeting for audio. Set aside a portion of the discretionary spending to save for audio gear. It will be well worth it and it will not impede more important things. 

You will be paying with time. While not having high end sound now, you will not be stressed financially. In time, you will have a much better system and better listening experiences than you ever thought possible.  :)