I am not sure, but I think high end companies making Uber expensive kit, may have a businiess model with which they can survive. They tend to be very small and hand make to order. They don't have large inventories and presumably no high fixed costs.
The problem I think is with the larger High End Companies like ARC, Conrad Johnson, who do make Statement, insanely expensive kit, but there bread and butter is the kit under say $15000. They are larger companies with higher fixed costs and if the lower end "mass market" sales dry up, they will be in trouble. I say mass market, but that is a relative term. HiFi is very much a small niche business now, compared with the heady days in the 70's, when most people aspired to some sort of HiFi system.
As you say though, the smaller companies are chasing each other up in price and down in sales numbers, all chasing the declining number of super rich, who can afford $500,000 systems