It can be a LOT of things. Are you pushing for absolute highest resolution? What filters? Your issue can be a function of the CPU in your mini, the amount of RAM in the Mini, your DAC's capabilities. And plenty of other stuff.
I run HQP into Roon. I upsample to 768kmhz using Sync-L1 and LNS15 dither. I'm using a new M1 iMac and have no problems. I did add 10ms "Buffer Time" in the HQP settings to help with very rare glitches.
I can run with DSD as the default output mode but I prefer the PCM 705.6/769k better.
Sometimes just demanding less upsampling can make a huge difference. Try 705.6k instead of 768k or try even 384k. These will tax your system less. Take it a step at a time.