My question is could this cause issues down the road?,
No.
should I just let go, or start a return?
There is a really important reason why you should not do that: There is no guarantee the next speaker will be better, or won't have other QC problems, like broken drivers, crossovers or cabinetry. If the speaker is performing well I'd leave it alone. :)