Unfortunately, this is one of those topics where you first have to define your terms. The term bypass cap is used quite differently by audiophiles and electrical engineers. Audiophiles typically use the term to refer to multiple capacitors wired in parallel in the same physical location, or nearly so. The purpose of this (at least within the realm of high-end audio) is to create a single, conjugate capacitor that outperforms ("sounds better than") any single cap in the same application. The mix of caps is usually arrived at through a series of listening evaluations, so the process is mainly subjective. For the purposes of high-end audio, bypass caps are almost exclusively film types (polyester, polypropylene, polystyrene, teflon, etc.) The theory is simple enough - smaller value caps may (for a variety of reasons) have better high-frequency performance than larger ones (especially when the larger caps are electrolytic), so combining large and small caps may (when everything works well) result in a single capacitance with outstanding full-range performance. As with many things in audio, this is easier said than done, but it remains a very popular practice among hobbyists.
Electrical engineers usually use the term to refer to multiple capacitors connected in parallel electrically, but distributed along the power supply path so that the smaller caps with superior high-frequency performance are located adjacent to critical circuit elements (particularly wide-bandwidth gain blocks). The intent is to improve the stability of these circuits by providing a superior low-impedance path to ground on the power supply lines. Many modern wide-bandwidth circuits will not operate properly (or at all) without these local bypass caps in place.