Where to buy fuse for fuse panel?

Sounds like this is this a large cartridge fuse intended for main disconnects, which is not the correct type fusebox for household circuits. But no don't overfuse; that is not safe & if you had a fire then your insurance might likely be voided. I believe for that you need the screw-in base Edison type fuses available in 20A size, etc. That is a Buss W20. The thread looks just like a light bulb's. Buss sells them in plastic housing now (no more ceramics) although 30 amp W30's are still available in ceramic.
The larger cartridge fuses may be available in a 20 amps version but I'm not sure about that. Try asking at an electrical supply warehouse.

Years ago I remember some type of metallic adapters that could be slipped over the smaller version cartridge fuses ends (2 required per fuse) that would allow the smaller fuses to be installed in the next size larger holder. Soory I don't remember a brand name or any other specific information. Check with electrical supply houses that carry both size fuses or an extremely good hardware store. Absolutely do not use a fuse larger than the wire (or recepticles) are/is rated for.

Is your fuse panel old? If it is, you probably have the newer S-Type fuses and are attempting to place them in Edison base fuse holders. S-types are smaller in diameter for a given amp rating than an Edison. Go to a Home Depot, ask for "Edison Base" plug fuses. Be very careful - all amperages are the same size.