Are all speaker relays removable? One receiver with two relays?

I'd go with that. If you cleaned front panel switches, one trouble-maker is the 'tape monitor' switch which is notorious for knocking out one channel mysteriously. You might have already sorted that out.

Does replacement require soldering in my case? (I didn't see any socket) My concern is that I am not sure if the relay is still good as the symptom is gone. A few days ago the left channel became very quiet but if I fed the right preamp-out to the left main-in (jumper), the left became normal. Then the problem disappeared "by itself" after, I believe, unplugging and plugging the power cord. After that I did a general cleaning of the front knobs and rear connectors. So far both channels have been working well with many deliberate plugging in and out and powering on and off. Maybe I should wait and observe for a while before taking any further action? "If it ain't broke, don't fix it"?

I've removed the covers to relays like that before. They aren't really intended to be removed, so its likely they will not snap back in place very well once you've cleaned the relay contacts.

IME cleaning relay contacts is problematic. When the contacts have got to the point that they are able to corrode, the relay is on borrowed time and you will find that the contacts will need service again, sooner rather than later. If you can find a replacement you are better off replacing the relay itself. IOW only clean the contacts when you have no other choice.

Do some speaker relays have fixed covers so they have to be desoldered for cleaning?

Do not try to clean the contacts, just replace it with a Fujitsu FTR-F1AA024V
https://www.mouser.ca/ProductDetail/Fujitsu/FTR-F1AA024V?qs=llYdsnK8Kc2ETH%252B7Ws971Q%3D%3D

I also find a black box of the same size and shape above the relay (please see the photo) but without any identification on it. What is that?

It is a similar relay, could be replaced by someone, maybe the data is printed on the side wall instead on top.

Thanks, Erik, judging by the photo I posted, do you think the black plastic cover of the relay can be pulled off? If so, I am thinking to clean the relay with a strip of thick paper with electrical cleaner without removing the relay itself.

Relays are only removable if socketed. 

I've had older Yamaha pro amps with relays and no sockets, so I added them. :)