Oscilloscopes - what specs to look for?

Yes, there is alway some antenna, but there is a pronounced increase in noise when shorted probe touches the other circuit (shorted with wire loop or two-pin adapter).
Battery powered scopes also show this to some extend, because of these tiny capacitances. Few pF becomes only few hundred ohm of reactive impedance at 100MHz frequency.

Most of what I was going to say has already been said here, mostly by @georgehifi. I do a lot of audio electronics, I use a QA401 for measurements (it's truly incredible value for money when you compare it to the other stuff out there) and an uncalibrated, analogue 100MHz Tektronix to 'see' the signal... usually square wave testing for oscillation which I find much easier on an analogue scope. It's worth remembering that although we can't hear higher than 20kHz, an amplifier can oscillate at any frequency in it's bandwidth so high resolution is a must.
I've used digital scopes which were useful at the time for the FFT but I don't need that now I have the QA401... if you set it up right you can measure distortion down to about 0.0001% along with all the other measurements you'd expect to run on an amp. Other than a 4 1/2 count Fluke DMM i could get by without any other measurement tools.

I made some searching on oscilloscopes and concluded that non-toy scope starts at about $200.  There are three companies wort consideration: Siglent, Rigol (about the same) and Owon or Uni-T at the lower end.  Cheapest you can get is Hanmatek exact copy of Owon, but it is unknown brand, scope has two channels but no external trigger (deal breaker) and no test/calibration points (deal breaker as well).  This 100MHz Hanmatek would cost about $230 ending up $250 total, including tax.  Rigol has special at $299 (free shipping, no tax) on 2-channel scope, that they normally sell for $369.  It is DS1202Z-E  - a 2 channel 200MHz scope with 1Gs/s.  Of course 1Gs/s for 200MHz bandwidth is not enough but still, 200MHz is more transparent, rise time is 1.75ns and the probes are 350MHz.  This scope also includes free options worth probably as much as scope itself - Advance triggering, Memory depth upgrade, Real Time Waveform Record/Relay and Serial Bus Analysis.  It has only two channels but 4 channel scope is $50 more for 50MHz only.  Either scope is good choice, but I used to work with 100MHz 2GS/s two channel Agilent, that was sufficient most of the time.  In addition Rigol now gives warranty extension from 3 to 5 years for direct buys (or authorized dealers).  I bought this scope and just finished playing with it.  It is wonderful, well build instrument on the par with $1k Agilent.  It is very intuitive, in spite of tons of options, and has immaculate user's guide in pdf (plus other guides).  Of course I don't have to have it, but it was my tool at work and I feel I need to have it (bucket list).  Unfortunately the cheapest protective case that makes sense (size and quality) is one made by Rigol that is $58 at Newegg.  I bought it as well.

@jaytor  2-channel 200MHz scope is still 200MHz per channel.  Sampling rate drops by half but analog amplifiers are still 200MHz.  Some scopes (including mentioned Hanmatek) don't even drop sampling rate (dual A/D converter).