are NET Switches worth considering?

Is a switch useful if the only ethernet cables connected to the router are the server and streamer?

+1 +1, I use the LHY-SW10. Absolutely worthy purchase but if you go there, you should go further. You can see how it is incorporated in my system with a pretty full list of equipment.

Only possible reason to do it is if your existing streamer and dac are not filtering out noise coming from your ethernet connection.  The current Stereophile (Brilliant Corners column) has a brief mention of an extremely cost effective solution: TP-Link MC220L ethernet to fiber converters, TP-Link TL-FM311 SFP modules, and a short run of fiber optic cable.  

Another option, a little more expensive, is to just get same basic setup through Small Green Computer:

Systemoptique

Adding a NA Tempus was very positive for my Innous Statement so yes it is worth considering a switch.  The Tempus replaced 3 switches I used to isolate my audio gear from the home network.

Now I feed the Tempus with a Sonore/SGC Systemoptique OpticalModule Deluxe with the upgrade Finisar SFP's.  Previously the Systemoptique was used between two of the switches.

I also added the Muon Pro Streaming System from NA.  Good thing about NA is the 60 day trial.

In my experience Audio switches are definitely beneficial, some more than others.

@rikkipuu sounds complicating and above my audio comprehensive skill set lol. You obviously have knowledge and experience in this area.  What about a simple unit like the Innuos PhoenixNET?