Help, cannot get 5.1 sound from ANTHEM AVM30

Connected ANTHEM AVM30 with OPPO95 with 6 RCA interconnects and it only pays 2 channel. Also using XLR for 2 channel and that works fine. I have had it work with Coax but somehow cant get that to work anymore but want to stick with the 6 RCA's as this is utilizing the DAC from the OPPO. Toslink connected to cable box works fine in 5.1
I have followed the OPPO set up instructions and also the ANTHEM's, but I still may have something wrong, unless of course something is wrong with the ANTHEM.
Any suggestions please? Thanks.