For HD audio on the PS3, you need to be hooked up via HDMI (PS3 can't send lossless over Toslink). Then, make sure your PS3 audio setting is set to LPCM.
You won't see anything light up on your receiver except "direct" or in the case of my 876, "multichannel"...The PS3 does all the decoding.
The best way on a PS3 to tell what codec it's using is to hit select on the controller when the movie plays. Look at the top ribbon, and it will tell you what audio it's putting out.