I agree with Matt's synopsis here, Damian. At lower listening levels, the 304 should do just fine.
I've made comparisons between the 502 and 702 (models from two years ago, but with the same power supply and rated power as today's 504 and 704) and the difference is considerable once you get above a lower listening level. Depending on your listening habits, a higher model like the 604 or the ever-so-popular 674 will really deliver for you.
Crank it.
Caleb Denison