Contrary to popular belief, the RS5 is not direct competitors with the M4 or anything in the compact executive class for that matter. It doesn't carry as much of a heritage nor does it offer any track versions. It's designed to be more comfortable than anything, it just happens to be a fast car.
The RS5 has no direct competitors. If anything, the two-door variant of the E550 (and its E53 AMG successor) as well as the 392-powered Dodge Challenger models are suited most closely with the Audi's personality. But even those serve a different purpose.
The M4 has no direct competitors either. The C63 AMG is supposed to be its rival but like the RS5 it also has not much of a racing pedigree nor is it very lightweight (not to mention even the current M4 xDrive coupe is lighter than the previous RWD C63 coupe!).