IMO there is no car like it on the planet.

overbuilt motor which can make almost 2x the hp/tq on a bone stock block and no head work!

torsen awd, no haldex bs.

only downside is open rear diff (EDL sucks) and its nose heavy.

Didn't even spot the difference!

The RS4 is loooovely, but I don't really have 10k to spend on a car..

I agree with prj. I sold my "Stage 3" S4 Avant two weeks ago and bought an RS4 right away. I had to spend a good amount of money on it right away just to replace the thermostat. (did the full timing belt kit while there). It is not fully mapped yet, but it does feel much much better than the S4 already even though my S4 was in a really tip-top condition. However if my friend would not be a mechanic and I would have to pay for labour besides getting him drunk, I would not have bought a 2.7TT.
Not to mention that if you want a powerful S4 you will need to either build one for a lot of $$$ or buy one which is built already for a lot of $$$. Save your money for a bit longer and just go for the RS4.

Or buy a built Inline 5 cyl., if you want to have the most fun 

The maintenance is not that different between a RS4 and a S4, only the body panels and brakes are what are more expensive really...
I see no point in getting a S4, if you're going to blow so much money on maintenance, at least dump it into a car that has some value.

I think PRJ's forgetting that not everyone digs station wagons.

Not to say that I don't, but that's a consideration.

Trying to compare driving dynamics is difficult to do unless you're doing it back to back. I've had a lot of cars in a short amount of time, including a 2nd generation CTS-V, and Pontiac G8 GT (holden commode). Currently I have a 2.0T ATS with the magneride suspension. In comparison to those, the B5 platform isn't particularly remarkable in terms of how it drives. To be painfully honest, it oversteers just about every time you want to push the car. Some people feel that in this generation of cars, Audi was more interested in the safe driving dynamics that AWD provides rather than it's handling prow less, and that certainly seems true. Only an idiot is going to upset this car enough to do some damage. That just doesn't make for a fun car to drive on a good canyon course to me though.

However, the B5 S4 is not without good points, and arguably that's not really what it's about either,I think most people(myself included) fall in love with the car for a few reasons;
Power delivery- the APB when modified comes on it's power like a freight train. This car, with K03's and boost problems, feels like it hits harder than my 500whp CTS-V. That quite simply, is extremely addicting, and maybe a slightly exaggeration since I don't have the V to compare back to back, but the LSA was buttery smooth where the APB feels like I might self induce whiplash.

Size- The B5 is small, and fairly light. The back seats are mostly a side-show attraction, but for someone like me, I like that, it makes the whole experience more personal. More importantly, it feels small and light(minus being nose heavy). The C4 I have is a 3200lb coupe(4-500lbs lighter than the B5), it's physically smaller in every dimension, but it feels about as heavy as my 4100lb CTS-V. It's not very fun to toss around. It feels as you put 'like driving the titanic'

Simplicity- I'm not going to sit here and tell you that the S4 is anything less than what it's hyped up to be when shit breaks or in terms of mantiance. I'm 20K into mine and it's not even driveable yet.
However, it's just old enough of a platform that it doesn't have any of the modern complex convince systems or powertrain subsystems to make it run. The drivetrain is bulletproof. Most engine problems turn out to be boost problems.It has leather seats, power doors/locks and that's it. No gimmicky garbage to go wrong. No adjustable suspension or 5 layers of traction control. Personally I like having a 'natural' driving experience.