I would 100% go down the 320d route.
My mums aging Rover 75 has the 2litre turb-deez from BMW and it is a monster of an engine compared to other deseasals i have driven!
Don't be put off by size, generally a 3 series isnt going to be much bigger than say a Focus or Golf for normal use unless you live in Paris?