Defense is kinda' meh, but going from ~20 to >100 is actually nice for the normal game (to stop weak things from papercutting you to death).
For high-damage things more prot is more useful. I sometimes swap titles along with wearing heavy armor to make high-level stuff hurt less. For some examples, let's say I have the following setups (with screenshots to show they're real) and I take an attack that would normally do 1000 damage.
Naked, no special titles.
[spoiler="90 def, 0 prot."]
[Image: http://i.imgur.com/EKo3xW9.png]
[/spoiler]
910 damage taken.
Armor on, no special titles.
[spoiler="142 def, 16 prot."]
[Image: http://i.imgur.com/wHRF58o.png]
[/spoiler]
721 damage taken.
Armor+shield, no special titles.
[spoiler="153(+7) def, 23 prot."]
[Image: http://i.imgur.com/KEc76xM.png]
[/spoiler]
647 damage taken.
Armor on, defensive titles.
[spoiler="159 def, 33 prot."]
[Image: http://i.imgur.com/eAwdzg4.png]
[/spoiler]
563 damage taken.
Armor+shield, defensive titles.
[spoiler="174(+7) def, 40 prot."]
[Image: http://i.imgur.com/J6MFykl.png]
[/spoiler]
491 damage taken.
[SIZE="1"](My armor has been around for a while and isn't specifically upgraded for prot, Royal Knight armor with prot upgrades or something would fare better. In addition the Rebis Guard Cylinder is a small shield, something like an Avon would do better in both def and prot but I'm here to show my variable setups as they're actually used.)[/SIZE]
So like, going from 30 prot to 50 prot isn't that important, but going from 5 prot to 30 provides a noticeable difference in how many of your organs stay inside of you after an attack.
And for def, going from 125 to 200 def won't cut nearly as much out of a high-level attack as more prot would, but going from 0 to 75 def will stop weaker things from being able to reliably damage you which can help when the dalies are weaker missions or something.
... so yes, unless your main damage stat relies on dexterity or you have a clothing enchant setup from an old event and GM typo that can't be replicated and gives insane gains in the membrane, I recommend most people to wear and rank heavy armor.