Depends slightly on whether or not it's a PvM or PvP character.
In PvM, you could probably get away with 110/115 swordsmanship, although your hit chance will be less as will your evasion than a legendary swordsman, obviously.
For PvP, you want the absolute maximum hit chance and evasion. Most people have legendary combat skills who are PvPing, therefore you'd be at a disadvantage if you tried running with less.
Some figures for you (ignoring any DCI and HCI that may apply)
120 swordsman vs. 120 swordsman = 50% chance to hit.
115 swordsman vs. 120 swordsman = 48% chance to hit.
110 swordsman vs. 120 swordsman = 46% chance to hit.
100 swordsman vs. 120 swordsman = 42% chance to hit.
As you can see, it is significant in terms of hit/defence chance but for PvM it wouldn't matter as much as it would in a PvP environment.
Would also suggest taking your tactics and anatomy higher than they are now to maximize your damage output, as well as resisting spells to 120 if you're PvPing.
Hope this helps. Take care.