Since I don't enjoy PvP, I doubt Siege would ever attract me much - but were I to want to start PvP, the main reason I'd not go to Siege is time. I have far too much I enjoy doing on my home shard already, and a long list of things I mean to try. I simply would never have enough playing time to split between two or more shards, and I've no inclination to abandon the friends, characters and homes I have already.
Guess I'm not so hard hit as your predicament, because although I'm not a PvPer (Completely PvM), I want the threat of it pushed at me. Sounds stupid, but there's nothing like shatting yur pants, and pegging it, heart thumping wild, to evade that Red name scattering around on the side of the screen - probably homing in on you at blazing speeds ^^;;
A lot of the folks I knew have left Europa now, and moved on, so I personally could handle raising a character each on two Servers. The house issue is off putting though, I'd like one on each shard, whether manual refreshes are needed or whatever...
I seriously want "real" danger in a game, where dieing matters. Sure, it'll get frustrating at times, but I really think it'd be well worth it. After some odd Felluca experiences over the years, I sincerely believe that.