A Mage Weapon lowers your Magery Skill by the amount listed.
In return, it will allow you to use that weapon as if your Magery Skill was the proper Weapon Skill for the weapon. So, if you have a Staff of Pyros, and your character has 120 Magery skill, they will swing the weapon as if they have 120 Macing skill, for purpose of determining chance to hit, and your bonuses to defend.
However, you cannot use special moves of the weapon (things like Whirlwind), if you do not have the actual weapon skill and tactics at the proper levels.
You can use items with Magery bonuses (like jewelry) to bring your skill back up to its normal level. So, If I use a "Mage Weapon -24" Bow, at 120 Magery, it would lower my Magery to 96. I could then put on jewelry that adds up to +24 or more magery, and be at 120 Magery again (which will also increase my chance to hit with the Bow)
If a Mage Weapon has Spell Channeling, it will remain in your hand when you cast spells. If it does not, it will fall into your backpack, and your Magery Skill will return to normal as you choose a target.
Holding a Spell Channeling Mage Weapon with a skill penalty, and not using anything to offset the skill reduction, will allow you to gain skill quicker, by casting lower level spells to gain skill. This only works if you have the room in your skill to go up, without the weapon.
One can also gain Magery from weapon combat, as the Magery skill is treated as the weapon skill. It will only do so, if you do not have the actual weapon skill at a higher level than the adjusted Magery skill. One can even gain magery using a Mage Weapon with 100% poison damage, by the method normally used to train weapon skills with a golem - and because you will be doing so little damage to the golem from not being a warrior type, the golem will probably regenerate its hit points faster than you can lower them with the weapon, even on one that does not have 100% poison resist.