Only when crafting the items. Works best when crafting items which need recipe scrolls. Also you dont need to have the skill your using on the talisman to be protected. i.e. you can use a blacksmith talisman with failure protection when using the tailoring skill.