Let me try to explain this in a way that is easier to understand. I've posted this before, but I think it applies here:
You start with 10,000 ingots and 10,000 wood. When you craft armor, you are looking for the exceptional and arms lore bonus to be more or less in specific categories. So, you will probably end up making hundreds, thousands of pieces looking for the right pieces to go forward with.
Say you are making a plate mail legs, it takes 20 ingots. Out of the 10,000 ingots, you will be able to make 500. You don't like any of those, so you smelt them down and then make 250 more. Then 125, 62, 31, 16, 8, 4, 2, 1 (something like that). Because of smelting, you were able to make an additional 499 (maybe 500, my math might be off). So, you had essentially an additional extra 10,000 ingots.
With wood, not only is the wood not recyclable, BUT you need 4 or 6 additional bark fragments with it. So, in order to get the same base results as you do with plate (or leather), you not only need twice as much wood (in this example, would be 20,000 as opposed to 10,000) but you need an additional 2,000 to 3,000 bark fragments- none of which you can get back. And those armor pieces thrown away.
And that is not taking into account if you try to enhance without the forged metal of artifacts.
Yes, the end result is pretty damn good, but really, for all you have to go through to get there, it's not over the top.