Yes, the color is lost... The only thing you can do is reset it to default color for the item (as if made from normal off-white cloth) or dye it yet another color.
Do note that, while expensive, there were ways on the Oriental shards to dye cloth the AoS colors, so if you can get that color cloth (expensive), you can replicate the look (but not the titles/names) of the original in a new item (then slap a Clothing Bless Deed tailor BOD reward on it to make it blessed).