If I remeber correct, you cant simply see to where the prisoner wants to go. He lies in chains and doubleclicking him (or her) releases him and he starts escorting you. Normal escorts do ask before they follow you and have to be accepted. Cant remember that with the prisoners. (I do them as often as I see them but i can forget really fast.
)
To see where he wants to go, you can choose the quest-button from your paperdoll (in 2D, not shure on KR). Any escort is listed as a normal quest there and you can get the correct destination for it.
There are many places for brigand camps with prisoners. Thought not every brigand has one. Worth a try could be those camps between Haven Moongate and Old Haven. I ve seen there several prisoners but was too busy trying to pich the chests there. Only take care not to kill those prisoners by accident. (I use to clear such a camp with a blade spirit. Saidly the prisoner often fells to it too if i havent lured the brigands away far enough. Btw the prisoners dont have any loot. Only what they carry on their paperdoll.)