Yes. The literal translation sounds a bit clumsy in English, so it's better to just add the possessive adjective 'your' to the English translation. The 'of you' that you also added would literally be 'to you' (indirect object) in the literal translation. What you're essentially dealing with, when presented in the infinitive, is 'ponérsele a alguien un reparo' ('one gives to someone a criticism').
|