Dejarme con el pastel/marrón

Dejarme con el pastel/marrón
 

how can I say dejarme con el pastel/marrón?.
When we are in a bad situation and someone leaves and let us alone to solve the problem.

One possibility is: to leave someone in the lurch.

If the phrase is not meant to be negative, as to leave me in a lurch is, the I think the best translation is to let me figure it out, or to keep it more idiomatic, to leave me with a tough nut to crack.