If it's turning, but not going back in, push in at the same time while you're turning. If the brake pads/rotors were way worn down, the piston may have come off the end of the jack-screw. Also, make sure you are turning clockwise.
Engine or brake caliper? if brakes,and it has seized up,cover it with a penatrating fluid,and leave over night,and then try again,remember if your talking about brakes,once its back together before driving it pump the brake pedal a couple of times also when u push the piston back,unscrew the master cylinder top and mop up any leakage quickly.