I would let the car downshift by itself. Whenever I play around with my select shift, I sometimes downshift but most of the time I let the car take care of the downshifting while I take care of the upshifting.
Whenever I do downshift, though... I don't experience a 'lurch'. It downshifts, and depending on the speed I'm at, you'll hear the engine working and the car slowing down but it never 'lurches'.