Okay, I'll bite.
This is a tough call and every clown on this site will have an opinion. This bozo say the biggest problem is that we are not in the same room with you and seeing these bikes. It is all theory and gestimation. Seat tube is nearly irrelevant to fit.
You sound like you have a long torso, have adapted your medium bike to your proportions, and become accustomed to it. Anything else will feel strange but that doesn't mean it is wrong.
Relative seat height/bar height is not so much an indication. Stem height can be limited by steering tube length. Barring that spacers and stem angles can expand bar height greatly. Seats can go as high or low as needed. This is largely independent of bike size.
Seat Fore and Aft are very adjustable as rails allow for adapting and Set-back seatposts expand that. Stems come in all sorts of lengths. This is also independent of frame size.
Seat position is GENERALLY set by placing the knee over the pedal (KOP). The pedal spindle runs across the midpoint between the ball at the base of the great toe and the ball just below the little toe. That set, the seat height is coarsely set high enough to get extension on the leg but low enough to avoid rocking your hips. (There are formulae and knee angles but this is a good thumbnail.) Seat fore/aft is set by placing your fixed foot on the pedal, placing the pedal at 3 o'clock and moving the seat forward until your patella (knee cap) just overt the pedal spindle. Subsequent fine tuning of height and fore/aft are needed.
ALL THIS IS INDEPEDENT OF BIKE SIZE!
The key point specific to bike size and your body is the twitchiness of the steering. Climb on a Small bike and you will see what I mean.
My call is that your torso overwhelms the TT on the medium and would be better suited to a bike for a taller man. The large is 2 cm longer in the wheelbase and TT and handling will relax under you.
That said, do not decide without riding both the medium and the large. Other models of the Giant line are not substantially different in the critical dimensions so try them. Bring your Giant in to the shop and compare them side-by-side. As we are not with you this is the best index I can offer. I use it every time I can when I change bikes for myself or friends and teen racers on my team.
