Technique can also be a killer to even the most fit cyclist. I regularly ride with triatheletes, the Ironman variety. When the trail is fast and straight they can kill me but i just let them go because as soon as it get twisty and turny and technical I catch right back up. At the end of the ride they are completely shelled and I, being in much less shape then them, am too but they shouldn't be. It is only because they are working so hard to make it through things that I just fly through.
Practice your cornering, if you are braking turning then accelerating through the corners, that is inefficient and you are wasting energy. Practice pumping through the rollers, using gravity to accelerate you down the hills and then attack over the top of the next hill.
If you are getting killed by guys you regularly ride with, watch what they are doing in the technical sections, emulate them and frankly nothing makes you faster than riding with people that are faster than you.