Big wheels just make more sense on rough terrain. Their advantage was held back by:
1) Crappy geo - usually long chainstays that made the front feel heavier, which resulted in poor handling. You might think that extra weight on the front is good for traction, but bikes works counter-intuitively. The front wheel will want to take-a-digger (push) in turns when there's too much weight on it, breaking traction due to the ground shearing. The body instinctively knows this and will subconsciously get its weight back, but that is not a strong position for cornering compared to having the chin over the stem (and shoulders over the bar).
People would judge bikes by how easy they were to manual. It was important to those in the know, but some others couldn't rationalize with this method of judgement. People tried to counter this issue with lightweight wheels, tires, handlebar, frame, etc. All this helped, but I'd say that it was finally addressed by extending the front wheel further out (slacker HA, longer reach, longer fork). This had the effect of making the body's CoG closer to the rear, further from the front. Balance was regained, with a longer front-center to match the longer CS. Some 29ers balanced things with a shorter CS instead, but they tended to not be as fast as long wheelbase options; they were more like big-wheeled BMX bikes (Honzo, Riot).
-----
2) Parts that were too flimsy. The wheels increased in size, but the rest of the bike did not increase proportionally. Weight weenie movement didn't help either. Carbon was selling due to the promise of stiffness. Carbon's too expensive for many, so new standards rolled out to address the issue - tapered steerer, bigger fork stanchions, boost spacing, wider rims...
People slowly opened up to trying heavier tires, rims, and even inserts, on top of living with other heavier parts. Long wheelbase geo masked how heavy the extra weight felt. Heavier-duty parts became more of a norm, seeing multiple brands offering options. For example, a Fox 36 seemed pretty heavy-duty until the 38 came out--now people consider a 36 a mid-travel option. The Fox 34 today seems like a very light-duty trail option compared to back in 2012.
---
29 wheels were adopted for XC race and trail bikes early on, since these crowds didn't really care much for balanced geo nor heavy-duty parts. The logic was that whatever worked for the winning XC racers, should have been good enough for the general populace. When everyone was on mostly the same equipment in the races, rider fitness and skill were mostly responsible for differences, so equipment progress was kind of slow, with racers simply demanding lower weight and more efficient power transfer.
The gravity scene wasn't willing to give up their dialed geo and stout parts. It wasn't until these things were solved that they found that bigger wheels were truly superior. Still, they notice some downsides like the rear tire buzzing their ass when they bottom out the bike's suspension and their own lower body's suspension. Gee Atherton had a high profile crash caused by slamming hit butt on his rear (mullet 27.5 rear) on a big huck, that was caught on film (
Athertons in "The Knife Edge").
----
Long story short, it's improper to generalize. People called 29ers a fad due to fanatic hyping. 29er wheels aren't so good that discerning people would choose them over an equivalent smaller wheeled option with better geo and stouter parts. For example, I'd take a Vitus Sentier 27 over a Vitus Sentier 29, since the 29er version has ~15mm longer chainstay which spoils the handling. The cheapo 29er wheels are also weaker than 27.5 wheels, and the 29er fork of equivalent travel will be flexier (less predictability and less confidence-inspiring). Among high-end expensive options, a vast majority of bikes that make it to my best-of-the-best list are long wheelbase 29ers.