This is pretty much where I stand right now. The only downside would be dropping a large amount of money on a 2nd wheel set.I'm busy researching the 27.5x4 wheels now but the bike I think I want is the 9.6. It's going to be ridden primarily in the winter and on tracks made by snowmobiles . Since we get some freeze/thaw here, I'll probably add studs to the tires. I don't need or want suspension at this point. The 27.5's are probably a good improvement over the 26x4's in terms of patch size.
If the 27.5's don't work, then I'll get a set of 26x5 wheels and have the best of both worlds. So that would be the downside.