I have a lot of different feelings on this issue.
Ultimately e-bikes on the trails I ride, where I literally rarely see another human being, and no hikers at all, seems like a non-issue (most of these trails I can not share the names or locations). I plan to get a mid-power ebike for these trails. I'll of course keep a bike for the easier trails, family rides, Crank & Drank (night XC ride), etc.
Another factor is full power ebikes have 2x more torque than a full powered 450F motocross e-bike at it's peak (85NM + 63 ft tq. A KX450F has 32 ft tq. People can argue all they want that it's only about 1 hp, but riding them with a light e-bike, fast rolling tires, and all that instant torque they feel more like a 10hp dirt bike.
On the other hand a guy on the Austin FB e-bike group posted a group ride, stating as being very fast (he actually said 'plan on going 20 mph), on the Saturday after New Years (70 degrees and blue skies, on a Holiday weekend) on the most beginner trails you can basically find in Austin at Bliss Spillar. These are trails where people set there infants down on the path, dogs run free, the occasional wheelchair runs around. Here lies the problem. To ride those trails even with a 10 mph average on a bicycle requires a lot of skill, fitness & experience that these e-bikers mostly don't possess. There is also a clear lack of judgement.