Fantasy Island is a step above a stationary bike.
Definitely don't plan a trip around Fantasy Island!! Like what was said above, most of the locals(count me in this) won't drive across town to ride it. It is flat and twisty and that is all. Spin, turn, spin, repeat as necessary. It is good if you live close to it and need a quick workout, as it is better than sitting in a gym looking at the wall and spinning, but there is not a bit of challenge to it unless you just started mountain biking yesterday.
50 year is worth driving for because of the unique terrain, and scenery.
I wouldn't make a special trip for Sweetwater. It is more fun than Fantasy Island for sure, with more hills, but again, no technical challenges.
Mt. Lemmon rides, have a lot more challenges, and are done as shuttles by most because it is up one way steep, down the other way steep. I do like trying to climb Milagrosa though, and Agua Caliente is a nice techy climb.
Arizona trail from Pistol hill is fun and scenic down to the bridges, then gets boring as hell from then on south.
Tucson Mountain Park has a lot of trails, but not easy to find them if you don't explore a lot there, or go with a guide. I explore every little goat trail I can find there, and am amazed at how many people ride there, and do the same loop, over and over with no clue how much more there is. Robles at the south end has quite a few trails, and Brown Mountain to the north end is a very fun technical trail. Inside the park I like John Krein(techy) Gates Pass around Golden Gate Mtn(techy) Hooligan(techy), and the east side of Cat Mtn. to Robles connector(techy), which I am now Henceforth by Special Decree Officially giving the name of "Cool Cat".