Nope that's not correct. Your caliper appears to be about 10mm or so to high.
What is the Shimano part number on the adapter that you are currently using on the rear? If it has an F in the number you've got a front adapter on the rear, example SM-MA-F180PPA. That number is for a front 180mm post mount brake to post mount fork adapter. The F in the part number designates a FRONT adapter. A rear adapter would have a part number like SM-SA-R180PPA, or something similar.
If that's the case, the answer is not having material machined off the adapter, but getting the correct adapter! However that might be an interesting exercise. I am not currently aware of anyone that carries a 180 pm to pm rear adapter, or if Shimano even makes one yet. Post mount frames are not that common, and it usually takes a bit of time for brake manufacturers to catch up with something new like that.
With most post mount frames the mounts are designed for a 160mm rotor and a direct mount of the caliper to the posts. This is the case with the Anthem. Since 180 rear PM to PM adapters are rather rare at the moment, I would suggest contacting Giant seeing what they recommend. Either that or contacting Shimano and finding out if the do indeed make a 180 PM to PM adapter for their brakes.
This is all assuming that you're rear brake caliper is indeed a post mount caliper. A picture of the caliper with the wheel installed would help a bit.
Good Dirt
What is the Shimano part number on the adapter that you are currently using on the rear? If it has an F in the number you've got a front adapter on the rear, example SM-MA-F180PPA. That number is for a front 180mm post mount brake to post mount fork adapter. The F in the part number designates a FRONT adapter. A rear adapter would have a part number like SM-SA-R180PPA, or something similar.
If that's the case, the answer is not having material machined off the adapter, but getting the correct adapter! However that might be an interesting exercise. I am not currently aware of anyone that carries a 180 pm to pm rear adapter, or if Shimano even makes one yet. Post mount frames are not that common, and it usually takes a bit of time for brake manufacturers to catch up with something new like that.
With most post mount frames the mounts are designed for a 160mm rotor and a direct mount of the caliper to the posts. This is the case with the Anthem. Since 180 rear PM to PM adapters are rather rare at the moment, I would suggest contacting Giant seeing what they recommend. Either that or contacting Shimano and finding out if the do indeed make a 180 PM to PM adapter for their brakes.
This is all assuming that you're rear brake caliper is indeed a post mount caliper. A picture of the caliper with the wheel installed would help a bit.
Good Dirt