1mm may be enough depending on your top cap. Only way to know for sure is to try it.
Leave the stem bolts loose so the stem turns easily on the steering tube. Torque the top cap to 3-4 nm
If it becomes harder to turn the stem on the steering tube, the top cap did not bottom out and you have enough space to adjust the headset correctly. All is good, adjust the headset like normal.
If the stem still turns easily, the top cap bottomed out on the steering tube before pre-setting the bearings. That's no good. Go with the 5mm gap for now, and get a 2mm spacer some day. You will be fine since you don't have a carbon steering tube.