Clearly pump site rotation is more important but the Dexcom manual does state:
Avoid using the same spot repeatedly for sensor insertion. Never use the same site for 2 sensor sessions in a row.
I'd say you're lucky if you don't have any irritation at all. I simply alternate between sides of my abdomen. Since the wire is so small it would be nearly impossible to insert in exactly the same place. Their recommendation is probably less about effectiveness and more about not building up scar tissue.
I also use a pump and I have to choose a new site that is not near a recent pump or CGM site. I have read that a CGM does not cause scar tissue, but a pump can. Something being delivered into your body without site rotation can cause scar tissue, but a CGM is not delivering anything, so it will not cause scar tissue. Existing scar tissue can cause problems at a CGM site, but the CGM will not cause any additional scar tissue.