Was puzzling over this for a while, and I think I found something that would work. There are a lot of more expensive solutions, but this was the cheapest as a starting point.
Sonic Alert Traditional System USS 360 Universal Sound Transmitter
You put this next to your Dexcom. There is a sensitivity dial, which you adjust so that it picks up the Dexcom alert. And you can adjust how many beeps it takes to turn on and off. (I'd guess you'd want a single Dex beep to set it on or off).
You basically plug something into the USS360. For example, if you use a light next to your bed (make sure the light switch is turned on), you can set this to turn the light on and off every time it hears a Dex beep! Or you could plug in a radio and have it cranked all the way up. The USS360 would turn the radio on and off for every beep.
It's $54, but it has a 30 day money back guarantee if it doesn't work. And the only other thing you'd need would be a light or something to plug into this that would wake you up.
So the Dex "wakes up" the USS360, which "wakes up" your light, which wakes up you...
Here are some links:
What do you think?