I kind of thought thats what that was. Still, you have a weak point where the GFI adapter plugs into the wall. For just a few dollars at home depot you can buy a new GFI outlet, turn off your breaker for that outlet, and just a few screws later its changed out.
I am sure it is one of those things an electrician would go nuts over. For me, I would probably build a housing around it, seal it to the wall, and call it a day since that wouldn't require me to run out to get an outlet. Then a year later I would have a problem and I would just think back on the day when I was told to change it out. That is just my luck with reefs. I always try to skimp a little here and there and it always comes back to bite me in the behind sometime in the future.