I actually figured out what is going on with me!!
Since I was a teenager I always dealt with periodic acne. When I avoided 
junkfood it wasn't too bad.  No matter what I ate I periodically would get 
pimples. When I was vegetarian I would get boil type pimples occasionally.
I tried every Paleo variation under the sun. When I ate very Paleo It 
wouldn't be too bad. For the past couple of months my skin was clear and I 
thought I finally had it under control.
I went skiing a few weeks ago and got sunburn on my face.  I put aloe gel on 
it for a few days and I broke out.
My complexion was never great even when I didn't have blemishes. Now I had 
pimples again.
I was looking at the redness on my face and It dawned on me... It looked 
similar to the couple times I had jock itch. It was some kind of fungus.  The 
stress of the sunburn and the moisture of the aloe let it get out of control 
again.

I went to the store, and put an anti-fungal cream on my face and............ 
It is all going away -- quickly!!!
When I ate a high carb diet -I must have set my body up for some kind of 
fungus that I kept in mostly in check with a Paleo low carb diet.

Does anyone else have any experience with any kind of fungus?

I appreciate everyone's suggestions. After I sent off the letter to the list, 
I figured it out.

-David