<<Disclaimer: Verify this information before applying it to your situation.>>

Mouths ulcers have been described in untreated celiacs and usually go away or
get much better after going on the gluten free diet.  Why these occur is not
known it may be a direct effect of the gluten on the lining of the mouth, a
nutritional deficiency , or an autoimmune process. Chelitis, or angular
stomatitis( sore corners to the mouth) are also common and relate to the iron
deficiency.  A sore tongue may be due to Vitamin def, esp B12 or Folate.
Not Medical advice
Joe Murray