Huge pages are virtual memory pages which are bigger than the default base page size of 4KB. Huge pages can improve memory performance for common access patterns on large data sets. Huge pages also increase the maximum size of data and text in a program accessible by the high speed network.
The Cray Aries interconnect can map more pages using fewer resources meaning communications may be faster. Your code may run with fewer TLB misses (hence faster) however, your code may load extra data and so run slower.
Only way to know is by experimentation. Use modules to change default page sizes (man intro_hugepages for more information). module load craype-hugepages2M and craype-hugepag8M are most commonly successfully on Cray XC.