So the Cortex M4 processor has an internal cache see the CMCC peripheral in the datasheet.
Try in your main code to disable the cache:
CMCC->CTRL.bit.CEN=0;
Then run your code and see if it works.
Again Atmel writes all their drivers with the cache turned off and if you turn on you are on your own... If you want to argue with the tech-less support about this please do, I have given up. Also check that the registers for the GMAC are defined correctly in the include files. The SAME70 had some of the GMAC register addresses wrong and I had to fix them before it would work. Took weeks before Microchip tech-less support would acknowledge their error, they kept saying the datasheets were wrong...
If the code runs better with cache off then it is time to review the drivers and address issues. I had to do this with the SAMD51 chip and flash driver, specifically after writing to flash it would read as being incorrect because the cache was caching the old data.
If you need help beyond this you can contact me off list.
Trampas