Since we don’t have access to the source code we can’t make any specific comments as to the performance difference.
As a general comment, the two compilers are developed by two very different teams with vastly different funding sources; ifort/ifx by Intel’s engineers and gfortran mostly by open-source volunteer contributors. The really impressive part is how gfortran, a completely free and open-source compiler, is able to complete so well with a compiler that up until a couple years ago was a paid, high-end product (parallel studio XE).