# Parsing contents of .mod files using Python

**URL:** <https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248>\
**Category:** Help\
**Created:** [April 16, 2022, 11:57pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248 "2022-04-16T23:57:58Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![gnikit](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/gnikit/32/1213_2.png) [@gnikit](https://fortran-lang.discourse.group/u/gnikit)\
**Post date:** [April 16, 2022, 11:57pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/1 "2022-04-16T23:57:58Z")

</div>

While developing the fortls Language Server I have always been troubled with how one deals with precompiled module files (.mod files). In contrast to normal modules the source code of the external module is not required to be present. This makes extracting information like public interfaces out of .mod files extremely complicated. Further complicating matters, .mod file contents vary between different compilers.

In `gfortran` at least, .mod files are just an optimised and gziped version of the Abstract Syntax Tree (AST) structure the compiler normally generates. Doing:

```nohighlight
zcat modulefile.mod

```

will output the AST in plain text. However, the problem is that gfortran optimises away the `PUBLIC`/`PRIVATE` attributes that are normally attached to the AST nodes, so the format of the extracted information (and maybe the information itself) are not very helpful

One can generate the normal AST and compare the differences

```nohighlight
gfortran -fdump-fortran-original mysrc.f90

```

Making matters worse, AST representations between compilers also differ so for fortls to be able to parse any .mod file it would also need to be able to parse all compiler AST representations to Python, which is unlikely to happen.

I would be interested in hearing your ideas about how one could parse some of the contents of a .mod file.

1. Are there any ways of getting the public information + attributes and types that do not involve parsing the compiler-generated AST?
2. Are there any Python APIs (GCC, Intel, LLVM, etc.) that could be used to obtain the ASTs in Python?
3. If not, would it be a good idea to create some?

---

<div class="post-metadata">

**Author:** ![rwmsu](https://avatars.discourse-cdn.com/v4/letter/r/48db29/32.png) [@rwmsu](https://fortran-lang.discourse.group/u/rwmsu)\
**Post date:** [April 17, 2022, 2:45pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/2 "2022-04-17T14:45:13Z")

</div>

One solution to the incompatible module file problem that plaques anyone who desires to write libraries and/or applications that can be used by multiple compilers without forcing the user to compile a different version for each compiler he/she choses to use just to have compatible module files is for the Fortran community in general and the standards committee in particular to define a standard module file format in some kind of markup like language (maybe XML etc) that can be read and/or generated as a option to the default vendor formats. This could be controlled by compiler flags etc. A precompile step (hidden from the user) would probably be needed but I’m willing to sacrifice a little compile time for the flexibility that transportable module files would give you.

just my 2 cents

---

<div class="post-metadata">

**Author:** ![gnikit](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/gnikit/32/1213_2.png) [@gnikit](https://fortran-lang.discourse.group/u/gnikit)\
**Post date:** [April 17, 2022, 6:15pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/4 "2022-04-17T18:15:05Z")

</div>

Yes that would be ideal, a strategy similar to C++'s headers would be nice. I think the closest Fortran has to this are submodules.

As for adding the .mod file generation to the standard, I suspect that would be a hard sell to the committee members. Other than allowing for compiler agnostic Language Servers to provide information for external modules, I don’t necessarily see a lot of other applications.

Maybe it will make the lives of compiler Devs easier (in the long run) and improve the available tooling for the language and maybe it won’t. My guess is that it would require a disproportionately greater amount of work to standardise and implement when compared to the potential perceived benefits.

If any J3 committee members want to share their thoughts on this it would be great.

---

<div class="post-metadata">

**Author:** ![rfarmer](https://avatars.discourse-cdn.com/v4/letter/r/3d9bf3/32.png) [@rfarmer](https://fortran-lang.discourse.group/u/rfarmer)\
**Post date:** [April 17, 2022, 7:12pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/5 "2022-04-17T19:12:58Z")

</div>

My library gfort2py [GitHub - rjfarmer/gfort2py: Library to allow calling fortran code from python](https://github.com/rjfarmer/gfort2py) parses most of a gfortran mod file (The in dev version [gfort2py/module\_parse.py at dev2 · rjfarmer/gfort2py · GitHub](https://github.com/rjfarmer/gfort2py/blob/dev2/gfort2py/module_parse.py) does a better job of tracking all the data present, even if I don’t do anything with it).

Mod files are definitely not the way to go if you need to support multiple compilers. In that case all you can do is parse the fortran source code (like f2py does). Anything else will be entirely compiler dependent.

---

<div class="post-metadata">

**Author:** ![everythingfunctional](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/everythingfunctional/32/176_2.png) [@everythingfunctional](https://fortran-lang.discourse.group/u/everythingfunctional)\
**Post date:** [April 18, 2022, 1:39pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/6 "2022-04-18T13:39:18Z")

</div>

> [@kargl](#):
>
> Information about a private subprogram is not recorded in a gfortran \*.mod file. In fact, a private subprogram might be completely in-lined, so you have no idea if a private subprogram existed or not.

This can’t (I believe) actually be true. It would prevent submodules from working correctly. When compiling a module, it cannot be known that a submodule will not be used. And because a submodule has access to private module entities, they must always be included in a `*.mod` file. I’ll admit that a submodule wouldn’t be of any use without any interface blocks, but the presence of an interface block also doesn’t mean that there will necessarily be a submodule either.

---

<div class="post-metadata">

**Author:** ![everythingfunctional](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/everythingfunctional/32/176_2.png) [@everythingfunctional](https://fortran-lang.discourse.group/u/everythingfunctional)\
**Post date:** [April 18, 2022, 2:15pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/8 "2022-04-18T14:15:41Z")

</div>

ok, now try and compile the following:

```fortran
submodule (foo) foo_s
  implicit none
contains
  function baz()
    integer baz
    baz = bah()
  end function
end submodule

```

It of course wouldn’t be useful for anything, as nothing can see `baz`, but it’s still valid Fortran.

---

<div class="post-metadata">

**Author:** ![gnikit](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/gnikit/32/1213_2.png) [@gnikit](https://fortran-lang.discourse.group/u/gnikit)\
**Post date:** [April 18, 2022, 2:19pm UTC](https://fortran-lang.discourse.group/t/parsing-contents-of-mod-files-using-python/3248/9 "2022-04-18T14:19:11Z")

</div>

I think it would help if one could compare with the unoptimised AST. Basically, some of these `UNKNOWN` correspond to the visibility attribute

```auto
Namespace: A-Z: (UNKNOWN 0)
procedure name = foo
  symtree: 'bah' || symbol: 'bah'          
    type spec : (INTEGER 4)
    attributes: (PROCEDURE PRIVATE MODULE-PROC FUNCTION IMPLICIT_PURE)
    result: bah
  symtree: 'bar' || symbol: 'bar'          
    type spec : (INTEGER 4)
    attributes: (PROCEDURE PUBLIC MODULE-PROC FUNCTION IMPLICIT_PURE)
    result: bar
  symtree: 'foo' || symbol: 'foo'          
    type spec : (UNKNOWN 0)
    attributes: (MODULE )

  code:
CONTAINS

  Namespace: A-Z: (UNKNOWN 0)
  procedure name = bah
    symtree: 'bah' || symbol: 'bah' from namespace 'foo'

    code:
    ASSIGN foo:bah 1
    

CONTAINS

  Namespace: A-Z: (UNKNOWN 0)
  procedure name = bar
    symtree: 'bah' || symbol: 'bah' from namespace 'foo'
    symtree: 'bar' || symbol: 'bar' from namespace 'foo'

    code:
    ASSIGN foo:bar bah[[()]]

```