You would need to embed a Python interpreter to fully match Fypp. Perhaps you could settle on a subset of Fypp syntax. The idea has been floated before here: Preprocessor support · Issue #78 · fortran-lang/fpm · GitHub
Yes, the initial parser should be limited to ifs and loops, and I guess that would cover most of the use cases.
1 Like
Why? as long as a sufficiently recent Python interpreter is visible from the command line, and that the user can do pip install fypp neither fpm, stdlib and for that matter any other package would need to embed anything.
I see zero difficulty in doing this: collect the fypp pre-preprocessing macros from the toml file just as for the C-preprocessor. Within fpm assemble them in a allocatable character string to then send them to execute_command_line
We would need only to transform parts of what is being done currently in the fypp_deployment.py script here within fpm:
pre_process_fypp
def pre_process_fypp(args):
"""use fypp to preprocess all source files.
Processed files will be dumped at <current_folder>/temp/<file.f90> or <file.F90>
Parameters
----------
args :
CLI arguments.
"""
kwd = []
kwd.append("-DMAXRANK="+str(args.maxrank))
kwd.append("-DPROJECT_VERSION_MAJOR="+str(args.vmajor))
kwd.append("-DPROJECT_VERSION_MINOR="+str(args.vminor))
kwd.append("-DPROJECT_VERSION_PATCH="+str(args.vpatch))
if args.with_qp:
kwd.append("-DWITH_QP=True")
if args.with_xdp:
kwd.append("-DWITH_XDP=True")
if args.with_ilp64:
kwd.append("-DWITH_ILP64=True")
optparser = fypp.get_option_parser()
options, leftover = optparser.parse_args(args=kwd)
options.includes = ['include']
if args.lnumbering:
options.line_numbering = True
tool = fypp.Fypp(options)
# Check destination folder for preprocessing. if not 'stdlib-fpm', it is assumed to be the root folder.
if not os.path.exists('src'+os.sep+'temp'):
os.makedirs('src'+os.sep+'temp')
if not os.path.exists('test'+os.sep+'temp'):
os.makedirs('test'+os.sep+'temp')
# Define the folders to search for *.fypp files
folders = ['src','test']
# Process all folders
fypp_files = [os.path.join(root, file) for folder in folders
for root, _, files in os.walk(folder)
for file in files if file.endswith(".fypp")]
def process_f(file):
source_file = file
root = os.path.dirname(file)
if not os.path.exists(root+os.sep+'temp'):
os.makedirs(root+os.sep+'temp')
basename = os.path.splitext(os.path.basename(source_file))[0]
sfx = 'f90' if basename not in C_PREPROCESSED else 'F90'
target_file = root+os.sep+'temp' + os.sep + basename + '.' + sfx
tool.process_file(source_file, target_file)
Parallel(n_jobs=args.njob)(delayed(process_f)(f) for f in fypp_files)
return
1 Like
It was my comment on @FedericoPerini’s idea to have a Fortran implementation of Fypp; the purpose of this being independence from Python. Personally, I don’t find dependence on Python anything bad, because I use it all the time and as @kimala has said it is so common at this point. I guess the root problem is that Windows does not have a system Python interpreter.
1 Like
Wouldn’t that end up being in competition with the Fortran preprocessor currently under the works by the standard committee?
Yes @hkvzjal: @ivanpribec got it right that I meant to actually implement the whole pre-processor including the parsing stage in Fortran. Of course a quicker first version is what you are saying i.e. to wrap calls to fypp in the same way that fpm already does e.g. for pkg-config or the compiler executable, etc., with the requirement that Python and fypp must be installed and findable.
no more than what fypp is already, imho.
Not really. The committee is standardizing the existing cpp-like syntax (formally-based on C23 with some Fortran amendments), and that preprocessor doesn’t come with looping constructs like Fypp does. The scope of the preprocessor undergoing standardization can be read here: https://j3-fortran.org/doc/year/25/25-114r2.txt
1 Like
I think this is the way to go, propose a “simply fortran” main branch and a “lightweight-dev” branch with the fypp files.
That would be extremely limiting and partially useless: there are already tools that do that (e.g. C-preprocessors) and would be terribly ironic if, in the attempt of building a tool to avoid the fortran user to reinvent the wheel every time, we would get lost in reinventing another wheel.
In my fypp workflow I have a python dictionary of functions, where by function I mean a really working python function. Can we match this with a fortran based preprocessor?
To be fair, I got a lot of complaints about my extensive usage of fypp but no other good alternative nor help in development. So yes, we can listen carefully to what the umarell say, but who’s doing the build?
2 Likes
I think this is a good idea, provided it doesn’t cause other complications elsewhere. Either that or increase the visibility of the Fortran-only branch, but putting it in main is probably the easiest way to achieve this visibility. The fact fypp easy to get into is not something potential contributors immediately know or see. I imagine the reality for a quite a few potential contributors is:
> check out main > see unfamiliar aspects of the code > unsure how much extra work it is to familiarise themselves with it > put plans to engage “on hold” due to unknown additional time investment.
P.S. I checked out the stdlib-fpm branch last night, then returned to main with the discourse comments at the back of my head, and it all started making a lot of sense.
2 Likes
Thanks to joining this community I have taken a very deep interest into modernizing the codebase I worked with during my PhD, an electronic structure software called GAMESS which is written in mostly F77 with little F90. I’ve been able to apply many of the things I’ve learned through reading this discourse in practice and have some very cool plans to develop things. I am also very grateful!
4 Likes
The source codes in stdlib vary in readability. stdlib_ascii.f90 has small, commented string functions. Besides using stdlib, a user could just grab a needed function. stdlib_stats_corr.f90 has no comments and starts with many functions that just return 1. Only on line 243
module function corr_2_rsp_rsp(x, dim, mask) result(res)
do I see a function that computes correlations. Stdlib supports a range of numeric types and strives to be generic, but this reduces readability. I think SciPy, R, and Matlab typically assume that calculations are done in double precision and wonder if a version of stdlib that only supports double precision variables for numerical calculations could be created automatically, for pedagogical reasons.
1 Like
That’s a good point about varying readability. I had primarily looked through the stats modules and it took me a while to figure out where computations are happening and I started extracting it from different modules to just have the core of it at a glance, but stdlib_ascii.f90 is really well commented.
1 Like
I am more than happy to support any activities, which brings additional benefits for the Fortran community.
A stripped down version of fypp, providing conditionals and loops would be rather easy in Fortran, I guess. We have to keep in mind, however, that fypp uses Python’s eval() function to evaluate expressions. (I had neither the time nor the desire to write my own domain specific language interpreter for a preprocessor which I actually never wanted to write; but had to after I gave up my futile attempts on generating my templates using m4 
). Therefore, full compatibility would be hard (almost impossible) to achieve as @ivanpribec pointed out.
I agree, that using lists, dictionaries and other Pythonic-stuff in preprocessor expressions is often a sign of overengineering. (Ironically, in our projects usually I am the one who tries to keep fypp-enthusiasts from using too much fypp-magic. ) But there are some cases, where it becomes handy, and it is hard to think about any other straightforward solution (e.g. try to make automatic test registration in a unit testing framework like Fortuno without pre-processor magic…)
This means, that if we develop a Fortran based fypp alternative, we might end up with a simple pre-processor, which does not cover all scenarios fypp is currently used for (so python-fypp will be still around and used). Additionally, it would offer mostly features, which the next Fortran standard (with generics and standardized pre-processor format for conditional compilation) would make superfluous anyway (at least once those features are implemented in various compilers). So I am not sure, whether its really worth the effort.
4 Likes
I think lack of macros and/or looping constructs were major downfalls of the ‘coco’ preprocessing that was an optional part of Fortran-95, then disappeared in later standards. Dan Nagle’s version of coco did support macros. However it seems to have disappeared from the web.
The Fortran portion of ESMF uses (Fortran-compatible subset of) cpp to do type/kind/rank replication of various procedures. Since cpp macros can only be ‘one liners’, it took some fiddling - with two preprocessor passes and some script magic between them, to get code to replicate the way we wanted it. (7 Conventions)
If I were still working on the project, I’d definitely want to consider using fypp instead - using loops rather than macros. But at this late point, they are probably happy just doing what has already been in place for the past ~20 years with cpp.
1 Like
stdlib is a terrific project for Fortran, and we can’t thank enough those who contribute to it.
The biggest drawback to me is the (apparent) complete lack of interest and support from the compiler vendors, and this is really a pity. Vendors shipping a compiled version of stdlib along with their compilers would be a huge breakthrough…
Without that, using stdlib in an industrial context can be quite difficult. For instance I tried compiling the sources (from the fpm branch, but without using fpm), but I faced a major problem: in our industrial software we are stuck with ifort 21, which is not in the list of compilers that are tested against. And as a matter of facts, I got a lot of compiler errors (which are not compilation errors…). Applying workarounds in the code was possible, but there were too many errors and I gave up, I didn’t have enough time to spare to that…
2 Likes
@PierU on my side, our reference compiler is ifort19 (because of how freaking delicate is the MVS and intel compilers integration on Windows…, move one finger and everything breaks). I also had to do a few modifications but like in two files. I took only the src folder from the stdlib-fpm branch to integrate in the industrial code-base. Maybe you could signal which errors did you find in an issue on stdlib? I’m pretty sure a not-that-annoying work around can be found 
And yes, support by compilers would be the ideal scenario!!
2 Likes
OK, I will give a new try. If you had only a couple of errors, maybe I did something wrong…
I was doing something wrong 
…
The problem is the dependency management of our internal build system, which doesn’t care about the submodules. It results into tons of errors, and among these errors there are a few errors that are not related to submodules. And I stopped investigating before understanding that there was a dependency issue…
So at the moment I have a built a classical makefile, and I am trying to sort out the real errors (there are only a few of them, indeed).
1 Like
Good news, I could finally compile the sources of stdlib with ifort 21 
! Thanks !
There are a few issues and I had to (quick and dirty) fix a couple of files, but I will post that on github (edit: Compilation errors with IFORT 21 · Issue #960 · fortran-lang/stdlib · GitHub)
3 Likes