These were the things discussed in today’s GSoC meeting with @milancurcic .
Discussion about what we have as preprocessors. They can be Fortran-based preprocessors like FMacro, Prep, and non-Fortan based preprocessors like Fypp and CPP.
We discussed the approaches for getting preprocessor support in FPM.
The approach was to treat pre processors as FPM Dependency/Plugin and allow the fpm to handle its installation if not installed. But, this approach would create problems for non-Fortran-based preprocessors like Fypp, and then it would be up to the user to install them.
Then, we agreed to start from cpp support for FPM initially and we are aiming to keep the FPM pre processor support open for other pre processors also. This week, I will be sharing a manifest syntax for macros and enabling preprocessors.
Here are the meeting notes for June 9, 2022 GSoC Meeting
We talked about Why and Why not we need to preprocess all the files in the source directory. We talked about the disadvantages of preprocessing all the files when the project contains a large number of files (100 files or more).
After that we agreed to propose a syntax for preprocessing all the files and also have an option to preprocess files using their suffixes in a specific directory. (Using Pattern Matching).
We also learned about specifying specific macros in the manifest files that would be necessary for certain projects.
In the end, we concluded that we will improve upon the current manifest syntax and update it with the syntax to choose a suffix to preprocess.
Also, find an approach to deal with the macros in the manifest file.
And at last, we agreed to keep in radar the rare case scenarios which may arise in certain project that has a file that uses both preprocessors cpp and fypp and how to preprocess those files.
Thanks @arteevraina and thanks @awvwgk for the helpful insights on the call.
One takeaway to add to Arteev’s notes is that pre-processing all source files as a default, including those that have nothing to pre-process, is likely to not add meaningful overhead to worry about at this time, as all sources need to be scanned by fpm.
Regarding custom CPP macros, here’s a minimal example to consider:
program p
use iso_fortran_env, only: real32, real64
implicit none
#ifdef REAL64
integer, parameter :: wp = real64
#else
integer, parameter :: wp = real32
#endif
real(wp), parameter :: pi = 4 * atan(1._wp)
print *, pi
end program p
I think it’d improve UX. Currently the only way to do it is via compiler flags, which for multiple macros and building many times (e.g. in development) gets quite tiresome.
Are there some other ways to provide macros to fpm?
Here are the meeting notes for June 16, 2022 meeting :
Finalizing the syntax for fpm.toml for preprocessor support, along with support for specifying the suffixes, directories to run the preprocessor and macros.
We will need to ask for feedback from the community if we need to consider the order of preprocessing if a project uses multiple preprocessors like cpp and fypp that means run cpp first and then fypp or vice-versa or is it unimportant.
My first code contribution will be around the manifest syntax, so that fpm will be able to parse it and don’t do anything as of now.
I will be looking in the fpm code base and try to figure out from where I can start defining the preprocessor table.
We will also be deciding on where the code contributions made by me will be merged. Will that be in main branch or a feature branch.
Here are the meeting notes for June 23, 2022 meeting
We have decided that our Pull Request will go against main branch. Discussion can be found here.
My first PR will be a small feature addition to parse the manifest file and add a cpp flag to the compiler flags if preprocess.cpp table is found. As of this PR, we will focus on cpp preprocessor. This is done in order to not push any code that does not do anything on to the main branch.
Here are the meeting notes for July 14, 2022 meeting
I will be researching and gathering feedback on the manifest syntax for defined macros for example: -DTESTMACRO and macros that have their value set in fpm.toml file like -DVERSIONMACRO = 2. Will be opening a github issue for this for gathering feedback from fpm maintainers.
After the Pull Request for this feature is completed. We will start with support for third party preprocessors like fypp in fpm.
Here are the meeting notes for July 21, 2022 meeting
We discussed the upcoming Mid Term Evaluation and its deadline.
We discussed the Pull Request and decided to add a test to check if the macros set in the flag in the top-level project should not propagate to the dependencies of the project.
We discussed about the feature flag issue created here and its scope with the current fpm project.
We also discussed about the support for preprocessors are not native to the compiler. We prioritized to add the support for fypp in the fpm project. My next steps with this will be add useful instructions for the user to install the fypp if it is not already installed.
Here are the meeting notes for July 28, 2022 meeting :
We discussed about the issue I was facing regarding reading the macros from the manifest and they were getting passed on to the dependency of the project.
I will update my Pull Request already opened with the new commits along with the implementation for the test cases.
Also, we discuss the potential issue while implementing preprocessor support. The issue is discussed here.
After completing the changes for the pull request I will proceed on with the implementation of the support for fypp preprocessor in the fpm.
Here are the meeting notes for August 4, 2022 meeting:
We discussed that fypp should be checked for dependencies as well. Since it might be the case that there will be a dependency that uses fypp preprocessor.
For fypp, the compilation will be two-stage process.
I will try to build stdlib using fpm to identify any loopholes that fpm as of now.
I guess this will be the case for any project using stdlib (and not stdlib-fpm) as depency, right?
If the source codes are generated by fypp, fpm can now be used with the current structure of stdlib. The script fpm-deployement.sh is not needed anymore in that case. In this branch, I added a small script that generates the files with fypp and removes others to prepare the stdlib directory for fpm. Maybe this script/branch can help you to identify possible issues.
The idea is to start with a compiler wrapper (fypp-gfortran.sh or similar) like we are using for the fpm single file version (https://github.com/fortran-lang/fpm/blob/main/ci/single-file-gfortran.sh). This would allow as to use fpm as is, intercept the compile command, add fypp preprocessing on demand and than invoke the compiler.
Of course this is not a final solution, but it would allow the first fpm only build of stdlib and we can identify whether there are still issue we have to address in the stdlib repo first. Afterwards we can implement the commands injected by the compiler wrapper into fpm to establish proper fypp support.
I will move the shell script for running fypp preprocessor to python syntax, because python feels more intuitive and it will be easier to do modifications like filtering the flags, macros, include paths.
I will also try to build the stdlib using the compiler wrapper script and report any problems related to changing the structure of stdlib inside the Discourse channel.
Then, keeping that script as a blueprint, I will make necessary changes in the fpm to establish complete preprocessing support.
We are able to build the source files, examples and tests inside stdlib with the help of compiler script except two tests that are not compiling correctly mentioned here.
We spent some time in the meeting to debug the issue with a failed compilation of these two test files and came to the conclusion that fpm does not compile .cpp files and that’s why we are not able to compile one test but on the other hand other test is failing due to some other reasons which need a debugging for some more time as we were not able to figure that out in the meeting completely. @jeremie.vandenplas and @lkedward will spare some time to help me out with that.
We are still getting * test/hash_functions/test_hash_functions.f90 which is giving me this error :undefined reference to generate_all_c_hash despite having the C++ files compilation support merged yesterday. Will open an issue with a minimum reproducible example on GitHub Repo to gather feedback and possible solutions.
I asked my doubts regarding how I am going to make the final work product for final evaluation in GSoC.
For this week, I will keep working on the issue that we are currently stuck on compiling the test in stdlib.
