[FPM] May be IO blocking problem feedback in fpm test

I run the fpm on my laptop ( CPU: i5-8250u ), after the fpm build, and then run the fpm test will spend nearly 3 seconds time on my laptop. From the test information below, it seems that the fpm test spends more time on the io blocking of the query file information. Can it be optimized in the subsequent development process ?

Perhaps the priority of fpm performance optimization is not very high, because there are many features worth increasing that need to be prioritized, but I want to try to feedback.

benchmark:

time <commond>
Command being timed: "fpm test --flag -Wall"

real    0m2.948s
user    0m0.000s
sys     0m0.031s
Command being timed: "./test"

real    0m0.060s
user    0m0.000s
sys     0m0.047s
1 Like

Thanks for the feedback @zoziha! Yes, unfortunately due to the primitive way that we query the filesystem (via a shell command), the listing of package files takes quite a while at the moment. The problem is particularly bad on MS Windows and for deep directory structures. We plan to move away from using these shell commands in the future which should speed things up.

As you point out, performance optimization of fpm isn’t a priority right now, however there is some low hanging fruit to optimize the current file listing routine if anyone is interested: we’re currently calling a shell command for every subdirectory when querying files (see fpm_filesystem::list_files). However we only need to call the shell command once with a recursive flag, which would go some way to speeding up the current implementation for deep folder hierarchies. (Even better if someone can put together some cross-platform code that doesn’t require any shell commands!)

2 Likes

Thank you for your reply and hope fpm is getting better and better.

1 Like

We can use M_system [1] by @urbanjost. If there are no objections, I will work on this issue next week.

[1] https://urbanjost.github.io/general-purpose-fortran/BOOK_M_system.html#DOCUMENT43

1 Like

Many thanks to @une, whose PR to fix this issue just got merged!
There should be a noticeable performance improvement, particularly for projects with many source files and for those working on MS Windows.

4 Likes

Kudos to @une

1 Like

Much thanks @une. As a bonus this also fixed a bug a client of mine encountered on Windows with a network drive. :fireworks:

1 Like