wmkdep runs out of open file descriptors

wmkdep.l runs out of open file descriptors, especially when using the -q option on wmake.

mentioned in commit b7f3ca53

I've added some error message. An alternative would be to wait for file descriptors to become available (only possible if running multi-threaded)

added discussion enhancement labels

Might be able to explicitly close them in yywrap() ?

Tried but that doesn't work - you cannot close a file that is on the stack of buffers . Now trying closing only if no buffer on stack:

    /* No more buffers on the stack:
     * Return 1 to inform lex to finish now that all buffers have been read
     */

    if (yyin)
    {
        fclose(yyin);
        yyin = NULL;
    }

forget above - does not help either - still runs out of 1024 descriptors. I have to set it to 4096.

Just a reminder that wmkdepend was an alternative that was still available until OpenFOAM 2.4.x: https://github.com/OpenFOAM/OpenFOAM-2.4.x/tree/master/wmake/src - at which point it was dropped, because it was based on an outdated code generator (I vaguely remember the name "Coco"-something) (I suspect that @mark was who contributed it several years ago, but I'm not certain of it).

Nonetheless, that old alternative did not have this limitation with the description limits, because it would parse each file and keep a local array with the file list. We are still using it on our port for Windows, given that there is no other way for us to remove the maximum open file limit per process.

If by any chance anyone wants to try using something more modern, Kitware uses the Ninja-Build system on VTK and ParaView: https://ninja-build.org/ - I haven't checked if it would even help much here, but they use it in conjunction with CMake for managing the file dependencies, if I remember correctly.

assigned to @mark and unassigned @Mattijs

mentioned in commit dda3397b

mentioned in merge request !198 (merged)

mentioned in commit 9d0b0e0a

I'll close this out with inclusion of the newly written wmkdepend program. Same program name that @wyldckat also mentioned, but using ragel for the lexing rather than coco/r, which isn't being developed and slower than the wkdep flex version.

closed

reopened

Mattijs noticed some flakey parsing of comments - taking a look.

mentioned in commit f3ab1a64

closed