wmkdep runs out of open file descriptors

wmkdep.l runs out of open file descriptors, especially when using the -q option on wmake.

mentioned in commit b7f3ca534c5c6c83c3e1d20e1278959e8274a44b

I've added some error message. An alternative would be to wait for file descriptors to become available (only possible if running multi-threaded)

added ~20 ~22 labels

Might be able to explicitly close them in yywrap() ?

Tried but that doesn't work - you cannot close a file that is on the stack of buffers . Now trying closing only if no buffer on stack:

    /* No more buffers on the stack:
     * Return 1 to inform lex to finish now that all buffers have been read
     */

    if (yyin)
    {
        fclose(yyin);
        yyin = NULL;
    }

forget above - does not help either - still runs out of 1024 descriptors. I have to set it to 4096.

Just a reminder that wmkdepend was an alternative that was still available until OpenFOAM 2.4.x: https://github.com/OpenFOAM/OpenFOAM-2.4.x/tree/master/wmake/src - at which point it was dropped, because it was based on an outdated code generator (I vaguely remember the name "Coco"-something) (I suspect that @mark was who contributed it several years ago, but I'm not certain of it).

Nonetheless, that old alternative did not have this limitation with the description limits, because it would parse each file and keep a local array with the file list. We are still using it on our port for Windows, given that there is no other way for us to remove the maximum open file limit per process.

If by any chance anyone wants to try using something more modern, Kitware uses the Ninja-Build system on VTK and ParaView: https://ninja-build.org/ - I haven't checked if it would even help much here, but they use it in conjunction with CMake for managing the file dependencies, if I remember correctly.

By Bruno Santos on 2018-04-04T14:25:59 (imported from GitLab project)

assigned to @mark and unassigned @Mattijs

mentioned in commit dda3397b229992606cc528670ecc491a0d06c6d1

mentioned in merge request !198 (merged)

mentioned in commit 9d0b0e0ab36280f6f2ccfd8877cfcf832a33bb6c

I'll close this out with inclusion of the newly written wmkdepend program. Same program name that @wyldckat also mentioned, but using ragel for the lexing rather than coco/r, which isn't being developed and slower than the wkdep flex version.

closed

reopened

Mattijs noticed some flakey parsing of comments - taking a look.

mentioned in commit f3ab1a64b59a398c9fbec9f859a3a89318f241fb

closed