Binary option for vtk surface format

In my understanding, there is no option to write surfaces from functionObjects, e.g. iso-surfaces, in a binary vtk format.

I believe this would be quite useful to reduce the size of each single file and speed up the reading in Paraview. For example, the iso-surface of Lambda2 in a 10M cells mesh from an external aero application is as big as 500M and it takes more than 10 seconds to be read.

Would this be possible or is there any constraint regarding the vtk format preventing this option to be implemented?

Thanks,

Riccardo

Which version are you using. My computer is turned off at the moment, so I can't check for certain but this should work with 1906. Note, however, that the "binary" format that I mean here is inline base64 encoded and not raw appended format. The trouble with appended format is that it is a bad fit when streaming on the fly since you either need a double-pass to handle the content or else some initial file padding and then use a rewind to rewrite portions later.

Binary ensight output could be another alternative.

I've been looking into the v1812 so far, but I'll give the v1906 a try tomorrow.

Would the option be called in the same way as for ensight? i.e.:

formatOptions { ensight { format binary; } }

By Riccardo Rossi on 2019-07-11T22:54:26 (imported from GitLab project)

assigned to @mark

Even better! (Or worse, depending on your opinion). The default output is base64 and xml, but you can change both:

• https://www.openfoam.com/documentation/guides/latest/api/classFoam_1_1surfaceWriters_1_1vtkWriter.html#details

The surface writers have been fully redone, and sampling benefits from it:

• https://www.openfoam.com/releases/openfoam-v1906/post-processing.php#post-processing-surface-sampling

This sounds actually quite nice.

I'll done some testing and comparison with the 500MB output per file in v1812 and I'll share it with you asap.

Thanks Mark.

By Riccardo Rossi on 2019-07-12T07:11:58 (imported from GitLab project)

I just tested the vtk binary format on a smaller case and the iso surface size went from 13MB to 11MB using:

formatOptions { vtk { format binary; legacy false; precision 10; } }

I was expecting something smaller based on my experience for example with ascii/binary STL files.

Any thought?

By Riccardo Rossi on 2019-07-15T12:51:10 (imported from GitLab project)

This will depend to an extent on what the input data looks like. The base64 encoding requires 16 bytes of storage for 12 bytes of raw binary data, so that means your 11MB (base64) should be about 8.5MB raw. We cannot get real raw for the XML format, without moving to the appended output (with all of the issues described previously).

Try using binary with legacy. Since you are using this output for sampled surfaces, you don't need the multi-piece capabilities of the VTK XML format. And the file-series support should also work fine for legacy files too.

Note that the total difference between VTK ASCII and binary (even 'raw') will probably not be anywhere near as dramatic as you know from STL files. STL files are just a bunch of points, so binary is (1 normal, 3 points, 1 attribute) per triangle == 50 bytes. The ASCII form has almost that just for a single x,y,z point.

Also in VTK, a chunk of the data is for mesh connectivity (represented as ints). If there aren't too many points in the mesh, the ASCII representation (digits and space separator) can be comparable to binary sizes. Same thing if the fields contain lots of zeros, ones, etc.

For interest sake, it would also be useful to see what size EnSight binary comes out with.

Thanks for the detailed explanation.

I just ran again the same case moving to isoSurfaceTopo type and writing the same surface in three formats, with the following results:

1 - Vtk binary, legacy, precision 10, 3.2 MB

2 - Vtk ascii, legagcy, precision 10, 6.8 MB

3 - Ensight binary, legacy, precision 10, 8.3 MB

The gain is much higher indeeed. A little surprised to see the Ensight one larger than the Vtk ascii, but perhaps is carrying more info with it.

Anyhow, later on I'll go back to the original case where the isoSurface file size was 500 MB and I'll try the same.

Let me know if you have additional thoughts or tests you want me to run.

By Riccardo Rossi on 2019-07-16T10:58:26 (imported from GitLab project)

I just performed the test on the original case, with the following results:

1 - Vtk ascii, legacy precision 10, iso-surface file size 832MB

2 - Vtk binary, legacy precision 10, iso-surface file size 390MB

During the same run, I have also tried to use the new isoSurfaceTopo method, which resulted in a troncated iso-surface near the object, as shown in the picture attached (orange is Lambda2 via standard isoSurface and blue Lambda2 via isoSurfaceTopo).

Also, when trying to compute the Lamb vector function object as suggested in the guide with:

postProcess -func lambVector

I receive the following error:

Cannot find functionObject file lambVector

Moreover, the tutorial suggested in the guide to test the Lamb vector does not exist in the path:

$FOAM_TUTORIALS/incompressible/pimpleFoam/LES/motorBike

Shall I open two new separate threads for these issues?

By Riccardo Rossi on 2019-07-24T15:15:04 (imported from GitLab project)

Yes, please open new issues. For the isoSurfaceTopo, it could be the "regularise" is too aggressive - worth checking as well. Your example might make a good test case @Prashant

Closing this topic - since the others are now being used.

closed