Memory leak in FileHelperEngine

[jens-at-anca]

It appears that there is a memory leak when creating a new instance of the FileHelperEngine with a record class created by DelimitedClassBuilder;

FileHelpers Version: 3.2.7.0
Visual Studio 2017

Pseudo code to reproduce the issue:

for (int i=0; i<10000; i++)
{
   DelimitedClassBuilder cb = new DelimitedClassBuilder(className, ",");
   // cb.AddField
   FileHelperEngine engine = new FileHelperEngine(cb.CreateRecordClass());
}

Observing the Process Memory in the diagnostic Tool in Visual Studio shows that the memory is continuously increasing.

[netniV]

From a normal coding point of view, I believe that this code would be the same for any given loop of new objects. Whilst memory will increase up to the point of saturation, garbage collection should kick in to dispose of objects that are no longer referenced.

Looking over the source involved in the class building, there are a few objects that could be disposed of within ClassBuilder (the base class) but these only exist within the Encrypt/Decrypt functions which you are not likely to be using. But I will generate a pull request for those changes.

The one thing that ClassBuilder doesn't check is that class you are trying to create a class already exists. No caching is performed of the class that is built by the class builders.

Pro's

• Any minor variations you may have in the source data used to build the class won't cause a problem (CsvClassBuilder is built on DelimitedClassBuilder, so different headers would generate different fields)

• The type can be unloaded by the calling program without causing a problem to ClassBuilder

• The dynamic assemblies can be utilised to handle multiple variations of the same data by caching these different types in the calling program, eg to show the differences between two different types.

Con's

• If you generate a lot of ClassBuilders and call CreateRecordClass(), a new assembly is generated for every call
• You can end up with multiple runtime types of the same name
• Memory usage will increase for every call because of the new dynamically generated assembly that is loaded in memory (no file is created)
• Calling program needs code to make sure that they definitely want to recreate the object type. The Type to be created should be cached if possible.

[netniV]

This is an example of the code you have updated to include a cache:

Dictionary<string,Type> dictTypeCache = new Dictionary<string,Type>();
for (int i=0; i<10000; i++)
{
    if (!dictTypeCache.ContainsKey(className)) {
        DelimitedClassBuilder cb = new DelimitedClassBuilder(className, ",");
        // cb.AddField
        Type ct = cb.CreateClassRecord();            
        dictTypeCache.Add(className, ct);
    }
    FileHelperEngine engine = new FileHelperEngine(dictTypeCache[className]);
}

Please note that I wrote this code by hand so there may be typo's. Can you retest with the type caching to see how the memory does then?

[jens-at-anca]

Hi netniV, I tested your suggestion to use a cache and the memory is not increasing.
It looks like the "leak" is in the DelimitedClassBuilder - CreateClassRecord() function.

thanks for your help

[mcavigelli]

merged netniV branch Issue-271.

This should resolve the described problems.

[netniV]

Was that still applicable ?? Quite an old fix that. 👍