Skip to content

Regression in extracting metadata from large files after dotnet switch #200

@lzivadinovic

Description

@lzivadinovic

Hi, Im having trouble extracting metadata from large file after installing version 2.0.0.

This is the output from version 1.4.4 using mono runtime

Apptainer> mono /opt/additional/ThermoRawFileParser/ThermoRawFileParser.exe --version
1.4.4
Apptainer> mono /opt/additional/ThermoRawFileParser/ThermoRawFileParser.exe --input /app/MYFILE.raw --metadata_output
_file /app/v144.json --excludeExceptionData --format 4 --metadata 0
2026-01-05 15:01:32 INFO Started parsing /app/MYFILE.raw
2026-01-05 15:01:47 INFO Finished parsing /app/MYFILE.raw
2026-01-05 15:01:47 INFO Processing completed 0 errors, 0 warnings
Apptainer> ls -latrh /app/
total 8.4G
-rw-r----- 1 ubuntu ubuntu 8.4G Jan  5 13:50 MYFILE.raw
drwxr-xr-x 1 ubuntu ubuntu  100 Jan  5 15:00 ..
drwxr-xr-x 2 ubuntu ubuntu 4.0K Jan  5 15:01 .
-rw-r--r-- 1 ubuntu ubuntu 2.8K Jan  5 15:01 v144.json

After installing version v2.0.0 (linux) it looks like that run never finishes...

Apptainer> ./ThermoRawFileParser --version
2.0.0.0
Apptainer> ./ThermoRawFileParser  --input /app/MYFILE.raw --metadata_output_file /app/v200.json --excludeExceptionDat
a --format 4 --metadata 0
2026-01-05 15:50:13 INFO Started parsing /app/MYFILE.raw

After looking a bit with strace, main difference is that for some reason kernel is invoking membarrier (from dotnet GC?)

And it just hangs in this loop. Its the same on multiple systems (and also when running directly on host, tested on arch/ubuntu and inside docker)

...
[pid  1229] 15:50:57.463476 membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED, 0) = 0 <0.000114>
[pid  1229] 15:50:57.463807 openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 99 <0.000102>
[pid  1229] 15:50:57.464001 fstat(99, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 <0.000109>
[pid  1229] 15:50:57.464209 read(99, "MemTotal:       24609988 kB\nMemF"..., 1024) = 1024 <0.000087>
[pid  1229] 15:50:57.464418 close(99)   = 0 <0.000063>
[pid  1229] 15:50:57.464596 openat(AT_FDCWD, "/sys/fs/cgroup/init.scope/memory.max", O_RDONLY) = 99 <0.000079>
[pid  1229] 15:50:57.464776 fstat(99, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 <0.000054>
[pid  1229] 15:50:57.464944 read(99, "max\n", 4096) = 4 <0.000051>
[pid  1229] 15:50:57.465092 close(99)   = 0 <0.000066>
[pid  1229] 15:50:57.465270 sysinfo({uptime=18762, loads=[42112, 46208, 38016], totalram=25200627712, freeram=7798878208, sharedram=1356152832, bufferram=442052608, totalswap=6442450944, freeswap=6442450944, procs=687, totalhigh=0, freehigh=0, mem_unit=1}) = 0 <0.000052>
[pid  1229] 15:50:57.467415 membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED, 0) = 0 <0.000087>
[pid  1229] 15:50:57.467571 openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 99 <0.000106>
[pid  1229] 15:50:57.467820 fstat(99, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 <0.000083>
[pid  1229] 15:50:57.468125 read(99, "MemTotal:       24609988 kB\nMemF"..., 1024) = 1024 <0.000121>
[pid  1229] 15:50:57.468428 close(99)   = 0 <0.000052>
[pid  1229] 15:50:57.468636 openat(AT_FDCWD, "/sys/fs/cgroup/init.scope/memory.max", O_RDONLY) = 99 <0.000085>
[pid  1229] 15:50:57.468851 fstat(99, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 <0.000033>
[pid  1229] 15:50:57.469058 read(99, "max\n", 4096) = 4 <0.000171>
[pid  1229] 15:50:57.469352 close(99)   = 0 <0.000098>
[pid  1229] 15:50:57.469571 sysinfo({uptime=18762, loads=[42112, 46208, 38016], totalram=25200627712, freeram=7798878208, sharedram=1356152832, bufferram=442052608, totalswap=6442450944, freeswap=6442450944, procs=687, totalhigh=0, freehigh=0, mem_unit=1}) = 0 <0.000054>
...

I've tried compiling from source but nothing changed.

I found this dotnet/extensions#5267 and forced cgroup memory limit size to actual number, but basically nothing changed, just that value is ok and then membarier again

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions