Windows 10 is faster with -fno-openmp than setDTthreads(1) on many repeated calls

[ColeMiller1]

OpenMP support may cause issues with Windows performance, especially with many repeated calls.

library(data.table)
setDTthreads(1L)
allIterations <- data.table(v1 = runif(1e5), v2 = runif(1e5))
DoSomething <- function(row) someCalculation <- row[["v1"]] + 1

system.time(for (r in 1:nrow(allIterations)) DoSomething(allIterations[r, ]))
##1.12.8 
##   user  system elapsed 
##  17.09    4.48   22.14 

##master
##   user  system elapsed 
##  17.50    4.53   22.51 

##master w/ OpenMP lines deleted from subset.c
##   user  system elapsed 
##  12.37    0.01   12.85 

Note, I would expect using setDTthreads(1L) would minimize any impacts to performance but that does not appear to be the case on Windows 10.

[jangorecki]

That is interresting, could you check if if (1<0) next to num_threads(1) will solve that? I just recently checked that we can safely rely on num_threads only, and we don't need to escape using if: #4484 (comment)
What is your compiler and version?

[ColeMiller1]

Using

#pragma omp parallel for if (1 < 0) num_threads(getDTthreads())

Does not help (I also changed the two _Pragma("...") lines). Based on performance, it seems like it is still trying to fork a process or whatever the equivalent is for Windows.

I use what is in Rtools 3.5.0.4 which appears to be MinGW gcc 4.9.3??? I am not versed in compilers. After googling, I will install Rtools 4.0 which is at gcc 8.3.0 and update this post later. Below is from the build-info.txt of both 32- and 64-bit version

version : MinGW-W64-builds-4.2.0
user    : Jeroen
date    : 09.12.2015- 9:12:03
args    : --mode=gcc-4.9.3 --rt-version=v3 --arch=i686 --exceptions=dwarf --static-gcc --threads=posix --enable-languages=c,c++,fortran
PATH    : /mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Program Files (x86)/MiKTeX 2.9/miktex/bin:/c/Program Files/Git/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

version : MinGW-W64-builds-4.2.0
user    : Jeroen
date    : 09.12.2015- 5:21:38
args    : --mode=gcc-4.9.3 --rt-version=v3 --arch=x86_64 --exceptions=seh --static-gcc --threads=posix --enable-languages=c,c++,fortran
PATH    : /mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Program Files (x86)/MiKTeX 2.9/miktex/bin:/c/Program Files/Git/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

Finally, when I do install.packages('data.table'), does my compiler matter then? Because I get identical performance with CRAN version.

[jangorecki]

if you install from binaries then it doesn't matter. For measuring performance install from source, use -O3 -mtune=native flags, and use recent gcc, 7+ should be fine.

[ColeMiller1]

I upgraded to R 4.0.1 / rtools 4.0 which includes gcc 8.3. I recompiled and same performance issues.

I am uncertain on how to change the -O3 -mtune=native flags. With install.packages(...), I have tried INSTALL_opts and configure.args. I have tried to compile directly from the Terminal with R CMB INSTALL data.table_1.12.9.tar.gz -03 -mtune=native. Note: cc() does not work on Windows 10.

[jangorecki]

if it doesn't exists, just create it

cat ~/.R/Makevars 

CC=gcc
CFLAGS=-O3 -mtune=native

[jangorecki]

So if none of if (1<0) and num_threads(1) will not eliminate openmp overhead on windows, then I think we have to ignore that, and document. If user wants to use single thread always, then it may be better to compile with openmp disabled.

[ColeMiller1]

That seems somewhat user unfriendly - I embarrassingly would not know how to do that. Is something like this not maintainable?

//global variables
bool get_nt = true;
int nt = 0;

...
// inside a function
  if (get_nt) nt = getDTthreads();
  get_nt = false;

  if (nt > 1) {
#pragma omp parallel for num_threads(nt)
    for (int i=0; i<n; i++) {
      if (stop) continue;
      int elem = idxp[i];
      if ((elem<1 && elem!=NA_INTEGER) || elem>max) stop=true;
    }
  }
  else {
    for (int i=0; i<n; i++) {
      if (stop) continue;
      int elem = idxp[i];
      if ((elem<1 && elem!=NA_INTEGER) || elem>max) stop=true;
    }
  }

I reworked subset.c to follow the flow above.

library(data.table)
allIterations <- data.table(v1 = runif(1e5), v2 = runif(1e4))
DoSomething <- function(row) someCalculation <- row[["v1"]] + 1
system.time(for (r in 1:nrow(allIterations)) DoSomething(allIterations[r, ]))

##setDTthreads(1L)
##   user  system elapsed 
##  16.09    0.14   16.61 

##setDTthreads(2L)
##   user  system elapsed 
##  58.11   32.06   78.61 

Obviously, my hack has negatively impacted multiple threads - I believe it's related to where I included it in the #define _Pragma("...") bit... I am looking at the resources and testing other things out. Ideally, your suggestion of #pragma omp parallel if(1<0) for num_threads(2) would have been great - maybe there's still something to that.

I will report back with progress.

[ColeMiller1]

Also, specific to subset.c, is it typical to design parallel teams to make more parallel teams? That is, subsetDT makes teams for each column and then each column in subsetvectorraw gets teams to subset the vector itself.

[jangorecki]

If you are asking for nested parallelism then we want to avoid it, collapsing nested loops is fine.

[ColeMiller1]

Sorry about the previous post about nested parallelism - I accidentally introduced it while editing subset.c. There are no issues in the actual master branch.

My experimentation has been unsuccessful. I am interested in compiling without OpenMP to verify that there is no performance issues. How do I compile without OpenMP...?

I see there are two Makevars files in src. This in .in:

PKG_CFLAGS = @openmp_cflags@
PKG_LIBS = @openmp_cflags@ -lz

all: $(SHLIB)
	if [ "$(SHLIB)" != "datatable$(SHLIB_EXT)" ]; then mv $(SHLIB) datatable$(SHLIB_EXT); fi
	if [ "$(OS)" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id datatable$(SHLIB_EXT) datatable$(SHLIB_EXT); fi

And this is .win:

PKG_CFLAGS = $(SHLIB_OPENMP_CFLAGS)
PKG_LIBS = $(SHLIB_OPENMP_CFLAGS) -lz

all: $(SHLIB)
	mv $(SHLIB) datatable$(SHLIB_EXT)

When using ```install.packages(..., type = "source"), this is what a line looks like:

"C:/rtools40/mingw64/bin/"gcc  -I"C:/PROGRA~1/R/R-40~1.1/include" -DNDEBUG       -fopenmp   -O2 -Wall  -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c wrappers.c -o wrappers.o

While I have attempted some research, I do not know where to begin.

[jangorecki]

I would try commenting out first or first two lines of Makevars file, or replacing $(SHLIB_OPENMP_CFLAGS) / @openmp_cflags@ with -fno-openmp.

After #4374 it is probably even easier, by putting -fno-openmp in ~/.R/Makevars. It was merged very recently.

[ColeMiller1]

Still working on this - tried the throttle threads PR but it still did not work.

What is an example where OpenMP should shine? Using 1.12.8 and x = sample(1e7L), order(x) is still 3x faster on 1 thread than forder(x). Note, using multiple threads helps but it still does not catch up to order(x).

library(data.table)

x = sample(1e7L)

setDTthreads(1L)

 bench::mark(data.table:::forder(x),
 order(x),
 min_iterations = 10L)

### A tibble: 2 x 13
##  expression               min median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc
 ## <bch:expr>             <bch> <bch:>     <dbl> <bch:byt>    <dbl> <int> <dbl>
##1 data.table:::forder(x) 923ms  929ms      1.07    38.1MB    0.459     7     3
##2 order(x)               354ms  355ms      2.79    38.1MB    1.19      7     3

setDTthreads(2L)

##  expression               min median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc
##  <bch:expr>             <bch> <bch:>     <dbl> <bch:byt>    <dbl> <int> <dbl>
##1 data.table:::forder(x) 614ms  657ms      1.50    38.1MB    0.376     8     2
##2 order(x)               359ms  368ms      2.73    38.1MB    1.17      7     3