Can't createArrowFiles on tutorial example

[nchernia]

I've gone as far as to delete R.framework and reinstall R; I cannot get ArchR to createArrowFiles. It's the dreaded HDF5 error, and when subThreading = F, it's a different one but also completely mysterious. I'm loading parallel first and I've checked the md5 on the tutorial data files.

Sys.setenv(HDF5_USE_FILE_LOCKING = FALSE)
Sys.setenv(RHDF5_USE_FILE_LOCKING = FALSE)
library(parallel)
library(magick)
library(ArchR)
inputFiles <- getTutorialData("Hematopoiesis")
addArchRThreads(1)
addArchRGenome("hg38")
ArrowFiles <- createArrowFiles(inputFiles = inputFiles, sampleNames = names(inputFiles))
Using GeneAnnotation set by addArchRGenome(Hg38)!
Using GeneAnnotation set by addArchRGenome(Hg38)!
ArchR logging to : ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log
If there is an issue, please report to github with logFile!
2022-06-02 10:33:41 : Batch Execution w/ safelapply!, 0 mins elapsed.
2022-06-02 10:33:41 : (scATAC_BMMC_R1 : 1 of 3) Reading In Fragments from inputFiles (readMethod = tabix), 0.001 mins elapsed.
2022-06-02 10:33:41 : (scATAC_BMMC_R1 : 1 of 3) Tabix Bed To Temporary File, 0.001 mins elapsed.

************************************************************
2022-06-02 10:33:41 : ERROR Found in .tabixToTmp for (scATAC_BMMC_R1 : 1 of 3) 
LogFile = ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log

<simpleError in H5Fcreate(file): HDF5. File accessibility. Unable to open file.>

************************************************************

2022-06-02 10:33:41 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..
ArchR logging successful to : ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log
ArrowFiles <- createArrowFiles(inputFiles = inputFiles, 
[ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log](https://github.com/GreenleafLab/ArchR/files/8824905/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log)
[ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log](https://github.com/GreenleafLab/ArchR/files/8824907/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log)
sampleNames = names(inputFiles), subThreading = FALSE)
Using GeneAnnotation set by addArchRGenome(Hg38)!
Using GeneAnnotation set by addArchRGenome(Hg38)!
ArchR logging to : ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log
If there is an issue, please report to github with logFile!
2022-06-02 10:34:01 : Batch Execution w/ safelapply!, 0 mins elapsed.
2022-06-02 10:34:01 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..
ArchR logging successful to : ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log
Warning message:
In mclapply(..., mc.cores = threads, mc.preschedule = preschedule) :
  3 function calls resulted in an error

Log files attached.

[rcorces]

Hi @nchernia! Thanks for using ArchR! Please make sure that your post belongs in the Issues section. Only bugs and error reports belong in the Issues section. Usage questions and feature requests should be posted in the Discussions section, not in Issues.

Before we help you, you must respond to the following questions unless your original post already contained this information:
1. If you've encountered an error, have you already searched previous Issues to make sure that this hasn't already been solved?
2. Can you recapitulate your error using the tutorial code and dataset? If so, provide a reproducible example.
3. Did you post your log file? If not, add it now.
4. Remove any screenshots that contain text and instead copy and paste the text using markdown's codeblock syntax (three consecutive backticks). You can do this by editing your original post.

[nchernia]

cat ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log

           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    
Logging With ArchR!

Start Time : 2022-06-02 10:34:01

------- ArchR Info

ArchRThreads = 1
ArchRGenome = Hg38

------- System Info

Computer OS = unix
Total Cores = 8

------- Session Info

R version 4.1.2 (2021-11-01)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 11.6.4

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Rsamtools_2.10.0                  BSgenome.Hsapiens.UCSC.hg38_1.4.4 BSgenome_1.62.0                  
 [4] rtracklayer_1.54.0                Biostrings_2.62.0                 XVector_0.34.0                   
 [7] ArchR_1.0.1                       magrittr_2.0.3                    rhdf5_2.38.1                     
[10] Matrix_1.4-1                      data.table_1.14.2                 SummarizedExperiment_1.24.0      
[13] Biobase_2.54.0                    GenomicRanges_1.46.1              GenomeInfoDb_1.30.1              
[16] IRanges_2.28.0                    S4Vectors_0.32.4                  BiocGenerics_0.40.0              
[19] MatrixGenerics_1.6.0              matrixStats_0.62.0                ggplot2_3.3.6                    
[22] magick_2.7.3                     

loaded via a namespace (and not attached):
 [1] tidyselect_1.1.2         purrr_0.3.4              lattice_0.20-45          colorspace_2.0-3        
 [5] vctrs_0.4.1              generics_0.1.2           yaml_2.3.5               utf8_1.2.2              
 [9] XML_3.99-0.9             rlang_1.0.2              pillar_1.7.0             glue_1.6.2              
[13] withr_2.5.0              BiocParallel_1.28.3      GenomeInfoDbData_1.2.7   lifecycle_1.0.1         
[17] stringr_1.4.0            zlibbioc_1.40.0          munsell_0.5.0            gtable_0.3.0            
[21] restfulr_0.0.13          fansi_1.0.3              Rcpp_1.0.8.3             scales_1.2.0            
[25] DelayedArray_0.20.0      rjson_0.2.21             stringi_1.7.6            dplyr_1.0.9             
[29] BiocIO_1.4.0             grid_4.1.2               cli_3.3.0                tools_4.1.2             
[33] bitops_1.0-7             rhdf5filters_1.6.0       RCurl_1.98-1.6           tibble_3.1.7            
[37] crayon_1.5.1             pkgconfig_2.0.3          ellipsis_0.3.2           rstudioapi_0.13         
[41] Rhdf5lib_1.16.0          R6_2.5.1                 GenomicAlignments_1.30.0 compiler_4.1.2          


------- Log Info


2022-06-02 10:34:01 : createArrowFiles Input-Parameters, Class = list

createArrowFiles Input-Parameters$inputFiles: length = 3
                                      scATAC_BMMC_R1                                  scATAC_CD34_BMMC_R1 
     "HemeFragments/scATAC_BMMC_R1.fragments.tsv.gz" "HemeFragments/scATAC_CD34_BMMC_R1.fragments.tsv.gz" 
                                      scATAC_PBMC_R1 
     "HemeFragments/scATAC_PBMC_R1.fragments.tsv.gz" 


createArrowFiles Input-Parameters$sampleNames: length = 3
[1] "scATAC_BMMC_R1"      "scATAC_CD34_BMMC_R1" "scATAC_PBMC_R1"     


createArrowFiles Input-Parameters$outputNames: length = 3
[1] "scATAC_BMMC_R1"      "scATAC_CD34_BMMC_R1" "scATAC_PBMC_R1"     


createArrowFiles Input-Parameters$validBarcodes: length = 0
NULL


createArrowFiles Input-Parameters$minTSS: length = 1
[1] 4


createArrowFiles Input-Parameters$minFrags: length = 1
[1] 1000


createArrowFiles Input-Parameters$maxFrags: length = 1
[1] 1e+05


createArrowFiles Input-Parameters$QCDir: length = 1
[1] "QualityControl"


createArrowFiles Input-Parameters$nucLength: length = 1
[1] 147


createArrowFiles Input-Parameters$promoterRegion: length = 2
[1] 2000  100


createArrowFiles Input-Parameters$excludeChr: length = 2
[1] "chrM" "chrY"


createArrowFiles Input-Parameters$nChunk: length = 1
[1] 5


createArrowFiles Input-Parameters$bcTag: length = 1
[1] "qname"


createArrowFiles Input-Parameters$gsubExpression: length = 0
NULL


createArrowFiles Input-Parameters$bamFlag: length = 0
NULL


createArrowFiles Input-Parameters$offsetPlus: length = 1
[1] 4


createArrowFiles Input-Parameters$offsetMinus: length = 1
[1] -5


createArrowFiles Input-Parameters$addTileMat: length = 1
[1] TRUE


createArrowFiles Input-Parameters$addGeneScoreMat: length = 1
[1] TRUE


createArrowFiles Input-Parameters$force: length = 1
[1] FALSE


createArrowFiles Input-Parameters$threads: length = 1
[1] 1


createArrowFiles Input-Parameters$parallelParam: length = 0
NULL


createArrowFiles Input-Parameters$subThreading: length = 1
[1] FALSE


createArrowFiles Input-Parameters$verbose: length = 1
[1] TRUE


createArrowFiles Input-Parameters$cleanTmp: length = 1
[1] TRUE


createArrowFiles Input-Parameters$logFile: length = 1
[1] "ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log"


createArrowFiles Input-Parameters$filterFrags: length = 1
[1] 1000


createArrowFiles Input-Parameters$filterTSS: length = 1
[1] 4



2022-06-02 10:34:01 : Cleaning Temporary Files
2022-06-02 10:34:01 : Batch Execution w/ safelapply!, 0 mins elapsed.

###########
2022-06-02 10:34:01 : Creating Arrow File scATAC_BMMC_R1.arrow (scATAC_BMMC_R1 : 1 of 3)
###########


###########
2022-06-02 10:34:01 : Creating Arrow File scATAC_CD34_BMMC_R1.arrow (scATAC_CD34_BMMC_R1 : 2 of 3)
###########


2022-06-02 10:34:01 : validBC, Class = NULL

2022-06-02 10:34:01 : validBC, Class = NULL

###########
2022-06-02 10:34:01 : Creating Arrow File scATAC_PBMC_R1.arrow (scATAC_PBMC_R1 : 3 of 3)
###########


validBC: length = 0

2022-06-02 10:34:01 : validBC, Class = NULL

NULL
validBC: length = 0



NULL
validBC: length = 0

NULL




2022-06-02 10:34:01 : (scATAC_BMMC_R1 : 1 of 3) Determining Arrow Method to use!

2022-06-02 10:34:01 : (scATAC_CD34_BMMC_R1 : 2 of 3) Determining Arrow Method to use!

2022-06-02 10:34:01 : (scATAC_PBMC_R1 : 3 of 3) Determining Arrow Method to use!
2022-06-02 10:34:01 : (scATAC_BMMC_R1 : 1 of 3) Reading In Fragments from inputFiles (readMethod = tabix), 0.001 mins elapsed.
2022-06-02 10:34:01 : (scATAC_CD34_BMMC_R1 : 2 of 3) Reading In Fragments from inputFiles (readMethod = tabix), 0.001 mins elapsed.
2022-06-02 10:34:01 : (scATAC_PBMC_R1 : 3 of 3) Reading In Fragments from inputFiles (readMethod = tabix), 0.001 mins elapsed.
2022-06-02 10:34:01 : (scATAC_BMMC_R1 : 1 of 3) Tabix Bed To Temporary File, 0.001 mins elapsed.
2022-06-02 10:34:01 : (scATAC_CD34_BMMC_R1 : 2 of 3) Tabix Bed To Temporary File, 0.001 mins elapsed.
2022-06-02 10:34:01 : (scATAC_PBMC_R1 : 3 of 3) Tabix Bed To Temporary File, 0.001 mins elapsed.

************************************************************
2022-06-02 10:34:01 : ERROR Found in .tabixToTmp for (scATAC_BMMC_R1 : 1 of 3) 
LogFile = ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log

<simpleError in H5Fcreate(file): HDF5. File accessibility. Unable to open file.>

2022-06-02 10:34:01 : errorList, Class = list


************************************************************
2022-06-02 10:34:01 : ERROR Found in .tabixToTmp for (scATAC_CD34_BMMC_R1 : 2 of 3) 
LogFile = ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log

errorList$tabixFile: length = 1

************************************************************
2022-06-02 10:34:01 : ERROR Found in .tabixToTmp for (scATAC_PBMC_R1 : 3 of 3) 
LogFile = ArchRLogs/ArchR-createArrows-17e16ff14ddc-Date-2022-06-02_Time-10-34-01.log

                                 scATAC_BMMC_R1 
"HemeFragments/scATAC_BMMC_R1.fragments.tsv.gz" 
<simpleError in H5Fcreate(file): HDF5. File accessibility. Unable to open file.
>
<simpleError in H5Fcreate(file): HDF5. File accessibility. Unable to open file.>


2022-06-02 10:34:01 : errorList, Class = list
errorList$sampleName: length = 1

2022-06-02 10:34:01 : errorList, Class = list
[1]
 "scATAC_BMMC_R1"

errorList$tabixFile: length = 1

errorList$tabixFile: length = 1
                                 scATAC_PBMC_R1 

"HemeFragments/scATAC_PBMC_R1.fragments.tsv.gz" 
                                 scATAC_CD34_BMMC_R1 
"HemeFragments/scATAC_CD34_BMMC_R1.fragments.tsv.gz" 
errorList$validBC: length = 0

NULL


errorList$sampleName: length = 1


[1] "scATAC_PBMC_R1"
errorList$sampleName: length = 1

[1] "scATAC_CD34_BMMC_R1"




errorList$validBC: length = 0
errorList$chromSizes: length = 24
errorList$validBC: length = 0
NULL
NULL




errorList$chromSizes: length = 24
errorList$chromSizes: length = 24
GRanges object with 6 ranges and 0 metadata columns:
GRanges object with 6 ranges and 0 metadata columnsGRanges object with 6 ranges and 0 metadata columns:
:
      seqnames      ranges strand
         <Rle>   <IRanges>  <Rle>
  [1]     chr1 1-248956422      *
  [2]     chr2 1-242193529      *
  [3]     chr3 1-198295559      *
  [4]     chr4 1-190214555      *
  [5]     chr5 1-181538259      *
  [6]     chr6 1-170805979      *
  -------
  seqinfo: 24 sequences from an unspecified genome


errorList$nChunk: length = 1
           seqnames seqnames      ranges      ranges strand strand
     
         <Rle>    <Rle>   <IRanges>   <IRanges>  <Rle>  <Rle>
  [1]
  [1]     chr1     chr1 1-248956422 1-248956422      *      *
  [2]
  [2]     chr2     chr2 1-242193529 1-242193529      *      *
  [3][1]
  [3]     chr3     chr3 5 1-198295559 1-198295559      *      *

  [4]
  [4]     chr4     chr4 1-190214555 1-190214555      *      *
  [5]
  [5]     chr5     chr5 1-181538259 1-181538259      *      *
  [6]
  [6]     chr6     chr6 1-170805979 1-170805979      *      *

  -------
  -------

  seqinfo:   seqinfo: 24 sequences from an unspecified genome24 sequences from an unspecified genome


errorList$threads: length = 1


[1] 1



errorList$nChunk: length = 1
errorList$nChunk: length = 1

[1] 5
errorList$gsubExpression: length = 0
[1] 5

NULL





errorList$threads: length = 1
errorList$prefix: length = 1
errorList$threads: length = 1
[1] 1[1] "(scATAC_BMMC_R1 : 1 of 3)"

[1] 1






errorList$verbose: length = 1
errorList$gsubExpression: length = 0
errorList$gsubExpression: length = 0
[1]NULL
 TRUE
NULL






errorList$prefix: length = 1
errorList$tstart: length = 1
errorList$prefix: length = 1
[1] "(scATAC_PBMC_R1 : 3 of 3)"
[1] "(scATAC_CD34_BMMC_R1 : 2 of 3)"
[1]
 "2022-06-02 10:34:01 EDT"





errorList$verbose: length = 1
errorList$verbose: length = 1

************************************************************

[1] TRUE
[1] TRUE




errorList$tstart: length = 1
errorList$tstart: length = 1
[1] "2022-06-02 10:34:01 EDT"
[1] "2022-06-02 10:34:01 EDT"




************************************************************



************************************************************


2022-06-02 10:34:01 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..

------- Completed

End Time : 2022-06-02 10:34:01
Elapsed Time Minutes = 0.0124211510022481
Elapsed Time Hours = 0.000207166936662462

-------

[nchernia]

cat ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log

           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    
Logging With ArchR!

Start Time : 2022-06-02 10:33:41

------- ArchR Info

ArchRThreads = 1
ArchRGenome = Hg38

------- System Info

Computer OS = unix
Total Cores = 8

------- Session Info

R version 4.1.2 (2021-11-01)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 11.6.4

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BSgenome.Hsapiens.UCSC.hg38_1.4.4 BSgenome_1.62.0                   rtracklayer_1.54.0               
 [4] Biostrings_2.62.0                 XVector_0.34.0                    ArchR_1.0.1                      
 [7] magrittr_2.0.3                    rhdf5_2.38.1                      Matrix_1.4-1                     
[10] data.table_1.14.2                 SummarizedExperiment_1.24.0       Biobase_2.54.0                   
[13] GenomicRanges_1.46.1              GenomeInfoDb_1.30.1               IRanges_2.28.0                   
[16] S4Vectors_0.32.4                  BiocGenerics_0.40.0               MatrixGenerics_1.6.0             
[19] matrixStats_0.62.0                ggplot2_3.3.6                     magick_2.7.3                     

loaded via a namespace (and not attached):
 [1] tidyselect_1.1.2         purrr_0.3.4              lattice_0.20-45          colorspace_2.0-3        
 [5] vctrs_0.4.1              generics_0.1.2           yaml_2.3.5               utf8_1.2.2              
 [9] XML_3.99-0.9             rlang_1.0.2              pillar_1.7.0             glue_1.6.2              
[13] withr_2.5.0              BiocParallel_1.28.3      GenomeInfoDbData_1.2.7   lifecycle_1.0.1         
[17] stringr_1.4.0            zlibbioc_1.40.0          munsell_0.5.0            gtable_0.3.0            
[21] restfulr_0.0.13          fansi_1.0.3              Rcpp_1.0.8.3             scales_1.2.0            
[25] DelayedArray_0.20.0      Rsamtools_2.10.0         rjson_0.2.21             stringi_1.7.6           
[29] dplyr_1.0.9              BiocIO_1.4.0             grid_4.1.2               cli_3.3.0               
[33] tools_4.1.2              bitops_1.0-7             rhdf5filters_1.6.0       RCurl_1.98-1.6          
[37] tibble_3.1.7             crayon_1.5.1             pkgconfig_2.0.3          ellipsis_0.3.2          
[41] rstudioapi_0.13          Rhdf5lib_1.16.0          R6_2.5.1                 GenomicAlignments_1.30.0
[45] compiler_4.1.2          


------- Log Info


2022-06-02 10:33:41 : createArrowFiles Input-Parameters, Class = list

createArrowFiles Input-Parameters$inputFiles: length = 3
                                      scATAC_BMMC_R1                                  scATAC_CD34_BMMC_R1 
     "HemeFragments/scATAC_BMMC_R1.fragments.tsv.gz" "HemeFragments/scATAC_CD34_BMMC_R1.fragments.tsv.gz" 
                                      scATAC_PBMC_R1 
     "HemeFragments/scATAC_PBMC_R1.fragments.tsv.gz" 


createArrowFiles Input-Parameters$sampleNames: length = 3
[1] "scATAC_BMMC_R1"      "scATAC_CD34_BMMC_R1" "scATAC_PBMC_R1"     


createArrowFiles Input-Parameters$outputNames: length = 3
[1] "scATAC_BMMC_R1"      "scATAC_CD34_BMMC_R1" "scATAC_PBMC_R1"     


createArrowFiles Input-Parameters$validBarcodes: length = 0
NULL


createArrowFiles Input-Parameters$minTSS: length = 1
[1] 4


createArrowFiles Input-Parameters$minFrags: length = 1
[1] 1000


createArrowFiles Input-Parameters$maxFrags: length = 1
[1] 1e+05


createArrowFiles Input-Parameters$QCDir: length = 1
[1] "QualityControl"


createArrowFiles Input-Parameters$nucLength: length = 1
[1] 147


createArrowFiles Input-Parameters$promoterRegion: length = 2
[1] 2000  100


createArrowFiles Input-Parameters$excludeChr: length = 2
[1] "chrM" "chrY"


createArrowFiles Input-Parameters$nChunk: length = 1
[1] 5


createArrowFiles Input-Parameters$bcTag: length = 1
[1] "qname"


createArrowFiles Input-Parameters$gsubExpression: length = 0
NULL


createArrowFiles Input-Parameters$bamFlag: length = 0
NULL


createArrowFiles Input-Parameters$offsetPlus: length = 1
[1] 4


createArrowFiles Input-Parameters$offsetMinus: length = 1
[1] -5


createArrowFiles Input-Parameters$addTileMat: length = 1
[1] TRUE


createArrowFiles Input-Parameters$addGeneScoreMat: length = 1
[1] TRUE


createArrowFiles Input-Parameters$force: length = 1
[1] FALSE


createArrowFiles Input-Parameters$threads: length = 1
[1] 1


createArrowFiles Input-Parameters$parallelParam: length = 0
NULL


createArrowFiles Input-Parameters$subThreading: length = 1
[1] TRUE


createArrowFiles Input-Parameters$verbose: length = 1
[1] TRUE


createArrowFiles Input-Parameters$cleanTmp: length = 1
[1] TRUE


createArrowFiles Input-Parameters$logFile: length = 1
[1] "ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log"


createArrowFiles Input-Parameters$filterFrags: length = 1
[1] 1000


createArrowFiles Input-Parameters$filterTSS: length = 1
[1] 4



2022-06-02 10:33:41 : Cleaning Temporary Files
2022-06-02 10:33:41 : Batch Execution w/ safelapply!, 0 mins elapsed.

###########
2022-06-02 10:33:41 : Creating Arrow File scATAC_BMMC_R1.arrow (scATAC_BMMC_R1 : 1 of 3)
###########


2022-06-02 10:33:41 : validBC, Class = NULL

validBC: length = 0
NULL



2022-06-02 10:33:41 : (scATAC_BMMC_R1 : 1 of 3) Determining Arrow Method to use!
2022-06-02 10:33:41 : (scATAC_BMMC_R1 : 1 of 3) Reading In Fragments from inputFiles (readMethod = tabix), 0.001 mins elapsed.
2022-06-02 10:33:41 : (scATAC_BMMC_R1 : 1 of 3) Tabix Bed To Temporary File, 0.001 mins elapsed.

************************************************************
2022-06-02 10:33:41 : ERROR Found in .tabixToTmp for (scATAC_BMMC_R1 : 1 of 3) 
LogFile = ArchRLogs/ArchR-createArrows-17e12937ea66-Date-2022-06-02_Time-10-33-41.log

<simpleError in H5Fcreate(file): HDF5. File accessibility. Unable to open file.>

2022-06-02 10:33:41 : errorList, Class = list

errorList$tabixFile: length = 1
                                 scATAC_BMMC_R1 
"HemeFragments/scATAC_BMMC_R1.fragments.tsv.gz" 


errorList$sampleName: length = 1
[1] "scATAC_BMMC_R1"


errorList$validBC: length = 0
NULL


errorList$chromSizes: length = 24
GRanges object with 6 ranges and 0 metadata columns:
      seqnames      ranges strand
         <Rle>   <IRanges>  <Rle>
  [1]     chr1 1-248956422      *
  [2]     chr2 1-242193529      *
  [3]     chr3 1-198295559      *
  [4]     chr4 1-190214555      *
  [5]     chr5 1-181538259      *
  [6]     chr6 1-170805979      *
  -------
  seqinfo: 24 sequences from an unspecified genome


errorList$nChunk: length = 1
[1] 5


errorList$threads: length = 1
[1] 1


errorList$gsubExpression: length = 0
NULL


errorList$prefix: length = 1
[1] "(scATAC_BMMC_R1 : 1 of 3)"


errorList$verbose: length = 1
[1] TRUE


errorList$tstart: length = 1
[1] "2022-06-02 10:33:41 EDT"



************************************************************


2022-06-02 10:33:41 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..

------- Completed

End Time : 2022-06-02 10:33:41
Elapsed Time Minutes = 0.00410616795221965
Elapsed Time Hours = 6.85544146431817e-05

-------

[rcorces]

Quite mysterious. Can you force file permissions on the entire project directory with chmod -R 777 and see if that does anything?

It would also be helpful if you could provide the output of traceback() after encountering the error.

The error you are getting is presumably caused by this line:

ArchR/R/CreateArrow.R

Line 1183 in 968e442

o <- h5createFile(tmpFile)

Can you try running the following two lines from your project directory and then also from your home directory:

tmpFile <- ArchR:::.tempfile(pattern = paste0("tmp-TEST-arrow"), fileext=".arrow")
o <- h5closeAll()
o <- h5createFile(tmpFile)
o <- h5createGroup(tmpFile, paste0("Fragments"))
o <- h5closeAll()

[nchernia]

Thanks so much for your quick reply. I ran chmod -R 777 with no difference. Traceback is mostly saying "No traceback available". In both current project directory and my home directory, I get the following errors when trying to create the tmpFile: ``` tmpFile <- ArchR:::.tempfile(pattern = paste0("tmp-TEST-arrow"),fileext=".arrow") o <- h5closeAll() o <- h5createFile(tmpFile) Error in H5Fcreate(file) : HDF5. File accessibility. Unable to open file. No traceback available o <- h5createGroup(tmpFile, paste0("Fragments")) Error in h5checktypeOrOpenLoc(file, native = FALSE) : Error in h5checktypeOrOpenLoc(). Cannot open file. File tmp/tmp-TEST-arrow-6a911cdb6ebe-Date-2022-06-02_Time-12-57-09.arrow does not exist. 2: H5Fcreate(file) 1: h5createFile(tmpFile) ```

[rcorces]

And I assume that this file tmp/tmp-TEST-arrow-6a911cdb6ebe-Date-2022-06-02_Time-12-57-09.arrow really does not exist on your system right?

Either way, this is not an ArchR problem per se. There is something about your environment that is incompatible with creating hdf5 files with rhdf5.

I would recommend posting an issue to the rhdf5 github page or to the bioconductor support forum. I'm closing this as I dont believe it is an ArchR problem.

https://support.bioconductor.org/
https://github.com/grimbough/rhdf5

It would be great if you find a solution and can report back in case others encounter this issue.

[nchernia]

I think there's an issue in your example at least in that you're expected that "tmp" doesn't exist or exists as a directory (I have a bad habit of writing any much to a file named "tmp" whereever I happen to be). When I create the "tmp" directory outside of this command, the h5 works.

Also the below closed issue is the same.

#1123

In any event, I have once again deleted everything R related (i.e. the /Library/Frameworks directory), then reinstalled gfortran, reinstalled R (v 4.1.2), and set paths as directed in this thread: https://www.cynkra.com/blog/2021-03-16-gfortran-macos/ then reinstalled ArchR as directed in instructions.

Running with subthreading = F did not work (checked logs and no info as to why not), but running without that flag did.

[rcorces]

@nchernia - thats interesting. I'm not sure exactly what you mean by you think there is an error in the example I provided but if I'm interpreting your response correctly, you're saying that the tmp/ directory isnt getting created?

This directory is supposed to be automatically created during the temp file creation process as shown here:

ArchR/R/HiddenUtils.R

Lines 326 to 338 in 968e442

	.tempfile <- function(pattern = "tmp", tmpdir = "tmp", fileext = "", addDOC = TRUE){
	dir.create(tmpdir, showWarnings = FALSE)
	if(addDOC){
	doc <- paste0("-Date-", Sys.Date(), "_Time-", gsub(":","-", stringr::str_split(Sys.time(), pattern=" ",simplify=TRUE)[1,2]))
	}else{
	doc <- ""
	}
	tempfile(pattern = paste0(pattern, "-"), tmpdir = tmpdir, fileext = paste0(doc, fileext))
	}

in this case showWarnings has been set to FALSE to avoid warnings when the directory is already created. But maybe in your case directory creation is failing and no warning message is being output (permission denied results in warning, not error for dir.create())

I added a check to make sure that the directory exists via 978cd3c in release_1.0.2. If you upgrade to release_1.0.2 then failure to create the directory should produce a more reasonable error message. Now why exactly you were unable to create the directory, I'm not sure.

[nchernia]

Unable to create because I already had a file (not directory) named tmp I’ll double check the new release though I’m less motivated now that it’s working. Thanks for the help!

…

-- Neva Cherniavsky Durand, Ph.D. | she, her, hers Senior Scientist | Gene Regulation Observatory Broad Institute of MIT and Harvard

[rcorces]

Thanks for clarifying. I've now added a check to make sure a file doesnt exist with the same name as the directory. Appreciate your help on this.

[jmmuncie]

Hi - this fix seems to be causing issues for me. When I run addGeneExpressionMatrix I am getting the following error:

Error in .tempfile(pattern = paste0("ArchR-", name), fileext = ".log",  : 
  Attempted to create temporary directory ArchRLogs but a file already exists with this name. Please remove this file and try again!

And traceback() produces the following:

5: stop(paste0("Attempted to create temporary directory ", tmpdir, 
       " but a file already exists with this name. Please remove this file and try again!"))
4: .tempfile(pattern = paste0("ArchR-", name), fileext = ".log", 
       tmpdir = logDir)
3: createLogFile("addGeneExpressionMatrix")
2: .validInput(input = logFile, name = "logFile", valid = c("character"))
1: addGeneExpressionMatrix(input = proj_Mef2c_v13_E9, seRNA = seRNA_Mef2c_v13_E9, 
       excludeChr = c("chrM", "chrY"), force = TRUE)

Previously, ArchR would simply write the new log file to the pre-existing "ArchRLogs" directory - i.e. all my logs were saved to one place. Even if I move the "ArchRLogs" folder out of my working directory, this error persists.

Thanks in advance for the help.