Precompilation error inside PackageCompiler

[bclyons12]

Describe the example

PackageCompiler.create_app() errors when RecursiveArrayTools is a dependency due to precompilation failure. It doesn't seem to see definitions of rand that should be loaded by default, even with filter_stdlibs=false. Admittedly I don't know if this is an error in RecursiveArrayTools or in PrecompilationTools.

Minimal Reproducible Example 👇

#!/usr/bin/env bash
set -e

DIR="$(mktemp -d)"
echo "DIR=$DIR"
cd "$DIR"

# 1) minimal app
julia --startup-file=no -e 'using Pkg; Pkg.generate("MyApp")'
cat > MyApp/src/MyApp.jl <<'EOF'
module MyApp
function julia_main()::Cint
    return 0
end
end
EOF

# 2) add the trigger dep (RecursiveArrayTools)
julia --startup-file=no --project=MyApp -e 'using Pkg; Pkg.add("RecursiveArrayTools"); Pkg.precompile()'

# 3) build the app (PackageCompiler only needed here)
julia --startup-file=no --project=. -e 'using Pkg; Pkg.add("PackageCompiler")'
julia --startup-file=no --project=. -e 'using Pkg; Pkg.status()'
julia --startup-file=no --project=. -e 'using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)'
julia --startup-file=no --project=. -e 'using InteractiveUtils; versioninfo()'
julia --startup-file=no --project=. -e 'using PackageCompiler; create_app("MyApp","MyAppCompiled"; force=true, filter_stdlibs=false)'

echo "Done. Output at: $DIR/MyAppCompiled"

Error & Stacktrace ⚠️

DIR=/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz
  Generating  project MyApp:
    MyApp/Project.toml
    MyApp/src/MyApp.jl
   Resolving package versions...
      Compat entries added for RecursiveArrayTools
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/MyApp/Project.toml`
  [731186ca] + RecursiveArrayTools v3.47.0
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/MyApp/Manifest.toml`
  [7d9f7c33] + Accessors v0.1.43
  [79e6a3ab] + Adapt v4.4.0
  [4fba245c] + ArrayInterface v7.22.0
  [a33af91c] + CompositionsBase v0.1.2
  [187b0558] + ConstructionBase v1.6.0
  [ffbed154] + DocStringExtensions v0.9.5
  [e2ba6199] + ExprTools v0.1.10
  [46192b85] + GPUArraysCore v0.2.0
  [3587e190] + InverseFunctions v0.1.17
  [1914dd2f] + MacroTools v0.5.16
⌅ [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.5.1
  [3cdcf5f2] + RecipesBase v1.3.4
  [731186ca] + RecursiveArrayTools v3.47.0
  [ae029012] + Requires v1.3.1
  [7e49a35a] + RuntimeGeneratedFunctions v0.5.17
  [1e83bf80] + StaticArraysCore v1.4.4
  [2efcf032] + SymbolicIndexingInterface v0.3.46
  [56f22d72] + Artifacts v1.11.0
  [ade2ca70] + Dates v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [fa267f1f] + TOML v1.0.3
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [4536629a] + OpenBLAS_jll v0.3.27+1
  [8e850b90] + libblastrampoline_jll v5.11.0+0
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  1 dependency successfully precompiled in 0 seconds. 31 already precompiled.
   Resolving package versions...
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/Project.toml`
  [9b87118b] + PackageCompiler v2.2.4
    Updating `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/Manifest.toml`
  [c27321d9] + Glob v1.4.0
  [9b87118b] + PackageCompiler v2.2.4
  [05181044] + RelocatableFolders v1.0.1
  [6c6a2e73] + Scratch v1.3.0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [deac9b47] + LibCURL_jll v8.6.0+0
  [e37daf67] + LibGit2_jll v1.7.2+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.6+0
  [14a3606d] + MozillaCACerts_jll v2023.12.12
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850ede] + nghttp2_jll v1.59.0+0
  [3f19e933] + p7zip_jll v17.4.0+2
Status `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/Project.toml`
  [9b87118b] PackageCompiler v2.2.4
Status `/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/Manifest.toml`
  [c27321d9] Glob v1.4.0
  [9b87118b] PackageCompiler v2.2.4
  [05181044] RelocatableFolders v1.0.1
  [6c6a2e73] Scratch v1.3.0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [83775a58] Zlib_jll v1.2.13+1
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2
Julia Version 1.11.7
Commit f2b3dbda30a (2025-09-08 12:10 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 12 × Apple M4 Pro
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m1)
Threads: 8 default, 0 interactive, 4 GC (on 8 virtual cores)
Environment:
  JULIA_NUM_THREADS = auto
PackageCompiler: bundled libraries:
  ├── Base:
  │    ├── libLLVM.dylib - 53.188 MiB
  │    ├── libatomic.1.dylib - 107.531 KiB
  │    ├── libdSFMT.dylib - 68.453 KiB
  │    ├── libgcc_s.1.1.dylib - 168.984 KiB
  │    ├── libgfortran.5.dylib - 1.787 MiB
  │    ├── libgmp.10.dylib - 490.641 KiB
  │    ├── libgmpxx.4.dylib - 111.547 KiB
  │    ├── libgomp.1.dylib - 420.031 KiB
  │    ├── libjulia-codegen.1.11.7.dylib - 2.003 MiB
  │    ├── libjulia-internal.1.11.7.dylib - 4.763 MiB
  │    ├── libmpfr.6.dylib - 564.875 KiB
  │    ├── libopenlibm.4.0.dylib - 173.781 KiB
  │    ├── libpcre2-8.0.dylib - 757.500 KiB
  │    ├── libquadmath.0.dylib - 372.688 KiB
  │    ├── libssp.0.dylib - 86.797 KiB
  │    ├── libstdc++.6.dylib - 3.416 MiB
  │    ├── libuv.2.dylib - 368.844 KiB
  │    ├── libz.1.2.13.dylib - 170.281 KiB
  │    ├── libunwind.1.0.dylib - 107.719 KiB
  │    ├── libjulia.1.11.7.dylib - 225.578 KiB
  ├── Stdlibs:
  │   ├── libblastrampoline_jll
  │   │   ├── libblastrampoline.5.dylib - 2.430 MiB
  │   ├── OpenBLAS_jll
  │   │   ├── libopenblas64_.0.3.27.dylib - 18.727 MiB
  │   │   ├── libopenblas64_.dylib - 18.727 MiB
  Total library file size: 109.138 MiB
⠋ [01m:22s] PackageCompiler: creating compiler sysimage (incremental=false)ld: warning: reexported library with install name '@rpath/libunwind.1.dylib' found at '/Users/lyons/.julia/juliaup/julia-1.11.7+0.aarch64.apple.darwin14/lib/julia/libunwind.1.0.dylib' couldn't be matched with any parent library and will be linked directly
✔ [01m:22s] PackageCompiler: creating compiler sysimage (incremental=false)
⠙ [00m:42s] PackageCompiler: compiling fresh sysimage (incremental=false)ld: warning: reexported library with install name '@rpath/libunwind.1.dylib' found at '/Users/lyons/.julia/juliaup/julia-1.11.7+0.aarch64.apple.darwin14/lib/julia/libunwind.1.0.dylib' couldn't be matched with any parent library and will be linked directly
✔ [00m:42s] PackageCompiler: compiling fresh sysimage (incremental=false)
Precompiling project...
  ✗ RecursiveArrayTools
  30 dependencies successfully precompiled in 19 seconds. 8 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

RecursiveArrayTools 

Failed to precompile RecursiveArrayTools [731186ca-8d62-57ce-b412-fbd966d074cd] to "/Users/lyons/.julia/compiled/v1.11/RecursiveArrayTools/jl_XEYbgp".
ERROR: LoadError: MethodError: no method matching rand(::Int64)
The function `rand` exists, but no method is defined for this combination of argument types.
Stacktrace:
  [1] (::RecursiveArrayTools.var"#172#173")(::Int64)
    @ RecursiveArrayTools ./none:0
  [2] iterate
    @ ./generator.jl:48 [inlined]
  [3] collect(itr::Base.Generator{UnitRange{Int64}, RecursiveArrayTools.var"#172#173"})
    @ Base ./array.jl:791
  [4] macro expansion
    @ ~/.julia/packages/RecursiveArrayTools/5VjiX/src/precompilation.jl:6 [inlined]
  [5] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
  [6] macro expansion
    @ ~/.julia/packages/RecursiveArrayTools/5VjiX/src/precompilation.jl:4 [inlined]
  [7] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:140 [inlined]
  [8] top-level scope
    @ ~/.julia/packages/RecursiveArrayTools/5VjiX/src/precompilation.jl:139
  [9] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [10] _include(mapexpr::Function, mod::Module, _path::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [11] include(mod::Module, _path::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [12] include(x::String)
    @ RecursiveArrayTools ~/.julia/packages/RecursiveArrayTools/5VjiX/src/RecursiveArrayTools.jl:5
 [13] top-level scope
    @ ~/.julia/packages/RecursiveArrayTools/5VjiX/src/RecursiveArrayTools.jl:145
 [14] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [15] _include(mapexpr::Function, mod::Module, _path::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [16] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [17] top-level scope
    @ stdin:6
 [18] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base /private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib:-1
 [19] include_string
    @ ./loading.jl:2744 [inlined]
 [20] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:321
 [21] _start()
    @ Base ./client.jl:531
in expression starting at /Users/lyons/.julia/packages/RecursiveArrayTools/5VjiX/src/precompilation.jl:3
in expression starting at /Users/lyons/.julia/packages/RecursiveArrayTools/5VjiX/src/RecursiveArrayTools.jl:2
in expression starting at stdin:
ERROR: failed process: Process(setenv(`/Users/lyons/.julia/juliaup/julia-1.11.7+0.aarch64.apple.darwin14/bin/julia --color=yes --startup-file=no --pkgimages=no --sysimage=/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_dsTc2c/sys.dylib -e 'using Pkg; Pkg.precompile()'`,["_CE_M=", "PATH=/Users/lyons/src/OMFIT-source/bin:/opt/homebrew/Caskroom/miniforge/base/bin:/opt/homebrew/Caskroom/miniforge/base/condabin:/Users/lyons/.juliaup/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pmk/env/global/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Library/TeX/texbin", "OPENBLAS_DEFAULT_NUM_THREADS=1", "USER=lyons", "CONDA_PROMPT_MODIFIER=(base) ", "SHELL=/bin/zsh", "OSLogRateLimit=64", "TMPDIR=/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/", "_CONDA_EXE=/opt/homebrew/Caskroom/miniforge/base/bin/conda", "HOMEBREW_CELLAR=/opt/homebrew/Cellar"  …  "JULIA_NUM_THREADS=auto", "PWD=/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz", "DISPLAY=/private/tmp/com.apple.launchd.ewRvnREedN/org.xquartz:0", "TERM_PROGRAM=Apple_Terminal", "XML_CATALOG_FILES=file:///opt/homebrew/Caskroom/miniforge/base/etc/xml/catalog file:///etc/xml/catalog", "CONDA_PREFIX=/opt/homebrew/Caskroom/miniforge/base", "TERM_SESSION_ID=9EF066B7-29F5-41D3-BDA8-FA4EA7327776", "OPENAI_API_KEY=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", "JULIA_LOAD_PATH=/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/tmp.BxKDkRtThz/MyApp:@stdlib", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:598 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:513
 [3] run
   @ ./process.jl:510 [inlined]
 [4] ensurecompiled(project::String, packages::Vector{String}, sysimage::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/LhoqT/src/PackageCompiler.jl:372
 [5] create_sysimage(packages::Vector{String}; sysimage_path::String, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, julia_init_h_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String, import_into_main::Bool)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/LhoqT/src/PackageCompiler.jl:666
 [6] create_sysimage
   @ ~/.julia/packages/PackageCompiler/LhoqT/src/PackageCompiler.jl:604 [inlined]
 [7] create_app(package_dir::String, app_dir::String; executables::Nothing, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, force::Bool, c_driver_program::String, cpu_target::String, include_lazy_artifacts::Bool, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, include_preferences::Bool, script::Nothing)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/LhoqT/src/PackageCompiler.jl:968
 [8] top-level scope
   @ none:1

Not Working Environment (please complete the following information):
See above

Working Environment:
Change script to Pkg.add(name="RecursiveArrayTools", version="3.42"), before the PrecompileTools workload was added in #515

[zhaoyachuanl]

hello, you can try this temporate solution if needed,it works on my project

1. copy RecursiveArrayTools.jl to project local folder
2. add Random to "[deps]" of Project.toml in RecursiveArrayTools.jl
3. add "using Random" to RecursiveArrayTools.jl/src/precompilation.jl
4. use pkg.develop to develop this local RecursiveArrayTools.jl
5. test again, it should work now.