Out of memory error crash in Axis Python OpenGL

[mbuesch]

When I start up the sim/axis_mm.ini example on my Debian Bookworm or Debian Sid machines with the LinuxCNC 2.9 or LinuxCNC master branch, I get a strange out of memory crash in Python OpenGL code.

$:~/develop/git/linuxcnc/scripts$ ./linuxcnc
LINUXCNC - 2.9.0~pre1
Machine configuration directory is '/home/mb/develop/git/linuxcnc/configs/sim/axis'
Machine configuration file is 'axis_mm.ini'
Starting LinuxCNC...
linuxcnc TPMOD=tpmod HOMEMOD=homemod EMCMOT=motmod
Note: Using POSIX realtime
Found file(lib): /home/mb/develop/git/linuxcnc/lib/hallib/core_sim.hal
Found file(lib): /home/mb/develop/git/linuxcnc/lib/hallib/sim_spindle_encoder.hal
Found file(lib): /home/mb/develop/git/linuxcnc/lib/hallib/axis_manualtoolchange.hal
Found file(lib): /home/mb/develop/git/linuxcnc/lib/hallib/simulated_home.hal
note: MAXV     max: 53.340 units/sec 3200.400 units/min
note: LJOG     max: 53.340 units/sec 3200.400 units/min
note: LJOG default: 30.480 units/sec 1828.800 units/min
note: jog_order='XYZ'
note: jog_invert=set()
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/OpenGL/latebind.py", line 43, in __call__
    return self._finalCall( *args, **named )
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mb/develop/git/linuxcnc/bin/axis", line 3964, in <module>
    get_coordinate_font(vars.dro_large_font.get())
  File "/home/mb/develop/git/linuxcnc/bin/axis", line 3842, in get_coordinate_font
    glnav.use_pango_font(coordinate_font, 0, 128)
  File "/home/mb/develop/git/linuxcnc/lib/python/glnav.py", line 65, in use_pango_font
    glBitmap(1, 0, 0, 0, 0, h-d, bytearray([0]*4))
  File "/usr/lib/python3/dist-packages/OpenGL/latebind.py", line 47, in __call__
    return self._finalCall( *args, **named )
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/OpenGL/wrapper.py", line 700, in wrapperCall
    raise err
  File "/usr/lib/python3/dist-packages/OpenGL/wrapper.py", line 693, in wrapperCall
    result = wrappedOperation( *cArguments )
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/OpenGL/platform/baseplatform.py", line 415, in __call__
    return self( *args, **named )
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/OpenGL/error.py", line 230, in glCheckError
    raise self._errorClass(
OpenGL.error.GLError: GLError(
	err = 1285,
	description = b'out of memory',
	baseOperation = glBitmap,
	pyArgs = (
		1,
		0,
		0,
		0,
		0,
		16.0,
		<OpenGL.arrays._buffers.Py_buffer object at 0x7fc4a2a1b5c0>,
	),
	cArgs = (
		1,
		0,
		0,
		0,
		0,
		16.0,
		<OpenGL.arrays._buffers.Py_buffer object at 0x7fc4a2a1b5c0>,
	),
	cArguments = (
		1,
		0,
		0,
		0,
		0,
		16.0,
		<OpenGL.arrays._buffers.Py_buffer object at 0x7fc4a2a1b5c0>,
	)
)
Shutting down and cleaning up LinuxCNC...

Here are the steps I follow to reproduce the issue:

1. Happens during start of axis sim/axis_mm.ini example.

Information about my hardware and software:

This happens on my older PC with 8G system RAM with a very old PCI-E graphics card from 2009. Not sure how much graphics RAM it has. But it probably only has a couple hundred MB.
But it also happens on my new Ryzen based system with 24G system RAM. I think I set the graphics RAM to 1G.

I doubt that this really is an out of memory problem unless LinuxCNC needs that much graphics memory now.

I even tried to raise memlock limit to 4G on the older PC. Didn't help.

On the old PC I use Debian Bookworm and on the Ryzen I use Debian Sid.

[Sigma1912]

looks like the same issue as this:
#1599

[mbuesch]

Well, I'm not so sure it is. The error text in #1599 talks about a boost error and not about out of memory. Whereas there's no mention of boost in my error message.
To me this looks unrelated.

[Sigma1912]

Ah yes, have a look at this one on the forum, seems to fit your error:
https://forum.linuxcnc.org/38-general-linuxcnc-questions/47910-opengl-out-of-memory-error#261447

[edit]
If you go back to the above linked issue you will see that the 'out of memory' error appears later in the discussion. Looking at this update from Seb this issue should be fixed since February though:

I think this issue talks about a number of problems, but the OpenGL "glBitmap out of memory" problem should be fixed by @jepler's https://github.com/LinuxCNC/linuxcnc/pull/2328. That fix is in 2.9 and master/2.10 now.

[mbuesch]

Yes, I know about these discussions, but this still seems to be a new problem.

I think this issue talks about a number of problems, but the OpenGL "glBitmap out of memory" problem should be fixed by @jepler's #2328. That fix is in 2.9 and master/2.10 now.

It doesn't fix it for me. I use latest 2.9 and master (as of the day I opened this Issue).

What I haven't tried is commenting out random pieces of code, like in all the suggestions.
But that doesn't really look like a solution to me.
I think the first step should be to understand what's really going on here.
From all those posts it seems like everybody is poking in the dark. That's why I opened this Issue.

If you have any suggestion for me to try, I'm completely open for it. It is 100% reproducible on two completely different machines. Therefore, I can try things out to track this down to the real root cause. But because I don't know anything about OpenGL, I'm a bit lost and need some help here.

[rene-dev]

this is mesa bug: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8904
more info here: #2446
replace the 2 calls with glBitmap(1, 1, 0, 0, w, -h+d, bytearray([0]*4)) as a workaround.

[mbuesch]

Thanks for the suggestion. Changing the second parameter from 0 to 1 fixed the problem for me. No more crashes (at least in axis demo mode. Now I need to run my actual setup and see...)

However, the change from #2446 changes the first parameter to 0 and the last one to None. I also tried that. That does not fix the problem. Therefore merging that PR would not fix it for me.

[rene-dev]

exactly. the PR does it correctly, but due to the bug in mesa that doesnt work.
afaik 1, 1 breaks it on some other systems, thats why it was changes to 1, 0.
it only moves the cursor, and should not need memory or a size at all.

[mbuesch]

What about catching the exception and trying again with the alternative parameter set? That way we could support buggy and fixed versions.

[petterreinholdtsen]

You mean something like this?

diff --git a/lib/python/glnav.py b/lib/python/glnav.py
index 77c6fbf7fc..c60cd5c001 100644
--- a/lib/python/glnav.py
+++ b/lib/python/glnav.py
@@ -2,6 +2,7 @@ import math
 import array, itertools
 import sys
 
+from OpenGL.error import GLError
 from OpenGL.GL import *
 from OpenGL.GLU import *
 
@@ -62,8 +63,10 @@ def use_pango_font(font, start, count, will_call_prepost=False):
         context.restore()
         w, h = int(w / Pango.SCALE), int(h / Pango.SCALE)
         glNewList(base+i, GL_COMPILE)
-        glBitmap(1, 0, 0, 0, 0, h-d, bytearray([0]*4))
-        #glDrawPixels(0, 0, 0, 0, 0, h-d, '');
+        try:
+            glBitmap(0, 0, 0, 0, 0, h-d, bytearray([0]*4))
+        except OpenGL.error.GLError:
+            glBitmap(1, 0, 0, 0, 0, h-d, bytearray([0]*4))
         if not will_call_prepost:
             pango_font_pre()
         if w and h: 
@@ -72,8 +75,11 @@ def use_pango_font(font, start, count, will_call_prepost=False):
                 glDrawPixels(w, h, GL_LUMINANCE, GL_UNSIGNED_BYTE, a.tobytes())
             except Exception as e:
                 print("glnav Exception ",e)
-
-        glBitmap(1, 0, 0, 0, w, -h+d, bytearray([0]*4))
+        try:
+            glBitmap(0, 0, 0, 0, w, -h+d, bytearray([0]*4))
+        except OpenGL.error.GLError:
+            glBitmap(1, 0, 0, 0, 0, h-d, bytearray([0]*4))
+        if not will_call_prepost:
         if not will_call_prepost:
             pango_font_post()
         glEndList()
diff --git a/scripts/linuxcnc.in b/scripts/linuxcnc.in
index 75abe58b78..051c9f357e 100644
--- a/scripts/linuxcnc.in
+++ b/scripts/linuxcnc.in
@@ -22,15 +22,6 @@ if test "xyes" = "x@RUN_IN_PLACE@"; then
     fi
 fi
 
-# Force pyopengl (python3-opengl) to use its GLX backend, even
-# if we're running on Wayland where it would normally use EGL.
-# Similar for the Gtk and Qt backends.
-if [[ ! -v LINUXCNC_OPENGL_PLATFORM || "${LINUXCNC_OPENGL_PLATFORM}" == "glx" ]]; then
-    export PYOPENGL_PLATFORM="x11"
-    export GDK_BACKEND="x11"
-    export QT_QPA_PLATFORM="xcb"
-fi
-
 ################################################################################
 # 0. Values that come from configure
 ################################################################################

Note, the patch is untested and is only intended as a starting point and to document the locations adjusted to fix the OpenGL issue.

[mbuesch]

Yes, something like that. Having the correct code for the non-buggy version in the try branch and then having one or more alternatives in the exception branch for buggy versions.

In my case (Debian Bookworm) I need these variants:

glBitmap(1, 1, 0, 0, 0, h-d, bytearray([0]*4))

glBitmap(1, 1, 0, 0, w, -h+d, bytearray([0]*4))

[coredump]

Had the same problem but with qtdragon_hd instead of axis. Using the patch from #2510 (comment) with mbuesch values for Bookworm fixed it, or at least fixed it enough that the interface opens again.

[andypugh]

Is anyone looking further at Petter's suggestion?