subprocess.Popen hang issues on "opencode run message --json format"?

[davidbernat]

Question

We are simply trying to launch opencode in the background via Python subprocess.Popen, and the execution hands indefinitely at the process.readline() moment. For clarity, this is the standard, very common, way to launch a subprocess (or opencode turn) into the command line from within Python, and yet there are a number of well-known peculiarities in Python between the process.readline() hanging indefinitely when new lines and empty byte strings are not encountered, potentially when the underlying command line process does something peculiar with its handling of newlines etc. I searched the opencode GitHub issues and found only one mention of subprocess.Popen (which really surprises me given how procedurally common this is a requirement to run from Python) so I figured it might be helpful for everyone others to debug this issue here.

This command works as intended when run directly from the command line in about 10 seconds, returning its first json formatted line (type=step_start) within about 7 seconds. All pretty common and in line with TUI expectations.

The use of --format json does not impact the situation, nor does simpler prompts, nor do prompts without files.

The replacement of cmd with trivial or simple cmds (e.g. `cmd=["echo", ""hello""] all work as expected with no issues.

I have searched my code repositories for how I have approached this the few times I remember this being an issue, and most of my implementations elsewhere are more or less identical, so I do not know what the issue is.

Do you?

cmd = ["opencode", "run", "\"Please complete the task using @test_TASK.md @test_PROJECT.md\"" "-f" "test_TASK.md" "-f" "test_PROJECT.md" "--format" "json"]

env = os.environ.copy()
proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while self.proc.poll() is None:
  # stdout, stderr = self.proc.communicate()  # tried this too
    for stream, is_error in [(self.proc.stdout, False), (self.proc.stderr, True)]:
        if stream and not stream.closed:
            try:
                line = stream.readline()  # <== HANGS HERE INDEFINITELY ON FIRST READ
                if line:
                    line = line.decode("utf-8", errors="replace").rstrip()
            except: pass

🔧 Executing command:
   opencode run "Please complete the task using @test_TASK.md @test_PROJECT.md" -f test_TASK.md -f test_PROJECT.md --format json
📝 Process 19836 started at 23:28:12
❤️ Heartbeat monitoring active (every 5 seconds)
️❤️ Heartbeat now=23:28:12    +0m 0s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:17    +0m 5s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:22    +0m10s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:27    +0m15s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:32    +0m20s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:37    +0m25s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:42    +0m30s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:47    +0m35s pid=19836 state=running_waiting
️❤️ Heartbeat now=23:28:52    +0m40s pid=19836 state=running_waiting

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• opencode run hangs forever on API errors (breaks CLI/automation integrations) #8203: opencode run hangs forever on API errors (breaks CLI/automation integrations) - directly addresses subprocess automation hanging
• Local MCPs fail to start - stdin closes immediately after spawn #8434: Local MCPs fail to start - stdin closes immediately after spawn - related subprocess communication and stdio handling
• TUI exits but process hangs (no shell return) when opencode.json is present #1717: TUI exits but process hangs (no shell return) when opencode.json is present - related process hanging and subprocess cleanup issues
• Embedded bun runtime hangs indefinitely on bun add during bootstrap on Linux (BunInstallFailedError) #11836: Embedded bun runtime hangs indefinitely on bun add during bootstrap on Linux - similar subprocess hanging indefinitely issues

Feel free to ignore if your specific case involves different circumstances or error conditions.

[justindra]

This could potentially be related to a fix in #11895 - if the model was attempting to use the question tool during opencode run, it would hang indefinitely waiting for a response that would never come (since there's no interactive TUI to answer questions).

The fix ensures that tools denied by session permissions (like the question tool in non-interactive mode) are properly filtered out before being sent to the model, preventing the model from ever calling them.

[davidbernat]

Thanks @justindra, definitely related, but I do not think that will resolve the issue.

1. running manually CLI the prompt executed correctly a dozens times, and never by prompting a question.
2. the issue only occurs when running the same CLI execution from subprocess.Popen

Notes:

1. I was using opencode v1.1.48 last night and upgraded to v1.1.49, which is the version with the question bug.
2. Using v1.1.49 does not result in question hanging and works properly, but not from subprocess.Popen

Let me know what to debug. Nothing jumps out at me immediately.
This is a blocker. I have other housekeeping I can work on.

[dogmatic69]

having the same issue trying to run from python in 1.1.52. I added "permission": "allow" to the config and then seems to work, so it's 100% getting stuck on a question.

[davidbernat]

@rekram1-node @dogmatic69 again, perhaps I am the For Dummies author of our future.

1. Why on earth would I permission a closed UI system to blanket permission=allow?
2. Can we find a diagnostic to confirm opencode serve and opencode run hang "for questions"?
3. I appreciate @dogmatic69 chiming in but can we get a member of the company to chime in? (Love IKEA, btw.)
4. The commands run in Terminal perfectly as expected. Only through subprocess.Popen and REST API do we see this.
5. Nothing in Terminal successes suggests the system is getting hung up on AskUserQuestion.
6. Using the CLI tool from Terminal works as expected, repeatedly, with no other indications.
7. I have very, very thoroughly built out our subprocess.Popen infrastructure this week waiting for an answer.

We are being very helpful. We are really disappointed in OpenCode.
I cannot think of a single post that was answered responsibly.

Here is a simple one: opencode/big-pickle seems to have started underperforming, did a backend change occur?
We are very serious practitioners here with a tremendous amount of sympathy to your investors over the years.

[suraciii]

I also ran into this issue, disabling supermemory plugin fixed it.

[davidbernat]

@suraciii absolutely not an issue for starlight. The system faults at OpenCode, and their teams have been multiply briefed. xo.

@dogmatic69 et al. What is "Tequila Sunrise" and how is that related to the engine OpenCode is using to automate its GitHub.
Also: please note that no answers are provided why yesterday Starlight was required to create a GitHub Login & Google Login to access OpenCode Zen, and include a $20 upfront payment, when both of these would have been required months ago at sign-up. For those reading along we are tracking the changes of Silicon Valley investor architectures in response to the Wars.

When a company does not provide a near-immediate upfront reasoning (an xAI) we quietly begin replacing that company.

[dogmatic69]

It's someones user name... 🤦🏻

[davidbernat]

We have updated our software and find that the opencode run CLI usage appears to be resolved by version 1.2.26.

We find that the CLI interface works with 1. short messages, 2. messages + trivial files, 3. messages + complex files.
This appears to resolve this ticket. The opencode serve + SDK still hangs for case 3. But that is a different ticket.

Thanks.

[davidbernat]

Learned something new today about an old problem about terminals and IDE emulators. For those who are interested in this problem, which was originally traced back to internal malfunctions of opencode run, but then continued to persist in some environments (specifically, IntelliJ IDE), the problem is core to the difference between the runtime environment of IDEs and Terminals. I had never come across this solution before, and the hiccup of the Python code is subtle enough that I can easily imagine a lot of other people having the same problem. So I am going to post my new understanding here and the resolution.

This is likely very simple to many of you who know how terminal stdin/stdout physics works, but the error creating code was included in the original February 2nd ticket 48 days ago, and nobody from any of the teams commented on the line or added any solution, so let us go deep.

THE PROBLEM

When "opencode run" is executed from macOS Terminal, it now works reliably. When the same code is invoked from a Python script creating a subprocess, it works reliably when the Python script is executed from macOS Terminal. When the same code is invoked from a Python script running inside IntelliJ IDEA's Python run or debugger configuration, the subprocess hangs indefinitely. This is not a simple bug to decode unless you know the innate terminal characteristics of the IDE, a consequence of how different execution environments handle standard input (stdin), and it's a well-known issue that affects many developers.

THE ROOT CAUSE

The issue originates from a line in the Python script :
self.proc = subprocess.Popen(self.cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

Note that stdin is not specified. When stdin is None (the default), Python's subprocess.Popen inherits the parent process's stdin. (The parent being whatever application is launching Python, i.e., macOS Terminal or the IntelliJ Python runtime environment.) This subprocess works correctly when the parent process runs in a terminal. However, in an IDE like IntelliJ, the execution environment behaves differently.

The TLDR fix is to directly set how you want the stdin to behave when the user (you, the non-existent, non-active typer) provides no input. The computation carried out by the software and its print statements are not part of the stdin physics.

self.proc = subprocess.Popen(self.cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

WHY TERMINAL AND INTELLIJ BEHAVE DIFFERENTLY

When you type into a terminal, your keystrokes go into a buffer. The computer processes tasks, then checks the buffer for any new keystrokes. This check-and-buffer interaction is fundamental to how programs handle input by trading off computing cycles between ongoing processes and new user engagement which might deviate existing computations (such as even exiting the process).

In a PTY (macOS terminal, which is a "pseudoterminal", or PTY), when the buffer is empty (no typing, or previous typing has been processed), the operating system immediately adds to the buffer its special "end of file [buffer]" (EOF) character telling the program "the user isn't typing right now, move on with your existing computation." The program continues processing, and the buffer remains on a side still open for future typing to pool up if the user starts typing again.

In an IDE console like IntelliJ's, stdin is connected to a pipe instead of a PTY. A pipe is a simple one-directional channel for data between processes (type on one side of the pipe and the characters arrive at the calculating process on the other). Unlike a PTY, a pipe does not automatically signal EOF when empty—it simply has no data waiting. So when the program's read() call checks stdin and finds nothing, it doesn't get a clear "nothing here, continue" from a PTY—instead it gets "well, there is no backlog, so, yeah, how you doing, there might be more data coming eventually, wanna hang" and the computer processing side is (in both cases) passive and does not (in both cases) close the connection until the input side says to continue. The program then waits (hangs), blocking indefinitely for input that will never arrive (because the Python runtime environment was not set up to be getting any user input). (This is the part that scrambles my brain: if we know on the IDE side the user is not going to input, why is input and stdin even part of the piping infrastructure? That is, in effect, why stdin=subprocess.DEVNULL says DEVNULL as in device is null but under the hood actually still contains the system to be read and thus needs to shut off the file reads with the end of file "carry on" message.

Why would IntelliJ configure itself this way? Because IDE consoles are designed primarily for displaying output, not for full terminal emulation. Creating a PTY requires more operating system resources and complexity, and historically IDEs didn't need interactive stdin for most tasks. Using a simple pipe is easier and adequate for most use cases—running tests, executing scripts, building projects, and so on. The tradeoff favors simplicity and performance over complete terminal compatibility.

This is standard behavior across most modern IDEs. IntelliJ, PyCharm, VS Code, Eclipse, and others all use pipe-based stdin in their internal consoles by default. JetBrains explicitly documents this as a known limitation of their IDE console (see JetBrains YouTrack issue IDEA-133811), and that finally came up in the search engine searches tonight for solutions to this bug. The workaround is either to run in an external terminal, or—as we see—to explicitly configure how stdin is handled. But to set stdin=subprocess.DEVNULL for any processes in which the user is not anticipated to engage directly with a keyboard-like device is standard smart enforcement to remove the issue of when and how the program is running. Learned something today.

WHY THIS BUG IN MY CODE WAS MISSED

Several factors conspired to hide this issue:

1. Environment-specific failure: The code works in the developer's primary workflow (Terminal), so it was assumed to be correct. Bugs that only appear in certain environments are easy to miss.

2. No error, no log: The subprocess doesn't crash—it simply hangs on a blocking read() call. This can appear as slow execution rather than a deadlock, leading engineers to suspect LLM interactions, API calls, or tool execution rather than the process-spawning mechanism.

3. Implicit vs. explicit: The bug exists in what's not specified (stdin defaults to None), rather than in explicit code. Default behaviors are easy to overlook, especially when they work in most circumstances.

4. Known IDE limitation: This is a documented issue with IDE consoles, but it's not widely advertised. Developers may not realize that IDE consoles handle stdin differently than terminals.

5. Lack of knowledge of how .read() interacts with a complex system of piping at a layer between two applications, which is also dependent on operating system and application design choices.

APPENDIX: TECHNICAL EXPLANATIONS (ELI15-FRIENDLY)

A. What is a Terminal, and What is a PTY?

Terminal originally referred to physical hardware—a screen and keyboard connected to a mainframe computer. You typed commands, the mainframe processed them, and text appeared on your screen.

A PTY (Pseudo-Terminal) is a software emulation of that physical terminal. When you open macOS Terminal, the operating system creates a PTY pair:

• Master side: Connected to Terminal.app (the window you're typing in)
• Slave side: Connected to your shell (bash, zsh, etc.)

The key property of a PTY is that it tells programs "you are connected to an interactive device" via the isatty() function. This allows programs to behave differently in interactive sessions versus scripted environments.

B. What Does "stdin is Empty" Mean? What is EOF?

stdin is a stream of bytes that flows into a program—whatever you type, or whatever is piped into the program (if you are configuring a process to be waiting for input from another program, which is not the case in this example).

EOF (End-of-File) is a special signal that says "there is no more data coming." When you press Ctrl+D in a terminal, you're sending EOF.

When a program calls read() on stdin, it asks: "Is there any input?" If stdin is a file or pipe, read() waits until either data arrives OR EOF is signaled. If stdin is a PTY and you're not typing, the OS immediately returns EOF rather than waiting.

The difference: In a PTY, EOF is signaled automatically when the input buffer is empty. In a pipe without an explicit close, EOF may never be signaled—the read() call just waits.

C. What Does isatty() Do?

isatty() is a system call that asks: "Is this file descriptor connected to a terminal?"

   #include <unistd.h>
   int isatty(int fd);  // Returns 1 if fd is a terminal, 0 otherwise

Many programs check isatty(STDIN_FILENO) to decide their behavior. If it returns true, they know they're in an interactive session. If false, they know they're in a script or pipe.

The Python documentation explicitly states: "The interpreter operates somewhat like the Unix shell: when called with standard input connected to a tty device, it prompts for commands and executes them." When stdin is not a tty (e.g., IDE console), Python may behave differently.

D. What is a Blocking read() Call?

When a program calls read(fd, buffer, size), it says: "Read up to size bytes from fd into buffer."

In blocking mode (the default), if no data is available, the call waits (blocks) until data arrives or EOF is received. In non-blocking mode, the call returns immediately with whatever is available (or nothing).

When stdin is a pipe without clear EOF, read() blocks indefinitely—the program hangs, waiting for input that never comes.

E. Why Doesn't This Interrupt stdout?

It doesn't "interrupt" stdout—it never reaches stdout at all. The process hangs during initialization, before it even produces output:

1. Child process starts
2. Child tries to read stdin (checking for configuration, interactive prompts, etc.)
3. read() blocks waiting for input
4. Child never reaches the point of producing stdout
5. Parent (IntelliJ) also waits for child to produce output
6. Deadlock: both wait forever

APPENDIX: BIBLIOGRAPHY

1. Python Documentation — subprocess module
   https://docs.python.org/3/library/subprocess.html
   Documents that stdin=None inherits from parent, and subprocess.DEVNULL behavior.

2. JetBrains IntelliJ IDEA — Run/Debug Configuration
   https://www.jetbrains.com/help/idea/run-debug-configuration-application.html
   Documents the "Redirect input from" option and known stdin limitations in IDE consoles.

3. JetBrains YouTrack — IDEA-133811
   https://youtrack.jetbrains.com/issue/IDEA-133811
   Known issue: "stdin doesn't work in IDE console" for interactive applications.

4. Stack Overflow — "Python interactive shell not responding to input when run in subprocess"
   https://stackoverflow.com/questions/66763957
   Technical explanation: "Note that you are connecting python to a non-tty standard input, and so it behaves differently... it waits until stdin is closed."

5. Python Bug Tracker — Issue 12786: subprocess wait() hangs when stdin is closed
   https://bugs.python.org/issue12786
   Historical discussion of stdin inheritance and pipe behavior in subprocesses.

6. Stack Overflow — "Why Doesn't IntelliJ Idea Console Pause for User Input"
   https://stackoverflow.com/questions/44672217
   Confirms: "The Run console does not wait for user input at the readLine() method."

7. Stack Overflow — "Communicating Java and Python processes freeze after a while when using readline()"
   https://stackoverflow.com/questions/78228951
   Discusses pipe buffer deadlocks between parent and child processes.

8. The Linux Programming Interface — Michael Kerrisk, Chapter 44
   PTY (pseudo-terminal) internals and how they differ from pipes.

9. Advanced Programming in the UNIX Environment — W. Richard Stevens, Chapter 9
   File descriptor inheritance in fork() and exec(), stdin/stdout/stderr handling.

There is probably one 22 year old kid out there that will find this very handy to get written down. Thanks.
SDK and opencode serve unchanged.