Consider dropping custom command line parsing to use “standard” rules

[ymeine]

While using the Everything CLI from both PowerShell and Node.js, I noticed a strange behavior: when starting to include spaces in some paths prefixed with a function, things would break.

Example: es folder:"folder/with some/spaces".

Turns out, by default, PowerShell or Node.js would quote and escape the argument to make sure it’s received as a single one, as long as the target application uses “standard” command line parsing to grab the arguments. This is not the case in Everything CLI.

So it receives "folder:\"folder/with some/spaces\"" (I omitted the beginning which contains the executable path). In the source code, _es_get_argv and _es_get_command_argv will not take into account the \" escape sequences and therefore count the second quote as a closing quote, ending up with two, weird arguments instead of one:

• "folder:\"folder/with
• some/spaces\""

I did some tests, changing the source code, and using argv from C instead seems to be working perfectly fine, at least for my simple use cases. CommandLineToArgv would probably work equally.

So the big question is: besides unfortunate backwards-compability requirements, what are the reasons Everything CLI is doing custom parsing? Could it be possible to use standard runtime behavior instead, in a future major version for instance? Or maybe an alternate build?

Context

• Everything version: 1.5.0.1404a
• Everything CLI version: 1.1.0.36

Reproduction

Example given for PowerShell, in a temporary, isolated directory:

mkdir 'folder/with some/spaces' # create a hierarchy with some spaces
es folder:"folder/with some/spaces" # search for it and see it break

There may be other use cases where it breaks, I haven’t taken the time to investigate more.

Workarounds

For those who are facing the same issue as me.

The list is not exhaustive, and solutions should be adapted to your context.

PowerShell

⚠️ It looks like this solution only works inside scripts, not in interactive shells. I haven’t pushed investigations further.

• use $PSNativeCommandArgumentPassing = 'Legacy', but don’t forget to revert it
• about_Preference_Variables - PowerShell | Microsoft Learn

function es {
	try {
	    $previous = $PSNativeCommandArgumentPassing
	    $PSNativeCommandArgumentPassing = 'Legacy'
	    $output = es.exe @args # don't forget the extension to avoid infinite recursion
	} finally {
	    $PSNativeCommandArgumentPassing = $previous
	}
	$output
}
# instead of: 
# es @args

Node.js:

• use windowsVerbatimArguments: true option
• Child process | Node.js v25.4.0 Documentation

import process from 'node:process';
import {spawn} from 'node:child_process';

const exe = 'es';
const args = process.argv.slice(2);

const proc = spawn(exe, args, {stdio: 'inherit', windowsVerbatimArguments: true});
// instead of: 
// const proc = spawn(exe, args, {stdio: 'inherit'});

Python

• use a single string input in Popen, instead of an array of strings. On Windows this would pass it as the whole command line, verbatim
• subprocess — Subprocess management — Python 3.14.2 documentation

import shutil
import sys
from subprocess import Popen

exe = shutil.which('es')
args = sys.argv[1:]

proc = Popen(' '.join([exe] + args))
# instead of: 
# proc = Popen([exe] + args)

proc.wait()

My workaround

What I actually did on my side is to use the Python wrapper (slightly improved compared to the snippet above), and bundle it with PyInstaller, then publish it as a release asset in my fork. That was also the perfect opportunity to use a more “standard” naming scheme for the asset, so that tools like mise-en-place would be able to find it more easily.

[voidtools]

Users come across this issue often.

I am adding a -argv command line option to use CommandLineToArgv.

CommandLineToArgv will not be used by default as it breaks too much.
A simple search for "c:\windows\" doesn't work because the trailing \" is converted to a literal "

If the calling process is powershell, I will look into using -argv by default.

A cmdlet is on my TODO list.

[ymeine]

Users come across this issue often.

Knowing that, maybe it would be a good opportunity to wait a bit and collect feedback/suggestions regarding how to handle that change then? Unless you already have a very clear and mature idea on your mind — which seems to be the case given the details you shared.

On my side, I opened the issue because I love this project and I think a change regarding that topic could make it even better. But there’s no rush, since I found workarounds which work for me.

I am adding a -argv command line option to use CommandLineToArgv.

Well, with that change I guess shell aliases would be easy to setup then. And for sure wrappers be easier to implement. And that keeps backwards-compatibility, which is the most important here.

CommandLineToArgv will not be used by default as it breaks too much.

Is it just pure legacy, or was there a specific use case where custom parsing was actually improving UX? I’m legitimately curious.

A simple search for "c:\windows\" doesn't work because the trailing \" is converted to a literal "

Yeah, to be honest the code was a bit much to digest among all the rest I had to work on, so I’m not even sure I understand how quotes are handled. I know it counts them, to know when there’s an opening quote and when there’s the corresponding closing one, but I didn’t go beyond that.

If the calling process is powershell, I will look into using -argv by default.
A cmdlet is on my TODO list.

That is interesting! I’m really looking forward to it.

Btw, is that TODO list private (which I would completely understand) or is it shared (maybe in the forum)?

---

That makes me think: would adding the possibility to enable stdin as input for the query be a good idea? Probably with an additional flag parameter. I can imagine cases where it would be easier for pipe processing. And it would also avoid this issue of encoding/decoding the command line, making all wrappers easier to implement as well.

[voidtools]

Btw, is that TODO list private (which I would completely understand) or is it shared (maybe in the forum)?

My TODO list is public and can be found in es.c at the top of the file:

https://github.com/voidtools/ES/blob/main/src/es.c

That makes me think: would adding the possibility to enable stdin as input for the query be a good idea?

I will consider a -search-file - command line option to read the search from stdin.
Thank you for the suggestion.

There is a -s* command line option that treats the rest of the search as literal, but it doesn't help with how Powershell escapes quotes.

[ymeine]

My TODO list is public and can be found in es.c at the top of the file:

https://github.com/voidtools/ES/blob/main/src/es.c

Great! I'll take time to check it.

I will consider a -search-file - command line option to read the search from stdin.
Thank you for the suggestion.

Happy to hear that. But again, no rush, it takes time to properly think through new features.

With that, it also means it will support regular on-disk files, which is another thing I had in mind.

But, be it a regular file or stdin, in both cases it raises the question: would it handle just one query, on one line? Would it treat multiple lines as multiple queries, like a OR, merging the results? Or should it just throw an error in this case? That's why I said above "it takes time to properly think through new features": I honestly think it is an opportunity to make something great, but because of backwards compatibility, it would be better to make the right shot from the start.

Having an input file to drive Everything is like opening the Pandora box, and I realize my small, additional suggestion of supporting stdin is the same. If we imagine "crazy" things, an input file could become "a script with a global context giving access to Everything API and run by Everything itself". Obviously, I didn't mean to go that far, and nor does your -search-file flag (otherwise it would be named -overkill-script 😄 ).

Now, it's a discussion that differs from the scope of the original issue here, so I won't digress more. Could be a separate issue/discussion on GitHub.

There is a -s* command line option that treats the rest of the search as literal, but it doesn't help with how Powershell escapes quotes.

I'll check that too, out of curiosity. So if I understand correctly, that was the reason of custom command line parsing.