Vertico is confused by TRAMP syntax "simplified"

[rrthomas]

Start Emacs 30.1 with C-u package-isolate using vertico-2.4.

This gives me:

;; - compat-30.1.0.1 (dependency)
;; - seq-2.24 (dependency)
;; - vertico-2.4

Then:

M-x vertico-mode RET, C-h v tramp-syntax RET (this is to load tramp-change-syntax); M-x tramp-change-syntax RET, simplified RET, C-x C-f /, and I get the message in the minibuf:

"Vertico detected an error: Press C-h e to see the stack trace"

The problem appears to be with the fact that vertico is trying to apply text properties to an empty (zero-length) string in the list of completions, starting at position 1.

[minad]

Hi, I could not reproduce this problem. (Maybe it is already fixed in the newer Tramp version which I am using right now with Emacs 31). Can you please show the stack trace you've got?

[rrthomas]

Sure (lightly edited to remove some hosts from the list of remote hosts, which comes from ~/.ssh/config AFAICT):

Vertico detected an error:
  backtrace-to-string()
  vertico--debug((args-out-of-range 0 1))
  put-text-property(0 1 completion--unquoted "" "")
  #f(compiled-function (completion) #<bytecode -0x94d61869b541de1>)("")
  mapcar(#f(compiled-function (completion) #<bytecode -0x94d61869b541de1>) ("codereview.qt-project.org:" ".compute.amazonaws.com:" ".sf.net:" ".local:" "localhost:" "" "tmp/" "srv/" "bin.usr-is-merged/" "../" "opt/" "sbin.usr-is-merged/" "debug" "cdrom/" "lib64/" "etc/" "lib.usr-is-merged/" "root/" "lost+found/" "home/" "proc/" "lib32/" "sys/" "mnt/" "usr/" "sbin/" "run/" "dev/" "var/" "lib/" "bin/" "libx32/" "./" ...))
  completion--twq-all("~//" "/" ("codereview.qt-project.org:" ".compute.amazonaws.com:" ".sf.net:" ".local:" "localhost:" "" "tmp/" "srv/" "bin.usr-is-merged/" "../" "opt/" "sbin.usr-is-merged/" "debug" "cdrom/" "lib64/" "etc/" "lib.usr-is-merged/" "root/" "lost+found/" "home/" "proc/" "lib32/" "sys/" "mnt/" "usr/" "sbin/" "run/" "dev/" "var/" "lib/" "bin/" "libx32/" "./" ...) 1 substitute-in-file-name completion--sifn-requote)
  #f(compiled-function (unquoted-result op) #<bytecode 0x11656cbe50c26661>)(("codereview.qt-project.org:" ".compute.amazonaws.com:" ".sf.net:" ".local:" "localhost:" "" "tmp/" "srv/" "bin.usr-is-merged/" "../" "opt/" "sbin.usr-is-merged/" "debug" "cdrom/" "lib64/" "etc/" "lib.usr-is-merged/" "root/" "lost+found/" "home/" "proc/" "lib32/" "sys/" "mnt/" "usr/" "sbin/" "run/" "dev/" "var/" "lib/" "bin/" "libx32/" "./" ...) 2)
  completion--nth-completion(2 "~//" read-file-name-internal file-exists-p 3 (metadata (category . file) (completion--unquote-requote . t)))
  completion-all-completions("~//" read-file-name-internal file-exists-p 3 (metadata (category . file) (completion--unquote-requote . t)))
  apply(completion-all-completions ("~//" read-file-name-internal file-exists-p 3 (metadata (category . file) (completion--unquote-requote . t))))
  vertico--filter-completions("~//" read-file-name-internal file-exists-p 3 (metadata (category . file) (completion--unquote-requote . t)))
  vertico--recompute(3 "~//")
  vertico--update(interruptible)
  #f(compiled-function () #<bytecode 0x12d0afe02f5576a8>)()
  #f(compiled-function () #<bytecode -0x12f2f4be403b917d>)()
  handler-bind-1(#f(compiled-function () #<bytecode -0x12f2f4be403b917d>) (error) vertico--debug)
  vertico--protect(#f(compiled-function () #<bytecode 0x12d0afe02f5576a8>))
  vertico--exhibit()
  read-from-minibuffer("Find file: " "~/" (keymap (keymap (32)) keymap (10 . minibuffer-complete-and-exit) (13 . minibuffer-complete-and-exit) keymap (menu-bar keymap (minibuf "Minibuf" keymap (tab menu-item "Complete" minibuffer-complete :help "Complete as far as possible") (space menu-item "Complete Word" minibuffer-complete-word :help "Complete at most one word") (63 menu-item "List Completions" minibuffer-completion-help :help "Display all possible completions") "Minibuf")) (M-down . minibuffer-next-completion) (M-up . minibuffer-previous-completion) (27 keymap (13 . minibuffer-choose-completion) (103 keymap (27 keymap (99 . switch-to-completions))) (118 . switch-to-completions)) (prior . switch-to-completions) (63 . minibuffer-completion-help) (32 . minibuffer-complete-word) (backtab . minibuffer-complete) (9 . minibuffer-complete) keymap (menu-bar keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" previous-history-element :help "Put previous minibuffer history element in the minibuffer") (next menu-item "Next History Item" next-history-element :help "Put next minibuffer history element in the minibuffer") (isearch-backward menu-item "Isearch History Backward" isearch-backward :help "Incrementally search minibuffer history backward") (isearch-forward menu-item "Isearch History Forward" isearch-forward :help "Incrementally search minibuffer history forward") (return menu-item "Enter" exit-minibuffer :key-sequence "\15" :help "Terminate input and exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (24 keymap (down . minibuffer-complete-defaults) (up . minibuffer-complete-history)) (13 . exit-minibuffer) (10 . exit-minibuffer) (7 . abort-minibuffers) (C-tab . file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . previous-history-element) (up . previous-line-or-history-element) (prior . previous-history-element) (XF86Forward . next-history-element) (down . next-line-or-history-element) (next . next-history-element) (27 keymap (60 . minibuffer-beginning-of-buffer) (114 . previous-matching-history-element) (115 . next-matching-history-element) (112 . previous-history-element) (110 . next-history-element))) nil file-name-history "~/" nil)
  #f(compiled-function (prompt collection &optional predicate require-match initial-input hist def inherit-input-method) "Default method for reading from the minibuffer with completion.\nSee `completing-read' for the meaning of the arguments." #<bytecode 0x1134b2b5392ba6f>)("Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/" nil)
  apply((#f(compiled-function (prompt collection &optional predicate require-match initial-input hist def inherit-input-method) "Default method for reading from the minibuffer with completion.\nSee `completing-read' for the meaning of the arguments." #<bytecode 0x1134b2b5392ba6f>) "Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/" nil))
  vertico--advice(#f(compiled-function (prompt collection &optional predicate require-match initial-input hist def inherit-input-method) "Default method for reading from the minibuffer with completion.\nSee `completing-read' for the meaning of the arguments." #<bytecode 0x1134b2b5392ba6f>) "Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/" nil)
  apply(vertico--advice #f(compiled-function (prompt collection &optional predicate require-match initial-input hist def inherit-input-method) "Default method for reading from the minibuffer with completion.\nSee `completing-read' for the meaning of the arguments." #<bytecode 0x1134b2b5392ba6f>) ("Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/" nil))
  completing-read-default("Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/" nil)
  completing-read("Find file: " read-file-name-internal file-exists-p confirm-after-completion "~/" file-name-history "~/")
  read-file-name-default("Find file: " nil "~/" confirm-after-completion nil nil)
  read-file-name("Find file: " nil "~/" confirm-after-completion)
  find-file-read-args("Find file: " confirm-after-completion)
  byte-code("\300\301\302 \"\207" [find-file-read-args "Find file: " confirm-nonexistent-file-or-buffer] 3)
  call-interactively(find-file nil nil)
  command-execute(find-file)

[minad]

Thanks! I wondered why you didn't post the stacktrace right away (as is my expectation), but sensitive data is obviously a problem for file completion :(

This is not a Vertico bug (and cannot be fixed here) since the error occurs inside completion--twq-all, or more precisely, inside the (lambda (completion) ...) there.

You could report the problem upstream via M-x report-emacs-bug. But maybe the issue has already been fixed since I do not see it on Emacs master? As far as I remember there have been recent changes to this part of minibuffer.el. On the other hand I still find the offending line:

(put-text-property 0 1 'completion--unquoted completion qcompletion)

[minad]

Another thing to consider - empty candidates are an edge case. I don't remember if they are actually valid by the definition of completing-read. I think they are not valid since empty candidates are indistinguishable from the so called "null completion", where you exit completion without any input, with the goal of either selecting the default value or of bailing out. So the Tramp file completion table could better avoid to produce empty candidates.

[rrthomas]

Thanks! I wondered why you didn't post the stacktrace right away (as is my expectation), but sensitive data is obviously a problem for file completion :(

I didn't post a stack trace because the bug report template didn't ask for one!

You could report the problem upstream via M-x report-emacs-bug. But maybe the issue has already been fixed since I do not see it on Emacs master? As far as I remember there have been recent changes to this part of minibuffer.el. On the other hand I still find the offending line:

(put-text-property 0 1 'completion--unquoted completion qcompletion)

Thanks, I'll report an Emacs bug.

[rrthomas]

I filed https://debbugs.gnu.org/cgi/bugreport.cgi?bug=79173

[rrthomas]

An Emacs maintainer agrees that the problem does not manifest in Emacs 31, also agrees that TRAMP needs fixing (and has provided a patch), and finally, after I found the patch gave rise to a different problem, diagnosed a bug in vertico, and provided a patch. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=79173#25

[rrthomas]

Hmm, it is not correct for Vertico to bind minibuffer-completing-file-name.

Is it clear at least that there's a problem in vertico, or is there more diagnostic work to do?

[minad]

I don't understand why Michael thinks that there is a problem in Vertico. Maybe the variable minibuffer-completing-file-name is not set correctly or somewhere changes its value, but I have no idea where that happens. I could not observe any changes to the variables in my brief tests. From the stack trace, it follows that the variable is let bound by read-file-name-default.

[rrthomas]

Would you like to reply to the Emacs bug, or should I relay?

[minad]

Anyway, here is not the right place to discuss this. Let's keep the discussion on the emacs bug report. The Tramp patch has not been installed yet as far as I can tell from the Emacs master git log. I can test this again when it lands.

[rrthomas]

I think this discussion can be resolved now. I'm still a bit confused about what's going on, but on retrying the patched TRAMP, everything seems fine.