`cider-enable-flex-completion` is not a sufficient replacement for `cider-company-enable-fuzzy-completion`

[alexander-yakushev]

After I enabled cider-enable-flex-completion instead of cider-company-enable-fuzzy-completion, I noticed that my completion was acting up. Certain prefixes that used to complete correctly now failed to complete, and weird suggestions were popping up. The weird suggestions are understandable because of flex aggressive fuzzy policy, but that's not that big of an issue. The issue is the following completion scenarios supported by compliment that don't work in flex:

1. Namespace completion by first letters of namespaces:

• (cider-complete "cji") => (#("clojure.java.io" 0 1 (ns nil type "namespace")))
• Tab with flex => no completions.

2. FQ classname completions by first word:

• (cider-complete "InputSt") => (#("java.io.InputStream" 0 1 (ns nil type "class")) #("java.io.InputStreamReader" 0 1 (ns nil type "class")) ...
• Tab with flex => no completions.

3. Fuzzy completion behind namespace prefix:

• (cider-complete "str/sl") => (#("str/split-lines" 0 1 (ns "clojure.string" type "function")))
• Tab with flex => no completions.

There are a couple of others but they are not as important as these.

Additionally, Emacs achieves this flex behavior by sending two requests to the backend: one with the normal prefix and one with empty prefix (""). Since we have no upper bound on the number of candidates we return, this may add some extra overhead/delay for the completion because CIDER has to deserialize all that output.

[alexander-yakushev]

Overall, the less the frontend (editor) tries to be smart about the completion candidates, the better. We can control the UX much better on the backend. And if there are users that want flex-like fuzzy matching, we can also add that to Compliment and hide behind a toggle. That is still much more efficient and makes more sense than filtering through everything on the frontend side.

[vemv]

Thanks for the issue!

We should tread carefully here because we shouldn't necessarily be smarter than (or reinvent) Emacs' mechanisms.

Most of all, it's a concern of simplicity - Emacs has an abundance of styles, and there's also an abundance of Emacs packages building up on those, so creating our own standards can possibly make things even messier for everyone involved.

I don't have a specific suggestion/stance except: we can try first hard at adhering at Emacs standards and squeezing performance out of them.

If we document reasoning/facts reflecting that that's impossible, we could keep exploring solutions.

[vemv]

Quick q - would it make sense to pass the completion style to Compliment?

In my mind it would make sense - then Compliment could infer what's wanted and return fewer / more relevant results.

[alexander-yakushev]

I'll repeat just in case: the problem is Emacs not displaying valid completion candidates returned by Compliment. Some of those are really important (like completing a class by its short name).

What you've proposed, if I understand correctly, would be to pass say flex to Compliment and an empty prefix and so that Compliment understands that Emacs will do the filtering and it should return everything it knows. But that would mean, for example, returning 600 thousand loaded classnames and I doubt that would be an efficient thing to do.

[vemv]

What if we passed flex and what the user has typed?

So it would include (among other stuff) these 3 things:

• What Emacs wants us to complete - ""
• The style - "flex"
• What CIDER knows that the user has actually typed - "InputSt"

[alexander-yakushev]

We should tread carefully here because we shouldn't necessarily be smarter than (or reinvent) Emacs' mechanisms.

I think we should be smarter than Emacs mechanisms. I don't see mainline Emacs as the frontier in completion solutions; e.g., company-mode steps away from it in many regards and there is a reason we use company and not just completion-at-point.

[alexander-yakushev]

What if we passed flex and what the user has typed?

Then Compliment doesn't really need to know that it was flex that triggered it, the returned candidates would be the same.

[alexander-yakushev]

I think I understood what you mean. So we would return the same "preferred" result twice, for normal prefix, and for the empty prefix, and let Emacs show the candidates that we want to show in the empty prefix case. I mean, that would work, but it feels like a bigger hack to me than what cider-company-enable-fuzzy-completion was doing.

[vemv]

company-mode steps away from it in many regards and there is a reason we use company and not just completion-at-point.

Well we (certainly you and me) use Company but many others user Corfu and whatnot.

That's where adhering to standards show their value - supporting a large divesity of users

Then Compliment doesn't really need to know that it was flex that triggered it, the returned candidates would be the same.

Then we have a potential solution, right? Have CIDER pass the actual user input as an additional k-v

(passing flex might be worthwhile for further trimming results, but I'd understand if you didn't want that duplicated work)

[vemv]

let Emacs show the candidates that we want to show in the empty prefix case.

Yes

I mean, that would work, but it feels like a bigger hack to me than what cider-company-enable-fuzzy-completion was doing

It seems a clean, low-effort, low-complection solution to me. Pass additional properties from the frontend and act on them on the backend.

To me (as someone who strives to support a diverse set of users) there's high value in adhering to Emacs standards, so I'd vote for this solution unless there was a huge factual drawback.

[alexander-yakushev]

Actually, I tried it again, and I see two identical messages are sent to nrepl for each completion attempt:

(-->
  id                        "570"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:42:27.829532000"
  context                   "nil"
  enhanced-cljs-completion? "t"
  prefix                    "InputS"
)
(<--
  id          "570"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:42:27.832275000"
  completions ((dict "candidate" "java.io.InputStream" "type" "class") ...)
  status      ("done")
)
(-->
  id                        "571"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:42:27.855899000"
  context                   "nil"
  enhanced-cljs-completion? "t"
  prefix                    "InputS"
)
(<--
  id          "571"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:42:27.858973000"
  completions ((dict "candidate" "java.io.InputStream" "type" "class") ...)
  status      ("done")
)

How would passing extra arguments to Compliment help here?

[vemv]

Please describe what's wrong in the nrepl interaction above

[alexander-yakushev]

The problem is that Emacs sends two identical requests to the backend (with prefix "InputS"). Compliment returns the correct result both times, but Emacs refuses to render it.

[vemv]

Why does that happen?

[alexander-yakushev]

Do you ask me? :)

However, if the prefix is all-lowercase, then it sends the second request with an empty prefix:

(-->
  id                        "658"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:55:04.272861000"
  context                   "(__prefix__)
"
  enhanced-cljs-completion? "t"
  prefix                    "cji"
)
(<--
  id         "659"
  session    "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp "2024-05-05 21:55:04.293445000"
  status     ("done" "no-eldoc")
)
(-->
  id                        "661"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:55:04.353659000"
  context                   "(__prefix__)
"
  enhanced-cljs-completion? "t"
  prefix                    ""
)
(<--
  id          "661"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:55:04.387442000"
  completions ((dict "candidate" "*" "ns" "clojure.core" "type" "function") ...)
  status      ("done")
)