Crash

[eed3si9n]

@ckipp01 @eed3si9n I went back in time trying different commits. The bug seemed to have been introduced in this commit that has some suspicious code: 1e70f7d

Originally posted by @gvolpe in nvim-treesitter/nvim-treesitter#4170 (comment)

Commit of tree-sitter-scala you tested this on

fce8f8c

A code sample showing the error

object Main

Show the error node

#0  0x000055e0c16039a0 in ts_decode_utf8 ()
#1  0x000055e0c160302e in ts_lexer.get_column ()
#2  0x00007f4a9064dfbe in tree_sitter_scala_external_scanner_scan () from /home/stephan/.local/share/nvim/site/pack/packer/start/nvim-treesitter/parser/scala.so
#3  0x000055e0c15e8afd in ts_parser_parse ()
#4  0x000055e0c13b7288 in parser_parse (L=0x7f4a937b1380) at /home/stephan/projects/neovim/src/nvim/lua/treesitter.c:421
#5  0x000055e0c163c1b6 in lj_BC_FUNCC ()
#6  0x000055e0c16261da in lua_pcall (L=0x20, nargs=<optimized out>, nresults=<optimized out>, errfunc=<optimized out>) at lj_api.c:1145
#7  0x000055e0c13af1b2 in nlua_pcall (lstate=0x7f4a937b1380, nargs=2, nresults=1) at /home/stephan/projects/neovim/src/nvim/lua/executor.c:153
#8  0x000055e0c13afc97 in nlua_call_ref (ref=5, name=0x55e0c16b8278 "buf", args=..., retval=true, err=0x7ffff60bd558) at /home/stephan/projects/neovim/src/nvim/lua/executor.c:1559
#9  0x000055e0c128d397 in decor_provider_invoke (ns_id=1, name=0x55e0c16b8278 "buf", ref=5, args=..., default_true=true, perr=0x55e0c17dcb30 <provider_err>) at /home/stephan/projects/neovim/src/nvim/decoration_provider.c:36
#10 0x000055e0c128e30c in decor_providers_invoke_buf (buf=0x55e0c2b70570, providers=0x7ffff60bd720, err=0x55e0c17dcb30 <provider_err>) at /home/stephan/projects/neovim/src/nvim/decoration_provider.c:184
#11 0x000055e0c12a39f3 in update_screen () at /home/stephan/projects/neovim/src/nvim/drawscreen.c:555
#12 0x000055e0c1398c1d in ins_compl_show_pum () at /home/stephan/projects/neovim/src/nvim/insexpand.c:1247
#13 0x000055e0c13998e5 in ins_compl_new_leader () at /home/stephan/projects/neovim/src/nvim/insexpand.c:1720
#14 0x000055e0c13999ea in ins_compl_addleader (c=120) at /home/stephan/projects/neovim/src/nvim/insexpand.c:1768
#15 0x000055e0c12a92cd in insert_execute (state=0x7ffff60bd948, key=120) at /home/stephan/projects/neovim/src/nvim/edit.c:570
#16 0x000055e0c15172fe in state_enter (s=0x7ffff60bd948) at /home/stephan/projects/neovim/src/nvim/state.c:99
#17 0x000055e0c12aa434 in insert_enter (s=0x7ffff60bd948) at /home/stephan/projects/neovim/src/nvim/edit.c:338
#18 0x000055e0c12a8fef in edit (cmdchar=65, startln=false, count=1) at /home/stephan/projects/neovim/src/nvim/edit.c:1268
#19 0x000055e0c1429a86 in invoke_edit (cap=0x7ffff60bdb10, repl=0, cmd=65, startln=0) at /home/stephan/projects/neovim/src/nvim/normal.c:6218
#20 0x000055e0c142454f in nv_edit (cap=0x7ffff60bdb10) at /home/stephan/projects/neovim/src/nvim/normal.c:6195
#21 0x000055e0c1421c44 in normal_execute (state=0x7ffff60bda90, key=65) at /home/stephan/projects/neovim/src/nvim/normal.c:1181
#22 0x000055e0c15172fe in state_enter (s=0x7ffff60bda90) at /home/stephan/projects/neovim/src/nvim/state.c:99
#23 0x000055e0c141dc94 in normal_enter (cmdwin=false, noexmode=false) at /home/stephan/projects/neovim/src/nvim/normal.c:478
#24 0x000055e0c11eb91f in main (argc=3, argv=0x7ffff60bdf28) at /home/stephan/projects/neovim/src/nvim/main.c:619

What do you expect the tree to look like

No crash

Where are you experiencing this error?

No response

[ckipp01]

Reopening this as it seems we're still getting crashes.

[gvolpe]

C is not my forte, but I got a compiler warning that could be saying something?

$ cargo build
   Compiling memchr v2.5.0
   Compiling tree-sitter v0.20.9
   Compiling tree-sitter-scala v0.20.7 (/home/gvolpe/workspace/tree-sitter-scala)
   Compiling aho-corasick v0.7.20
   Compiling regex v1.7.1
warning: src/scanner.c: In function ‘tree_sitter_scala_external_scanner_scan’:
warning: src/scanner.c:153:30: warning: comparison of integer expressions of different signedness: ‘uint32_t’ {aka ‘unsigned int’} and ‘int’ [-Wsign-compare]
warning:   153 |     lexer->get_column(lexer) == stack->last_column) {
warning:       |                              ^~
    Finished dev [unoptimized + debuginfo] target(s) in 4.90s

Here's the relevant code:

tree-sitter-scala/src/scanner.c

Lines 152 to 155 in 8079aed

	if (stack->last_newline_count > 0 &&
	lexer->get_column(lexer) == stack->last_column) {
	newline_count += stack->last_newline_count;
	}

[eed3si9n]

C is not my forte, but I got a compiler warning that could be saying something?

Currently we serialize everything using int * pointer, so I chose int for last_column. We can add explicit cast before the comparison to be sure, but I don't think this would cause illegal access of memory.

[gvolpe]

Can't really tell whether that's a good thing or not, but I trust you know your C-fu 😄

To really know what's going on, one approach is to do as Stephan indicates here: nvim-treesitter/nvim-treesitter#4170 (comment)