Dashes in Wiki page titles are replaced with spaces

[qwertfisch]

• Gitea version (or commit ref): 1.9.0+
• Git version: 2.22
• Operating system: Linux
• Database:
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes: https://try.gitea.io/qwertfisch/TestRepo/wiki/Page-with-dash-here
  • No
  • Not relevant

Description

The Gitea internal wiki does not allow dash characters in page titles. I can enter it on creation, but it is replaced with a space on display.

I know that the direct cause is that spaces are replaced with dashes on filename level (also in the URL), so there is no distinction between spaces and dashes when reopening a page. Hence it will all be escaped and displayed as a space character.

Attempted workaround with URL-escaped character

All non-ASCII Unicode characters are displayed as URL-escaped variants of their Unicode code point, and even / is replaced with %2F. So I would expect that a filename containing %2D (stands for the dash) could be loaded and displayed with its correct title. I had manually created such a file, pushed it, but the %2D-escaped dash is still further escaped to a space. Even worse: the URL with the page won’t load anymore.

This looks like a deeper error in escape handling with the dash character.

[zeripath]

If what I suspect is happening is, then I bet if you use %252D it will work.

However that's not really a solution...

[lunny]

It seems we already have a PR to fix this.

[qwertfisch]

@lunny In the list of open PRs I could not find anything related. Can you reference the PR in mind?

[qwertfisch]

@zeripath Sorry, that does not work either. The replacement of a dash character to a space seems to be executed right after URI decoding, but there is no double decoding.

I made an overview table to see the different behaviour. The last column means that a page can be loaded on the given URL, which does not work in the second and third case. It is only visible in the list of all pages.

Description	Filename	Resulting URL	Display	Page is found
Normal URI encoded chars	colon%3A-try-it.md	colon%3A-try-it	colon: try it	✔
Space URI encoded	space%20test.md	space-test	space test	✘
Dash URI encoded	page-with-one%2Ddash.md	page-with-one-dash	page with one dash	✘
Dash double URI encoded	page-with-one%252Ddash.md	page-with-one%2Ddash	page with one%2Ddash	✔

The second and third rows are kind of confusing. It looks as if %20 and %2D are URI decoded from the filename, then there is the default dash/space conversion to create a page title. But when creating the URL, it seems as if it’s not created from filename, but from the (previously decoded) page title. Which of course results in - characters for all spaces (and no dash at all), so neither the correct file with %2D nor %20 can be found.

Suggestion

Can’t we just remove skip the errornous space/dash conversion and store the page with URI encoding? This would solve every problem case, with the slight disadvantage of spaces encoded %20 in the URL …

[mrsdizzie]

Haven't looked too close but this is probably happening here:

gitea/models/wiki.go

Lines 43 to 53 in d4667a4

	func WikiFilenameToName(filename string) (string, error) {
	if !strings.HasSuffix(filename, ".md") {
	return "", ErrWikiInvalidFileName{filename}
	}
	basename := filename[:len(filename)-3]
	unescaped, err := url.QueryUnescape(basename)
	if err != nil {
	return "", err
	}
	return NormalizeWikiName(unescaped), nil
	}

So its first unescaping the file name and then running it through something that just replaces - with " " which matches described behavior.

I agree the filenames should have always been stored encoded but unfortunately they weren't and changing that would put dashes in the title of everyones page where they weren't there before. Would be a breaking change to consider.

[zeripath]

Make it a per repo configurable?

[mrsdizzie]

I hesitate to add yet more config options, particularly to keep a behavior that is not good (current replacing of space with dash).

I'd rather try and have some sort of code to handle legacy cases, or even just consider all literal dashes as spaces (breaking the less common case of a dash in the title) and then escape everything going forward and handle that properly.

I guess that would maybe look like first replacing all literal dashes and then escaping the filename. It would only break existing titles with a dash in them which already seem to not work properly anyways per this issue

[zeripath]

Hmm what happens to + ?

[mrsdizzie]

In what situation? I think in the situation of saving a new filename we should make sure to encode + to %2B and not leave it as +

Are you asking if somebody already has a + in their filename? If so that should just still work since any unencoding wouldn't mess with it.

[mrsdizzie]

Probably the real problem to solve would be preserving current links or making sure they still work somehow

[qwertfisch]

@zeripath + and ? are escaped and can be used properly. In fact every ASCII special character is usable. Only -._~ are not encoded but put directly in the filename / URL. (And of course the space is converted to dash before storing as file. This should be %20 normally.)

• page title: chartest !"#$%&'()*+, ./:;<=>?@[]^_`{|}~§
• filename: chartest-%21%22%23%24%25%26%27%28%29%2A%2B%2C-.%2F%3A%3B%3C%3D%3E%3F%40%5B%5C%5D%5E_%60%7B%7C%7D~%C2%A7.md

[zeripath]

@mrsdizzie @qwertfisch I was meaning that a plain '+' in an url should map to ' ' by convention, and through my testing it appears that this doesn't get mapped to '-' but rather retains the ' ' when passed through - so you should be able to reach a file with spaces in that way.

Yeesh this is so broken. I've been ignoring the wiki, as like the diff page, as I have been suspicious that it needs a thorough overhaul. This proves my fears.

@mrsdizzie I agree that adding configuration should be avoided as much as is possible. If we don't want to add configuration another option is to from 1.10 do things correctly but fallback to the previous behaviour if the old way would have a different file?

[mrsdizzie]

Perhaps -- probably something to think about if redesigning this. I'll note that it appears Gitea has just copied this same exact behavior from Github, which does the same thing with dashes and titles/filenames and you can't create a page title with dashes in the title there either.

[icetiger1974]

hello
Is this issue solved?
Or remain in the same situation?