Port code to Python 3

[bdamele]

Currently there is no pressure on Python projects to switch to the new version of Python interpreter, as the process of switching, especially on larger projects can be cumbersome (due to the few backward incompatibilities). The switch will take place eventually, but currently it is a very low priority task.

[stamparm]

Good reference: http://python3porting.com/noconv.html

[stamparm]

As I've collected information regarding this over time, this will be maybe an impossible task to do because of one feature: "unicode strings". We currently extensively use unicode everywhere together with encoding/decoding for page responses. Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

Good reference: http://lucumr.pocoo.org/2010/1/7/pros-and-cons-about-python-3/

[badbob]

Is there any news for this issue? Does Python 3.4 have required functionality?

[stamparm]

@badbob I would say no, but that's my opinion. Only small projects are prone to conversion from 2 to 3. I have a feeling that this is a futile task for sqlmap.

[badbob]

Looks like the only way to reuse sqlmap code in Python 3 solutions is a REST API. Thanks for developers sqlmap has it. I can't see any other variant.

[Flimm]

Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

I would highly disagree with this statement. In Python 2, strings represented either textual data or raw data (often raw data of textual data in an assumed or unknown encoding), while unicode objects represented textual data. Strings in Python 2 were by nature ambiguous. In Python 3, bytes objects represent raw data, and string objects represent textual data. It's crystal clear in Python 3, and it's enforced with exceptions when mixing strings with bytes objects. This crystal clear distinction exposes bad design choices in certain systems (for instance, the ambiguous encodings of URLs, or of filenames), and it causes the programmer to have to consider these problems upfront. Python 2 just let you stuff everything into a string, and it gives you runtime exceptions sometime in the future when you try anything not in ASCII.

I'm not trying to argue that sqlmap should necessarily be ported to Python 3. I just felt someone needed to speak up and defend Python 3's explicitness when it comes to strings and bytes.

Python 2.7 will no longer be supported past 2020, as advertised by this website: https://pythonclock.org/ Many scientific libraries for Python have announced that they will also no longer support Python 2.x http://www.python3statement.org/

If the owners of this project would welcome efforts to port this project to Python 3, perhaps create an issue saying so, with the tag help-wanted.

[stamparm]

I'll just leave this here. It is not an argument for anything, just to get the clear picture of current situation in Python world.

Reference: https://hynek.me/articles/python3-2016/

[stamparm]

@Flimm sqlmap currently has around 64K LoC (excluding comments and blank lines). Running simple 2to3 is not an option here (if you think that 2to3 is enough then you are utterly wrong). As I can see from other projects, estimates for conversion goes from 3-6 months on projects of this size. As I've solely done the unicode support here, I believe that I have a right to say that switching of unicode related stuff from Python 2 to Python 3 is one of the biggest obstacles for this whole switch. This is not only my opinion. Just Google around.

Let us all together wait for the 2020. I have a feeling that it will pass as any other (Python) year. I bet that nothing will change as there were couple of similar (failed) attempts in past couple of years.

[Flimm]

So you wouldn't welcome efforts to port this to Python 3?

[stamparm]

@Flimm efforts of porting sqlmap in one go? Impossible. sqlmap is a dynamic project with steady commits. Making a parallel branch and keeping up the pace with HEAD Python 2 changes would be a Sisyphean task. Also, there are parts of sqlmap that maybe couple of non-sqlmap developers would understand. If a converter (person doing the conversion) would get stuck inside, he'll have a hell of a time.

My conclusion. Let's wait for the 2020. If Python 2 will die I promise that I'll do the whole conversion thing - as I believe that nobody else would be able to do it.

[Flimm]

OK. Fair enough. I trust your experience in this project.