Enable more linting

[jayvdb]

WIP-ish PR. Interested to see if there are any parts that are objectionable.

I'm happy to split it into smaller chunks for different types of lint issues, with nice commit messages.

Also, I can add more fine tuning by implementing my own flake8-putty, which allows rules to be disabled per-file and even per-line without modifying the source, so new rules can be enforced for new code while existing lint errors in old code are ignored.

---

This change is 

[gsnedders]

In general, <3. Thanks for doing this! I'm pretty sure I'd rather not have some of the changes because I disagree with the lint, but I think that's a small number of the changes; will have a close look through tomorrow.

FWIW, there was some vague intent to just get rid of flake8-run.sh given it's pretty gratuitous now. Also, when it comes to pylint, if we're going to have it in Travis/tox we should probably pin it to one exact version to avoid breakage when it gets updated (this really goes for flake8 and its dependencies too, but I've far more experience of pylint changing things like the disable strings from version-to-version).

[jayvdb]

No worries. Im just waking up, so I'll focus on cleaning up only the actual provable user-facing bugs herein until I hear more from you.

[jayvdb]

Testing this change turned into a mini project #294 .

[jayvdb]

I split off the CI and pylint changes to be #293.

[willkg]

@jayvdb We've landed a bunch of changes, but I'm a little fuzzy on whether all the important things in this PR have been covered now or not. Are there outstanding things we want in this PR or can we close it out?

[jayvdb]

Quite a few parts are now simpler, but still heaps to do. I rebased and revised this last week, but was waiting for the other PRs to be merged, and was doing Google Code-in launch prep.
I'll push my revised version asap.

[willkg]

I want to get 1.0.0 out, so I'm going to drop this from the milestone.

Thank you for working on it! We can definitely land this in the future.

[hugovk]

At this point in time, I'd suggest just running Black and let it decide on the style choices.

• https://github.com/psf/black

[gsnedders]

@hugovk Let's try and minimise the number of open PRs when we do that (i.e., wait till after we've merged many of the currently open PRs), given it'll cause conflicts with everything, but yes, applying black seems sane. (Keeping this open in lieu of opening a new issue because I'm about to go to bed 🙃)

[ambv]

I think this is too outdated with the current state of the code to recover. Too many conflicts. Re-open if you think I'm wrong.

List of conflicts from an attempt to merge

CONFLICT (modify/delete): .travis.yml deleted in master and modified in HEAD.  Version HEAD of .travis.yml left in tree.
Auto-merging debug-info.py
Auto-merging html5lib/_inputstream.py
CONFLICT (content): Merge conflict in html5lib/_inputstream.py
Auto-merging html5lib/_tokenizer.py
Auto-merging html5lib/_utils.py
CONFLICT (content): Merge conflict in html5lib/_utils.py
Auto-merging html5lib/filters/sanitizer.py
CONFLICT (content): Merge conflict in html5lib/filters/sanitizer.py
Auto-merging html5lib/html5parser.py
CONFLICT (content): Merge conflict in html5lib/html5parser.py
Auto-merging html5lib/serializer.py
Auto-merging html5lib/tests/conftest.py
CONFLICT (content): Merge conflict in html5lib/tests/conftest.py
Auto-merging html5lib/tests/support.py
Auto-merging html5lib/tests/test_encoding.py
Auto-merging html5lib/tests/test_parser2.py
Auto-merging html5lib/tests/test_serializer.py
CONFLICT (content): Merge conflict in html5lib/tests/test_serializer.py
Auto-merging html5lib/tests/test_stream.py
Auto-merging html5lib/tests/test_treewalkers.py
CONFLICT (content): Merge conflict in html5lib/tests/test_treewalkers.py
Auto-merging html5lib/tests/tokenizer.py
Auto-merging html5lib/treeadapters/__init__.py
Auto-merging html5lib/treeadapters/genshi.py
Auto-merging html5lib/treeadapters/sax.py
CONFLICT (content): Merge conflict in html5lib/treeadapters/sax.py
Auto-merging html5lib/treebuilders/__init__.py
CONFLICT (content): Merge conflict in html5lib/treebuilders/__init__.py
Auto-merging html5lib/treebuilders/base.py
CONFLICT (content): Merge conflict in html5lib/treebuilders/base.py
Auto-merging html5lib/treebuilders/etree_lxml.py
Auto-merging parse.py
CONFLICT (content): Merge conflict in parse.py
Auto-merging requirements-test.txt
CONFLICT (content): Merge conflict in requirements-test.txt
Auto-merging setup.py
Auto-merging tox.ini
CONFLICT (content): Merge conflict in tox.ini
Auto-merging utils/entities.py
CONFLICT (modify/delete): utils/spider.py deleted in master and modified in HEAD.  Version HEAD of utils/spider.py left in tree.
Automatic merge failed; fix conflicts and then commit the result.

---

In general, while I'm a big supporter of linting, I feel like flake8 and pylint cover a lot of the same ground and using both in a project is too much.

I agree with @hugovk that after we go over the other open PRs in the repo we should adopt auto-formatting, which would further cut down on the needed lint rules.

Then I'd figure out what the lowest supported Python version should be for html5lib going forward.

Finally, after all that we can return to refine the adopted linting.