bugfix issue #152 plus test case

[pkrolikowski]

Generated output:

hyper --debug GET https://google.com:443/ authority:google.ru
...
HPACK encoding [(':method', 'GET'), (':scheme', 'https'), (':authority', 'google.com'), (':path', '/'), ('authority', 'google.ru')]

hyper --debug GET https://google.com:443/ :authority:google.ru
...
HPACK encoding [(':method', 'GET'), (':scheme', 'https'), (':path', '/'), (':authority', 'google.ru')]

hyper --debug GET https://google.com:443/ authority:google.ru accept-charset:utf-8
...
HPACK encoding [(':method', 'GET'), (':scheme', 'https'), (':authority', 'google.com'), (':path', '/'), ('accept-charset', 'utf-8'), ('authority', 'google.ru')]

So input authority:google.ru is not changing default ':authority' header value which I think is OK.
Based on https://http2.github.io/http2-spec/compression.html ("A. Static Table Definition" section) ':authority' is predefined header whereas 'authority ' is just additional header.

But when I run something like this:
hyper --debug GET https://google.com:443/ :authority:google.ru accept-charset:utf-8
the following error occurs:

Url Info: {'secure': True, 'fragment': None, 'netloc': 'google.com:443', 'host': 'google.com', 'query': None, 'path': '/', 'scheme': 'https', 'port': 443}
Commandline Argument: Namespace(_url='https://google.com:443/', body=None, debug=True, h2=False, headers={'accept-charset': 'utf-8', ':authority': 'google.ru'}, items=[<hyper.cli.KeyValue object at 0x7f3dc105b910>, <hyper.cli.KeyValue object at 0x7f3dc105b950>], method='GET', url=<hyper.cli.UrlInfo object at 0x7f3dc105b2d0>)
Selected protocol: h2
Sending frame SettingsFrame on stream 0
Received frame SettingsFrame on stream 0
Sending frame SettingsFrame on stream 0
Received frame WindowUpdateFrame on stream 0
HPACK encoding [(':method', 'GET'), (':scheme', 'https'), (':path', '/'), ('accept-charset', 'utf-8'), (':authority', 'google.ru')]
Adding (':method', 'GET') to the header table
Encoding 2 with 7 bits
Adding (':scheme', 'https') to the header table
Encoding 7 with 7 bits
Adding (':path', '/') to the header table
Encoding 4 with 7 bits
Adding ('accept-charset', 'utf-8') to the header table
Encoding 15 with 6 bits
Encoding 4 with 7 bits
Adding (':authority', 'google.ru') to the header table
Encoding 1 with 6 bits
Encoding 7 with 7 bits
Encoded header block to ���O��2��A��皂�e�
Sending frame HeadersFrame on stream 1
Received frame SettingsFrame on stream 0
Received frame GoAwayFrame on stream 0
Close stream 1
Sending frame GoAwayFrame on stream 0
Traceback (most recent call last):
File "/usr/local/bin/hyper", line 9, in
load_entry_point('hyper==0.5.0', 'console_scripts', 'hyper')()
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/cli.py", line 249, in main
data = request(args)
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/cli.py", line 239, in request
response = conn.get_response()
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/common/connection.py", line 124, in get_response
return self._conn.get_response(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/connection.py", line 207, in get_response
return HTTP20Response(stream.getheaders(), stream)
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/stream.py", line 318, in getheaders
self._recv_cb()
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/connection.py", line 727, in _recv_cb
self._consume_single_frame()
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/connection.py", line 623, in _consume_single_frame
self._consume_frame_payload(frame, data)
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/connection.py", line 707, in _consume_frame_payload
self.receive_frame(frame)
File "/usr/local/lib/python2.7/dist-packages/hyper-0.5.0-py2.7.egg/hyper/http20/connection.py", line 452, in receive_frame
raise ConnectionError(error_string)
hyper.http20.exceptions.ConnectionError: Encountered error INTERNAL_ERROR 0x2: Implementation fault

HPACK looks good, headers were added any ideas...

[Lukasa]

Ah, the problem there is likely to be with hyper: the special headers all need to come first in the header block. That's a new bug. =)

[pkrolikowski]

:D I'll try to fix it

[pkrolikowski]

It seems that headers order is OK now. It's without test case (I don't know where is the best place to put it).

[Lukasa]

@pkrolikowski So there's nothing wrong with the fix there, but it's in a difficult place: the hpack sub-module actually lives in a different codebase, and I'm disinclined to diverge from it.

I suspect the best place for this is alongside HTTPHeaderMap.replace(), which currently uses a shortcut I originally proposed to use other helper methods that does not replace "in-place". We may want a method called replace_in_place that has the same signature, but ensures that the headers are replaced in the location of the first entity. We can then call that from H2Stream.add_header.

That's potentially a subtle change: would you like me to make it myself and then you can rebase on top of it?

[pkrolikowski]

@Lukasa that's a good idea, let's do it that way

[Lukasa]

Ok @pkrolikowski, I've merged a fix in #204. You should be able to rebase onto the development branch and that should fix the misbehaviour (I hope).

[pkrolikowski]

@Lukasa Now it looks OK. So what next, should I make a pull request (with the changes enables override default headers).

[Lukasa]

@pkrolikowski You can keep pushing to this branch and GitHub will keep track. In this case, because you rebased, you'll just need to force push (git push -f).