Device test scripts not Python 3 compatable

[johnm545]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: ESP-12E
• Core Version: latest git
• Development Env: Arduino IDE
• Operating System: Ubuntu on WSL

Settings in IDE

• Module: NodeMCU
• Flash Mode: [qio|dio|other]
• Flash Size: 4MB
• lwip Variant: |v2 Lower Memory
• Reset Method: [ck|nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz
• Upload Using: Serial
• Upload Speed: [115200|

Problem Description

The on-device test scripts are not working since #6378 due to some Python 3 incompatibilities

But I went down the road of trying to get them working with python 3 and found a few problems which were easily fixed:
Poster lib not python 3 compatible: use poster3 instead
Some of the usual python 2-3 dramas, bytes/strings, print statements, urllib imports

I can't, however, see an easy fix to the below, short of hacking the xmerge.py file (getting rid of the encode() on line 68 does the job). It looks like xunitmerge is a bit broken with python3.

I can post a WIP of what I've done so far if someone wants to take a crack at fixing it.

:/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device$ make test_report
make[1]: Entering directory '/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest'
make[1]: 'virtualenv' is up to date.
make[1]: Leaving directory '/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest'
Traceback (most recent call last):
  File "libraries/BSTest/virtualenv/bin/xunitmerge", line 26, in <module>
    merge_xunit(args.report, args.output)
  File "/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest/virtualenv/lib/python3.6/site-packages/xunitmerge/xmerge.py", line 130, in merge_xunit
    merged.write(output, encoding='utf-8', xml_declaration=True)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 776, in write
    short_empty_elements=short_empty_elements)
  File "/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest/virtualenv/lib/python3.6/site-packages/xunitmerge/xmerge.py", line 70, in _serialize_xml
    original_serialize(write, elem, *args, **kwargs)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 941, in _serialize_xml
    short_empty_elements=short_empty_elements)
  File "/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest/virtualenv/lib/python3.6/site-packages/xunitmerge/xmerge.py", line 70, in _serialize_xml
    original_serialize(write, elem, *args, **kwargs)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 941, in _serialize_xml
    short_empty_elements=short_empty_elements)
  File "/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest/virtualenv/lib/python3.6/site-packages/xunitmerge/xmerge.py", line 70, in _serialize_xml
    original_serialize(write, elem, *args, **kwargs)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 941, in _serialize_xml
    short_empty_elements=short_empty_elements)
  File "/usr/share/arduino-1.8.9/hardware/esp8266com/esp8266/tests/device/libraries/BSTest/virtualenv/lib/python3.6/site-packages/xunitmerge/xmerge.py", line 68, in _serialize_xml
    ).encode('utf-8'))
TypeError: write() argument must be str, not bytes
Makefile:107: recipe for target 'test_report.xml' failed
make: *** [test_report.xml] Error 1

[earlephilhower]

I broke it, so I'll try and fix it.

Separately, we're glad to see people running device tests outside the core team!

[earlephilhower]

Can you please take a look at #6670, @johnm545, and see if you had a better way of handling the changes than I came up with?