(PUP-8243) Strip leading BOM from ERB templates

[joshcooper]

Previously, if a template contained a BOM, then it was preserved by the template function, and would end up in the resulting file or powershell command. Now we pass the bom option when reading the file, which strips the BOM as it is read.

For example, given a template with a leading BOM:

$ cat test.ps1.erb
<%= 42 %>
$ od -t x1 test.ps1.erb
0000000 ef bb bf 3c 25 3d 20 34 32 20 25 3e

Puppet used to include the BOM in the rendered output:

C:\Users\josh\projects\puppet>bundle exec puppet apply -e "notify { 'thing': message => template('C:\Users\josh\projects\puppet\test.ps1.erb') }"
...
Notice: /Stage[main]/Main/Notify[thing]/message: defined 'message' as ' 42'

Note the unprintable character before 42 above:

With this change:

C:\Users\josh\projects\puppet>bundle exec puppet apply -e "notify { 'thing': message => template('C:\Users\josh\projects\puppet\test.ps1.erb') }"
...
Notice: /Stage[main]/Main/Notify[thing]/message: defined 'message' as '42'

[joshcooper]

jenkins please test this with servertests

[joshcooper]

jenkins please test this with servertests on redhat8-64a,windows2019-64a

[ekohl]

I've started to see opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/file_impl.rb:83: warning: BOM with non-UTF encoding US-ASCII is nonsense recently. Could this be causing it?

I see it here with Puppet 6 on CentOS 7.

[joshcooper]

Yes looks like it. Apparently, ruby will warn if you specify bom and the Encoding.default_external encoding isn't utf-. We should check the encoding before adding the bom prefix. I filed this as https://tickets.puppetlabs.com/browse/PUP-11196