Issue 217 in html5lib: invalid characters in tag and attribute names

[h***@public.gmane.org]

Status: New
Owner: ----

New issue 217 by szepe.vi...-***@public.gmane.org: invalid characters in tag and
attribute names
http://code.google.com/p/html5lib/issues/detail?id=217

original "HTML" code:

pedig meg is szólaltattuk. <FOTÓGALÉRIA, VIDEÓ</b>

after html5lib.HTMLParser:

pedig meg is szólaltattuk. <fotÓgalÉria, b="" videÓ<=""></fotÓgalÉria,>

then HTMLParser can't handle tag and attributes with accented characters
and commas (etc.) in them:

Exception in html2text: <class 'HTMLParser.HTMLParseError'>; malformed
start tag, at line 89, column 154


"What is the expected output? What do you see instead?"

<fotgalria b="" vide=""></fotgalria>

Thank you!

[h***@public.gmane.org]

Updates:
Status: Invalid

Comment #1 on issue 217 by ja...-***@public.gmane.org: invalid characters in
tag and attribute names
http://code.google.com/p/html5lib/issues/detail?id=217

This behaviour is correct per spec; tag names and so on are converted to
ascii lowercase.

[h***@public.gmane.org]

Comment #2 on issue 217 by szepe.vi...-***@public.gmane.org: invalid characters in tag
and attribute names
http://code.google.com/p/html5lib/issues/detail?id=217

As I see, html5lib returns "<fotÓgalÉria>" as a tagname.
So "Ó" and "É" are NOT dropped.

Please implement this.

Thank you!!

[h***@public.gmane.org]

Comment #3 on issue 217 by ja...-***@public.gmane.org: invalid characters in
tag and attribute names
http://code.google.com/p/html5lib/issues/detail?id=217

Ó and É are not characters in ascii, therefore they don't get lowercased.
Compare with what browsers do:
http://software.hixie.ch/utilities/js/live-dom-viewer/saved/1922

[h***@public.gmane.org]

Comment #4 on issue 217 by szepe.vi...-***@public.gmane.org: invalid characters in tag
and attribute names
http://code.google.com/p/html5lib/issues/detail?id=217

Thank you for the demonstration!
After all I use lxml to strip invalid characters.