About this blog…

I am employed by Netnod as head of engineering, research and development and am among other things chair of the Security and Stability Advisory Committee at ICANN. You can find CV and photos of me at this page.

As I wear so many hats, I find it being necessary to somewhere express my personal view on things. This is the location where that happens. Postings on this blog, or at Facebook, Twitter etc, falls under this policy.

The views expressed on this post are mine and do not necessarily reflect the views of Netnod or any other of the organisations I have connections to.

IDNA

Version 0.0.5 of createtables.rb was released.

It resolves:

  • Ability to handle Unicode version numbers with 2-digit main number

A bug was found in version 0.0.3, so I have released version 0.0.4 of createtables.rb

It resolves:

  • When there is a comma in the name of a codepoint, the last column is within double quotes for -ianacsv
  • Earlier versions  did not handle EXTENSION B, EXTENSION C etc, but only EXTENSION A.

Version 0.0.3 of createtables.rb is just released. It adds an option to create CSV files in the same format IANA version of the derived property values has. It also includes a -fetch flag that downloads all files needed from the website of the Unicode Consortium. It creates one directory per version (6.0.0, 6.1.0 etc) and downloads the files into those directories.


Thanks to help from Yoshiro Yoneya at JPRS in Japan and his team, here is version 0.0.2 of createtables.rb, the software described below.

Changelog can be found here.

It has also been validated that it works with the early versions of Unicode 6.2 that can be found at https://www.unicode.org/Public/6.2.0/ucd/. At the time of writing of this some files where up to version 6.2.0d6.

To make it easier for people to access the tables created with this tool, here are the files created:

allcodepoints-6.2.0d6.txt 36MB
byscript-6.2.0d6.html 12MB
bygc-6.2.0d6.html 29MB
xmlrfc-6.2.0d6.xml 153kB
idnabis-tables-6.2.0d6.xml 419kB

Some people have asked me what software I use to do calculations according to the IDNA2008 standard. I have after some clean up of the code decided to show to the community what I use.

You can find the ruby program as createtables-0.0.1.rb

It is a ruby program and should not need any external ruby libraries. It does though require you have downloaded some unicode files, for example for beta version of Unicode 6.1 you download files from https://www.unicode.org/Public/6.1.0/ucd/. You can download several versions of the files, but as it is now, every time you do, you must reinitialize the cache by using argument -i. I should add detection of the version of the files…not done yet. If you look in the code, you see towards the end of the file how to directly show results of tests with the -e argument. The reason for a cache is simply to re-run the manual tests of specific code points faster.

I suggest you download the files to a subdirectory, for example called “6.1.0” and then you run the software as ./createtables-0.0.1.rb 6.1.0, i.e. with the name of the directory as an argument.

It takes a long time to run.

If you find problems, bugs or have other kind of improvements, please let me know.