Localization Work (v.0.1.4)

May 18
Localization Work (v.0.1.4)

Changes

  • Support for Internationalized domain names, punycode xn--* domains (see notes)
  • Exported CSV's now support Unicode and have BOM added for correct display of characters in Excel
  • Translate function added to all pages using Google translate widget
  • Translate tool added to domain lists (see notes)
  • New shortcut URL, TLD.CX (see notes)
  • Yet more updates to the interface, default columns at different screen sizes and a change of font

Release Notes

Internationalized domain names (IDNs)

All domains that are punycode encoded will now display with the correct character set. To see this in action view the .COM Alexa 1 Million list and enter 'xn--' (the ASCII Compatible Encoding prefix (ACE)) in the search box to show all encoded domains:

Internationalized domain names

Domain names (labels) are stored in the DNS as ASCII. However Internationalized domain names cannot be stored as ASCII and so have to be encoded. The encoding is achieved with the Punycode algorithm which encodes the international character set label into a string of ASCII characters. This code is then prefixed with the ACE prefix 'xn--' marking it as being encoded and then stored in the Domain Name System (DNS). Browsers and other software and sites (such as this one) then interrogate the DNS to receive domain name labels in ASCII and decode any names (labels) that are prefixed with 'xn--' back to their intended character sets and displayed correctly.

Incidentally entering the 'xn--' prefix in the home page search box filters by Top Level domains encoded and containing international character sets. You can achieve a similar effect by using the IDN category. However this shows all IDN domains which is not necessarily as useful. If the TLD category chosen is 'Country TLDs' 'In Root Zone', at time of writing 303 results are returned e.g. 303 country-code top level domains. Entering 'xn--' into the search box filters this list to IDN country code domains only, of which there are 56. The top in this list in terms of volume of registrations is .рф (.xn--p1ai), the Cyrillic country code top-level domain for the Russian Federation. The next two are IDN's for Taiwan both containing the exact same number of domains (37,045) as their registrations are mirrored: .台灣 (.xn--kpry57d, Taiwan in traditional Chinese characters) and .台湾 (.xn--kprw13d, Taiwan in simplified Chinese characters).

I don't know about anyone else but I find IDN's and their uses fascinating (really should get out more).

Translate tool & Google translate

Aside from Punycode decoding I've also added a link to Google translate which appears for any domain that has international characters. This link utilises Google's language detection algorithm to translate second level domain labels into English, the default language of the site. I've also added the Google widget for localization of DomainNameStats.com (but not the blog yet) to any number of different languages. Whilst not perfect this seemed like a good halfway house before full site localization at a later date. By default the domain translate link translates to English. However if a different site language is chosen (using the widget) then the domain names are translated into that language instead (a neat little hack which I was quite proud of, for all of a minute).

But thinking about it, all domain names (not just those with an xn-- prefix) may need to be translated from any language to any other, depending on the audience. Will look at implementing this in a later release.

New shortcut URL, TLD.CX

I've had this domain for some time but have never got around to using it properly. .CX is the country code Top Level Domain (ccTLD) for Christmas Island which is located in the Indian Ocean just south of Java and Sumatra with a population of 1,843 according to a census in 2016. We don't have a great deal of domain data on .CX (hopefully this will change soon) other than some pricing data and Alexa footprint. I like the extension though, short with a trailing 'X' and the domain as a whole has a nice rhyming rhythm to it (the D.C part). The trailing 'X' (to me anyway) implies the concept of exchange, so this kind of fits with the mission of DomainNameStats.com as a centre of information and ideas about domain names.

I should point out however I'm no domaining expert. I have particular ideas about what makes a good domain name, experts may roll their eyes at some of these ideas (like the above). But that contention I think is what makes working in this area fun. I'll explore some of these ideas in more depth and why I got into domains and the DNS at a later date (whether you like it or not).

To use the new shortcut just replace DomainNameStats.com with TLD.CX in for any page. So TLD.CX/com (top page for .COM detail), TLD.CX/com/new-domains (new domains just registered today) etc. Interested to see if anyone uses it:)

Interface & Session Updates (v0.1.3)

Mar 08
Interface & Session Updates (v0.1.3)

Changes

  • Filter, search and column visibility is now persistent
  • New index column added, '#', indicating the position of that TLD for today (across all TLDs) based on the total number of domains. To show the column access 'SetColumns'.
  • The columns ZoneNew, ZoneBack and ZoneDrop renamed to New, Backs, Drops respectively. Each column links to the list of domains for that category on the individual TLD page e.g. https://domainnamestats.com/com/new-domains. The 'New' column is now shown by default, next to the 'Change' column. See the Release notes below for more info on New/Drops and Backs and how they relate to the 'Change' figure.
  • The default columns shown (for a new user) now includes percentage changes for the last Day/Month/Quarter/Year. If we've just started tracking that TLD or the percentage is close to zero, then nothing is shown.
  • Percentage changes over 1000 are now shown as '>999'. Usually such a high percentage change indicates a TLD that has just been released (or has Verisign as their backend provider, see below).
  • More tweaks to layout

Release Notes

Where we have access to the TLD zonefile we track daily domain changes and then categorise the changed domains into three tables:

  • New - domains new to the zone today (never seen before)
  • Drops - domains dropped from zone (either not renewed (about to delete) or no proper nameservers)
  • Backs - domains that have re-appeared in the zone (either re-registered after deletion, renewed or now properly configured) The total of (New + Backs) - Drops equals the net change to the zonefile size for that day (e.g. the Change column). Large drops can often mask healthy growth in new original domains. You can see this by ordering by the New column, descending (see below)

Ordered by New domains today

Here .XYZ gained 4,407 domains (domains our system has never seen before) but also on the same day lost a further 6,681 domains from its zonefile. But the net change for the day was -1,201. The size of the drop has masked a healthy rise in new domains for .XYZ.

Note: Where we have no access to a zonefile there is no breakdown for New/Backs/Drops. This screenshot was taken before the legacy TLDs were updated hence no .COM/.NET etc

TLDs with Verisign as their backend provider and with low numbers of domains (typically about 20) show consistently high percentage changes. This is because Verisign adds and drops a few test domains (of the form 'emt-t-826666453-1520075851295-2-ho' etc) daily for these TLDs. We'll add an exclusion for these percentage changes at a later date, should help clean up the view.

Zone Work (v0.1.1)

Feb 14

Changes

  • Order pricing ascending (TLD pages)
  • Daily zone file changes archived for every TLD we track:
    • New Domains List - domains new to the zone today (never seen before)
    • Drops List - domains dropped from zone (either not renewed (about to delete) or no proper nameservers)
    • Backs List - domains that have re-appeared in the zone (either re-registered after deletion, renewed or properly configured)
  • Drop down list added to TLD pages allowing last 5 days (plus today) of zone file changes

Release Notes

The addition of historical zone file changes (for the TLDs we have zonefile access to) can be used to help identify recent registration trends and patterns. In addition insights on past trends that have gone out of favour can be gained through analysis of drop lists.

Beginnings (v0.1.0)

(Soft Launch at NamesCon 2018)

Jan 27

Changes

  • TLD list with sortable and configurable columns (home page)
  • Filter dropdown (home page)
    • Page for each TLD with:
    • Latest Pricing from popular registrars
    • New Domains List - domains new to the zone today (never seen before)
    • Drops List - domains droppped from zone (either not renewed (about to delete) or no proper nameservers)
    • Backs List - domains that have reappeared in the zone (either re-registered after deletion, renewed or properly configured)
    • Sites List - latest list of sites for this TLD in Alexa
  • Info panel detailing data collection progress (home page)
  • Data collection framework
  • Site framework (responsive)
  • Simple contact button

Release Notes

A good start.