[db-wg] Proposal to allow non-ASCII characters in "org-name:", "person:" and "role:" attributes

On Fri, Nov 24, 2023 at 10:03:15AM +0100, Edward Shryane via db-wg wrote:

Dear Ed, DB-WG Members,

> Currently the RIPE database only allows a subset of ASCII characters in the "org-name:", "person:" and "role:" attributes, for a few reasons including:
> 
> * These attributes are also a look-up key and the Whois protocol does not allow specifying character sets in queries.
> * RPSL names are ASCII according to RFC2622
> * Using a normalised name makes the object easier to query
> * Reading a normalised name is easier to interpret

I beg to differ with the last one.

> However there are some drawbacks to forcing names to only use a subset of ASCII characters:
> 
> * Organisations, roles and persons cannot use their actual name if it includes characters outside this subset.
> * Normalisation is not standard, but is an interpretation done by each maintainer, e.g. characters could be excluded or converted in different ways.

I too would like to see these two points addressed.

> Since we support the Latin-1 character set in the RIPE database, I propose we also allow non-ASCII Latin-1 characters in these attributes.
> 
> Querying for a name can be done either using the latin-1 characters (proposed) or a normalised, ASCII representation (currently). The normalised version will be generated by Whois and stored in a database index for querying. The primary key will also be generated from the normalised version.
> 
> Please let me know your feedback.

When in Rome, do as the Romans do ... or get back to UTF-8 topic ;-)

Time flies fast, and it occurs that it was exactly 13 years ago, in
Rome, during RIPE61 when I gave the presentation about non-ASCII
characters in DB. One can find the archives here:
https://ripe61.ripe.net/programme/meeting-plan/database-agenda/ Five
years later I formally proposed to allow UTF-8 in all free text
attributes of all DB objects except in primary keys
(https://www.ripe.net/ripe/mail/archives/db-wg/2015-April/004516.html).
This topic has been also discussed in very good RIPE Labs article by Ed
(https://labs.ripe.net/author/ed_shryane/impact-analysis-for-utf-8-in-the-ripe-database/).
Yet, we still discuss Latin-1 instead of UTF-8. And I'm puzzled why. :-)

I would like to see the topic addressed in more general way, solving the
issue(s) for the entire service region instead of adding characters used
by few Western countries only. Considering this, I would like to ask you
to switch to UTF-8 instead of using Latin-1.

Thanks for looking into this.

Best,
Piotr

-- 
Piotr Strzyżewski