A Spectre Is Haunting Unicode

(dampfkraft.com)

28 points | by sensanaty 3 hours ago

3 comments

  • hnfong 14 minutes ago
    Well, vast swaths of the Kangxi dictionary (which serves as "sources" for probably most of the CJK characters) are such "ghost" characters as described in the article...

    The peculiar properties of CJK characters and the philosophy (apparently the Japanese did not like Unicode's tendencies towards Aristotelian essentialism) under which they were implemented in Unicode probably singlehandedly forced unicode to expand beyond the BMP....

    • CodesInChaos 3 minutes ago
      > apparently the Japanese did not like Unicode's tendencies towards Aristotelian essentialism

      Do you mean Han Unification? I.e. that conceptually equivalent characters which are written differently in Japan and China received only a single unicode code-point, and are rendered the Chinese way by default on most computers?

  • joshdavham 46 minutes ago
    The author Paul McCann (polm) is one of my favourite programmers out there!

    He’s done awesome work in the Japanese NLP space over the last decade which has really helped me in my language learning projects.

    He maintains a mecab (Japanese tokenizer) wrapper for Python [0], has a book on Japanese NLP written for English speakers [1] and also worked on Spacy at one point [2].

    [0] https://github.com/polm/fugashi [1] https://www.japanesenlp.com/ [2] https://spacy.io/

  • philipov 6 minutes ago
    "- the spectre of communism. All the powers of old encoding have entered into a holy alliance to exorcise this spectre..."