掲示板 Forums - [HELP] Japanese font caching

The majority of renshuu's Japanese fonts come from a Japanese font maker's API called typesquare.com (English page available). They do have a free usage tier that doesn't require payment if you want to try setting it up on your own.

Except for the highest tier (which offers access to their WebAPI), everything is piped through their Javascript API.

Reference here: https://typesquare.com/service...

The API is designed for static page websites, with the idea that you have a page of text, you call the Javascript API, and it'll scan the page for characters, then download the chunk of the font needed for that page.

renshuu, on the other hand, frontloads this by having a temporary div with all of the Japanese characters in it, which results in about 4-5 calls to the api. You can easily see these in the web inspector in a web browser.

It then injects 4-5 style tags into the head of the document (this forum is not set up to handle code snippets, but you should find these if you search for elements in the document with the ts-font class)

So, here's what I'd like to do (I've already received permission to do this from typesquare).

Once it's fully loaded, I'd love for a way to cache this font data into localStorage, then use that instead when present (saving a good chunk of loading when the app/website is opened for the first time each day).

However, I have not been able to take the above font-face and get it into a form that can be saved, much less combine the 4-5 of them into a single file.

Some of these details are over my head, but it seems like you should be able to get by with a much smaller fraction of the font than most of it. The smallest font I could see on their website was over 6000 characters. The average renshuu user would be unlikely to use more than 50% of them, or so it seems to me.

For the purpose of testing assumptions on kanji frequency:

This appears to be a list of the kanji based on their frequency (based on Japanese Wikipedia articles) with their unicode values: https://github.com/scriptin/ka...

One approach to possibly consider is:

-take 1st 1000 or so most frequent kanji from that list, and put that in the div for front loading. According to the supporting site graphs, that will give you coverage of 90%+

-have a map/set with the unicodes for those 1000 kanji

-do a linear scan of the document text to check for membership of each character in the set (should only take linear time, so it should be fast)

-as soon as the linear scan encounter a kanji not in the set (should happen much less than 10% of the time), just call the Javascript API to scan the whole page and load the appropriate fonts

That should be fast for the majority of pages, but a bit slower for complex pages

You could even up the number of kanji in the div to lower the frequency of times it ends up finding a missed kanji

The original organization of the basic CJK block was by radical, with increasing stroke counts getting higher values. As an experiment, taking the low half would be an easy thing to try.

Would using mb_str_split and iterating over the resulting array not work for scanning ?

If you ideally want to have the entire font available, and are okay with around a 5mb download (as you note the 5mb limit of localstorage to be a limitation), have you considered just serving the full font file as a static file, rather than using an API to fetch specific characters?

NotoSans Japanese for example has a variable font available which includes italics and all font weights at 5.6mb gzipped. Or if you don't need italics and font weights, the non-variable font is 3.2mb gzipped.

It's unclear to me what frontend framework you're using, but if you're doing server-side rendering with templating, you could conditionally template in the css to load whichever font the user has selected in their settings.

If you convert it to woff2 format (which all major browsers support now that IE11 is dead), you can get it down to 4.1mb for the variable font, or 2.2mb for non-variable.