Hi. I maintain the Shaw Imperial and Shaw QWERTY keyboards. Over the last month or so I’ve found the keyboards have become almost unusable on iOS due to the keyboard regularly and randomly freezing. I’ve attached a screenshot of it actually frozen. If I change keyboards and delete a few characters, then I can go back and type in Keyman again but if I try the same combination of characters (type the same word) the problem recurs. I’ve been unable to discern any pattern in what will trigger the freeze. It often happens with tough-and-hold keys but is not restricted to that. It should be easy to reproduce since it happens a lot. I tried turning off predictive text using the Shavian lexical mode but this made no difference, and am using the latest version of Keyman through TestFlight (17.0.100-alpha) but it was happening with the release version as well.
As of today I’m using iOS 16.4.1(a). I’ve had the same problem on my iPhone 11 and my iPad Pro 2nd generation. EDIT: I have allow full access on.
Happy to provide any further details that may help.
Can you provide some additional info to help us reproduce the issue?
Does the freeze happen soon, or after typing long paragraphs?
Do you see this typing in several different apps?
I just tested it. It happened across all the apps I tried (Mail, Keyman itself, Discord) and it seems to be certain combinations of characters. For example if I try to type 𐑣𐑬 𐑤𐑪𐑙 [how long] as the only text at the start of a field it seemed always to freeze on the 𐑙. But I could type the same combination here in a longer block of text, strangely. It’s also not just this letter or combination but I haven’t been able to discern a pattern.
I’m wondering if it is connected with Shavian being in a higher Unicode plane made up of surrogate pairs. That is often an issue in other apps.
Anyway, I realise Shavian shouldn’t be a priority compared with under-served languages but thought it worth mentioning if it points to other issues that might come up.
EDIT: Further testing reveals that it doesn’t matter what Shavian letter follows 𐑣𐑬 𐑤𐑪, the next letter will freeze.
Thanks for the extra detail – that really helps us in trying to figure out the cause of the issue. I have tried to reproduce the issue here but so far without success – as in, “it just works”, which is not terribly helpful!
The symptoms you are experiencing are consistent with a Javascript error occurring within the keyboard or KeymanWeb itself – can you reproduce this issue on https://keymanweb.com/#en-shaw,Keyboard_english_shavian_igc on your phone – I have tried to repro here too without any luck.
The problem doesn’t occur if I use the online keyboard on either my phone or iPad. I haven’t heard a lot of reports of other people experiencing this so it’s a bit mysterious. It happens with the installed Keyman keyboard on both my iPad and iPhone though, with the same character combinations causing the keyboard to freeze.
Okay, so I think I’ve found the source of the problem, even if I don’t fully understand why.
Changing the following in the touch keyboard layout json file for Shavian IGC from:
"id": "U_10450",
"text": "𐑐"
to
"id": "U_10450",
"text": ""
seems to fix the problem. At some point I’m guessing the Shavian letter is being interpreted as a surrogate pair rather than a single Unicode code point. And for some reason in iOS 16 this is causing a problem. (The problem emerged around the same time as the iOS16.4.1 update, so I’m guessing here).
In my second keyboard Shavian QWERTY (yet to be fixed), the problem is even more explicit — for some reason the ‘text’ field is represented with surrogate pairs. This wasn’t a deliberate choice - I was using Keyman Developer (v10, I think) rather than looking at the raw code:
"id": "U_10450",
"text": "\uD801\uDC50"
It appears that either way, this breaks my keyboard in iOS16. Surrogate pairs usually seem to be the problem.
In my own testing with Shavian IGC, the problem was resolved by simply deleting the information in the ‘text’ field where the ‘id’ field already gives the Unicode point. I’m submitted a pull request for Shavian IGC to confirm this works in a release version. I’ll update Shavian QWERTY once confirmed.
That’s good investigation. I have checked our repository of keyboards for others which may have surrogate pairs misencoded in their .keyman-touch-layout file and found the following:
basic_kbdadlam
basic_kbdosa
tirhuta
english_shavian_qwerty
If you wanted to try those other three on your device, that would be an interesting test.
However, Shavian IGC in the keyboards repository does not appear to contain surrogate pairs – as far as I can tell, the binary representation of that particular character 𐑐 (which appears twice, once for each form factor in the file) is F0 90 91 90, which is the correct UTF-8 sequence for that character, and not a mis-encoded surrogate pair in UTF-8.
It definitely happens with english_shavian_qwerty (for the same letter combinations). I couldn’t get it to happen with the Adlam or Osage keyboards you referred to.
Thanks for checking on that. That suggests that the problem lies elsewhere. We’ll still cleanup the keyboards that have surrogate pairs in their keyman-touch-layout files because that’s not great, but it probably won’t make a difference.
I am slowly getting my machine setup to investigate this – takes a long time…
I’ve done some further tests. Simply changing the “text” field in english_shavian_qwerty to remove the surrogate pairs didn’t fix the problem. However, when I changed the font (as I did with english_shavian_igc as well), the problem seemed to be fixed. I tried just changing the font and leaving the the keyman-touch-layout file otherwise the same (with surrogate pairs) and it worked.
So, maybe it was a problem with the original font, but I’m still unsure why it only appeared in April 2023 after using it for some years.
I think it’s most likely something specific to the font rather than Shavian Unicode characters, possibly something non-compliant in the font itself?
Describe how it impacts you, let them worry about the technical details, but I’d be reporting that using the font HenryHigginsSans-Regular.ttf locks up the keyboard webview.
