Linux keyman 11 beta log error when keyboard selected

keyman 11 beta for Wasta Linux 18.04 does nothing (except error to log file) when I select my GE keyboard.
([ 619.668546] ibus-engine-key[4477]: segfault at 14 ip 00007f36850cc6ca sp 00007ffcd9dd1650 error 4 in libkmnkbp0.so.0.0.0[7f36850bd000+1d000])

Using the old kmfl-ibus, the source file (GE.kmn) file worked fine, but now, after creating a keyman package (GE.kmp) containing this and the GE.kmx compiled keyboard (which we tested on Windows and it works fine), it doesn’t work in my Wasta Linux. Originally this was an old .kmx file compiled in 2013, but I just recompiled it in Keyman 11 Developer and got the same error. Which keyboard is keyman for linux using? I presume it is the .kmx file, and no longer the kmn file.

Package: keyman Version: 11.0.103-1~sil2~bionic

Another issue: I notice that the icon which I see in Windows is substituted for the generic one in Linux. (My package didn’t specify an image file, because scales terribly and is very blurry in the installation.)

I noticed that just doing an “apt install keyman” likely isn’t good enough, because it didn’t update several ibus or kmfl libraries. So, I’ve done an apt dist-upgrade (but that didn’t fix the problem either).

Hi @Justin_Luth

I’ve recently inherited development of Keyman for Linux, so I’ll try to look into this.
Can you zip up the entire folder of your GE keyboard and send it to me in a PM?

Does this recap what you’ve done?

  1. install Keyman for Linux
  2. In Keyman Developer 11 beta
    a. added “linux” as a target for your GE.kmn keyboard.
    b. added GE.kmx to the keyboard package and created GE.kmp
  3. In keyman-config, you then installed GE.kmp
  4. When you try to select GE in ibus-keyman, you get the error

Which keyboard is keyman for linux using? I presume it is the .kmx file, and no longer the kmn file.

Corrrect. Keyman for Linux uses a new keyboard processor which handles the .kmx file, so you no longer need to include the .kmn file in the keyboard package.

Hi Darcy,

Thanks for helping. One thing I immediately see is that I did not do anything with “targets”, so I’ll test that to see if it makes any difference. It sounds like what you are saying is that keyman 11 for Linux only accepts SPECIFICALLY DESIGNATED/RECENT VERSION kmx files. That probably needs to be stated fairly clearly somewhere (like in the packaging guide) and certainly in any linux news.

Things that have come as a surprise to me so far have been (from what I can remember)

  1. it is not backwards compatible. It replaces a working kmfl system and breaks it.

    a. It only installs .kmp packages
    b. It cannot use .kmn files, but only uses recompiled-specifically-for-linux .kmx files (if I understood you correctly).

  2. the install (and uninstall) of keyman over top of a working kmfl doesn’t REQUIRE updates to various related libraries. There probably need to be a few version dependencies in the debian packaging. (I don’t know whether they are absolutely required since my GE.kmp didn’t work anyway, but I DO know that reverting back to kmfl was troubling because I had to uninstall by hand things like “apt remove kmfl” in order to get my system working - and at one point I even had one keyman file isolated, so that I had to re-install keyman before I could remove that library). I’ve done almost no .deb packaging myself, but I might try tinkering with this just to see if I can make some practical suggestions.

The recap sounds about right. Here is a lengthened version of it.

  1. Installed keyman 11 a while back to test, lost all keyboards and noticed that keyman11 only accepts .kmp installs which we didn’t have, and then walked away and forgot about my broken keyman. A few days ago I rolled back to ibus-kmfl from Ubuntu/SIL/Wasta after lots of head scratching and “trying various things” since error messages didn’t help at all. Finally got my old .kmn files working again.

  2. Installed keyman developer 11 in wine: seemed to mostly work for packaging, but had some problems with the “source” pane crashing, so didn’t bother to do any real work that way. Installed developer in Windows 7. Reviewed the packager guide (since I’ve never made a package before) and put all the files together. (And laughed at one point made in the forums which complained that distributing .kmx files was a bad thing because the source code is missing, but then see that .kmn files are discouraged from being bundled in .kmp as well - so you still loose your keyboard definition… I included the .kmn file very deliberately into my package.) I did NOTHING about recompiling the keyboards - never saw any hint saying that would be a good thing to do.

  3. Tested all of the .kmp packages in Windows 7 - verified that they worked fine - just like I would have expected (with Keyman 10).

  4. Re-installed keyman over top of my working Wasta system (apt install keyman onboard) and confirmed that all my .kmn files were broken

  5. ran km-config installing GE.kmp (probably as root, based on subsequent observations). Didn’t see any keyboards, restarted my computer, still no icons and no keyboard listed in km-config for me (justinl).

    b. now I am a little hazy on exactly the steps I took - so I might be wrong about this. I DON’T think that I added GE.kmp at this point.

    c. quickly assumed that I had run km-config as root. Ran again as root - confirmed GE was listed (and saw that it was install in /root/.config/share/keyman or something like that.)

    d. ran km-config as myself (justinl) and was surprised to see GE listed this time (see 4b - maybe I DID install GE.kmp at that step).

    ???. lots of fiddling around trying to add other-GE to the ibus input method throughout this time period. I don’t remember when it showed up as an option anymore. Eventually, it did show up in my ibus list of keyboards to switch to.

  6. try to use GE keyboard, but it doesn’t switch - just stays on EN.

  7. ran dist-upgrade (just to see what needed to be installed - too much to download so only installed one or two ibus-kmfl related items that I saw.) Rebooted, but still didn’t work, so ran a full dist-upgrade and rebooted. Still didn’t work.

  8. ran dmesg and saw the log errors.

  9. compile a new .kmx file (just load the kmn file in developer and hit compile - nothing more). By hand, replace the .kmx file in ~/.local/share/keyman/GE. Reboot and still get the error message.

  10. ran out of ideas.

Thanks for giving me more ideas,

Justin

Hi Justin,

A few comments that might help clarify some things. Hoping Darcy can work with you to get to the bottom of the errors!

  1. We intend to continue supporting KMFL keyboards in Keyman for Linux. So .kmn keyboards should continue to work (and if they are not, then that’s a bug or config issue).
  2. Keyman Developer has not been tested in WINE by us and I would have been surprised if it worked. However, the command line compiler (kmcomp) does work. That’s not enough for the long term of course.
  3. We encourage the separation of source and binary distribution. Source should be in a version-controlled repository, ideally at https://github.com/keymanapp/keyboards/ (see https://help.keyman.com/developer/keyboards/ for details on how to get a keyboard included there). We don’t support the distribution of plain .kmx files any longer because they cannot include documentation, or on screen keyboards. It’s not about source, so I’m not sure where you saw that in the forums?
  4. The segfault is clearly a bug and we need to get to the bottom of it. Given you had the same problem installing a brand new empty keyboard, I would suggest that there is something environmental causing it. There’s probably not enough information in the logfile to track it down, but hopefully Darcy can work with you to isolate the cause.
  5. From my understanding, apt install keyman will work in the future, but we are still in beta right now and so the package is available as a PPA for now.
  6. Keyman for Linux should accept any .kmx keyboard that works on Keyman Desktop in Windows (we encourage using linux as a target for new keyboards for completeness, but it’s not required).
  7. I am not sure if we have tested the KMFL->Keyman for Linux upgrade scenario completely. Your feedback is helpful.

I’ve created a package for the GE.kmn file Justin sent me in a PM and sent the ge.kmp file for him to try.
Justin, did that work for you?

Were you able to associate a language in your original keyboard package?
In the Package Editor --> Keyboards tab, click “Add”

In the ge.kmp file I sent you, I used “en” for English.

If language is blank, the package still compiles, but I wonder if that causes keyman-config to throw exceptions when trying to install.

From the log you sent me, it sure looks like keyman-config can’t handle a package where the language list is empty.

File "/usr/lib/python3/dist-packages/keyman_config/install_kmp.py", line 254, in install_keyboards_to_ibus

      logging.debug(kb["languages"][0])

  IndexError: list index out of range

I’ll try to get that fixed in time for stable release.

Following up in exchanging PM’s with @Justin_Luth

So I tried that with my GE keyboard. First I converted the BMP to .ico and created a new package. That didn’t help.
Then I just renamed the files in ~/.local/share/keyman/GE to be all lowercase, and then I could switch.
The custom icon didn’t show up correctly - just using the generic one. I just renamed GE.ico.jpg to ge.ico.jpg, restarted ibus-daemon, and the icon showed up properly.

After lowercasing all the keyboard file names, it appears the packaging works, the icon shows, and the keyboard works.

I’ll still try to improve the user-experience of installing .kmp packages in km-config.

Hi Mark (responding to your 7 point message),
1.) Glad to hear that the intention is to continue to support /usr/share/kmfl/.kmn. That will help with moving forward. Somehow, after I read that comment from you, then my old .kmn files started working. Why I struggled with it so much earlier I don’t know, but I failed to reproduce the problem just now. So thanks.
3.) The quote from “Keyman 11 Beta Feedback” is “plain .kmx files were a continual headache for us to support as in many cases the original developer was no longer available and we had to reverse engineer them in order to help users.” The same will be true for .kmp if they aren’t passed through github.
4.) Segfault appears to be due to a non-lowercase name for the .kmx file.
5,7.) Yeah, no problem with it being in a PPA. I think my point is that if you JUST follow the instructions and do “apt install keyman onscreen”, that in insufficient. Doing that causes at least kmfl to break (and likely more). I also had to do an apt dist-upgrade to update these four related packages: gir1.2-ibus-1.0 ibus ibus-kmfl libkmfl0.
6.) Great, and I can confirm that you are correct. After lower-casing the filename, my 2013 compiled keyboard works. One remaining problem is that Linux ignores the bmp inside the compiled .kmx file. It seems to require an .ico.png file which the package didn’t create.

After lowercasing all the keyboard file names, it appears the packaging works, the icon shows, and the keyboard works.

You can avoid user error in this regard by automatically lower-casing the names in Developer when the user imports files. I tested that in Windows. Even though the files on the disk are uppercase (GE.kmx), if the package code refers to it in lowercase (ge.kmx), then in the .kmp they will be lowercased.

I see. That’s true – but the backstory here was that the .kmp files were much more likely to include documentation so we didn’t need to reverse engineer in order to know how to use the keyboard. But yes, I can see how that’s a little obtuse … source on GitHub is our “gold standard” for sure.

Righto. I am guessing there is a mismatch in the package metadata and the filenames. One of our recommendations that we enforce in the keyboards repository is that all filenames must be in lower case (matching the pattern /^[a-z][a-z0-9_]+$/ to avoid these kinds of problems. We test this somewhat more thoroughly in very latest builds of Keyman Developer. Given Windows is case insensitive, we do have issues with legacy packages that have mismatching metadata here.

I would like to see Keyman for Linux being more robust in handling this of course; either treating package contents as case-insensitive, maybe transforming to lowercase on install? But we have to be careful because HTML documentation references images that may have case assumptions too :frowning: