Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting... Please check out the Community Guidelines in the
Announcements and Administration Category, below.

Keyboard device driver for custom keyboard?

Hello,

We have a custom HID keyboard hardware with approx 20 keys extra than the normal keyboard keys. Also this keyboard will support multiple script modes (For example Devanagari, English).
I don't have good understanding of how exactly keyboard device driver and its device stack works (but i have good understanding and experience in writing other windows kernel drivers like minifilter, network filter driver, virtual audio driver).From some initiale google search i understood that,keyboard device reports scan codes, these scan codes are translated into virtual keys and then virtual keys are translated into characters.

After some study, I came to conclusion that I need to write new device driver for this custom keyboard because it has some extra keys whose scan codes will not be recognized by standard driver and hence these extra keys will not work if used with standard driver.

Please let me know if my conclusion above is correct or not.

Thank you,
Gaurav

Comments

  • I think i should have mentioned new keyboard HID client driver instead of new device driver for custom keyboard, as custom keyboard will be detected as HID device.
  • R0b0t1R0b0t1 Posts: 129
    On Mon, Jul 9, 2018 at 7:58 AM, xxxxx@gmail.com
    <xxxxx@lists.osr.com> wrote:
    > Hello,
    >
    > We have a custom HID keyboard hardware with approx 20 keys extra than the normal keyboard keys. Also this keyboard will support multiple script modes (For example Devanagari, English).
    > I don't have good understanding of how exactly keyboard device driver and its device stack works (but i have good understanding and experience in writing other windows kernel drivers like minifilter, network filter driver, virtual audio driver).From some initiale google search i understood that,keyboard device reports scan codes, these scan codes are translated into virtual keys and then virtual keys are translated into characters.
    >
    > After some study, I came to conclusion that I need to write new device driver for this custom keyboard because it has some extra keys whose scan codes will not be recognized by standard driver and hence these extra keys will not work if used with standard driver.
    >
    > Please let me know if my conclusion above is correct or not.
    >

    It doesn't seem like a device driver will solve any problems you wish
    to solve. You have not explained the problems you wish to solve very
    well.

    Most things I could imagine you wish to do would be best done by
    adding elements to your keyboard's HID descriptor to create settings
    which can be changed from an application running on the PC. This
    would, unfortunately, require users of your keyboard to install
    software.

    Other problems you can solve in the firmware of the keyboard, by
    having it reconfigure its layout, potentially while running and
    without reenumerating.

    Keep in mind that the USB HID specification intends a keyboard to look
    like a matrix of buttons. There is not really supposed to be a concept
    of language.

    On Mon, Jul 9, 2018 at 8:25 AM, xxxxx@gmail.com
    <xxxxx@lists.osr.com> wrote:
    > I think i should have mentioned new keyboard HID client driver instead of new device driver for custom keyboard, as custom keyboard will be detected as HID device.
    >

    What are you trying to do? If existing scan codes don't cover the
    input you wish to produce then adding new scan codes (how?) won't
    help, because no existing application (namely the Windows input stack)
    would know about them.

    Look at http://www.usb.org/developers/hidpage/Hut1_12v2.pdf, usage
    page 0x07. There are reserved keys you *could* use. It may be best to
    create a non-keyboard control for the keys. Read the HID
    specification, you can create collections that work like keyboards.

    Cheers,
    R0b0t1
  • Doron_HolanDoron_Holan Posts: 10,353
    Not quite correct. Are you expecting windows to consume and use these extra scan codes directly or are you going to consume them from a special app? You can author a keyboard language mapping dll that maps scan codes to keys without a custom driver if you want windows to consume these keys directly. If you are going to consume them in your special app, put these extra keys in a separate top level collection with a custom usage page and you can open the HID directly from your app.

    d

    Bent from my phone

    ________________________________
    From: 30131445600n behalf of
    Sent: Monday, July 9, 2018 5:59 AM
    To: Windows System Software Devs Interest List
    Subject: [ntdev] Keyboard device driver for custom keyboard?

    Hello,

    We have a custom HID keyboard hardware with approx 20 keys extra than the normal keyboard keys. Also this keyboard will support multiple script modes (For example Devanagari, English).
    I don't have good understanding of how exactly keyboard device driver and its device stack works (but i have good understanding and experience in writing other windows kernel drivers like minifilter, network filter driver, virtual audio driver).From some initiale google search i understood that,keyboard device reports scan codes, these scan codes are translated into virtual keys and then virtual keys are translated into characters.

    After some study, I came to conclusion that I need to write new device driver for this custom keyboard because it has some extra keys whose scan codes will not be recognized by standard driver and hence these extra keys will not work if used with standard driver.

    Please let me know if my conclusion above is correct or not.

    Thank you,
    Gaurav

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • Thank you Robot1 and Doron for the response.

    Sorry i was not able to address correctly the problem that i want to solve.
    Actually i want that custom keyboard along with extra keys to work properly
    on Windows ( Exactly similar to what Doron mentioned that i am expecting
    windows to consume input of extra keys along with default keys that my
    keyboard has). Also along with normal combination of keys, i am expecting
    that combination of letter keys along with those extra keys should also
    produce required result in applications.

    @Doron: so are you saying that i should write new keyboard layout dll which
    will contain mapping of virtual keys to required characters? If this is
    true, then i am not clear who will translate scan codes of new keys to
    virtual keys as standard driver will not understand new scan codes.? As i
    said, i am not much aware of how exactly keyboard stack works, so my
    questions might be silly. I would love to get better understanding of
    keyboard stack before solving the problems that i am trying.

    Along with above expectations i also want to achieve more like if keyboard
    is switched to Devanagari script mode then i would like to produce some
    specific output if user pressed two keys simultaneously. Likewise there
    will be couple of combination of keys which have some specific meaning to
    each combination. Please point me to the direction to solve this.

    Thank you in advance.



    On Mon, 9 Jul 2018 at 8:50 PM, xxxxx@microsoft.com <
    xxxxx@lists.osr.com> wrote:

    > Not quite correct. Are you expecting windows to consume and use these
    > extra scan codes directly or are you going to consume them from a special
    > app? You can author a keyboard language mapping dll that maps scan codes
    > to keys without a custom driver if you want windows to consume these keys
    > directly. If you are going to consume them in your special app, put these
    > extra keys in a separate top level collection with a custom usage page and
    > you can open the HID directly from your app.
    >
    > d
    >
    > Bent from my phone
    >
    > ------------------------------
    > *From:* 30131445600n behalf of
    > *Sent:* Monday, July 9, 2018 5:59 AM
    > *To:* Windows System Software Devs Interest List
    > *Subject:* [ntdev] Keyboard device driver for custom keyboard?
    >
    > Hello,
    >
    > We have a custom HID keyboard hardware with approx 20 keys extra than the
    > normal keyboard keys. Also this keyboard will support multiple script modes
    > (For example Devanagari, English).
    > I don't have good understanding of how exactly keyboard device driver and
    > its device stack works (but i have good understanding and experience in
    > writing other windows kernel drivers like minifilter, network filter
    > driver, virtual audio driver).From some initiale google search i understood
    > that,keyboard device reports scan codes, these scan codes are translated
    > into virtual keys and then virtual keys are translated into characters.
    >
    > After some study, I came to conclusion that I need to write new device
    > driver for this custom keyboard because it has some extra keys whose scan
    > codes will not be recognized by standard driver and hence these extra keys
    > will not work if used with standard driver.
    >
    > Please let me know if my conclusion above is correct or not.
    >
    > Thank you,
    > Gaurav
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/showlists.cfm?list=ntdev&amp;data=02|01|[email protected]|176b557ff30f4011a5cb08d5e59bc35c|72f988bf86f141af91ab2d7cd011db47|1|0|636667379519189951&amp;sdata=Buwy1ch3hprI6GtTuK00M9zk/ghZJCn95HPJ+9Ps8X0=&amp;reserved=0&gt;
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at <
    > https://na01.safelinks.protection.outlook.com/?url=http://www.osr.com/seminars&amp;data=02|01|[email protected]|176b557ff30f4011a5cb08d5e59bc35c|72f988bf86f141af91ab2d7cd011db47|1|0|636667379519189951&amp;sdata=bRSuNJLQ2IOo2tcWvyfstM0H1/884uX3gZ66otW5EF4=&amp;reserved=0&gt;
    >
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/page.cfm?name=ListServer&amp;data=02|01|[email protected]|176b557ff30f4011a5cb08d5e59bc35c|72f988bf86f141af91ab2d7cd011db47|1|0|636667379519189951&amp;sdata=p8hNJjNSbx9dfA4fO0SMsdyIJLM4m2AahYo+cEvvghY=&amp;reserved=0&gt;
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • @Doron: I was trying to understand more about how exactly keyboard scan codes are converted. From the MSDN link https://docs.microsoft.com/en-us/windows/desktop/inputdev/about-keyboard-input , it looks like that keyboard device driver receives the scan codes from keyboard and then it converts those scan codes into virtual key codes. These translated virtual key codes are then processed by keyboard layout to translate into messages and to post to appropriate window.

    Above documentation confuses me with the possible solution you suggested above of authoring keyboard layout for my custom keyboard device. What confuses to me is that, as documentation says that keyboard device driver will map scan codes to virtual key codes then in my custom keyboard device (which has some extra new keys, so it will have new scan codes which are not familiar to existing keyboard device driver), how existing device driver understands new scan codes sent by my keyboard device? Or is it like that keyboard device driver just passes received scan codes to keyboard layout which then translates to virtual key codes and messages?

    Please help me to understand this.

    Thank you in advance.
  • Doron_HolanDoron_Holan Posts: 10,353
    The keyboard driver does not map to virtual keys. All it reports is scan codes. You can see this with KEYBOARD_INPUT_DATA.

    d

    Bent from my phone

    ________________________________
    From: 30551575500n behalf of
    Sent: Tuesday, July 24, 2018 5:11 AM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] Keyboard device driver for custom keyboard?

    @Doron: I was trying to understand more about how exactly keyboard scan codes are converted. From the MSDN link https://na01.safelinks.protection.outlook.com/?url=https://docs.microsoft.com/en-us/windows/desktop/inputdev/about-keyboard-input&amp;data=02|01|[email protected]|2b2b485094964d1ed3d708d5f15e8f44|72f988bf86f141af91ab2d7cd011db47|1|0|636680310786058295&amp;sdata=p6G7O1+b6F7aN991/NwsfWpKNxuj+bnEQYMdXZmcsbM=&amp;reserved=0 , it looks like that keyboard device driver receives the scan codes from keyboard and then it converts those scan codes into virtual key codes. These translated virtual key codes are then processed by keyboard layout to translate into messages and to post to appropriate window.

    Above documentation confuses me with the possible solution you suggested above of authoring keyboard layout for my custom keyboard device. What confuses to me is that, as documentation says that keyboard device driver will map scan codes to virtual key codes then in my custom keyboard device (which has some extra new keys, so it will have new scan codes which are not familiar to existing keyboard device driver), how existing device driver understands new scan codes sent by my keyboard device? Or is it like that keyboard device driver just passes received scan codes to keyboard layout which then translates to virtual key codes and messages?

    Please help me to understand this.

    Thank you in advance.

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • R0b0t1R0b0t1 Posts: 129
    On Tue, Jul 24, 2018 at 10:32 AM, xxxxx@microsoft.com
    <xxxxx@lists.osr.com> wrote:
    > The keyboard driver does not map to virtual keys. All it reports is scan
    > codes. You can see this with KEYBOARD_INPUT_DATA.
    >

    To explain further there should be differences in PS2 and USB HID
    drivers as to scan codes. Everything that is not PS2 seems to use the
    HID usage codes whether or not the transport is USB.
  • Doron_HolanDoron_Holan Posts: 10,353
    YES, everything not ps2 goes through HID (at least in box driver stacks). Kbdhid maps from hid usages to scan codes.

    d

    Bent from my phone

    ________________________________
    From: 30551534300n behalf of
    Sent: Tuesday, July 24, 2018 8:47 AM
    To: Windows System Software Devs Interest List
    Subject: Re: [ntdev] Keyboard device driver for custom keyboard?

    On Tue, Jul 24, 2018 at 10:32 AM, xxxxx@microsoft.com
    wrote:
    > The keyboard driver does not map to virtual keys. All it reports is scan
    > codes. You can see this with KEYBOARD_INPUT_DATA.
    >

    To explain further there should be differences in PS2 and USB HID
    drivers as to scan codes. Everything that is not PS2 seems to use the
    HID usage codes whether or not the transport is USB.

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • Pavel_APavel_A Posts: 2,640
    Is it possible to have a HID device report Unicode characters directly, instead of "scan codes" (or whatever the usages for kbd keys emulate) ?

    Regards,
    -- pa
  • Doron_HolanDoron_Holan Posts: 10,353
    No, not through the inbox stack. The raw input thread reads scan codes from keyboards

    d



    Bent from my phone

    ________________________________
    From: 30621506400n behalf of
    Sent: Tuesday, July 24, 2018 4:44 PM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] Keyboard device driver for custom keyboard?

    Is it possible to have a HID device report Unicode characters directly, instead of "scan codes" (or whatever the usages for kbd keys emulate) ?

    Regards,
    -- pa


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • Pavel_APavel_A Posts: 2,640
    After quick googling, lack of direct Unicode character input in HID is a known issue...

    https://github.com/kiibohd/KiiConf/issues/30

    https://deskthority.net/workshop-f7/hid-io-sideband-input-device-protocol-spec-t12927.html

    Looks like can be resolved only at the level of a USB standard. Now when UTF-8 is a clear winner, is it a good time to push this through?

    /* I have a similar project of HID device which would better produce cooked characters rather than scan codes & relying on host-side conversion. But IMHO the deskthority folks over-complicate the issue. All what is needed is just pack one or more UTF-8 characters in a HID report and define a new usage. If their hardware is not capable for this, find better hardware, there's a plenty of choice these days. */

    Regards,
    -- pa
  • Hello,

    Thank you Doron and Robot1 for the response. So it looks like i have to concentrate on having my own Keyboard layout and not device driver which will map all scan codes to virtual key code.

    Thank you again.
  • Pavel_APavel_A Posts: 2,640
    > So it looks like i have to concentrate on having my own Keyboard layout

    With a keyboard layout, there's another hitch. How will the device convince the host to activate your layout? Ok, we can define a shortcut key to switch directly to a specific language and layout. This is in Advanced keyboard settings ->Language bar options. The device can send this shortcut to activate the layout. It worked well since WinXP up to early Win10. But starting from RS2 it broke. Something resets these shortcuts after reboot, sometimes even after logoff. I could not find where they are stored, maybe need to search harder. So the bottom line, some helper service or driver still is needed.


    -- pa
  • R0b0t1R0b0t1 Posts: 129
    On Fri, Jul 27, 2018 at 2:45 PM, xxxxx@fastmail.fm
    <xxxxx@lists.osr.com> wrote:
    >> So it looks like i have to concentrate on having my own Keyboard layout
    >
    > With a keyboard layout, there's another hitch. How will the device convince the host to activate your layout? Ok, we can define a shortcut key to switch directly to a specific language and layout. This is in Advanced keyboard settings ->Language bar options. The device can send this shortcut to activate the layout. It worked well since WinXP up to early Win10. But starting from RS2 it broke. Something resets these shortcuts after reboot, sometimes even after logoff. I could not find where they are stored, maybe need to search harder. So the bottom line, some helper service or driver still is needed.
    >

    It's for reasons like this I am going with a helper program and then a
    driver with my product. It turns out you can fake almost all input
    from userspace, though there are some few programs that check to see
    if input is injected and if so they ignore it (or report to their
    developer in case of some MMORPGs). Consequently I recommend
    prototyping with a helper program and then moving to a driver. Such a
    program needs to run in your main desktop.

    You can just send default en-US scancodes but have your keyboard's
    firmware generate them based on a keymap in the device. This actually
    gives you a lot of flexibility.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!