Windows, HID, Multitouch Digitizers, Touch Screen and WPF

NTDEV
1

Hi,

In the last couple of months I have been dissecting all the aspects for HID for multitouch devices and its implementation.

The reason for this its to understand the different behavior of my device compared to other manufacturers devices, specially regarding to usage in WPF applications.

It happens that for some reason, WPF doesn’t handle touch with my device but it handles the touch (and other kind of gestures) with other devices, specifically the 3M 22" LCD monitor with integrated touch.

In order to understand if something was wrong with my HID descriptors specification I have analyse 3M’s descriptor and compared with mine and there is no change that justifies a different behavior.

Of course that the two descriptors are not equal because the 3M is an hardware HID solution and I use a miniport / virtual HID driver, however I don’t think that’s the case.

A few months ago I have asked something about understanding the relation between hid descriptors and the structs in memory and I have only realized that Tim Roberts had replied to me (thanks and sorry Tim, I have just read that today).

The most awkward aspect of all this, is that the official Microsoft examples of touch handling with managed code (MTScratchpadWMTouch) work perfectly.

Howerver, the ones for manipulation handling with WPF doesn’t. This makes me believe that there is somehow a get capabilities called by WPF that my device is not correctly answering and thus the touch handling is disabled.

Has someone crossed with a identical problem? Any single idea or suggestion?

Thanks,

With my best regards,

Nuno Santos

2

>In order to understand if something was wrong with my HID descriptors
specification I have analyse 3M’s descriptor and compared with mine and
there is no change that justifies a different behavior.

Why dont you post the diff

Of course that the two descriptors are not equal because the 3M is an
hardware HID solution and I use a miniport / virtual HID driver

I dont think this is necessarily so. The descriptor describes functionality
(not origin)

On Tue, Jul 3, 2012 at 9:29 PM, wrote:

> Hi,
>
> In the last couple of months I have been dissecting all the aspects for
> HID for multitouch devices and its implementation.
>
> The reason for this its to understand the different behavior of my device
> compared to other manufacturers devices, specially regarding to usage in
> WPF applications.
>
> It happens that for some reason, WPF doesn’t handle touch with my device
> but it handles the touch (and other kind of gestures) with other devices,
> specifically the 3M 22" LCD monitor with integrated touch.
>
> In order to understand if something was wrong with my HID descriptors
> specification I have analyse 3M’s descriptor and compared with mine and
> there is no change that justifies a different behavior.
>
> Of course that the two descriptors are not equal because the 3M is an
> hardware HID solution and I use a miniport / virtual HID driver, however I
> don’t think that’s the case.
>
> A few months ago I have asked something about understanding the relation
> between hid descriptors and the structs in memory and I have only realized
> that Tim Roberts had replied to me (thanks and sorry Tim, I have just read
> that today).
>
> The most awkward aspect of all this, is that the official Microsoft
> examples of touch handling with managed code (MTScratchpadWMTouch) work
> perfectly.
>
> Howerver, the ones for manipulation handling with WPF doesn’t. This makes
> me believe that there is somehow a get capabilities called by WPF that my
> device is not correctly answering and thus the touch handling is disabled.
>
> Has someone crossed with a identical problem? Any single idea or
> suggestion?
>
> Thanks,
>
> With my best regards,
>
> Nuno Santos
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

3

Hi Dan,

First of all, thanks for your response. It’s really hard to get someone to exchange ideas regarding this topic.

I can’t publish a diff because my descriptor cannot be sniffed by USBlyzer, as it seems it doesnt capture miniport drivers. I’ll post the two below, but with different read formats.

Semantical they are the same, or at least I can’t find any difference.

3M (without the pointer descriptor, 1 touch ony, as it was not necessary to past the whole 10 parallel contact points, and without vendor defined features):

Usage Page (Digitizer) 05 0D
Usage (Undefined) 09 0E
Collection (Application) A1 01
Report ID (17) 85 11
Usage (Undefined) 09 23
Collection (Logical) A1 02
Usage 09 52
Usage 09 53
Logical Minimum (0) 15 00
Logical Maximum (10) 25 0A
Report Size (8) 75 08
Report Count (2) 95 02
Feature (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) B1 02
End Collection C0
End Collection C0
Usage (Touch Screen) 09 04
Collection (Application) A1 01
Report ID (16) 85 10
Usage (Finger) 09 22
Collection (Logical) A1 02
Usage (Tip Switch) 09 42
Logical Minimum (0) 15 00
Logical Maximum (1) 25 01
Report Size (1) 75 01
Report Count (1) 95 01
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage (In Range) 09 32
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage 09 47
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Report Count (5) 95 05
Input (Cnst,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 03
Report Size (8) 75 08
Usage 09 51
Report Count (1) 95 01
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Logical Maximum (32767) 26 FF 7F
Report Size (16) 75 10
Unit Exponent (-2) 55 0E
Unit (Eng Lin: in^3) 65 33
Usage (X) 09 30
Physical Minimum (0) 35 00
Physical Maximum (1594) 46 3A 06
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage (Y) 09 31
Physical Maximum (1000) 46 E8 03
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
End Collection C0
Usage Page (Digitizer) 05 0D
Usage 09 54
Report Count (1) 95 01
Report Size (8) 75 08
Logical Minimum (0) 15 00
Logical Maximum (10) 25 0A
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Report ID (18) 85 12
Usage 09 55
Report Count (1) 95 01
Report Size (8) 75 08
Logical Minimum (0) 15 00
Logical Maximum (10) 25 0A
Feature (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) B1 02
End Collection C0

My own descriptor in the same conditions as the above (without point, 1 touch only)

0x05, 0x0d, // USAGE_PAGE (Digitizers)
0x09, 0x0E, // USAGE (Configuration)
0xa1, 0x01, // COLLECTION (Application)
0x85, REPORTID_FEATURE, // REPORT_ID (Feature)
0x09, 0x23, // USAGE (Device Settings)
0xa1, 0x02, // COLLECTION (logical)
0x09, 0x52, // USAGE (Device Mode)
0x09, 0x53, // USAGE (Device Index)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x0a, // LOGICAL_MAXIMUM (10)
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x02, // REPORT_COUNT (2)
0xb1, 0x02, // FEATURE (Data,Var,Abs)
0xc0, // END_COLLECTION
0xc0, // END_COLLECTION

0x09, 0x04, // USAGE (Touch Screen)
0xa1, 0x01, // COLLECTION (Application)
0x85, REPORTID_TOUCH, // REPORT_ID (Touch)
0x09, 0x22, // USAGE (Finger)

0x05, 0x0d, /* USAGE_PAGE (Digitizers) */ \
0xa1, 0x02, /* COLLECTION (Logical) */ \
0x09, 0x42, /* USAGE (Tip Switch) */ \
0x15, 0x00, /* LOGICAL_MINIMUM (0) */ \
0x25, 0x01, /* LOGICAL_MAXIMUM (1) */ \
0x75, 0x01, /* REPORT_SIZE (1) */ \
0x95, 0x01, /* REPORT_COUNT (1) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0x09, 0x32, /* USAGE (In Range) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0x09, 0x47, /* USAGE (Confidence) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0x95, 0x05, /* REPORT_COUNT (5) */ \
0x81, 0x03, /* INPUT (Cnst,Ary,Abs) */ \
0x09, 0x51, /* USAGE (Contact Identifier) */ \
0x75, 0x08, /* REPORT_SIZE (8) */ \
0x95, 0x01, /* REPORT_COUNT (1) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0xa1, 0x00, /* COLLECTION (Physical) */ \
0x05, 0x01, /* USAGE_PAGE (Generic Desktop) */ \
0x15, 0x00, /* LOGICAL_MINIMUM (0) */ \
0x26, 0x9c, 0x18, /* LOGICAL_MAXIMUM (6300) */ \
0x75, 0x10, /* REPORT_SIZE (16) */ \
0x55, 0x0e, /* UNIT EXPONENT (-2) */ \
0x65, 0x33, /* UNIT (Eng. Lin: in^3) */ \
0x35, 0x00, /* PHYSICAL MINIMUM (0) */ \
0x46, 0xe8, 0x03, /* PHYSICAL MAXIMUM (1594) */ \
0x09, 0x30, /* USAGE (X) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0x09, 0x31, /* USAGE (Y) */ \
0x35, 0x00, /* PHYSICAL MINIMUM (0) */ \
0x46, 0x3a, 0x06, /* PHYSICAL MAXIMUM (1000) */ \
0x81, 0x02, /* INPUT (Data,Var,Abs) */ \
0x45, 0x00, /* PHYSICAL MAXIMUM (reset) */ \
0xc0, /* END_COLLECTION */ \
0xc0, /* END_COLLECTION */ \

0x05, 0x0d, // USAGE_PAGE (Digitizers)
0x09, 0x54, // USAGE (Actual count)
0x25, 0x00, // LOGICAL_MAXIMUM (0)
0x25, 0x0c, // LOGICAL_MAXIMUM (12)
0x95, 0x01, // REPORT_COUNT (1)
0x75, 0x08, // REPORT_SIZE (8)
0x81, 0x02, // INPUT (Data,Var,Abs)

0x85, REPORTID_MAX_COUNT, // REPORT_ID (Feature)
0x09, 0x55, // USAGE(Maximum Count)
0x25, 0x00, // LOGICAL_MAXIMUM (0)
0x25, 0x0c, // LOGICAL_MAXIMUM (12)
0x95, 0x01, // REPORT_COUNT (1)
0x75, 0x08, // REPORT_SIZE (8)
0xb1, 0x02, // FEATURE (Data,Var,Abs)
0xc0, // END_COLLECTION

4

Hmm… I do not know why your device should work with MTScratchpad but not
WPF

I have a virtual MT HID at http://code.google.com/p/vmulti/ I havent tried
it specifically with any WPF apps but if it works you could see what (or
if) there are any differences there.

Otherwise perhaps look at the timing of the change to in-range and
tip-switch bits in your HID compared to others.

Good luck :slight_smile:

On Wed, Jul 4, 2012 at 8:58 PM, wrote:

> Hi Dan,
>
> First of all, thanks for your response. It’s really hard to get someone to
> exchange ideas regarding this topic.
>
> I can’t publish a diff because my descriptor cannot be sniffed by
> USBlyzer, as it seems it doesnt capture miniport drivers. I’ll post the two
> below, but with different read formats.
>
> Semantical they are the same, or at least I can’t find any difference.
>
> 3M (without the pointer descriptor, 1 touch ony, as it was not necessary
> to past the whole 10 parallel contact points, and without vendor defined
> features):
>
> Usage Page (Digitizer) 05 0D
> Usage (Undefined) 09 0E
> Collection (Application) A1 01
> Report ID (17) 85 11
> Usage (Undefined) 09 23
> Collection (Logical) A1 02
> Usage 09 52
> Usage 09 53
> Logical Minimum (0) 15 00
> Logical Maximum (10) 25 0A
> Report Size (8) 75 08
> Report Count (2) 95 02
> Feature (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) B1 02
> End Collection C0
> End Collection C0
> Usage (Touch Screen) 09 04
> Collection (Application) A1 01
> Report ID (16) 85 10
> Usage (Finger) 09 22
> Collection (Logical) A1 02
> Usage (Tip Switch) 09 42
> Logical Minimum (0) 15 00
> Logical Maximum (1) 25 01
> Report Size (1) 75 01
> Report Count (1) 95 01
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Usage (In Range) 09 32
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Usage 09 47
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Report Count (5) 95 05
> Input (Cnst,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 03
> Report Size (8) 75 08
> Usage 09 51
> Report Count (1) 95 01
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Usage Page (Generic Desktop) 05 01
> Logical Maximum (32767) 26 FF 7F
> Report Size (16) 75 10
> Unit Exponent (-2) 55 0E
> Unit (Eng Lin: in^3) 65 33
> Usage (X) 09 30
> Physical Minimum (0) 35 00
> Physical Maximum (1594) 46 3A 06
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Usage (Y) 09 31
> Physical Maximum (1000) 46 E8 03
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> End Collection C0
> Usage Page (Digitizer) 05 0D
> Usage 09 54
> Report Count (1) 95 01
> Report Size (8) 75 08
> Logical Minimum (0) 15 00
> Logical Maximum (10) 25 0A
> Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
> Report ID (18) 85 12
> Usage 09 55
> Report Count (1) 95 01
> Report Size (8) 75 08
> Logical Minimum (0) 15 00
> Logical Maximum (10) 25 0A
> Feature (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) B1 02
> End Collection C0
>
> My own descriptor in the same conditions as the above (without point, 1
> touch only)
>
> 0x05, 0x0d, // USAGE_PAGE (Digitizers)
> 0x09, 0x0E, // USAGE (Configuration)
> 0xa1, 0x01, // COLLECTION (Application)
> 0x85, REPORTID_FEATURE, // REPORT_ID (Feature)
> 0x09, 0x23, // USAGE (Device Settings)
> 0xa1, 0x02, // COLLECTION (logical)
> 0x09, 0x52, // USAGE (Device Mode)
> 0x09, 0x53, // USAGE (Device Index)
> 0x15, 0x00, // LOGICAL_MINIMUM (0)
> 0x25, 0x0a, // LOGICAL_MAXIMUM (10)
> 0x75, 0x08, // REPORT_SIZE (8)
> 0x95, 0x02, // REPORT_COUNT (2)
> 0xb1, 0x02, // FEATURE (Data,Var,Abs)
> 0xc0, // END_COLLECTION
> 0xc0, // END_COLLECTION
>
> 0x09, 0x04, // USAGE (Touch Screen)
> 0xa1, 0x01, // COLLECTION (Application)
> 0x85, REPORTID_TOUCH, // REPORT_ID (Touch)
> 0x09, 0x22, // USAGE (Finger)
>
> 0x05, 0x0d, /* USAGE_PAGE
> (Digitizers) / <br>> 0xa1, 0x02, / COLLECTION (Logical)
> / <br>> 0x09, 0x42, / USAGE (Tip Switch)
> / <br>> 0x15, 0x00, / LOGICAL_MINIMUM (0)
> / <br>> 0x25, 0x01, / LOGICAL_MAXIMUM (1)
> / <br>> 0x75, 0x01, / REPORT_SIZE (1)
> / <br>> 0x95, 0x01, / REPORT_COUNT (1)
> / <br>> 0x81, 0x02, / INPUT (Data,Var,Abs)
> / <br>> 0x09, 0x32, / USAGE (In Range)
> / <br>> 0x81, 0x02, / INPUT (Data,Var,Abs)
> / <br>> 0x09, 0x47, / USAGE
> (Confidence) / <br>> 0x81, 0x02, / INPUT (Data,Var,Abs)
> / <br>> 0x95, 0x05, / REPORT_COUNT (5)
> / <br>> 0x81, 0x03, / INPUT (Cnst,Ary,Abs)
> / <br>> 0x09, 0x51, / USAGE (Contact
> Identifier) / <br>> 0x75, 0x08, / REPORT_SIZE (8)
> / <br>> 0x95, 0x01, / REPORT_COUNT (1)
> / <br>> 0x81, 0x02, / INPUT (Data,Var,Abs)
> / <br>> 0xa1, 0x00, / COLLECTION (Physical)
> / <br>> 0x05, 0x01, / USAGE_PAGE (Generic
> Desktop) / <br>> 0x15, 0x00, / LOGICAL_MINIMUM (0)
> / <br>> 0x26, 0x9c, 0x18, / LOGICAL_MAXIMUM
> (6300) / <br>> 0x75, 0x10, / REPORT_SIZE (16)
> / <br>> 0x55, 0x0e, /
> UNIT EXPONENT (-2) / <br>> 0x65, 0x33, /
> UNIT (Eng. Lin: in^3) / <br>> 0x35, 0x00, /
> PHYSICAL MINIMUM (0) / <br>> 0x46, 0xe8, 0x03, /
> PHYSICAL MAXIMUM (1594) / <br>> 0x09, 0x30, / USAGE (X)
> / <br>> 0x81, 0x02, / INPUT (Data,Var,Abs)
> / <br>> 0x09, 0x31, / USAGE (Y)
> / <br>> 0x35, 0x00, /
> PHYSICAL MINIMUM (0) / <br>> 0x46, 0x3a, 0x06, /
> PHYSICAL MAXIMUM (1000) / <br>> 0x81, 0x02, / INPUT
> (Data,Var,Abs) / <br>> 0x45, 0x00, /
> PHYSICAL MAXIMUM (reset) / <br>> 0xc0, / END_COLLECTION
> / <br>> 0xc0, / END_COLLECTION
> */ <br>>
> 0x05, 0x0d, // USAGE_PAGE (Digitizers)
> 0x09, 0x54, // USAGE (Actual count)
> 0x25, 0x00, // LOGICAL_MAXIMUM
> (0)
> 0x25, 0x0c, // LOGICAL_MAXIMUM
> (12)
> 0x95, 0x01, // REPORT_COUNT (1)
> 0x75, 0x08, // REPORT_SIZE (8)
> 0x81, 0x02, // INPUT (Data,Var,Abs)
>
> 0x85, REPORTID_MAX_COUNT, // REPORT_ID (Feature)
> 0x09, 0x55, // USAGE(Maximum Count)
> 0x25, 0x00, // LOGICAL_MAXIMUM
> (0)
> 0x25, 0x0c, // LOGICAL_MAXIMUM
> (12)
> 0x95, 0x01, // REPORT_COUNT (1)
> 0x75, 0x08, // REPORT_SIZE (8)
> 0xb1, 0x02, // FEATURE (Data,Var,Abs)
> 0xc0, // END_COLLECTION
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

5

Hi,

First of all thanks for your reply.

What I have discovered so far is that somehow, WPF doesn’t use WM_TOUCH messages to retrieve touch information. The only reason I find for this to happen is because WPF is older than Windows 7 and this was the way they implemented to support touch devices and WM_TOUCH was not being propagated by sytem by that time.

I have compared descriptors and reports in detail and I can’t find anything relevant. The descriptors are OK. Even the bits of in range and tip switch report.

My device provides touch functionality in all Windows environment and it only doesn’t work on WPF. That’s for sure!

I have a small test application that I use as test and it works with LUMIO and 3M touch devices. These devices are native hardware USB HID devices.

I’ll try to use your driver and compare somethings. In the mean while, can you test if it works out of the box for you? You can download the test app here:

www.imaginando.net/Test.exe

It is a small WPF applicatio which shows a globe and it provides pinch and pan gestures to interact with the globe. By default it doesn’t work with mouse, only with Touch. It also provides a checkbox to provide mouse emulation. Without the checkbox toggled, it works out of the box for 3M and Lumio touchscreens, but not with mine.

Thanks,

Nuno

6

Dan,

I have found the following code to list multitouch input devices on WPF:

bool foundMT = false;

foreach (TabletDevice tdc in Tablet.TabletDevices)
{
if (tdc.StylusDevices.Count > 1)
{
Console.WriteLine("Found: " + tdc.Name);

foundMT = true;
}
}

It happens that this code lists your driver, but when I plug mine it doesnt list even one!!!

My driver makes something that makes it not work with WPF and most of all, it stops the other from being seen.

Once again, I underline, my driver provides touch input to Windows. Every Windows single touch feature is enabled: touch, gestures, flicks, etc

Only WPF doesn’t work! Once again, your driver gets listed.

I have given a look to your code and everything you have, I have. However, I have much more than that. Maybe i’m doing something extra that is not being helpfull.

Does anyone have a clue of what might be happening here?

Thanks,

Nuno

7

Dan,

I have found the problem. My driver was not responding to IOCTL_HID_GET_STRING’s.

It seems that WPF stop questioning devices that don’t respond to IOCTL_HID_GET_STRING.

It was not a descriptors problem, it was not a RAW DEVICE problem…

Finally! This was a huge of a problem for me, but it’s finally solved!!! :slight_smile:

I hope this helps someone with the same problem because it is not an easy problem to determine the reason.

Thanks for your replies,

With my best regards,

Nuno