WHQL\HCK\HLK of OpenVPN TUNTAP driver

Hi all.

This is not a question or ask for help, I just want to share the fact that I managed to get WHQL certificate for OpenVPN TUNTAP driver (tap0901) and make it pass all the required tests for Windows 7, 8, 8.1, 10 both 32-bit and 64-bit versions with LAN as Product Type. In case someone googles for it :).

I have done it 4 times by now. Last time it took me 1 week to get it certified.

HCK test list for your reference (Windows 7, 8, 8.1):

[DevFund]
DF - Concurrent Hardware And Operating System (CHAOS) Test (Certification)
DF - Embedded Signature Verification Test (Certification)
DF - Fuzz Misc API test (Certification)
DF - Fuzz misc API with zero length query test (Certification)
DF - Fuzz open and close test (Certification)
DF - Fuzz Query and Set File Information Test (Certification)
DF - Fuzz Query and Set Security Test (Certification)
DF - Fuzz random FSCTL test (Certification)
DF - Fuzz random IOCTL test (Certification)
DF - Fuzz sub-opens test (Certification)
DF - Fuzz sub-opens with streams test (Certification)
DF - Fuzz zero length buffer FSCTL test (Certification)
DF - Fuzz zero length buffer IOCTL test (Certification)
DF - PNP Cancel Remove Device Test (Certification)
DF - PNP Cancel Stop Device Test (Certification)
DF - PNP DIF Remove Device Test (Certification)
DF - PNP Disable And Enable Device Test (Certification)
DF - PNP (disable and enable) with IO Before and After (Basic)
DF - PNP Rebalance Fail Restart Device Test (Certification)
DF - PNP Rebalance Request New Resources Device Test (Certification)
DF - PNP Remove Device Test (Certification)
DF - PNP Stop (Rebalance) Device Test (Certification)
DF - PNP Surprise Remove Device Test (Certification)
DF - Reinstall with IO Before and After (Certification)
DF - Sleep and PNP (disable and enable) with IO Before and After (Certification)
Device Driver INF Verification Test (Certification)
Wdf - Verify Driver Load Order Group is not WdfLoadGroup

[6.0]
1c_64BitOIDs
1c_FaultHandling
1c_IOCTLCoverage
1c_KernelCalls
1c_Mini6PerfSend
1c_Mini6RSSOids
1c_Mini6Send
1c_NdisRequestCov
1c_OidsDeviceIoControl
1c_OidsWMI
1c_Registry
1c_WMICoverage
2c_Mini6CheckConnections
2c_Mini6Performance
2c_Mini6SendRecv
2c_Mini6SendRecvReply
2c_Mini6Stress
2c_OidsNdisRequest
2c_Priority

[6.5]
1 Machine - GenericMiniportRequirements
1 Machine - OffloadMisc
1 Machine - PM_Requirements
1 Machine - StandardizedKeywords
2 Machine - AddressChange
2 Machine - CheckConnectivity
2 Machine - CheckConnectivity with AutoFail
2 Machine - ConfigCheck
2 Machine - E2EPerf
2 Machine - GlitchFreeDevice
2 Machine - InterruptModeration
2 Machine - InvalidPackets
2 Machine - KeepAlive
2 Machine - LinkCheck
2 Machine - MPE_Ethernet.xml
2 Machine - MultiCast Address
2 Machine - OffloadChecksum
2 Machine - PacketFilters
2 Machine - Reset
2 Machine - SelectiveSuspend
2 Machine - ShortPackets
2 Machine - SingleEtherType
2 Machine - Stats
2 Machine - TxFlowControl
2 Machine - VlanSendRecv

[LAN.PM]
PM_WakeOnLinkChange
PM_WolMagicPacket
PM_WolPattern
PM_PowerStateTransition

HLK test list for your reference (Windows 10):

[DevFund]
DevFund Broker Test
DF - Concurrent Hardware And Operating System (CHAOS) Test (Development and Integration)
DF - Concurrent Hardware And Operating System (CHAOS) Test (Reliability)
DF - Embedded Signature Verification Test (Tuning and Validation)
DF - Fuzz Misc API test (Reliability)
DF - Fuzz misc API with zero length query test (Reliability)
DF - Fuzz open and close test (Reliability)
DF - Fuzz Query and Set File Information Test (Reliability)
DF - Fuzz Query and Set Security Test (Reliability)
DF - Fuzz random FSCTL test (Reliability)
DF - Fuzz random IOCTL test (Reliability)
DF - Fuzz sub-opens test (Reliability)
DF - Fuzz sub-opens with streams test (Reliability)
DF - Fuzz zero length buffer FSCTL test (Reliability)
DF - Fuzz zero length buffer IOCTL test (Reliability)
DF - InfVerif INF Verification
DF - PCI Root Port Surprise Remove Test (PCI devices only) (Reliability)
DF - PNP (disable and enable) with IO Before and After (Bring Up)
DF - PNP Cancel Remove Device Test (Development and Integration)
DF - PNP Cancel Remove Device Test (Reliability)
DF - PNP Cancel Stop Device Test (Development and Integration)
DF - PNP Cancel Stop Device Test (Reliability)
DF - PNP DIF Remove Device Test (Development and Integration)
DF - PNP DIF Remove Device Test (Reliability)
DF - PNP Disable And Enable Device Test (Development and Integration)
DF - PNP Disable And Enable Device Test (Reliability)
DF - PNP Rebalance Fail Restart Device Test (Development and Integration)
DF - PNP Rebalance Fail Restart Device Test (Reliability)
DF - PNP Rebalance Request New Resources Device Test (Development and Integration)
DF - PNP Rebalance Request New Resources Device Test (Reliability)
DF - PNP Remove Device Test (Development and Integration)
DF - PNP Remove Device Test (Reliability)
DF - PNP Stop (Rebalance) Device Test (Development and Integration)
DF - PNP Stop (Rebalance) Device Test (Reliability)
DF - PNP Surprise Remove Device Test (Development and Integration)
DF - PNP Surprise Remove Device Test (Reliability)
DF - Reboot restart with IO before and after (Reliability)
DF - Reboot Restart with IO During (Development and Integration)
DF - Reboot Restart with IO During (Reliability)
DF - Reinstall with IO Before and After (Development and Integration)
DF - Reinstall with IO Before and After (Reliability)
DF - SimpleIO stress test with IO process termination (Reliability)
DF - Sleep and PNP (disable and enable) with IO Before and After (Development and Integration)
DF - Sleep and PNP (disable and enable) with IO Before and After (Reliability)
DF - Sleep with IO Before and After (Bring Up)
DF - Sleep with IO During (Development and Integration)
DF - Sleep with IO During (Reliability)
TDI filters and LSPs are not allowed

[6.0]
1c_64BitOIDs
1c_FaultHandling
1c_IOCTLCoverage
1c_KernelCalls
1c_Mini6PerfSend
1c_Mini6RSSOids
1c_Mini6Send
1c_NdisRequestCov
1c_OidsDeviceIoControl
1c_OidsWMI
1c_Registry
1c_WMICoverage
2c_Mini6CheckConnections
2c_Mini6Performance
2c_Mini6SendRecv
2c_Mini6SendRecvReply
2c_Mini6Stress
2c_OidsNdisRequest
2c_Priority

[6.5]
1 Machine - GenericMiniportRequirements
1 Machine - OffloadParity
1 Machine - PM_Requirements
1 Machine - StandardizedKeywords
2 Machine - AddressChange
2 Machine - CheckConnectivity
2 Machine - CheckConnectivity with AutoFail
2 Machine - ConfigCheck
2 Machine - E2EPerf
2 Machine - GlitchFreeDevice
2 Machine - InterruptModeration
2 Machine - InvalidPackets
2 Machine - KeepAlive
2 Machine - LinkCheck
2 Machine - MultiCast Address
2 Machine - OffloadMisc
2 Machine - PacketFilters
2 Machine - Reset
2 Machine - SelectiveSuspend
2 Machine - ShortPackets
2 Machine - SingleEtherType
2 Machine - Stats
2 Machine - TxFlowControl
2 Machine - VlanSendRecv

[LAN.PM]
PM_WakeOnLinkChange
PM_WolMagicPacket
PM_WolPattern
PM_PowerStateTransition

Thanks for sharing, mate!
Should prove useful in the long run.


Alexandra from W2C

@Andrii_Chabykin said:

Hi all.

This is not a question or ask for help, I just want to share the fact that I managed to get WHQL certificate for OpenVPN TUNTAP driver (tap0901) and make it pass all the required tests for Windows 7, 8, 8.1, 10 both 32-bit and 64-bit versions with LAN as Product Type. In case someone googles for it :).

I have done it 4 times by now. Last time it took me 1 week to get it certified.

It is good to hear. May I ask if you changed driver’s source code much?
As far as I can see it won’t pass at least "2 Machine - AddressChange, 2 Machine - VlanSendRecv and 2c_Priority.

Glad to see this working. One thing to note, not all of the HLK tests that start with DF - are required to run for WHQL certificate, only the ones with (Reliability) in the name. The other tests will be filtered out when the HLK playlist is applied. If done before testing this could save a lot of time. Here are some links for more information on the playlist and how to apply them.

https://docs.microsoft.com/en-us/windows-hardware/design/compatibility/
The latest playlist can be downloaded at: http://aka.ms/HLKPlaylist
https://docs.microsoft.com/en-us/windows-hardware/test/hlk/getstarted/step-6-select-and-run-tests

@Cymon said:
Glad to see this working. One thing to note, not all of the HLK tests that start with DF - are required to run for WHQL certificate, only the ones with (Reliability) in the name. The other tests will be filtered out when the HLK playlist is applied. If done before testing this could save a lot of time. Here are some links for more information on the playlist and how to apply them.

The hardest part so far was to pass NDIS specific tests. If it was possible to certify it as “Other device” instead of “Network device” it would be much easier…

Hi Andrii,

Have you run WHCK test for tap-windows6 project (https://github.com/OpenVPN/tap-windows6) anytime. By any chance are your sharing code modification required to pass hck test.

Thanks,

@niki88 said:
The hardest part so far was to pass NDIS specific tests. If it was possible to certify it as “Other device” instead of “Network device” it would be much easier…

Out of curiosity what is making the NDIS specific tests difficult? Is it setup/configuration, executing them, or triaging the failures?

@Cymon said:

@niki88 said:
The hardest part so far was to pass NDIS specific tests. If it was possible to certify it as “Other device” instead of “Network device” it would be much easier…

Out of curiosity what is making the NDIS specific tests difficult? Is it setup/configuration, executing them, or triaging the failures?

Well… one of them is supposed to take 5 hours long and I never got it finished even leaving running overnight. Several are more than an hour. Some require two machines talking via your virtual device and there issues with that, some just don’t pass without changes in the code like (2 Machine - VlanSendRecv for example). Also, actual failing tests seem to depend on “PhysicalMediaType”.

niki88 wrote:

>
> Out of curiosity what is making the NDIS specific tests difficult? Is it setup/configuration, executing them, or triaging the failures?
Well… one of them is supposed to take 5 hours long and I never got it finished even leaving running overnight. Several are more than an hour. Some require two machines talking via your virtual device and there issues with that, some just don’t pass without changes in the code like (2 Machine - VlanSendRecv for example). Also, actual failing tests seem to depend on “PhysicalMediaType”.

I assume the NDIS test suite needs to run the generic WHQL power
management and sleep tests.  Those can certainly take 18 to 24 hours.