Anybody automating the HLK signing process?

The process of setting up HLK devices, running the tests, collecting the results, packaging for all variants, submitting the package, retrieving the signed package, and integrating the package into your installation/redistributable software is anything but concisely documented. Many of the individual steps have various levels of documentation, but for the most part we all have the same workflow needs (ex: make a change and get it to our customer). I’ve got all the steps between code change and making an installer automated (with the exception of the HLK signing process).

Are people actually automating that process successfully? If so, are you able to integrate it into the rest of your CI process? I’d be really curious how folks are pulling this off. What does the device topology look like (ex: build machine(s), HLK server/client(s), how do they interact, etc. I find doing basic things like creating a proper .cab file for submission a PITA every time I need to make a new one. I’d love to replace the entire manual process with an automated process that has a single configuration file to account for anything unique to my driver.

Interested in automating the HLK process as well - it’s a pain having to manually go through it each release and it only looks like it will get worse given the number of OSes that I’ll want to test will only increase. Anyone doing this?

Edit: Also, I did find this github project: https://github.com/HCK-CI/AutoHCK although I haven’t tried to use it yet (other priorities rn).

@Shane_Corbin said:
Are people actually automating that process successfully?

Signing process, yes. We are using the previously cited “Microsoft Surface Dev Center Manager” code on github, https://github.com/microsoft/SDCM This supports both Attested Signing submissions as well as HLK and HCK submissions.

If you mean HLK testing (prior to HLK signing), I am not a good resource for that. I do recall someone here mentioning the https://github.com/HCK-CI/toolsHCK wrappers to enable some automation of certain operations, but I can’t say whether that enables full automation of the process or not.

I’m actually working on automation of these processes in our company, and AutoHCK (along with related projects, like toolsHCK, rtoolsHCK, VirtHCK) is indeed the one we are using. Although our system is based on quite an old version of it (they’ve done a lot of refactoring since we’ve adopted it, and it would be too much work to learn all the code from scratch and port our internal patches on top of the latest version). And for signing I’m using a Python script written by me, which uses the dashoard API.

I can tell, that quite a high level of automation is obtainable, both with testing, and signing. At the moment we already have a “one button” solution for running the HCK/HLK tests for all the drivers under all the supported OS versions, plus a button for sending the packages for signing. I could even merge these stages together, if I wanted, but from my experience, that would not be a good idea.

The problem is, it’s all not stable and reliable enough for maintaining a 100% automation. Many HCK/HLK tests can fail randomly (and then suddenly pass after like 20 or 40 restarts); some require very special environment to pass; weird issues occur that have to be resolved manually; MS filters and playlists are updated regularly, adding new tests or making some failures passable… I keep working on adding workarounds for more and more such cases, but it looks like a never ending battle. Therefore, the whole system is much more maintainable if kept split in stages. You need to be able to jump “inside” the whole solution, tweak something, and then contiue from where you’ve stopped, instead of restarting it all from scratch - a full run takes way too much time. Besides, it’s generally useful to be able to review the test results, restart specific tests or targets that need tweaking, repackage it the way you like, and only then send it all for signing (which also quite regularly returns weird server errors, so the sending has to be retried - and that can also be automated, of course).

Well, that’s partially the essence of what I was going to say: You’re never going to be able to really automate running the HLKs cuz they’re so squirrelly. And don’t forget you’ll need a second non-EV cert on file with the Dashboard, to avoid the fun of using an EV Cert.

I’m sure you can make things “simpler”… but the way I look at it: How many times do you need to logo your driver? Do you really want a CI/CD pipeline that ends in a release signed, logo’ed, driver package? Ugh…. I sure don’t. Attestation sign those intermediate updates, at best.

And if I’m gonna do a logo release once each quarter, it’s easier in the long run to let the test team handle it.

Peter