Containerized WDK Build Environments?

So now that I’ve played around with this for a few days, I have to say that
“windows containers” appear to be a moving target on WIn10. Building the
container on a current release works, building on an insider build is a
nightmare of inconsistent results. Once the container is built and
validated I think it is good to go on any supported platform, regardless of
where it was originally generated, but I haven’t verified that.

Of course I could do all of this on S2016 instead, but then I’d have to
deal with nested hyper-visors and that is another nightmare of what works
and what doesn’t…

Mark Roddy

On Fri, Jun 29, 2018 at 3:23 PM Mark Roddy wrote:

> Oh that is just the ticket. The container works great. Greatest container
> ever. Now I have to go convince it to build our stuff.
>
> Thanks Jakob!
>
> Mark Roddy
>
>
> On Wed, Jun 27, 2018 at 2:26 PM Mark Roddy wrote:
>
>> And I should read the thread to the bottom :-). That docker script looks
>> great. Will hack away at it.
>> Mark Roddy
>>
>>
>> On Wed, Jun 27, 2018 at 2:20 PM Mark Roddy wrote:
>>
>>> I looked at the EWDK as a starting point for a containerized build
>>> solution and gave up because it was just too huge to be a viable container
>>> image and there appears to be no reasonable way to get docker containers to
>>> use network share mount points. A msft supported docker image for each
>>> released wdk with reasonable size would be a huge win from my perspective.
>>>
>>> Mark Roddy
>>>
>>>
>>> On Thu, Jun 21, 2018 at 10:15 AM xxxxx@osr.com
>>> wrote:
>>>
>>>> (For those who may not be familiar with Mr. Lichtenberg… he’s the dev
>>>> lead for the WDK and related tools, and a LONG time friend of the driver
>>>> development community).
>>>>
>>>> Thank you, Mr. Lichtenberg, for jumping into the conversation. Your
>>>> help and insight is always appreciated.
>>>>
>>>>


>>>>
>>>> Honestly… the problem is that it’s very hard to narrow down. “Back
>>>> in the day” we might have had a driver and a (native) C/C++ test app in one
>>>> solution and that’s it. THESE days, that test app MIGHT be C/C++, or it
>>>> might equally be written in C#, or there could be a service in the Solution
>>>> (in C/C++ or C#), or it might be some UWP management app (OK, not really
>>>> the last of those… but someday, maybe).
>>>>
>>>> We could be convinced to build the user-mode parts separately (as a
>>>> separate Solution)… but we still have to build both parts (the driver
>>>> Solution and the UM Solution) as part of our CI process. So, bottom line,
>>>> we need to have a full-blown VS (to accommodate the various user-mode
>>>> options) and the driver part.
>>>>
>>>>


>>>>
>>>> That’s good news.
>>>>
>>>> One approach might be to (a) ship along with the WDK the pieces/parts
>>>> that are needed to ensure that tools work on Windows Server, (b) provide a
>>>> container with VS Community and the WDK pre-installed. Extra points for
>>>> having such a container on Docker Hub. How cool would THAT be??
>>>>
>>>> ANYhow… We still plan to give this a try, and see how it works.
>>>>
>>>> AND I hope there are others in the community who have tried this, and
>>>> can share the details of their experiences.
>>>>
>>>> Regardless, as I said, we’ll write-up whatever we come up with to help
>>>> those who come after us.
>>>>
>>>> Peter
>>>> OSR
>>>> @OSRDrivers
>>>>
>>>>
>>>> —
>>>> NTDEV is sponsored by OSR
>>>>
>>>> Visit the list online at: <
>>>> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>>>>
>>>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>>>> software drivers!
>>>> Details at http:
>>>>
>>>> To unsubscribe, visit the List Server section of OSR Online at <
>>>> http://www.osronline.com/page.cfm?name=ListServer&gt;
>>>>
>>></http:>

Thanks for the follow-up. Mr. Roddy. That’s very cool.

Did you see that there’s a new Windows Server Container image that’s supposed to have “more stuff” in it to support more apps?
https:</https:>

Also… My Friend Google tells me that VS versions PRIOR to VS 2017 can’t be installed in a container (because too much stuff they need is missing, or something)… which would, for us, completely defeat the need for using Containers. If anybody has info on this, please go public.

Having pre-built containerized WDK build environments would be a Very Good Thing.

Peter
OSR
@OSRDrivers

" Please note that for compatibility reasons we recommend running the same
build version for the container host and the container itself. ’

Arghhhhhhh!!!

No. I want my containers isolated from OS dependencies as much as possible.

Mark Roddy

On Tue, Jul 3, 2018 at 1:17 PM xxxxx@osr.com wrote:

> Thanks for the follow-up. Mr. Roddy. That’s very cool.
>
> Did you see that there’s a new Windows Server Container image that’s
> supposed to have “more stuff” in it to support more apps?
> <
> https://blogs.technet.microsoft.com/virtualization/2018/06/27/insider-preview-windows-container-image/
> >
>
> Also… My Friend Google tells me that VS versions PRIOR to VS 2017 can’t
> be installed in a container (because too much stuff they need is missing,
> or something)… which would, for us, completely defeat the need for using
> Containers. If anybody has info on this, please go public.
>
> Having pre-built containerized WDK build environments would be a Very Good
> Thing.
>
> Peter
> OSR
> @OSRDrivers
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

Silly me. Here, all along, I thought that was the point of containers.

Check this out:

https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility

So… really… how would this work? I need to have a host system that?s the newest rev of windows im order to host a container that?s on the newest rev of windows. Even if I?m using Hyper-V isolation.

That feels… impractical.

Or, you know… perhaps I?m missing something fundamental. I?m not ?Mr. Container? by any means.

Peter
OSR
@OSRDrivers

No it is a huge wtf. The point of my endeavors was to have this all tied
into a git server that enables distributed CI/CD (gitlab bitbucket etc)
such that there is no need for some giant server farm someplace with some
huge queue of build jobs on it, instead the build system can be distributed
to any available system that can host the desired docker image, like you
know your own dev system. And if by chance you have to build stuff for both
linux and windows wow, it’s all good, you can do that too.

I’ll continue down this path for a while 'cause it really does have to
happen, but …

Mark Roddy

On Tue, Jul 3, 2018 at 5:36 PM xxxxx@osr.com wrote:

>


>
> Silly me. Here, all along, I thought that was the point of containers.
>
> Check this out:
>
>
> https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility
>
> So… really… how would this work? I need to have a host system that?s
> the newest rev of windows im order to host a container that?s on the newest
> rev of windows. Even if I?m using Hyper-V isolation.
>
> That feels… impractical.
>
> Or, you know… perhaps I?m missing something fundamental. I?m not ?Mr.
> Container? by any means.
>
> Peter
> OSR
> @OSRDrivers
>
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

Hello @OSRDrivers /Peter, I have managed to install both VS2013 and VS2015 on a windows server docker container. I have also installed a couple of SDKs and WDKs on the container. However, my learning has been that, container is a different beast altogether. If something can be installed on a VM or a physical server, it necessarily does not mean that the same can be installed on a container. For example: just today, i hit an error installing a certain version of WDK that otherwise would install flawlessly on a VM or a physical system.
If you need further details on how i could get these things installed, please let me know and i will be happy to share information about the same.

FYI: i was even able to build our driver projects on this container without issues.

Regards,
Kiran Hegde

Just for pedantic’s sake, “necessarily does not mean” is a very different statement than “does not necessarily mean”.

Just for the sake of pedantry you mean :wink:

Or for the sake of being pedantic, but pedantic cannot be a possessive in English

RE the actual problem, I have been going the other way. Not even trying to maintain isolated environments, but to try to ensure that the projects all build with the latest tools. Of course I have the luxury of needing only to support versions I designate since our software is not for general distribution but only within our firm. That means Windows 10 & server 2016 x64 except for that annoying VS 2019 add-in that still has to be x86 when even our Excel 2010 compatible one can be x64 but I digress

I’d love to know more about how containers can be made to work for the EWDK, so @Kiran_Hegde please share if you will.

/door flies open

Build environments? I’ve pretty much fucking had it with the VS build environments.

The build environments are a mess. There are WDK issues that go unsolved and there’s no central “clearinghouse” to document them or the workarounds for them. There are never fixes, and new versions are tied to only one version of VS, forcing perpetual upgrades. The ever-evolving and almost-always-broken and continually updating Visual Studio is absolutely hideously terrible in terms of a central tool on which to depend. Code Analysis work so randomly in VS 2019 that it probably shouldn’t even be considered an existing feature.

What’s really amusing is that I’m working on a project with some C# code, and for C# VS seems to be pretty solid. Code analysis even works (well, modulo the whole “please upgrade to the Roslyn FxCop packages… but we won’t let you easily determine or control which rules are active in each of those packages so you’re stuck with either getting 76,000 warnings that your code style and the one FxCop arbitrarily chooses are different, or you use the old post build analysis and be content with seeing nag messages during your builds… guess which I chose).

VS was a strength for MSFT. The constant updates, lack of quality, and unpredictability in a tool that’s central to most dev’s work is inexcusable, and is an unending source of frustration and annoyance.

This is exactly what happens when you create a build environment that only customers use. Nobody internally uses it for anything serious. So it never gets fully and thoroughly tested in real-world use. This reminds me of a company where I worked long, long, ago where we famously had “one email system we use, and one email system that we ship.” We used to get complaints about the one we shipped all the time. Because it sucked. But we didn’t care. Because our email worked just fine thank you.

I would pay actual money to go back to Razzle. It never was fucked up. Because it was used by thousands of people, every day, for months before anybody in the driver community ever saw it.

Peter

Build environments? I’ve pretty much fucking had it with the VS build environments.

Amen. Unless my customers demand something else, I’m still using Visual Studio 2015 for everything, because I know it works.

However, I NEVER use the IDE. Maybe it’s different after all of its DLLs have been cached, but on a system where it hasn’t been run for a while, it literally takes 60 seconds for the IDE to launch and get to a point where a project can be loaded. I say it jokingly, but I can literally make change in vim, save it, compile it, test it, and be back in vim making the next change before the IDE has even launched. I don’t have the patience.

On the other hand, I have grown quite fond of VSCode. It is actually starting to draw me away from my friend of 30 years, vim. It’s quick, it’s friendly, it’s pretty, and I love the “project” view that vim doesn’t offer. Plus, it works the same on Windows, Mac, and Linux, and I do as much on the other platforms as I do in Windows. It bothers me a little bit that it is an Electron app, which means it’s really the whole Chrome browser running Javascript, but it’s hard to argue with success.

What’s really amusing is that I’m working on a project with some C# code, and for C# VS seems to be pretty solid.

Yep. I often see the C# folks in the MSDN forums talking about tools and fancy add-ons that aren’t available to VS C++ programmers. It’s not terribly surprising; C# is Microsoft’s baby.

And as a side note: ReSharper … for either C# or C++ … is totally worth it to me. For C# it continues to work miracles and practically write my code for me. For C++ it’s still reasonably handy, and having it run Clang-Tidy in the background and give me hints/tips is an awesome bonus.

I used Visual Assist X for years for C++… but the ReSharper people really stepped-up their game in C++ support over the past year.

If there’s any doubt, this is not a paid endorsement or the result of a free license. We pay retail for our ReSharper license (now that I’m no longer an MVP).

@“Peter_Viscarola_(OSR)” said:
I’d love to know more about how containers can be made to work for the EWDK, so @Kiran_Hegde please share if you will.

/door flies open

Build environments? I’ve pretty much fucking had it with the VS build environments.

Couldn’t agree more. I have a project on the backlog to chuck the whole mess and move to the EWDK toolchain. I looked briefly at a container for that, and two years ago there were issues that made it difficult, but perhaps they have been resolved. Also I have mostly ditched using VS for anything other than C#. It is really trivial to use the delightful vscode as your code editor and to integrate it with a script to run your builds.

I’m looking to stand up some sort of lab environment for driver build & test. Reading this thread has been insightful.

Seems like the most recent consensus is that VMs are hard (pain to setup, annoying to maintain), Docker Containers are brittle (install issues).

Regardless, as I said, we’ll write-up whatever we come up with to help those who come after us.


Source: https://community.osr.com/discussion/comment/289446/#Comment_289446

Has there been any recent developments? I’d love to learn from those with more experience here. Thanks! :smile:

I don’t think you can say that is a consensus. I have certainly not found VMs to be hard to create or maintain. I have not used Docker myself, but the whole point is that you pay the install penalty once, and then users of the container get a free ride.

I think we need more stories from the real world.

1 Like

There’s another big advantage of VM’s over containers: for certain regulated industries, you need to be able to reproduce the exact environment (OS, toolchain, etc.) used to create a particular binary years since it was first built. Containers can’t do that because they are hosted inside of an OS which changes year over year, month over month and for MS week over week.

VM’s can be snapshotted, exported and burned onto a BDXL disk and they are frozen … and if you use the right VM (I use VMWare Professional) it’s able to remount images made decades ago.

I’m very firmly in the VM camp … :slight_smile:

1 Like

@falcon This is a necropost.

I think the conclusions are:

  1. The EWDK makes life much easier… but it won’t build apps.
  2. It is possible to use containers, but it is fraught with issues and ultimately not worth it.
  3. Just use VMs… they’re large, but they work and can be stored and “replayed” easily as Mr. @craig_howard said.

So… there you have it.

And Don’t Necropost!

Peter

Apologies for indulging the necropost. But my experience with VMs is just the opposite.
Almost always my VMs that are several years long refused to run on modern versions of VMware or VirtualBox,
either because the emulated hardware platform has changed or the host ceased to understand outdated guest tools.
So I have to keep old versions of host software and hope that these old versions will still install and work on Win10… or wait… is it Win11 already?

On the other hand, Docker containers are great because they can be re-created from a plain text recipe.
Every manually crafted VM is unique :frowning:

1 Like

I’m using virtual machines (VMWare) for such purpose, too. So far so good. One general issue in these days is, that tools, IDE, … are more and more depended on being always online for license retrieving and stuff. Who can guarantee that your account for managing the license is still available over the next ten years? Beside that, it is not always a good idea to give your VM online access to prevent automatic updates that would contradict to have a frozen toolchain and OS.

As a rule I don’t use products that have online licenses, just for that reason: you are boned if you try to reproduce the environment in the future and that license has changed or gone away. That is a pretty easy rule to follow with drivers/ UI’s for me, very difficult with FPGA and board design tools but it’s doable if you use node locked licenses or you just don’t use floating license products …

It’s pretty common to have the OS attempt to update itself in a VM (and annoying, as I’ve ranted about in other threads), but I do a snapshot of the VM when I first spin it up and always do a periodic restore from snapshot to make sure something didn’t update while I was away (and another reason you’re boned if you use a container, if the host OS updates and there’s a component in that update that the container uses then you have lost the reproducibility)

As I’ve said, others seem to lead a very blessed life compared to mine … for me, I need to be able to guarantee a perfect level of consistency in the toolchain going into the end product, and be able to reproduce that exact toolchain years down the road. Some sort of online license, OS and toolchain updates, even internet access looking for an IP address that may no longer be there or which may point to content which is no longer there or in the same state breaks my world …

((sigh))

As this apparently is de-necroposted now:

  1. it is entirely possible to run a containerized EWDK build, in fact it is trivial.
  2. containers, unlike some opinions expressed here, are explicitly a much more defined and reproducible build environment than vms or physical systems, as one can specify, by git commit id, precisely which binary container image you wish to use.

The only caveats are that the container host is of course not under similar control, but in general this feature of containers is ignored, as containers by definition, isolate themselves from their hosts, although not as fully as vms do, and windows containers are pretty much an abomination in the container world.