Display Mirror Driver

NTDEV
21

Hi Tim,

If I set up a GDI, with EngCreateBitmap and EngAssociateSurface I cannot hook the functions I need. I haven’t had much sleep reading up on all this and that, how is it possible to carry on hooking while using GDI. Someone mentioned that you can’t and you have to have device managed and GDI. I wanted to run this past you before I read further - just so I know the direction I am going in.

I thought the Service could call the DrvEscape routines in the graphics device? It doesn’t matter if not, I can always bypass this problem.

22

I think I’ve answered my own question.

Can someone let me know if I have this right.

Inside DrvSurfaceEnable.
I need to use EngCreateBitmap and EngAssoicateSurface to create my own surface, which is just a link to a bitmap memory location.

I need also to use EngCreateDeviceSurface so that I can hook all calls made to the graphics device. (which is also made to my mirror driver.)

If I change the Desktop, (say for example, the user desktop (1440x900x32) to the service desktop (1024x768x32)) Which functions are called to detect this change?

When the hooked EngCreateDeviceSurface called DrvBitBlt, I then run EngBitBlt on the Source OBJSURF, to my own OBJSURF surface. (EngBitBlt(&MySurface, psoSrc, …) so I don’t have to forfill the drawing myself.

The information for the bitmap is stored in MySurface and I use this to generate the screenshot of what is currently on screen. This would be simple but even with the suggested changes I seem to only pick up some images and not everything that appears in a screenshot.If I picked up everything from psoSrc->pvBits I would understand and be able to move on from here.

This does seem to be a lot harder than I thought it would be but I don’t usually give up so please excuse me here.

What are the requirements for PPDEV ppdev = (PPDEV) dhpdev; ? because it seems like this is a custom struct type where I could add / remove types.

Tim, I’m understanding and getting my head around a lot, thank you very much for everything so far.

The only error I am recieving is sometimes when I deattach the mirror driver, the screen goes blank. (sometimes and not always constant for the same compiled code.) It is only the graphics driver, any music, mouse / keyboard operations are fine, windows is still running. I am wondering also if there is a way to detect this and have the graphics driver restart itself.

23

xxxxx@hotmail.com wrote:

If I set up a GDI, with EngCreateBitmap and EngAssociateSurface I cannot hook the functions I need.

Why not? You pass the flags for the routines you want to hook to
EngAssociateSurface.

I thought the Service could call the DrvEscape routines in the graphics device? It doesn’t matter if not, I can always bypass this problem.

No. Services do not have access to GDI.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

24

xxxxx@hotmail.com wrote:

Inside DrvSurfaceEnable.
I need to use EngCreateBitmap and EngAssoicateSurface to create my own surface, which is just a link to a bitmap memory location.

I need also to use EngCreateDeviceSurface so that I can hook all calls made to the graphics device. (which is also made to my mirror driver.)

You shouldn’t need to do this. You should be able to pass the hook
flags to EngAssociateSurface. You’ll get first shot at the routines you
hook.

If I change the Desktop, (say for example, the user desktop (1440x900x32) to the service desktop (1024x768x32)) Which functions are called to detect this change?

You should see DrvEnablePDEV on the new surface, and DrvAssertMode to
tell you which one is being disabled and which is being enabled.

The information for the bitmap is stored in MySurface and I use this to generate the screenshot of what is currently on screen. This would be simple but even with the suggested changes I seem to only pick up some images and not everything that appears in a screenshot.If I picked up everything from psoSrc->pvBits I would understand and be able to move on from here.

That suggests you are still missing some API hooks.

This does seem to be a lot harder than I thought it would be but I don’t usually give up so please excuse me here.

It IS hard. You are a display driver. That’s a complicated
environment, mostly because it has a long history, back to the very
beginning of Windows. There are traditions and legacy contracts at work
here that were not thought out quite carefully enough.

What are the requirements for PPDEV ppdev = (PPDEV) dhpdev; ? because it seems like this is a custom struct type where I could add / remove types.

That structure belongs entirely to your device. You can put whatever
you want in there.

The only error I am recieving is sometimes when I deattach the mirror driver, the screen goes blank. (sometimes and not always constant for the same compiled code.) It is only the graphics driver, any music, mouse / keyboard operations are fine, windows is still running. I am wondering also if there is a way to detect this and have the graphics driver restart itself.

You’re saying that when you detach your mirror driver, the primary
graphics drive goes blank? That shouldn’t happen. Does it go black, or
does it go to a background color?

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

25

Hi Tim,

Thankyou very much for your time.

I think I understand a lot more than I did yesterday, I still have a couple of questions.

What is the difference between .pvScan0 and .pvBits?

I have uploaded two images, the first is how it should be the second is how it is. What hooks do you think I am missing? There is a colour problem too which I am unsure about.

http://img257.imageshack.us/img257/4946/image1dq.jpg
http://img839.imageshack.us/img839/2273/image0p.jpg

The second image is correct being upside down.

> The only error I am recieving is sometimes when I deattach the mirror driver,
the screen goes blank. (sometimes and not always constant for the same compiled
code.) It is only the graphics driver, any music, mouse / keyboard operations
are fine, windows is still running. I am wondering also if there is a way to
detect this and have the graphics driver restart itself.

You’re saying that when you detach your mirror driver, the primary
graphics drive goes blank? That shouldn’t happen. Does it go black, or
does it go to a background color?

It goes black / blank. I haven’t had it for a while but I cannot be sure because as far as Windows is concerned there is no error and the same code sometimes causes it and sometime does not. I will investigate it on my own next it occurs.

To be honest I think I am getting there, I began sumrising all the information I could get my hands and certainly know a lot more. Thankyou very much.

26

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-427533-
xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: 12 October 2010 14:23
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Display Mirror Driver

http://img257.imageshack.us/img257/4946/image1dq.jpg
http://img839.imageshack.us/img839/2273/image0p.jpg

The second image is correct being upside down.

You have the green and blue values swapped, so the OSR logo in red, black and white is correct, but the blue-ish webpage has become green-ish.

Tim.

27

Hi Tim Green,

I should have thought about that, thankyou very much.

28

xxxxx@hotmail.com wrote:

I think I understand a lot more than I did yesterday, I still have a couple of questions.

What is the difference between .pvScan0 and .pvBits?

pvBits points to the first byte of the bitmap – the lowest address.
pvScan0 points to the first byte of the top scanline (where Y=0).
Remember that, because of some academically-oriented engineer at IBM,
DIBs are stored bottom up. That is, the pixel with the lowest address
is the bottom-left pixel. The top-left pixel is very near the end of
the memory buffer.

In that case, lDelta will be negative. To traverse the scans in
top-down order, you should start at pvScan0, and add lDelta to the
address for each scanline.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

29

> xxxxx@hotmail.com wrote:

I think I understand a lot more than I did yesterday, I still have a couple of
questions.

What is the difference between .pvScan0 and .pvBits?

pvBits points to the first byte of the bitmap – the lowest address.
pvScan0 points to the first byte of the top scanline (where Y=0).
Remember that, because of some academically-oriented engineer at IBM,
DIBs are stored bottom up. That is, the pixel with the lowest address
is the bottom-left pixel. The top-left pixel is very near the end of
the memory buffer.

In that case, lDelta will be negative. To traverse the scans in
top-down order, you should start at pvScan0, and add lDelta to the
address for each scanline.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I knew they were stored upside down I never knew by who or why. I assumed it was something to do with speed or importance of data. I’ll add some extra header information to my Mapped File to transfer that data to the user application because everything needs to be sent across the desktop so to save CPU I’ll save that data for another application.

Thankyou. I’m still getting black squares and I am missing the whole sidebar in Windows Vista. (including its gargets.) Do you know if this is DirectDraw if this is why I am unable to pick it up or if there is another reason I cannot.

If I’m bothering you let me know and I’ll really read into stuff before I ask, thank you very much so far though.

30

xxxxx@hotmail.com wrote:

I knew they were stored upside down I never knew by who or why. I assumed it was something to do with speed or importance of data.

Nope, it’s the mathematical view. Think about a graph with Cartesian
coordinates. Now consider the first quadrant, where both X and Y are
positive. (0,0) is at the lower left. X increases to the right, Y
increases going up. That’s the model. IBM introduced that for OS/2
Presentation Manager, which is where the Windows DIB concept came from.
Postscript uses the exact same concept.

Thankyou. I’m still getting black squares and I am missing the whole sidebar in Windows Vista. (including its gargets.) Do you know if this is DirectDraw if this is why I am unable to pick it up or if there is another reason I cannot.

I don’t know, actually. I haven’t stepped through a display driver with
a debugger since Vista. Adding a mirror driver should disable Aero
Glass, and I thought that disabled gadgets. If they are being drawn
with Direct3D, then you’ll have trouble grabbing them.

If I’m bothering you let me know and I’ll really read into stuff before I ask, thank you very much so far though.

If you were bothering me, you can bet that I would stop replying. I
always have that option.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

31

> I don’t know, actually. I haven’t stepped through a display driver with
a debugger since Vista. Adding a mirror driver should disable Aero
Glass, and I thought that disabled gadgets. If they are being drawn
with Direct3D, then you’ll have trouble grabbing them.

How would I go about re-enabling Aero Glass? The good news is after a quick browse on google it is more likely the sidebar is Aero Glass rather than DirectDraw.

If you were bothering me, you can bet that I would stop replying. I
always have that option.

Thankyou, just let me know first. ; )

32

I’ve been readign another one of your posts Tim, and you say it isn’t possible. That is such a disapointment because I’ve been learning this for two weeks. I’ll have to come back to it later.

As an alternative to taking a picture of the users desktop, what if I were to write a keyboard filter driver (I have already done this.) send to the driver the print screen button to execute, Windows intercepts this and I take a copy of the clipboard. Would this be possible inside a service? Would Windows know which desktop to take an image of and do you know what I would (c++ question sorry.) need to do to grab the current logged in users clipboard.

I will add a note here. When I was experimenting with taking images, I was successfully picking up at least one Aero Glass image. (I wasn’t getting much else, just a copy of the desktop image but no taskbar.) The clock showing the correct time on a pink background. So it obviously had been rendered and the bitmap passed to the mirror driver. I didn’t get much else just a few question marks. I didn’t know how to phase that data at the time.

33

How does print screen capture the whole image?

34

xxxxx@hotmail.com wrote:

As an alternative to taking a picture of the users desktop,…

OK, please remember that a mirror driver DOES NOT take a picture of the
user’s desktop. A mirror driver is merely being handed the same drawing
calls that are going to the main display driver. It can use those calls
to draw on its own bitmap, and if done carefully enough that bitmap
should be a reasonable facsimile of the main desktop, but it is not
connected to the desktop in any way. This is a common misunderstanding.

If you can solve your problem by taking a periodic BitBlt from the
desktop (which is exactly what Print Screen does), then by all means
just go do that. You could have that working in an hour or two, with no
kernel code at all.

what if I were to write a keyboard filter driver (I have already done this.) send to the driver the print screen button to execute, Windows intercepts this and I take a copy of the clipboard. Would this be possible inside a service? Would Windows know which desktop to take an image of and do you know what I would (c++ question sorry.) need to do to grab the current logged in users clipboard.

Actually, I don’t know whether the system clipboard is available from a
service, but this is the wrong way to go anyway. You can do whatever
Print Screen can do (although not from a service).

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

35

I hope people realize that D3D is used by Internet Explorer 9 to get
hardware accelerated web page rendering (the beta is happening NOW). This
pretty much means mirror/capture/remote desktop solutions that don’t work on
D3D output are about to be (or already are) close to useless.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-427577-
xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Tuesday, October 12, 2010 1:41 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Display Mirror Driver

> I don’t know, actually. I haven’t stepped through a display driver
> with
a debugger since Vista. Adding a mirror driver should disable Aero Glass,
and
I thought that disabled gadgets. If they are being drawn with Direct3D,
then
you’ll have trouble grabbing them.

How would I go about re-enabling Aero Glass? The good news is after a
quick
browse on google it is more likely the sidebar is Aero Glass rather than
DirectDraw.

> If you were bothering me, you can bet that I would stop replying. I
always have that option.

Thankyou, just let me know first. ; )

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

36



Windows intercepts this and I take a copy of the clipboard

Services running as local system on Vista & later don’t have full access to the clipboard associated with the user desktop.  You can usually get text, but the richer formats are, in general, not available to the service.  You’re probably best off by grabbing a bitmap image of the desktop window.

----- Original Message -----
From: xxxxx@hotmail.com
To: “Windows System Software Devs Interest List”
Sent: Tuesday, October 12, 2010 2:58:31 PM
Subject: RE:[ntdev] Display Mirror Driver

I’ve been readign another one of your posts Tim, and you say it isn’t possible. That is such a disapointment because I’ve been learning this for two weeks. I’ll have to come back to it later.

As an alternative to taking a picture of the users desktop, what if I were to write a keyboard filter driver (I have already done this.) send to the driver the print screen button to execute, Windows intercepts this and I take a copy of the clipboard. Would this be possible inside a service? Would Windows know which desktop to take an image of and do you know what I would (c++ question sorry.) need to do to grab the current logged in users clipboard.

I will add a note here. When I was experimenting with taking images, I was successfully picking up at least one Aero Glass image. (I wasn’t getting much else, just a copy of the desktop image but no taskbar.) The clock showing the correct time on a pink background. So it obviously had been rendered and the bitmap passed to the mirror driver. I didn’t get much else just a few question marks. I didn’t know how to phase that data at the time.

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

37

One other note … as others have mentioned, all in this discussion is under the XPDM display model – which has been deprecated in Win7 and already doesn’t work for some MS applications (such as Media Player 12 in a second monitor) even in XPDM mode.

Win7 and later use WDDM 1.0 and 1.1 which function in a completely different manner, depending on the hardware GPU to do the “where does the pixel go” computations rather than XPDM which uses OS calls to make those computations. This makes a mirror driver useless as it’s the GPU that puts down the pixels, not some software in the GDI … specifically, those GDI calls for all the various clients are marshalled through the DWM (Desktop Window Manager) which then tells the GPU what needs to happen. Two examples of this are the Aero features like the glassy borders and Media Player, both of which use the GPU to actually do the shading operations.

Jake and Doron have commented in another thread earlier that this makes a mirror driver a no-op for WDDM, and making a software-driver GPU emulator also a no-op due to some design decisions by the MS video guys which appear to be specifically designed to prevent this sort of GPU emulation …

So, a mirror driver is basically going to be something only useful for XP, or for Win7 running in XPDM mode using a single monitor … just so we’re all up on that …

Cheers!

38

Thanks Tim,

Unfortunately I am running this from a service. I should have worded my reply much better. I’ve know it is a mirror driver before asking questions.

I had thought about using the RDP to capture the desktop, render it in the service and then send it to the client but I am not quite how well this will work. The one alternative that is left is writing a user client application and executing that every 30ms, or it is executed, opens a pipe and sends the data to the service every 30ms with the current desktop image. This means time would have to be spend determining where the changes have occurred rather than pre-knowing them.

I am wondering if RDP is another lawsuit for Microsoft from those who want to make their own Remote Desktop Servers - if this is the only way for a service to correctly grab a desktop image. : )

I hope people realize that D3D is used by Internet Explorer 9 to get
hardware accelerated web page rendering (the beta is happening NOW). This
pretty much means mirror/capture/remote desktop solutions that don’t work on
D3D output are about to be (or already are) close to useless.

I am beginning to think they are.

39

I’m totally confused at this point, but regarding RDP, it’s publicly
documented. A while back I sent a link to the document (PDF, somewhere in
the MSDN library).

mm
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Tuesday, October 12, 2010 5:57 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Display Mirror Driver

Thanks Tim,

Unfortunately I am running this from a service. I should have worded my
reply much better. I’ve know it is a mirror driver before asking questions.

I had thought about using the RDP to capture the desktop, render it in the
service and then send it to the client but I am not quite how well this will
work. The one alternative that is left is writing a user client application
and executing that every 30ms, or it is executed, opens a pipe and sends the
data to the service every 30ms with the current desktop image. This means
time would have to be spend determining where the changes have occurred
rather than pre-knowing them.

I am wondering if RDP is another lawsuit for Microsoft from those who want
to make their own Remote Desktop Servers - if this is the only way for a
service to correctly grab a desktop image. : )

I hope people realize that D3D is used by Internet Explorer 9 to get
hardware accelerated web page rendering (the beta is happening NOW). This
pretty much means mirror/capture/remote desktop solutions that don’t work on
D3D output are about to be (or already are) close to useless.

I am beginning to think they are.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

40

Jan Bottorff wrote:

I hope people realize that D3D is used by Internet Explorer 9 to get
hardware accelerated web page rendering (the beta is happening NOW). This
pretty much means mirror/capture/remote desktop solutions that don’t work on
D3D output are about to be (or already are) close to useless.

And that is a serious problem. I hope it is utterly obvious to everyone
with an interest in graphics that this is something people really,
really want. I, for one, am very surprised that we find ourselves 5
years beyond the introduction of Vista, and there is still no plan on
record to support WDDM mirroring.

Maybe we just need to restate the problem. The XPDM mirror driver
concept was a cute accident that happened to fall out of the way
multimonitor was implemented in Windows 2000. The fact that it could be
used for remoting was just a bonus. What people really seem to want is
remoting. There are many ways to implement that in WDDM that are more
efficient than a mirror driver. I would be happy to steer the mirror
driver newbies to such a solution, if it existed.

choward wrote:

Jake and Doron have commented in another thread earlier that this makes a
mirror driver a no-op for WDDM, and making a software-driver GPU emulator
also a no-op due to some design decisions by the MS video guys which appear
to be specifically designed to prevent this sort of GPU emulation …

Calling it a “no-op” has connotation that it is a “piece of cake”. It
would be more correct to call it a “boat anchor”.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.