ETW?

(NOTE: the first post had a small bug. Replace “LEVEL” with “FLAG” in
the sources file. I’ll be happy to describe level/flag differences in a
later post.)

Is there interest in my posting more information about ETW? If so,
please respond directly to me (please - not to the list), with the
following:

What would you like more info on:
A) how ETW works under the covers? (macros, WPP side, etc)?
B) how to convert to using ETW?
C) “best practices” using ETW?
D) Special format strings, etc.?
E) LEVEL, FLAG, and how to use both?
F) Tools usage? (traceview, logman, …)

How often you’d like to see me toss something over the wall to you guys:
G) Once a week
H) Once per two weeks
I) Once a month
J) Never. :slight_smile:

I’m not claiming to be an expert in any of the above, but I’ve been
using it long enough to let you know the common problems I’ve seen,
common usage, and stuff that’s still not included (so you can compain
about it too). If there’s lots of interest, I may even bug the trace
guys to review my docs in the future. :slight_smile:

Thanks,
.

IMHO ETW is absolutely useless.

After a brief look at its description, I have noticed that learning this
over-complicated stuff just to do KdPrint is really amazing. I can spend time
learning, but some real new technologies which will add features to the
product.

As about Event Log - I have 2-3 routines to print stuff to log, and reuse
them where I feel the need to do so.

Also I cannot catch the connection between debug traces and event log.
Debug traces are to understand what is going wrong due to my bugs. Event Log is
to write some failures due to “interesting” behaviour of the surrounding
environment. How do they intersect?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Henry Gabryjelski”
To: “Windows System Software Devs Interest List”
Sent: Monday, February 02, 2004 9:19 PM
Subject: [ntdev] ETW?

>
> (NOTE: the first post had a small bug. Replace “LEVEL” with “FLAG” in
> the sources file. I’ll be happy to describe level/flag differences in a
> later post.)
>
> Is there interest in my posting more information about ETW? If so,
> please respond directly to me (please - not to the list), with the
> following:
>
> What would you like more info on:
> A) how ETW works under the covers? (macros, WPP side, etc)?
> B) how to convert to using ETW?
> C) “best practices” using ETW?
> D) Special format strings, etc.?
> E) LEVEL, FLAG, and how to use both?
> F) Tools usage? (traceview, logman, …)
>
> How often you’d like to see me toss something over the wall to you guys:
> G) Once a week
> H) Once per two weeks
> I) Once a month
> J) Never. :slight_smile:
>
> I’m not claiming to be an expert in any of the above, but I’ve been
> using it long enough to let you know the common problems I’ve seen,
> common usage, and stuff that’s still not included (so you can compain
> about it too). If there’s lots of interest, I may even bug the trace
> guys to review my docs in the future. :slight_smile:
>
> Thanks,
> .
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

I’d say the general idea is good. Tracing isn’t just simple KdPrint. Within
past years I started to use traces more and more for all debugging for both
PC and embedded development (hey, I even understand why Linus doesn’t like
debuggers :). Now I use debuggers mainly as trace viewers and error catchers
(asserts, BSOD etc). For these purposes we developed trace library which
resembles ETW in some aspects but is IMHO much easier to use. It is set of
macros enabled by compiler defines and can be used in both debug and release
builds. Every trace has assigned an area which is logical unit as source
file, class or functionality (power management for example). Next, every
trace has level (trace, debug, warning…). If something has to be debugged,
current level for given area is changed in registry. That’s all in
principle. We use traces for both debugging on our machines and internal and
field testing. If there is a problem, QA or customer enables requested
areas, reproduces problem and send us captured traces. Usually, that’s
enough. Saves a lot of time and makes software more reliable.

The idea for separating traces from formatting is also good one. Formatting
is slow and some problems are dependent on timing. Most of problems I had to
debug in the past. Debug output can also cause unwanted synchronization.
Later formatting also allows better filtering; sometimes it may be necessary
to see info about TID and PID, sometimes it is just useless and wastes
space. Not speaking about embedded apps where may not be enough memory for
formatted traces.

I wanted to see if ETW can be used instead of our library using the same
macros as we use now. It is important because we aren’t willing to change
thousands lines. After brief examination I don’t know. Whole stuff seems
overcomplicated and bloated. Maybe I’m lazy but I haven’t found if above
mentioned concept of areas and levels can be simulated some reasonable way
(having one GUID per area doesn’t seem reasonable). I would spent time with
converting if can presume better results that we have with our library. From
docs I read it is unclear. The only real advantage I see are debug strings
separated from binary. But I’m not sure if the price isn’t loss of other
important features (real time output, extracting traces from crashdump
etc.).

Maybe ETW is useful and maybe it is better than what we use now. Maybe.

Well, lets compare. If somebody here uses ETW for development, tell me,
please, exact way how to solve following real-life problem: I have a testing
application which communicates with a development board via USB. Something
went wrong and communication doesn’t work. Application is built with traces
so I go to registry, find key for the application (easy) and change USB area
from warning to trace. Then I start DbgView, run app and examine traces. Now
what I’d need to do when want to have the same traces with ETW (presume
application is built for ETW)?

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


From: xxxxx@storagecraft.com[SMTP:xxxxx@storagecraft.com]
Reply To: xxxxx@lists.osr.com
Sent: Monday, February 02, 2004 7:40 PM
To: xxxxx@lists.osr.com
Subject: Re: [ntdev] ETW?

IMHO ETW is absolutely useless.

After a brief look at its description, I have noticed that learning
this
over-complicated stuff just to do KdPrint is really amazing. I can spend
time
learning, but some real new technologies which will add features to the
product.

As about Event Log - I have 2-3 routines to print stuff to log, and
reuse
them where I feel the need to do so.

Also I cannot catch the connection between debug traces and event log.
Debug traces are to understand what is going wrong due to my bugs. Event
Log is
to write some failures due to “interesting” behaviour of the surrounding
environment. How do they intersect?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Henry Gabryjelski”
> To: “Windows System Software Devs Interest List”
> Sent: Monday, February 02, 2004 9:19 PM
> Subject: [ntdev] ETW?
>
>
> >
> > (NOTE: the first post had a small bug. Replace “LEVEL” with “FLAG” in
> > the sources file. I’ll be happy to describe level/flag differences in a
> > later post.)
> >
> > Is there interest in my posting more information about ETW? If so,
> > please respond directly to me (please - not to the list), with the
> > following:
> >
> > What would you like more info on:
> > A) how ETW works under the covers? (macros, WPP side, etc)?
> > B) how to convert to using ETW?
> > C) “best practices” using ETW?
> > D) Special format strings, etc.?
> > E) LEVEL, FLAG, and how to use both?
> > F) Tools usage? (traceview, logman, …)
> >
> > How often you’d like to see me toss something over the wall to you guys:
> > G) Once a week
> > H) Once per two weeks
> > I) Once a month
> > J) Never. :slight_smile:
> >
> > I’m not claiming to be an expert in any of the above, but I’ve been
> > using it long enough to let you know the common problems I’ve seen,
> > common usage, and stuff that’s still not included (so you can compain
> > about it too). If there’s lots of interest, I may even bug the trace
> > guys to review my docs in the future. :slight_smile:
> >
> > Thanks,
> > .
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: michal.vodicka@st.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

All my debug prints were something like
Debug(DEBUG_MODULE,(“Hello there %p\n”,p));

So changing to ETW was mostly easy. Here are some of the issues I ran into

  1. Some of my Debug() code was bitmasked, so I had stuff like
    Debug(DEBUG_MODULE|DEBUG_PROC_BEGIN,()) if I wanted to get call maps. This
    doesn’t work for ETW. I just created a new flag for proc calls and modified
    the 50 or so I had.
  2. I had code like "Debug(DEBUG_MODULE,(“The pointer is %08x\n”,p)) which
    doesn’t work so well, I spent several hours going through the code and
    changing the %08x to %p. Good news was that I needed to do it for 64bitness
    also.
  3. I have over 50 files in my source dir, and I had to modify all of them to
    include the #include at the top of the file, easy but a pain in the butt.
    #ifdef RUN_WPP
    extern “C” {
    #include “bit.tmh”
    }
    #endif
  4. I still cant get “%-*.*s” to work in ETW, so I just don’t use it any more
    (I commented out the lines that used it)
  5. Getting it to work for W2k and having a single binary was a pain in the
    butt. Then on top of that, compiling for AMD64 and IA64 required using the
    WXP way of using ETW. Doing all of this required modifying the source to
    where it can be compiled both ways, and then modifying the SOURCES to where
    it detected the environment, and picked which way to use ETW.
  6. I don’t use tracelog to enable tracing as I didn’t want to ship it, or
    have the customer find it. In my command line portion of the product, I
    added commands to enable and disable the logging. This could also have been
    a UI choice, but didn’t feel it was needed. I spent the more time writing
    the code to enable and disable than actually converting the driver to use
    ETW. This was because it was severely under documented.
  7. I have it that when I compile a debug version of the driver it uses the
    old style debug stuff and when in release uses ETW. I do this because it is
    easier to use the debugger this way than to try and get it working through
    the debugger. This may be because I am familiar with the old code, but in
    any case it works for me. Doing this required some checking in the SOURCES
    file to not define RUN_WPP if doing a debug build.

Thanks,
Rob

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-165978-
xxxxx@lists.osr.com] On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 3:39 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

I’d say the general idea is good. Tracing isn’t just simple KdPrint.
Within
past years I started to use traces more and more for all debugging for
both
PC and embedded development (hey, I even understand why Linus doesn’t like
debuggers :). Now I use debuggers mainly as trace viewers and error
catchers
(asserts, BSOD etc). For these purposes we developed trace library which
resembles ETW in some aspects but is IMHO much easier to use. It is set of
macros enabled by compiler defines and can be used in both debug and
release
builds. Every trace has assigned an area which is logical unit as source
file, class or functionality (power management for example). Next, every
trace has level (trace, debug, warning…). If something has to be
debugged,
current level for given area is changed in registry. That’s all in
principle. We use traces for both debugging on our machines and internal
and
field testing. If there is a problem, QA or customer enables requested
areas, reproduces problem and send us captured traces. Usually, that’s
enough. Saves a lot of time and makes software more reliable.

The idea for separating traces from formatting is also good one.
Formatting
is slow and some problems are dependent on timing. Most of problems I had
to
debug in the past. Debug output can also cause unwanted synchronization.
Later formatting also allows better filtering; sometimes it may be
necessary
to see info about TID and PID, sometimes it is just useless and wastes
space. Not speaking about embedded apps where may not be enough memory for
formatted traces.

I wanted to see if ETW can be used instead of our library using the same
macros as we use now. It is important because we aren’t willing to change
thousands lines. After brief examination I don’t know. Whole stuff seems
overcomplicated and bloated. Maybe I’m lazy but I haven’t found if above
mentioned concept of areas and levels can be simulated some reasonable way
(having one GUID per area doesn’t seem reasonable). I would spent time
with
converting if can presume better results that we have with our library.
From
docs I read it is unclear. The only real advantage I see are debug strings
separated from binary. But I’m not sure if the price isn’t loss of other
important features (real time output, extracting traces from crashdump
etc.).

Maybe ETW is useful and maybe it is better than what we use now. Maybe.

Well, lets compare. If somebody here uses ETW for development, tell me,
please, exact way how to solve following real-life problem: I have a
testing
application which communicates with a development board via USB. Something
went wrong and communication doesn’t work. Application is built with
traces
so I go to registry, find key for the application (easy) and change USB
area
from warning to trace. Then I start DbgView, run app and examine traces.
Now
what I’d need to do when want to have the same traces with ETW (presume
application is built for ETW)?

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

> ----------
> From: xxxxx@storagecraft.com[SMTP:xxxxx@storagecraft.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Monday, February 02, 2004 7:40 PM
> To: xxxxx@lists.osr.com
> Subject: Re: [ntdev] ETW?
>
> IMHO ETW is absolutely useless.
>
> After a brief look at its description, I have noticed that learning
> this
> over-complicated stuff just to do KdPrint is really amazing. I can
spend
> time
> learning, but some real new technologies which will add features to the
> product.
>
> As about Event Log - I have 2-3 routines to print stuff to log, and
> reuse
> them where I feel the need to do so.
>
> Also I cannot catch the connection between debug traces and event
log.
> Debug traces are to understand what is going wrong due to my bugs. Event
> Log is
> to write some failures due to “interesting” behaviour of the surrounding
> environment. How do they intersect?
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Henry Gabryjelski”
> > To: “Windows System Software Devs Interest List”
> > Sent: Monday, February 02, 2004 9:19 PM
> > Subject: [ntdev] ETW?
> >
> >
> > >
> > > (NOTE: the first post had a small bug. Replace “LEVEL” with “FLAG” in
> > > the sources file. I’ll be happy to describe level/flag differences in
> a
> > > later post.)
> > >
> > > Is there interest in my posting more information about ETW? If so,
> > > please respond directly to me (please - not to the list), with the
> > > following:
> > >
> > > What would you like more info on:
> > > A) how ETW works under the covers? (macros, WPP side, etc)?
> > > B) how to convert to using ETW?
> > > C) “best practices” using ETW?
> > > D) Special format strings, etc.?
> > > E) LEVEL, FLAG, and how to use both?
> > > F) Tools usage? (traceview, logman, …)
> > >
> > > How often you’d like to see me toss something over the wall to you
> guys:
> > > G) Once a week
> > > H) Once per two weeks
> > > I) Once a month
> > > J) Never. :slight_smile:
> > >
> > > I’m not claiming to be an expert in any of the above, but I’ve been
> > > using it long enough to let you know the common problems I’ve seen,
> > > common usage, and stuff that’s still not included (so you can compain
> > > about it too). If there’s lots of interest, I may even bug the trace
> > > guys to review my docs in the future. :slight_smile:
> > >
> > > Thanks,
> > > .
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: michal.vodicka@st.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@cdp.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

> ----------

From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
Reply To: xxxxx@lists.osr.com
Sent: Monday, February 02, 2004 10:17 PM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

  1. I have it that when I compile a debug version of the driver it uses the
    old style debug stuff and when in release uses ETW. I do this because it
    is
    easier to use the debugger this way than to try and get it working through
    the debugger.

It is important. I need an easy way how to enable traces to be visible by
debugger. The same code has to run on mine and testing machines. Is there a
way how to format and send traces to debug output? In realtime? Capturing by
kd/windbg is insufficient because I prefer one machine debugging with
SoftICE or better, with DbgView which has highlight filters.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

SoftICE will catch debug prints, no ? If you have a second machine,
DriverMonitor will too. Or do I misunderstand you ?

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 5:34 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?


From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
Reply To: xxxxx@lists.osr.com
Sent: Monday, February 02, 2004 10:17 PM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

  1. I have it that when I compile a debug version of the driver it uses the
    old style debug stuff and when in release uses ETW. I do this because it
    is
    easier to use the debugger this way than to try and get it working through
    the debugger.

It is important. I need an easy way how to enable traces to be visible by
debugger. The same code has to run on mine and testing machines. Is there a
way how to format and send traces to debug output? In realtime? Capturing by
kd/windbg is insufficient because I prefer one machine debugging with
SoftICE or better, with DbgView which has highlight filters.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

Yes, you misunderstand. I wanted like to know if ETW can be redirected to
debug output and captured standard way. By SoftICE or anything else.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:05 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

SoftICE will catch debug prints, no ? If you have a second machine,
DriverMonitor will too. Or do I misunderstand you ?

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 5:34 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

> ----------
> From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Monday, February 02, 2004 10:17 PM
> To: xxxxx@lists.osr.com
> Subject: RE: [ntdev] ETW?
>
> 6. I have it that when I compile a debug version of the driver it uses
the
> old style debug stuff and when in release uses ETW. I do this because
it
> is
> easier to use the debugger this way than to try and get it working
through
> the debugger.
>
It is important. I need an easy way how to enable traces to be visible by
debugger. The same code has to run on mine and testing machines. Is there
a
way how to format and send traces to debug output? In realtime? Capturing
by
kd/windbg is insufficient because I prefer one machine debugging with
SoftICE or better, with DbgView which has highlight filters.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose
it to anyone else. If you received it in error please notify us
immediately
and then destroy it.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: michal.vodicka@st.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

I don’t think you can use SoftIce or DbgView as the output isn’t actually
sent to the debugger, just the raw data.

To view it in windbg you must do the following:
Tracelog (whatever params) -kd

Then in the debugger
!load winxp\wmitrace
!searchpath c:\MyTmfFiles
!dynamicprint 1

Then you will see the output as before. I usually need to do debug prints
during boot up as my driver is a system boot driver, and there is no way to
turn on ETW for the driver until the system is completely booted. This is
the main reason I left the old debug prints in the driver for debug builds.
If I need the debug info for a boot in the field, I am back to sending a
debug driver to them, but this has cut down on the number I do send
significantly.

Thanks,
Rob

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-165988-
xxxxx@lists.osr.com] On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 5:34 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

> ----------
> From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Monday, February 02, 2004 10:17 PM
> To: xxxxx@lists.osr.com
> Subject: RE: [ntdev] ETW?
>
> 6. I have it that when I compile a debug version of the driver it uses
the
> old style debug stuff and when in release uses ETW. I do this because
it
> is
> easier to use the debugger this way than to try and get it working
through
> the debugger.
>
It is important. I need an easy way how to enable traces to be visible by
debugger. The same code has to run on mine and testing machines. Is there
a
way how to format and send traces to debug output? In realtime? Capturing
by
kd/windbg is insufficient because I prefer one machine debugging with
SoftICE or better, with DbgView which has highlight filters.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@cdp.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t even
registered as a debugger. But if you want to do your own tracing, you can
write it in a way that SoftICE catches it. Also, BoundsChecker in DS 3.1
allows you to monitor your own driver’s functions, so you can keep tabs on
the full range of events that Bchkd tracks, and those will not only go to
the Workbench GUI in the host, but also to SoftICE where it’s reachable by
the “event” command.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 6:18 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

Yes, you misunderstand. I wanted like to know if ETW can be redirected to
debug output and captured standard way. By SoftICE or anything else.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:05 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

SoftICE will catch debug prints, no ? If you have a second machine,
DriverMonitor will too. Or do I misunderstand you ?

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 5:34 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

> ----------
> From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Monday, February 02, 2004 10:17 PM
> To: xxxxx@lists.osr.com
> Subject: RE: [ntdev] ETW?
>
> 6. I have it that when I compile a debug version of the driver it uses
the
> old style debug stuff and when in release uses ETW. I do this because
it
> is
> easier to use the debugger this way than to try and get it working
through
> the debugger.
>
It is important. I need an easy way how to enable traces to be visible by
debugger. The same code has to run on mine and testing machines. Is there
a
way how to format and send traces to debug output? In realtime? Capturing
by
kd/windbg is insufficient because I prefer one machine debugging with
SoftICE or better, with DbgView which has highlight filters.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose
it to anyone else. If you received it in error please notify us
immediately
and then destroy it.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: michal.vodicka@st.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

> ----------

From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:36 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

I don’t think you can use SoftIce or DbgView as the output isn’t actually
sent to the debugger, just the raw data.

Yes, it is clear. I just wondered if there isn’t something in the kernel
which would optionally format data and send to debug output.

To view it in windbg you must do the following:
Tracelog (whatever params) -kd

Then in the debugger
!load winxp\wmitrace
!searchpath c:\MyTmfFiles
!dynamicprint 1

Then you will see the output as before. I usually need to do debug prints
during boot up as my driver is a system boot driver, and there is no way
to
turn on ETW for the driver until the system is completely booted.

Really? Does it mean you always have to run an usermode app to enable
tracing? If so it is nearly useless. Trace settings as I use it has to be
persistent accross boots and available in DriverEntry().

This is
the main reason I left the old debug prints in the driver for debug
builds.
If I need the debug info for a boot in the field, I am back to sending a
debug driver to them, but this has cut down on the number I do send
significantly.

OK, thanks, I’m probably discouraged enough for next several months :wink:

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

> ----------

From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:43 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t even
registered as a debugger.

Maybe it should catch ETW in the future. It is standard OS feature, anyway.

But if you want to do your own tracing, you can
write it in a way that SoftICE catches it.

Which is what I do all the time. I just wondered if ETW can give me some
advantages.

Also, BoundsChecker in DS 3.1
allows you to monitor your own driver’s functions, so you can keep tabs on
the full range of events that Bchkd tracks, and those will not only go to
the Workbench GUI in the host, but also to SoftICE where it’s reachable by
the “event” command.

Sorry, I still prefer my own traces. Event command is helpful, thought. And
you know how I like your GUI apps :wink:

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

Maybe you’re right, but so far we don’t support ETW. Now, you don’t need to
use our GUI apps to benefit from BoundsChecker, because you have the SoftICE
“event” command. Anything that you ask bchkd.sys to catch ends up right
inside SoftICE.

The other thing is, I heard you complain a lot about the DS 2.5 Workbench,
but the GUI is now quite different in DS 3.1. Also, if you want, you can
operate the tools directly inside MSVS.NET and avoid our GUI shell
altogether !

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 7:13 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?


From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:43 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t even
registered as a debugger.

Maybe it should catch ETW in the future. It is standard OS feature, anyway.

But if you want to do your own tracing, you can
write it in a way that SoftICE catches it.

Which is what I do all the time. I just wondered if ETW can give me some
advantages.

Also, BoundsChecker in DS 3.1
allows you to monitor your own driver’s functions, so you can keep tabs on
the full range of events that Bchkd tracks, and those will not only go to
the Workbench GUI in the host, but also to SoftICE where it’s reachable by
the “event” command.

Sorry, I still prefer my own traces. Event command is helpful, thought. And
you know how I like your GUI apps :wink:

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

Michal,

If you disable BoundsChecker when you reboot, and if you have previously
turned file logging on, the log file will survive a reboot. I know it’s a
bit inconvenient, but it can be done - I just found out that we overwrite
the log file at boot time if file logging is turned on, maybe we should
change it to append instead.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 7:07 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

Really? Does it mean you always have to run an usermode app to enable
tracing? If so it is nearly useless. Trace settings as I use it has to be
persistent accross boots and available in DriverEntry().

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

BTW - will SoftICE have native PDB support without converting it to NMS?
This would increase the user experience a bit.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Moreira, Alberto”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, February 03, 2004 6:40 PM
Subject: RE: [ntdev] ETW?

> Maybe you’re right, but so far we don’t support ETW. Now, you don’t need to
> use our GUI apps to benefit from BoundsChecker, because you have the SoftICE
> “event” command. Anything that you ask bchkd.sys to catch ends up right
> inside SoftICE.
>
> The other thing is, I heard you complain a lot about the DS 2.5 Workbench,
> but the GUI is now quite different in DS 3.1. Also, if you want, you can
> operate the tools directly inside MSVS.NET and avoid our GUI shell
> altogether !
>
> Alberto.
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
> Sent: Monday, February 02, 2004 7:13 PM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] ETW?
>
>
> > ----------
> > From:
> > xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
> > Reply To: xxxxx@lists.osr.com
> > Sent: Tuesday, February 03, 2004 12:43 AM
> > To: xxxxx@lists.osr.com
> > Subject: RE: [ntdev] ETW?
> >
> > Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t even
> > registered as a debugger.
> >
> Maybe it should catch ETW in the future. It is standard OS feature, anyway.
>
> > But if you want to do your own tracing, you can
> > write it in a way that SoftICE catches it.
> >
> Which is what I do all the time. I just wondered if ETW can give me some
> advantages.
>
> > Also, BoundsChecker in DS 3.1
> > allows you to monitor your own driver’s functions, so you can keep tabs on
> > the full range of events that Bchkd tracks, and those will not only go to
> > the Workbench GUI in the host, but also to SoftICE where it’s reachable by
> > the “event” command.
> >
> Sorry, I still prefer my own traces. Event command is helpful, thought. And
> you know how I like your GUI apps :wink:
>
> Best regards,
>
> Michal Vodicka
> STMicroelectronics Design and Application s.r.o.
> [michal.vodicka@st.com, http:://www.st.com]
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@compuware.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or disclose
> it to anyone else. If you received it in error please notify us immediately
> and then destroy it.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

SoftICE must be able to totally function when the Operating System is not
functional. Also, we use NMS to get source code into SoftICE. This is
because the OS may have gone out to lunch.

SoftICE must be able to run on the bare iron !

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Maxim S. Shatskih
Sent: Tuesday, February 03, 2004 11:39 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] ETW?

BTW - will SoftICE have native PDB support without converting it to NMS?
This would increase the user experience a bit.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Moreira, Alberto”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, February 03, 2004 6:40 PM
Subject: RE: [ntdev] ETW?

> Maybe you’re right, but so far we don’t support ETW. Now, you don’t need
to
> use our GUI apps to benefit from BoundsChecker, because you have the
SoftICE
> “event” command. Anything that you ask bchkd.sys to catch ends up right
> inside SoftICE.
>
> The other thing is, I heard you complain a lot about the DS 2.5 Workbench,
> but the GUI is now quite different in DS 3.1. Also, if you want, you can
> operate the tools directly inside MSVS.NET and avoid our GUI shell
> altogether !
>
> Alberto.
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Michal Vodicka
> Sent: Monday, February 02, 2004 7:13 PM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] ETW?
>
>
> > ----------
> > From:
> > xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
> > Reply To: xxxxx@lists.osr.com
> > Sent: Tuesday, February 03, 2004 12:43 AM
> > To: xxxxx@lists.osr.com
> > Subject: RE: [ntdev] ETW?
> >
> > Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t
even
> > registered as a debugger.
> >
> Maybe it should catch ETW in the future. It is standard OS feature,
anyway.
>
> > But if you want to do your own tracing, you can
> > write it in a way that SoftICE catches it.
> >
> Which is what I do all the time. I just wondered if ETW can give me some
> advantages.
>
> > Also, BoundsChecker in DS 3.1
> > allows you to monitor your own driver’s functions, so you can keep tabs
on
> > the full range of events that Bchkd tracks, and those will not only go
to
> > the Workbench GUI in the host, but also to SoftICE where it’s reachable
by
> > the “event” command.
> >
> Sorry, I still prefer my own traces. Event command is helpful, thought.
And
> you know how I like your GUI apps :wink:
>
> Best regards,
>
> Michal Vodicka
> STMicroelectronics Design and Application s.r.o.
> [michal.vodicka@st.com, http:://www.st.com]
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@compuware.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
disclose
> it to anyone else. If you received it in error please notify us
immediately
> and then destroy it.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

  1. I believe the format string %08x would work if you were passing a
    non-pointer. ETW is more exacting about matching types, which I found
    to be a nice side benefit. It also will complain if you don’t have
    enough arguments for the given format string, which caught me a few
    times.

(what is “%-*.*s” supposed to do?)

  1. This is great feedback. I personally didn’t realize the problem of
    *requiring* the WXP method of enabling tracing. I can imagine a method
    to work around this, I’ll see if I can convince the trace guys that its
    worthwhile. As an added bonus, it won’t require new kernel/os changes.

  2. logman.exe, which uses different syntax than tracelog, is shipped in
    XP and higher. It’s also available for download, IIRC. I haven’t dealt
    with auto-enabling traces in the app/driver, but I’ll pass along that
    more docs are required for this.

6b) I also do the same thing regarding having a compile-time switch for
use DebugPrint. It’s more work in SOURCES and you need to wrap the
init/unint/.TMH inclusion, but overall I’ve found it to be workable.

I’ll pass you comments to the tracing team, see if they’re able to make
it better.
.

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Rob Green [mailto:xxxxx@cdp.com]
Sent: Monday, February 02, 2004 1:18 PM
Subject: RE: ETW?

All my debug prints were something like
Debug(DEBUG_MODULE,(“Hello there %p\n”,p));

So changing to ETW was mostly easy. Here are some of the issues I ran
into

  1. Some of my Debug() code was bitmasked, so I had stuff like
    Debug(DEBUG_MODULE|DEBUG_PROC_BEGIN,()) if I wanted to get call maps.
    This doesn’t work for ETW. I just created a new flag for proc calls and
    modified the 50 or so I had.
  2. I had code like "Debug(DEBUG_MODULE,(“The pointer is %08x\n”,p))
    which doesn’t work so well, I spent several hours going through the code
    and changing the %08x to %p. Good news was that I needed to do it for
    64bitness also.
  3. I have over 50 files in my source dir, and I had to modify all of
    them to include the #include at the top of the file, easy but a pain in
    the butt.
    #ifdef RUN_WPP
    extern “C” {
    #include “bit.tmh”
    }
    #endif
  4. I still cant get “%-*.*s” to work in ETW, so I just don’t use it any
    more (I commented out the lines that used it) 5. Getting it to work for
    W2k and having a single binary was a pain in the butt. Then on top of
    that, compiling for AMD64 and IA64 required using the WXP way of using
    ETW. Doing all of this required modifying the source to where it can be
    compiled both ways, and then modifying the SOURCES to where it detected
    the environment, and picked which way to use ETW.
  5. I don’t use tracelog to enable tracing as I didn’t want to ship it,
    or have the customer find it. In my command line portion of the
    product, I added commands to enable and disable the logging. This could
    also have been a UI choice, but didn’t feel it was needed. I spent the
    more time writing the code to enable and disable than actually
    converting the driver to use ETW. This was because it was severely
    under documented.
  6. I have it that when I compile a debug version of the driver it uses
    the old style debug stuff and when in release uses ETW. I do this
    because it is easier to use the debugger this way than to try and get it
    working through the debugger. This may be because I am familiar with
    the old code, but in any case it works for me. Doing this required some
    checking in the SOURCES file to not define RUN_WPP if doing a debug
    build.

Thanks,
Rob

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-165978-
xxxxx@lists.osr.com] On Behalf Of Michal Vodicka
Sent: Monday, February 02, 2004 3:39 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ETW?

I’d say the general idea is good. Tracing isn’t just simple KdPrint.
Within
past years I started to use traces more and more for all debugging for

both PC and embedded development (hey, I even understand why Linus
doesn’t like debuggers :). Now I use debuggers mainly as trace viewers

and error catchers (asserts, BSOD etc). For these purposes we
developed trace library which resembles ETW in some aspects but is
IMHO much easier to use. It is set of macros enabled by compiler
defines and can be used in both debug and release builds. Every trace
has assigned an area which is logical unit as source file, class or
functionality (power management for example). Next, every trace has
level (trace, debug, warning…). If something has to be debugged,
current level for given area is changed in registry. That’s all in
principle. We use traces for both debugging on our machines and
internal and field testing. If there is a problem, QA or customer
enables requested areas, reproduces problem and send us captured
traces. Usually, that’s enough. Saves a lot of time and makes software

more reliable.

The idea for separating traces from formatting is also good one.
Formatting
is slow and some problems are dependent on timing. Most of problems I
had to debug in the past. Debug output can also cause unwanted
synchronization.
Later formatting also allows better filtering; sometimes it may be
necessary to see info about TID and PID, sometimes it is just useless
and wastes space. Not speaking about embedded apps where may not be
enough memory for formatted traces.

I wanted to see if ETW can be used instead of our library using the
same macros as we use now. It is important because we aren’t willing
to change thousands lines. After brief examination I don’t know. Whole

stuff seems overcomplicated and bloated. Maybe I’m lazy but I haven’t
found if above mentioned concept of areas and levels can be simulated
some reasonable way (having one GUID per area doesn’t seem
reasonable). I would spent time with converting if can presume better
results that we have with our library.
From
docs I read it is unclear. The only real advantage I see are debug
strings separated from binary. But I’m not sure if the price isn’t
loss of other important features (real time output, extracting traces
from crashdump etc.).

Maybe ETW is useful and maybe it is better than what we use now.
Maybe.

Well, lets compare. If somebody here uses ETW for development, tell
me, please, exact way how to solve following real-life problem: I have

a testing application which communicates with a development board via
USB. Something went wrong and communication doesn’t work. Application
is built with traces so I go to registry, find key for the application

(easy) and change USB area from warning to trace. Then I start
DbgView, run app and examine traces.
Now
what I’d need to do when want to have the same traces with ETW
(presume application is built for ETW)?

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

> ----------
> From: xxxxx@storagecraft.com[SMTP:xxxxx@storagecraft.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Monday, February 02, 2004 7:40 PM
> To: xxxxx@lists.osr.com
> Subject: Re: [ntdev] ETW?
>
> IMHO ETW is absolutely useless.
>
> After a brief look at its description, I have noticed that
> learning this over-complicated stuff just to do KdPrint is really
> amazing. I can
spend
> time
> learning, but some real new technologies which will add features to
> the product.
>
> As about Event Log - I have 2-3 routines to print stuff to log,
> and reuse them where I feel the need to do so.
>
> Also I cannot catch the connection between debug traces and
> event
log.
> Debug traces are to understand what is going wrong due to my bugs.
> Event Log is to write some failures due to “interesting” behaviour
> of the surrounding environment. How do they intersect?
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Henry Gabryjelski”
> > To: “Windows System Software Devs Interest List”
> >
> > Sent: Monday, February 02, 2004 9:19 PM
> > Subject: [ntdev] ETW?
> >
> >
> > >
> > > (NOTE: the first post had a small bug. Replace “LEVEL” with
> > > “FLAG” in the sources file. I’ll be happy to describe level/flag
> > > differences in
> a
> > > later post.)
> > >
> > > Is there interest in my posting more information about ETW? If
> > > so, please respond directly to me (please - not to the list), with

> > > the
> > > following:
> > >
> > > What would you like more info on:
> > > A) how ETW works under the covers? (macros, WPP side, etc)?
> > > B) how to convert to using ETW?
> > > C) “best practices” using ETW?
> > > D) Special format strings, etc.?
> > > E) LEVEL, FLAG, and how to use both?
> > > F) Tools usage? (traceview, logman, …)
> > >
> > > How often you’d like to see me toss something over the wall to you
> guys:
> > > G) Once a week
> > > H) Once per two weeks
> > > I) Once a month
> > > J) Never. :slight_smile:
> > >
> > > I’m not claiming to be an expert in any of the above, but I’ve
> > > been using it long enough to let you know the common problems I’ve

> > > seen, common usage, and stuff that’s still not included (so you
> > > can compain about it too). If there’s lots of interest, I may
> > > even bug the trace guys to review my docs in the future. :slight_smile:
> > >
> > > Thanks,
> > > .
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to
> > > xxxxx@lists.osr.com
> > >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: michal.vodicka@st.com To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@cdp.com To unsubscribe
> send a blank email to xxxxx@lists.osr.com

Michal, thanks for answering the below questions so well.

There is, indeed, a method to provide what you are asking below. This
is called using LEVEL and FLAG in ETW. The simple way is:

Add the following lines to a common include file:

// the next two lines enable tracing by level and flag
#define WPP_LEVEL_FLAGS_LOGGER(lvl,flags) WPP_LEVEL_LOGGER(flags)
#define WPP_LEVEL_FLAGS_ENABLED(lvl, flags) (WPP_LEVEL_ENABLED(flags) &&
WPP_CONTROL(WPP_BIT_ ## flags).Level >= lvl)

Then, change your SOURCES file’s RUN_WPP line from ‘-func:Dbg(NULL,
FLAG, MSG, …)’ to ‘-func:Dbg(LEVEL, FLAG, MSG, …)’. The LEVEL
applies to all FLAGs (global per-GUID setting), while you can use the
FLAG setting to separate your traces into logical groups. The LEVEL
argument is not munged, so you can use literals (1,2,etc), defines
(TRACE_LEVEL_NONE, TRACE_LEVEL_FATAL, etc.), or local variables.

Real-world experience: If a customer calls PSS with a CD burning
problem, often times a trace is generated as the customer repro’s the
problem. If the PSS folks can’t determine the problem, I get the trace
and review it. With this method, I’ve been able to resolve 99%+ of
issues without anything other than a trace.

How to start a trace in the below scenario? Many options. My favorite
is to use TraceView (in the DDK) and point it to the private symbol
file. Traceview auto-extracts the GUID(s) for tracing, and has a lot of
options and a decent UI.

For customer sites w/o symbols, you can use the cmd-line tool LOGMAN.EXE
that ships with XP Pro RTM. There has also been some discussion about
it going into a service pack (SP1? SP2?), but I am not sure what
happened there. Here’s a pointer to some docs on LOGMAN.EXE:
http://www.microsoft.com/windowsxp/home/using/productdoc/en/default.asp?
url=/WINDOWSXP/home/using/productdoc/en/ntcmds_new_tools.asp

I hope this answers some of your questions. LEVEL/FLAG use is something
I expect to go into in greater detail in a later post.
.

This posting is provided “AS IS” with no warranties, and confers no
rights.
Use of included code samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

-----Original Message-----
From: Michal Vodicka [mailto:xxxxx@veridicom.cz.nospam]
Sent: Monday, February 02, 2004 12:39 PM
Subject: RE: ETW?

I’d say the general idea is good. Tracing isn’t just simple KdPrint.
Within past years I started to use traces more and more for all
debugging for both PC and embedded development (hey, I even understand
why Linus doesn’t like debuggers :). Now I use debuggers mainly as trace
viewers and error catchers (asserts, BSOD etc). For these purposes we
developed trace library which resembles ETW in some aspects but is IMHO
much easier to use. It is set of macros enabled by compiler defines and
can be used in both debug and release builds. Every trace has assigned
an area which is logical unit as source file, class or functionality
(power management for example). Next, every trace has level (trace,
debug, warning…). If something has to be debugged, current level for
given area is changed in registry. That’s all in principle. We use
traces for both debugging on our machines and internal and field
testing. If there is a problem, QA or customer enables requested areas,
reproduces problem and send us captured traces. Usually, that’s enough.
Saves a lot of time and makes software more reliable.

The idea for separating traces from formatting is also good one.
Formatting is slow and some problems are dependent on timing. Most of
problems I had to debug in the past. Debug output can also cause
unwanted synchronization.
Later formatting also allows better filtering; sometimes it may be
necessary to see info about TID and PID, sometimes it is just useless
and wastes space. Not speaking about embedded apps where may not be
enough memory for formatted traces.

I wanted to see if ETW can be used instead of our library using the same
macros as we use now. It is important because we aren’t willing to
change thousands lines. After brief examination I don’t know. Whole
stuff seems overcomplicated and bloated. Maybe I’m lazy but I haven’t
found if above mentioned concept of areas and levels can be simulated
some reasonable way (having one GUID per area doesn’t seem reasonable).
I would spent time with converting if can presume better results that we
have with our library. From docs I read it is unclear. The only real
advantage I see are debug strings separated from binary. But I’m not
sure if the price isn’t loss of other important features (real time
output, extracting traces from crashdump etc.).

Maybe ETW is useful and maybe it is better than what we use now. Maybe.

Well, lets compare. If somebody here uses ETW for development, tell me,
please, exact way how to solve following real-life problem: I have a
testing application which communicates with a development board via USB.
Something went wrong and communication doesn’t work. Application is
built with traces so I go to registry, find key for the application
(easy) and change USB area from warning to trace. Then I start DbgView,
run app and examine traces. Now what I’d need to do when want to have
the same traces with ETW (presume application is built for ETW)?

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]


From: xxxxx@storagecraft.com[SMTP:xxxxx@storagecraft.com]
Reply To: xxxxx@lists.osr.com
Sent: Monday, February 02, 2004 7:40 PM
To: xxxxx@lists.osr.com
Subject: Re: [ntdev] ETW?

IMHO ETW is absolutely useless.

After a brief look at its description, I have noticed that
learning this over-complicated stuff just to do KdPrint is really
amazing. I can spend time learning, but some real new technologies
which will add features to the product.

As about Event Log - I have 2-3 routines to print stuff to log,
and reuse them where I feel the need to do so.

Also I cannot catch the connection between debug traces and event
log.
Debug traces are to understand what is going wrong due to my bugs.
Event Log is to write some failures due to “interesting” behaviour of
the surrounding environment. How do they intersect?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Henry Gabryjelski”
> To: “Windows System Software Devs Interest List”
> Sent: Monday, February 02, 2004 9:19 PM
> Subject: [ntdev] ETW?
>
>
> >
> > (NOTE: the first post had a small bug. Replace “LEVEL” with “FLAG”
> > in the sources file. I’ll be happy to describe level/flag
> > differences in a later post.)
> >
> > Is there interest in my posting more information about ETW? If so,
> > please respond directly to me (please - not to the list), with the
> > following:
> >
> > What would you like more info on:
> > A) how ETW works under the covers? (macros, WPP side, etc)?
> > B) how to convert to using ETW?
> > C) “best practices” using ETW?
> > D) Special format strings, etc.?
> > E) LEVEL, FLAG, and how to use both?
> > F) Tools usage? (traceview, logman, …)
> >
> > How often you’d like to see me toss something over the wall to you
guys:
> > G) Once a week
> > H) Once per two weeks
> > I) Once a month
> > J) Never. :slight_smile:
> >
> > I’m not claiming to be an expert in any of the above, but I’ve been
> > using it long enough to let you know the common problems I’ve seen,
> > common usage, and stuff that’s still not included (so you can
> > compain about it too). If there’s lots of interest, I may even bug
> > the trace guys to review my docs in the future. :slight_smile:
> >
> > Thanks,
> > .
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: michal.vodicka@st.com To
> unsubscribe send a blank email to xxxxx@lists.osr.com
>

There are tools which allow you to schedule traces to occur, including
all the time (“at boot”). Each OS revision has enabled tracing at an
earlier and earlier time during the boot process. LOGMAN.EXE (XP PRO,
Server 2003, etc) already has this capability exposed to enable the
trace. Then, you just need to enable the debugger as Rob showed below
in order to capture/display the events.

I’ve never claimed it’s perfect, but realistically only us storage folks
are really affected by the slight delay in trace events being available.
If you’d like, I can get more info on exactly when traces start working
for each OS.
.

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Michal Vodicka [mailto:xxxxx@veridicom.cz.nospam]
Sent: Monday, February 02, 2004 4:07 PM
Subject: RE: ETW?


From: xxxxx@cdp.com[SMTP:xxxxx@cdp.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:36 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

I don’t think you can use SoftIce or DbgView as the output isn’t
actually sent to the debugger, just the raw data.

Yes, it is clear. I just wondered if there isn’t something in the kernel
which would optionally format data and send to debug output.

To view it in windbg you must do the following:
Tracelog (whatever params) -kd

Then in the debugger
!load winxp\wmitrace
!searchpath c:\MyTmfFiles
!dynamicprint 1

Then you will see the output as before. I usually need to do debug
prints during boot up as my driver is a system boot driver, and there
is no way to turn on ETW for the driver until the system is completely

booted.

Really? Does it mean you always have to run an usermode app to enable
tracing? If so it is nearly useless. Trace settings as I use it has to
be persistent accross boots and available in DriverEntry().

This is
the main reason I left the old debug prints in the driver for debug
builds.
If I need the debug info for a boot in the field, I am back to sending

a debug driver to them, but this has cut down on the number I do send
significantly.

OK, thanks, I’m probably discouraged enough for next several months :wink:

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

“I just wondered if ETW can give me some advantages.”

I will eventually post some of the “advanced” features. How about a
nice hexdump feature, something like “%!HEXDUMP!” resulting in a dump of
data similar to how the debugger shows data?

.

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Michal Vodicka [mailto:xxxxx@veridicom.cz.nospam]
Sent: Monday, February 02, 2004 4:13 PM
Subject: RE: ETW?


From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 12:43 AM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

Oh, I see. Well, no, we don’t catch ETW stuff, in fact, SoftICE isn’t
even registered as a debugger.

Maybe it should catch ETW in the future. It is standard OS feature,
anyway.

But if you want to do your own tracing, you can write it in a way that

SoftICE catches it.

Which is what I do all the time. I just wondered if ETW can give me some
advantages.

Also, BoundsChecker in DS 3.1
allows you to monitor your own driver’s functions, so you can keep
tabs on the full range of events that Bchkd tracks, and those will not

only go to the Workbench GUI in the host, but also to SoftICE where
it’s reachable by the “event” command.

Sorry, I still prefer my own traces. Event command is helpful, thought.
And you know how I like your GUI apps :wink:

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

> ----------

From:
xxxxx@compuware.com[SMTP:xxxxx@compuware.com]
Reply To: xxxxx@lists.osr.com
Sent: Tuesday, February 03, 2004 4:40 PM
To: xxxxx@lists.osr.com
Subject: RE: [ntdev] ETW?

Maybe you’re right, but so far we don’t support ETW. Now, you don’t need
to
use our GUI apps to benefit from BoundsChecker, because you have the
SoftICE
“event” command. Anything that you ask bchkd.sys to catch ends up right
inside SoftICE.

Yes, I use “event” command. This is the only way how I use BC.

The other thing is, I heard you complain a lot about the DS 2.5 Workbench,
but the GUI is now quite different in DS 3.1.

You haven’t heard my complains about 3.1? :wink: Maybe GUI is different but
apparently designed by the same people. Their design is incompatible with
me. The only way how to create useful and efficient GUI is to actually use
it for daily work. I don’t believe it is the case. Instead it seems
designers never tried to debug any complicated problem or analyse real-life
crashdump. Compare with text mode SoftICE UI. The difference is enormous.

I really tried to analyse several crashdumps with 3.1. After about 15
minutes of desperation I always resigned and switched to windbg and
continued with no problem. Sorry, ignore my GUI comments. I hope there are
people who are satisfied with your tools.

Also, if you want, you can
operate the tools directly inside MSVS.NET and avoid our GUI shell
altogether !

Apage, satanas!

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]