Reading redirected stdout...

Hi all.

I have encountered a problem when reading from a redirected stdout…

Here’s the setup:
My app launches a console application. Before calling CreateProcess, it
creates an anonymous pipe, and uses STARTUPINFO’s hStdOutput, hStdInput and
hStdError to redirect console app’s std handles. After that it uses ReadFile
to read from stdout (all of this is done according to “Creating a Child
Process with Redirected Input and Output” article from MSDN docs)

The problem is that writing to stdout is buffered, and ReadFile blocks until
either the console app exits or fflush(stdout) is called in the console app,
and I need to read data as it’s being written to stdout.
Since I don’t have access to the source code of the console apps that have to
be started, I can’t use setvbuf to set non buffered mode for stdout.

Any help would be greatly appreciated…

Marko
ICQ: 5990814

Coach: How’s a beer sound, Norm?
Norm: I dunno. I usually finish them before they get a word in.
– Cheers, Fortune and Men’s Weights

> Since I don’t have access to the source code of the console apps that have
to

be started, I can’t use setvbuf to set non buffered mode for stdout.

MS C runtime library comes with source.

Max

“Maxim S. Shatskih” wrote:

> Since I don’t have access to the source code of the console apps that have
to
> be started, I can’t use setvbuf to set non buffered mode for stdout.

MS C runtime library comes with source.

I know that, but I don’t see how that can help…

The idea behind all of this is that my app after starting a console app waits
for a string on the console, and when it appears (before a timeout), it
concludes that the console app has finished with initialization, and then my
app moves on…


Marko
ICQ: 5990814

I’d like to meet the guy who invented beer and see what he’s working on now.

> > > Since I don’t have access to the source code of the console

> > apps that have to be started, I can’t use setvbuf to set
> > non buffered mode for stdout.

Here is an idea:

Use detours (http://research.microsoft.com/sn/detours/) and write some
detours for the C runtime APIs and whatever other APIs you think the app may
be using to do this buffering. That will hopefully allow you to figure out
what API the app is using (whether it’s msvcrt.dll or whatever). Once you
do that, intercept once of the calls and use that to set the output to
non-buffered within that API. HTH.

  • Danilo

> The idea behind all of this is that my app after starting a console app
waits

for a string on the console, and when it appears (before a timeout), it
concludes that the console app has finished with initialization, and then
my
app moves on…

Use C runtime library functions to redirect the parent app’s stdin to the
pipe
created by pipe(), then run the child app with this pipe as stdout, then
wait for
the string.
This is done practically in the same way as in UNIX - by using C runtime
(not
Win32!) functions.

Max

From: “Maxim S. Shatskih”
Sent: Sunday, March 19, 2000 11:53 PM

> > The idea behind all of this is that my app after starting a console app
> waits
> > for a string on the console, and when it appears (before a timeout), it
> > concludes that the console app has finished with initialization, and
then
> my
> > app moves on…
>
> Use C runtime library functions to redirect the parent app’s stdin to the
> pipe
> created by pipe(), then run the child app with this pipe as stdout, then
> wait for
> the string.
> This is done practically in the same way as in UNIX - by using C runtime
> (not
> Win32!) functions.

However, this is a bit difficult if your C runtime does not support pipe()
(is this even an ANSI call)?

While it would be “better” to use the C runtime, any program can use the
Win32 functions to accomplish the same thing. Namely, by creating a pipe or
pipes with the Win32 function CreatePipe() and setting the handles in the
appropriate members of STARTUPINFO before calling CreateProcess(), the
standard I/O of any child process can be redirected (regardless of which C
runtime it uses, BTW). This works even if the child is a 16-bit DOS app.

If fact, there was article a few years ago an an issue of “Windows
Developer’s Journal” that illustrated how to do exactly this.
Unfortunately, I can’t seem to locate it online at http://www.wdj.com at the
moment (it may no longer be archived) and I don’t have the issue handy. The
title was something like “Two Methods for Capturing Win32 Console Output”.

Regards,

Matt Arnold
Professional Music Products
Mark of the Unicorn, Inc.
http://www.motu.com

----- Original Message -----
From: Matt Arnold
To: NT Developers Interest List
Sent: Tuesday, March 21, 2000 12:45 AM
Subject: [ntdev] Re: Reading redirected stdout…
> runtime it uses, BTW). This works even if the child is a 16-bit DOS app.
>
> If fact, there was article a few years ago an an issue of “Windows
> Developer’s Journal” that illustrated how to do exactly this.

It has come in WDJ May '97. In addition to above, u have to set the dwflags
= STARTF_USESTDHANDLES.
Alternatively, U can call GetStdHandle(STD_INPUT_HANDLE) and use ReadFile( )
on the above handle to read the data.
Also, anybody in the group know abt any Win95 driver user groups?

> The title was something like “Two Methods for Capturing Win32 Console
Output”.
>
>
> Regards,
>
> Matt Arnold
> Professional Music Products
> Mark of the Unicorn, Inc.
> http://www.motu.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@cmcltd.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)

> However, this is a bit difficult if your C runtime does not support pipe()

(is this even an ANSI call)?

MSVCRT supports pipe().

While it would be “better” to use the C runtime, any program *can* use the
Win32 functions to accomplish the same thing. Namely, by creating a pipe
or
pipes with the Win32 function CreatePipe() and setting the handles in the
appropriate members of STARTUPINFO before calling CreateProcess(), the
standard I/O of any child process can be redirected (regardless of which C
runtime it uses, BTW). This works even if the child is a 16-bit DOS app.

Yes. But you will need to call far more complex functions with lots of
usually-
unneccessary-and-set-to-NULL parameters. CRT way is much more elegant.

I used this approach to run a command-line app from the webserver (from
the COM DLL called by ASP), to pipe data to app’s stdion & to read the
app’s stdout from my code.
Note that for NT service, stdin/stdout/stderr are usually closed - not
opened
at all. You can open them if you want.

Max

From: “Bhaava Chaitanya Kancherla”
Sent: Tuesday, March 21, 2000 1:57 AM

> From: Matt Arnold
> Sent: Tuesday, March 21, 2000 12:45 AM
> > runtime it uses, BTW). This works even if the child is a 16-bit DOS
app.
> >
> > If fact, there was article a few years ago an an issue of “Windows
> > Developer’s Journal” that illustrated how to do exactly this.
>
> It has come in WDJ May '97. In addition to above, u have to set the
dwflags
> = STARTF_USESTDHANDLES.

Thanks, that’s correct. Athough I thought any competent programmer would
figure out the required flag after checking the STARTUPINFO documentation.
:wink: But, if we’re going to nit-pick, then let me also point out that the
handles supplied in the STARTUPINFO struct must created with the inhertible
attribute (otherwise the child process won’t be able to use them).

> Alternatively, U can call GetStdHandle(STD_INPUT_HANDLE) and use
eadFile( )
> on the above handle to read the data.

But GetStdHandle() doesn’t allow you to redirect the I/O of a child
process, which is what passing handles in STARTUPINFO (with the
STARTF_USESTDHANDLES flag) is all about. GetStdHandle() simply returns the
handles being used by the calling process for standard I/O.

> Also, anybody in the group know abt any Win95 driver user groups?

A good starting place for Windows driver programming resources on the Web
in general is http://www.chsw.com/ddk/. You’ll find information about a DDK
mailing list (including VxD topics at http://www.chsw.com/DDK-L/.

- Matt

KB article Q190351 covers how to do this in detail and supplies source.

-----Original Message-----
From: Matt A. [mailto:xxxxx@motu.com]
Sent: Tuesday, March 21, 2000 10:38 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Reading redirected stdout…

From: “Bhaava Chaitanya Kancherla”
Sent: Tuesday, March 21, 2000 1:57 AM

> From: Matt Arnold
> Sent: Tuesday, March 21, 2000 12:45 AM
> > runtime it uses, BTW). This works even if the child is a 16-bit DOS
app.
> >
> > If fact, there was article a few years ago an an issue of “Windows
> > Developer’s Journal” that illustrated how to do exactly this.
>
> It has come in WDJ May '97. In addition to above, u have to set the
dwflags
> = STARTF_USESTDHANDLES.

Thanks, that’s correct. Athough I thought any competent programmer would
figure out the required flag after checking the STARTUPINFO documentation.
:wink: But, if we’re going to nit-pick, then let me also point out that the
handles supplied in the STARTUPINFO struct must created with the inhertible
attribute (otherwise the child process won’t be able to use them).

> Alternatively, U can call GetStdHandle(STD_INPUT_HANDLE) and use
eadFile( )
> on the above handle to read the data.

But GetStdHandle() doesn’t allow you to redirect the I/O of a child
process, which is what passing handles in STARTUPINFO (with the
STARTF_USESTDHANDLES flag) is all about. GetStdHandle() simply returns the
handles being used by the calling process for standard I/O.

> Also, anybody in the group know abt any Win95 driver user groups?

A good starting place for Windows driver programming resources on the Web
in general is http://www.chsw.com/ddk/. You’ll find information about a DDK
mailing list (including VxD topics at http://www.chsw.com/DDK-L/.

- Matt


You are currently subscribed to ntdev as: xxxxx@microsoft.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

Just for the record the MSVCRT version of pipe (source is included with VC)
simply:
builds a NULL security descriptor
Calls CreatePipe
converts Win2k handles to CRT handles

about a page of code, of which about 60-70% is the handle converstion. The
rest is really quite trivial.

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Monday, March 20, 2000 10:54 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Reading redirected stdout…

However, this is a bit difficult if your C runtime does not support pipe()
(is this even an ANSI call)?

MSVCRT supports pipe().

While it would be “better” to use the C runtime, any program *can* use the
Win32 functions to accomplish the same thing. Namely, by creating a pipe
or
pipes with the Win32 function CreatePipe() and setting the handles in the
appropriate members of STARTUPINFO before calling CreateProcess(), the
standard I/O of any child process can be redirected (regardless of which C
runtime it uses, BTW). This works even if the child is a 16-bit DOS app.

Yes. But you will need to call far more complex functions with lots of
usually-
unneccessary-and-set-to-NULL parameters. CRT way is much more elegant.

I used this approach to run a command-line app from the webserver (from
the COM DLL called by ASP), to pipe data to app’s stdion & to read the
app’s stdout from my code.
Note that for NT service, stdin/stdout/stderr are usually closed - not
opened
at all. You can open them if you want.

Max


You are currently subscribed to ntdev as: xxxxx@microsoft.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)