AVStream multi filter

Hi,
I have a requirement to support multiple filters that can be connected from different applications and grab the same video (the same sensor).
please don’t ask why, and what that good for.

In the first step I extended the AVSHWS example to support two filters. each filter creates its Pin. One of them (first Pin) is chosen to be the "Real?- it?s connected to the sensor and gets DMA.
All other Pin (could be connected to different process) is ?dummy? ? gets frames from the first one by copy operation in DPC time of the ?Real? Pin. The ?dummy? pin has not any ISR, but only ?real? pin has.
please ignore all possible sync and PNP problems.
Since the filters are ?separate instances? of the driver, I use the linked list of ready frames pointers (comes from pin->Proccess()) in the Device class that is global for them.
In the DPC routine (of “real” pin) I copy the frame from the Real Pin frame to the dummy pin frame pointer. In such way the same frame is shown separately in two separate renderers.
Result:
When I create two filters and render two same frames in the same GraphEdit , I have no problem.
When I open two separate GraphEdits and create filter in each graph , I face strange situation:
The first pin is working properly , the second Pin > CCapturePin::Process () isn?t.
The frame pointer : Leading->StreamHeader->Data , ClonePointer->StreamHeader->Data address isn’t valid. I remind that it is received by KsPinGetLeadingEdgeStreamPointer , KsStreamPointerAdvance. The pointer itself is valid (it is not NULL) but when I try copy to this address I get BSOD. I gess the pointer points to address that isn’t allocated by KSProxy.
The interesting & strange thing is that this bad address appearance isn?t permanent. Sometimes the address is OK and copy operation success and sometimes it is bad.
I remind that this behaivior happen only when I use two separate graphs (each filer in separate procces).

I will thank you a lot if you have any idea.

xxxxx@hotmail.com wrote:

I have a requirement to support multiple filters that can be connected from different applications and grab the same video (the same sensor).
please don’t ask why, and what that good for.

That phrase is one that really ticks me off. You’re asking us for free
technical advice – the kind of advice most of us charge real money for
in our work. We have every right to ask “why”. We have repeatedly,
over the years, proven why it is beneficial for you to TELL us why.
Very often, people come in here with off-the-wall questions on some
strange solution they have concocted, when in fact the real problem they
were trying to solve had a much easier solution. Without knowing what
led to your question, we cannot possibly offer you reasonable advice.

In the first step I extended the AVSHWS example to support two filters. each filter creates its Pin. One of them (first Pin) is chosen to be the "Real?- it?s connected to the sensor and gets DMA.
All other Pin (could be connected to different process) is ?dummy? ? gets frames from the first one by copy operation in DPC time of the ?Real? Pin. The ?dummy? pin has not any ISR, but only ?real? pin has.

That’s seems unnecessarily complicated. I think I would have put all of
the DMA processing in the Device class, so that ALL of the pins were
just equal clients of the device. That way, the code is much more
symmetric. No real/dummy distinctions.

please ignore all possible sync and PNP problems.

Why not. Most posters do.

Since the filters are ?separate instances? of the driver, I use the linked list of ready frames pointers (comes from pin->Proccess()) in the Device class that is global for them.
In the DPC routine (of “real” pin) I copy the frame from the Real Pin frame to the dummy pin frame pointer. In such way the same frame is shown separately in two separate renderers.
Result:
When I create two filters and render two same frames in the same GraphEdit , I have no problem.
When I open two separate GraphEdits and create filter in each graph , I face strange situation:

Remember that you are crossing process boundaries here. That is, your
frame pointers refer to addresses in two different processes. That’s
almost certainly related here.

The first pin is working properly , the second Pin > CCapturePin::Process () isn?t.
The frame pointer : Leading->StreamHeader->Data , ClonePointer->StreamHeader->Data address isn’t valid.

Are you absolutely sure that you are locking the leading edge when you
fetch it and make your clones? If you don’t lock the leading edge, then
the kernel addresses won’t necessarily be valid from another process
context.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Tim thank you,

> We have every right to ask “why”.
You absolutely right. Sometimes we write thing coming to the mind fast and without deep consideration! Sorry.
Probably I wrote it only because I see sometimes unconstructive critics when things are not like it is accustomed. This critic of course could be right but as one who reads many years this from the side I see that mostly this critic is coming from fixation without any detailed discussion. Some people can?t accept unusual things or ideas. Reasonable critics are welcome when it is first well considered.
In addition, in the current case I just want to set focus to this approach and do not take this thread to other directions.
But basically I agree with every word you sad!

Actually I seriously examine the requirement, of multiplying one video to many renderers. I checked some other direction, like SplitCam , manycam and some other ideas I have , and found also limitations (for me) in each one. I decided to check splitting frames in kernel.
The avstream splitter is good only for one process . what I need is to split to several processes.
Since I?m new with AVStream I started some experiments. I deal with proof of concept this moment ,this why I put the sync & PNP issue outside this moment ? it is serious issue but noisy details this moment and out of pos focus.

You write :

> pointers refer to addresses in two different processes.
Indeed it is , but why this could be problem ? I see the Device class as Device Extension where I can put global things !
The Flow is :
In CCapturePin::Process () the call to KsPinGetLeadingEdgeStreamPointer / KsStreamPointerAdvance
Gets the pointer then KsStreamPointerClone clones …

The ClonePointer -> StreamHeader -> Data pointer moved to ProgramScatterGatherMappings()
Where the pointer inserted by the InsertTailList to LIST_ENTRY. The list is in Device class,
So I expect it should be OK.

Are you absolutely sure that you are locking the leading edge when you
fetch it and make your clones?
Please correct me if I miss something but as I understand KsPinGetLeadingEdgeStreamPointer locks it ! and Once the address is locked in kerenl I can use it even in different context.

May be the problem that the unlock is done in the end of the
CCapturePin::Process ()
{
?
…
…
if (NT_SUCCESS (Status) && Leading)
{
KsStreamPointerUnlock (Leading, FALSE);
}
else
{
if (Status == STATUS_DEVICE_NOT_READY) Status = STATUS_SUCCESS;
}
return Status;
}

Thank you in advance

xxxxx@hotmail.com wrote:

You write :
>> pointers refer to addresses in two different processes.
Indeed it is , but why this could be problem ? I see the Device class as Device Extension where I can put global things !

Well, yes, but you’re working with user-mode addresses here, and (for
example) a user-mode virtual address is only valid when you are in that
process’ context. You can save a virtual address from process 1 in your
device extension, but if you later try to use that address when process
2 is current, that’s disaster.

Now, to a certain extent, my comments here are silly, because the
AVStream framework has actually converted the addresses to kernel
addresses, and those are global. However, I think you give a clue to
your problem at the end:

> Are you absolutely sure that you are locking the leading edge when you fetch it and make your clones?
Please correct me if I miss something but as I understand KsPinGetLeadingEdgeStreamPointer locks it ! and Once the address is locked in kerenl I can use it even in different context.

Yes, but…

May be the problem that the unlock is done in the end of the
CCapturePin::Process ()

Absolutely. Think of it this way. Locking the leading edge locks the
pages in memory and returns to you a kernel address that maps to the
user-mode buffer. Unlocking the leading edge releases that lock, so the
pages can be paged out and might reappear at a different physical
address. Your kernel address is then broken.

You must keep the stream pointer locked until you no longer need the
buffer. As long as you need to access it, it must remain locked.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Hi Tim , Thank you,

converted the addresses to kernel addresses, and those are global.

exactly , this what i meant but didn’t express enough.

You must keep the stream pointer locked until you no longer need the buffer. As long as you need to access it, it must remain locked.

But, this is the AVSHWS example proccess function , I didn’t change this function.
Also I don’t understand why this is OK when I run the same code of two filters from same proccess. I remind that the problem only when I use two different graphEdits .

I think the Cloning should increment the reference counter, so the unlocking of leading pointer is OK.
The address still resident and locked because the clone exist. the real unlock will happen when
while (MappingsRemaining && Clone)
{
PKSSTREAM_POINTER NextClone = KsStreamPointerGetNextClone (Clone);
…
KsStreamPointerDelete (Clone);
}

Otherwise a buffer will not be released and a given IRP will not be completed…
do I think right ?

If so this means that i most probably have any syncronization problem than any architecture one.
thank you in advance

here is the proccess method :

NTSTATUS CCapturePin::Process ()
{
PAGED_CODE();

NTSTATUS Status = STATUS_SUCCESS;
PKSSTREAM_POINTER Leading;
Leading = KsPinGetLeadingEdgeStreamPointer (m_Pin,KSSTREAM_POINTER_STATE_LOCKED);

while (NT_SUCCESS (Status) && Leading)
{
PKSSTREAM_POINTER ClonePointer;
PSTREAM_POINTER_CONTEXT SPContext;

if ( NULL == Leading -> StreamHeader -> Data )
{
Status = KsStreamPointerAdvance(Leading);
continue;
}

if (!m_PreviousStreamPointer)
{
Status = KsStreamPointerClone (Leading,NULL,sizeof(STREAM_POINTER_CONTEXT),&ClonePointer);
ClonePointer->OffsetOut.Remaining,ClonePointer->OffsetOut.Count,ClonePointer->Context));

if (NT_SUCCESS (Status))
{
ClonePointer -> StreamHeader -> DataUsed = 0;

SPContext = reinterpret_cast <pstream_pointer_context> (ClonePointer -> Context);

SPContext -> BufferVirtual =
reinterpret_cast (ClonePointer -> StreamHeader -> Data);
}
}
else
{
ClonePointer = m_PreviousStreamPointer;
SPContext = reinterpret_cast <pstream_pointer_context> (ClonePointer -> Context);
Status = STATUS_SUCCESS;
}

if (!NT_SUCCESS (Status))
{
KsStreamPointerUnlock (Leading, FALSE);
break;
}
SPContext -> BufferVirtual,Leading -> OffsetOut.Remaining));

ULONG MappingsUsed = m_Device->ProgramScatterGatherMappings (&(SPContext -> BufferVirtual),
Leading -> OffsetOut.Mappings,
Leading -> OffsetOut.Remaining,
m_CapPinID);

if (MappingsUsed == Leading->OffsetOut.Remaining)
{
m_PreviousStreamPointer = NULL;
}
else
{
m_PreviousStreamPointer = ClonePointer;
}

if (MappingsUsed)
{
Status = KsStreamPointerAdvanceOffsets (Leading,0,MappingsUsed,FALSE);
}
else
{
Status = STATUS_PENDING;
break;
}

} // while

if (!Leading)
{
m_PendIo = TRUE;
Status = STATUS_PENDING;
}

// If we didn’t run the leading edge off the end of the queue, unlock it.
if (NT_SUCCESS (Status) && Leading)
KsStreamPointerUnlock (Leading, FALSE);
else
if (Status == STATUS_DEVICE_NOT_READY) Status = STATUS_SUCCESS;

if (!NT_SUCCESS (Status) || Status == STATUS_PENDING)
m_PendIo = TRUE;

return Status;
}</pstream_pointer_context></pstream_pointer_context>

xxxxx@hotmail.com wrote:

> You must keep the stream pointer locked until you no longer need the buffer. As long as you need to access it, it must remain locked.
But, this is the AVSHWS example proccess function , I didn’t change this function.
Also I don’t understand why this is OK when I run the same code of two filters from same proccess. I remind that the problem only when I use two different graphEdits .

Well, when you have two filters in a single process, all of the
addresses you get are in the “current” process. There isn’t much
opportunity for the pages to be swapped out as there is when you’re
bouncing back and forth between multiple processes.

I think the Cloning should increment the reference counter, so the unlocking of leading pointer is OK.
The address still resident and locked because the clone exist. the real unlock will happen when
while (MappingsRemaining && Clone)
{
PKSSTREAM_POINTER NextClone = KsStreamPointerGetNextClone (Clone);
…
KsStreamPointerDelete (Clone);
}

No. There’s one very important point you are missing – the reference
count is not related to locking. The reference count decides when the
IRP can be completed and sent back to user mode. Locking decides
whether the pages are locked into memory and can be used for DMA. You
can have a referenced pointer that is unlocked.

I think the avshws code is wrong. It works because it doesn’t really
have any DMA hardware – no one cares whether the memory is really
locked. There is no separate “lock” bit inside a KSSTREAM_POINTER. The
clone inherits the lock state from the original pointer. Both pointers
point to the same memory. When you unlock the original pointer, that
unlocks the memory, whether or not there are other references.

I think you need to remove the KsStreamPointerUnlock call in
CCapturePin::Process. It will be unlocked when the clone is eventually
deleted.

Otherwise a buffer will not be released and a given IRP will not be completed…
do I think right ?

You’re thinking about the reference count right, but that’s completely
unrelated to locking.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Tim Roberts wrote:

I think the avshws code is wrong. …

I think you need to remove the KsStreamPointerUnlock call in
CCapturePin::Process. It will be unlocked when the clone is eventually
deleted.

For what it’s worth, I just checked the code in my biggest AVStream
capture driver. I never unlock the leading edge once I make a clone.
The pointer remains locked until the last clone is deleted.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

> I never unlock the leading edge once I make a clone.

The pointer remains locked until the last clone is deleted.

If I understand you right , I never need to do KsStreamPointerUnlock in my case.
When I delete the clone by KsStreamPointerDelete (Clone); the leading will be unlocked.
If so, I don;t understand why I need clones at all ?
Anyway , I will check the corrected code after weekend.

Thnak you very much

xxxxx@hotmail.com wrote:

If I understand you right , I never need to do KsStreamPointerUnlock in my case.
When I delete the clone by KsStreamPointerDelete (Clone); the leading will be unlocked.

My big AVStream capture driver, for a series of USB web cams, contains
zero calls to KsStreamPointerUnlock.

If so, I don;t understand why I need clones at all ?
Anyway , I will check the corrected code after weekend.

Probably because you need multiple buffers queued up. You need to grab
a clone so you can advance the edge and get the next buffer.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Tim,

the problem is still exist even after I got rid of KsStreamPointerUnlock .
unfortunately, I must switch to another task for a while, and I will update this thread when I back to it and solve it.

Thank you very much for your help, anyway it was helpful to disscuss with you.
E.E