I referred to avssamp and added an audio pin into avshws. Now on graphedit it operates normally but there is no sound output when I play it. Some difference between avshws and avssamp are some advanced audio pin properties, and the clock status when playing. I just suspect that I might not be setting the time of the audio pin correctly. However, I actually have no idea what the real problem is. Could anyone suggest some topics or directions that I should look into?
current playback situation:
current audio properties:
following is the audio process:
NTSTATUS
CAudioCapturePin::
AVStrMiniPinProcess(
In PKSPIN Pin
)
{
DBG_ENTER("(DeviceState=%d)", m_Pin->DeviceState);
NT_ASSERT(Pin == m_Pin);
// PLIST_ENTRY ListEntry;
PKSSTREAM_POINTER StreamPointer;
PKSSTREAM_HEADER StreamHeader;
PUCHAR DataPointer;
ULONG BytesToProcess;
NTSTATUS Status = STATUS_SUCCESS;
StreamPointer = KsPinGetLeadingEdgeStreamPointer (
Pin,
KSSTREAM_POINTER_STATE_LOCKED
);
while (NT_SUCCESS (Status) && StreamPointer) {
//
// If no data is present in the Leading edge stream pointer, just
// move on to the next frame
//
if ( NULL == StreamPointer -> StreamHeader -> Data ) {
Status = KsStreamPointerAdvance(StreamPointer);
continue;
}
StreamHeader = StreamPointer->StreamHeader;
// refer to video
BytesToProcess = StreamPointer->Offset->Remaining;
DataPointer = (PUCHAR)StreamHeader->Data;
DBG_TRACE("BytesToProcess=%u", BytesToProcess);
if (BytesToProcess > 0) {
ULONG BytesUsed = m_WaveObject -> SynthesizeFixed (
// TimerInterval,
10000000, // 1 sec
(PVOID)DataPointer,
BytesToProcess
);
DBG_TRACE("BytesUsed=%u", BytesUsed);
if (m_Clock){
StreamHeader->PresentationTime.Time = m_Clock->GetTime();
DBG_TRACE("PresentationTime=%I64d", m_Clock->GetTime());
StreamHeader->PresentationTime.Numerator =
StreamHeader->PresentationTime.Denominator = 1;
StreamHeader->OptionsFlags |=
KSSTREAM_HEADER_OPTIONSF_TIMEVALID;
}
}
Status = KsStreamPointerAdvance(StreamPointer);
if (Status != STATUS_SUCCESS) {
break;
}
} // end while (TRUE)
// refer to video
//
// If we didn't run the leading edge off the end of the queue, unlock it.
//
if (NT_SUCCESS (Status) && StreamPointer) {
KsStreamPointerUnlock (StreamPointer, FALSE);
} else {
//
// DEVICE_NOT_READY indicates that the advancement ran off the end
// of the queue. We couldn't lock the leading edge.
//
if (Status == STATUS_DEVICE_NOT_READY) Status = STATUS_SUCCESS;
}
//
// If we failed with something that requires pending, set the pending I/O
// flag so we know we need to start it again in a completion DPC.
//
if (!NT_SUCCESS (Status) || Status == STATUS_PENDING) {
m_PendIo = TRUE;
}
DBG_LEAVE("()=0x%08X", Status);
return Status;
}
Why did you choose to do it this way, instead of just starting from avssamp? WIth avshws, you have two unrelated pins.
You’re telling SynthesizeFixed to advance the syntheses one full second each time, even though the buffers you’ll be getting for audio are quite small. Have you done debugging to see how large your audio buffers are?
Hi,
My next task is to develop a video capture card driver. The data source comes from hardware, and the pin-centric/avshws seems to be the suitable sample. However, avshws only demonstrates the video part, while my work needs to handle both video and audio.
Passing one second to SynthesizeFixed is just a temporary approach. I modified its internal logic so that it copies audio data based on BytesToProcess and recalculates the time progression accordingly. Please refer to the attached log.
The reason I implemented it this way is that I have no clear idea how much audio data should be filled for each StreamPointer. You mentioned that this is because the two pins are unrelated, so regarding synchronization between the two pins, which topic should I study further? (The data source will be HDMI, which includes both video and audio.) Thanks in advance for any suggestions.
The following is a section of windbg log. CCapturePin is the video pin of avshws. CBasePin is the original CCapturePin of avssamp, it is inherited by video and audio pin class in avssamp. CAudioCapturePin is the audio pin from avssamp. The audio format is 44.1 KHz, 24 bit, stereo.
10:17:59.908 WDM_Driver: Entering: CCapturePin::SetState(0->1)
10:17:59.908 WDM_Driver: Leaving: CCapturePin::SetState()=0x00000000
10:17:59.909 WDM_Driver: Entering: CBasePin::SetState(0->1)
10:17:59.909 WDM_Driver: Entering: CAudioCapturePin::Acquire()
10:17:59.909 WDM_Driver: Leaving: CAudioCapturePin::Acquire()=0x00000000
10:17:59.910 WDM_Driver: Leaving: CBasePin::SetState()=0x00000000
10:17:59.910 WDM_Driver: Entering: CCapturePin::SetState(1->2)
10:17:59.910 WDM_Driver: Leaving: CCapturePin::SetState()=0x00000000
10:17:59.912 WDM_Driver: Entering: CBasePin::SetState(1->2)
10:17:59.913 WDM_Driver: Leaving: CBasePin::SetState()=0x00000000
10:17:59.914 WDM_Driver: Entering: CCapturePin::SetState(2->3)
10:17:59.914 WDM_Driver: Leaving: CCapturePin::SetState()=0x00000000
10:17:59.915 WDM_Driver: Entering: CCapturePin::Process(DeviceState=2)
10:17:59.915 WDM_Driver: Leaving: CCapturePin::Process()=0x00000000
10:17:59.916 WDM_Driver: Entering: CBasePin::SetState(2->3)
10:17:59.916 WDM_Driver: Leaving: CBasePin::SetState()=0x00000000
10:17:59.917 WDM_Driver: Entering: CAudioCapturePin::AVStrMiniPinProcess(DeviceState=2)
10:17:59.917 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.917 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=2666666
10:17:59.917 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.918 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=-58971
10:17:59.918 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.918 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=5333332
10:17:59.918 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.919 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=-42947
10:17:59.919 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.920 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=7999998
10:17:59.920 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.920 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=-32413
10:17:59.920 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.921 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=10666664
10:17:59.921 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.921 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=-23848
10:17:59.921 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.922 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=13333330
10:17:59.922 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.923 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=-989
10:17:59.924 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.924 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=15999996
10:17:59.924 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.924 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=8495
10:17:59.924 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.925 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=18666662
10:17:59.925 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.925 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=17893
10:17:59.925 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.926 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=21333328
10:17:59.926 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.926 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=27804
10:17:59.926 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.927 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=23999994
10:17:59.928 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.928 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=46691
10:17:59.928 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.929 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=26666660
10:17:59.929 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.929 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=57787
10:17:59.929 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.930 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=29333326
10:17:59.930 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.930 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=70752
10:17:59.931 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.931 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=31999992
10:17:59.931 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.932 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=82876
10:17:59.933 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.933 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=34666658
10:17:59.933 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.934 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=101454
10:17:59.934 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.934 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=37333324
10:17:59.934 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.935 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=111789
10:17:59.935 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.935 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=39999990
10:17:59.935 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.936 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=122350
10:17:59.936 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.936 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=42666656
10:17:59.936 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.937 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=132705
10:17:59.938 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.938 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=45333322
10:17:59.938 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.938 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=148401
10:17:59.939 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.939 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=47999988
10:17:59.939 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.939 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=159367
10:17:59.940 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.940 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=50666654
10:17:59.940 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.941 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=171115
10:17:59.941 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.941 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=53333320
10:17:59.941 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.941 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=180588
10:17:59.942 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.943 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=55999986
10:17:59.943 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.943 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=198070
10:17:59.944 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.944 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=58666652
10:17:59.944 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.944 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=209354
10:17:59.945 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.945 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=61333318
10:17:59.945 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.945 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=220206
10:17:59.946 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.946 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=63999984
10:17:59.946 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.946 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=229935
10:17:59.949 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesToProcess=70560
10:17:59.950 WDM_Driver: In: CWaveObject::SynthesizeFixed m_SynthesisTime=66666650
10:17:59.950 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess BytesUsed=70560
10:17:59.950 WDM_Driver: In: CAudioCapturePin::AVStrMiniPinProcess PresentationTime=267195
10:17:59.950 WDM_Driver: Leaving: CAudioCapturePin::AVStrMiniPinProcess()=0x00000000
10:17:59.971 WDM_Driver: Entering: SimulatedInterrupt()
10:17:59.972 WDM_Driver: Leaving: SimulatedInterrupt()
10:17:59.975 WDM_Driver: Entering: CCapturePin::Process(DeviceState=3)
10:17:59.975 WDM_Driver: Leaving: CCapturePin::Process()=0x00000000
10:17:59.982 WDM_Driver: Entering: SimulatedInterrupt()
10:17:59.983 WDM_Driver: Leaving: SimulatedInterrupt()
10:17:59.985 WDM_Driver: Entering: CCapturePin::Process(DeviceState=3)
10:17:59.985 WDM_Driver: Leaving: CCapturePin::Process()=0x00000000
10:18:00.016 WDM_Driver: Entering: SimulatedInterrupt()
10:18:00.017 WDM_Driver: Leaving: SimulatedInterrupt()
...
If this is coming from a device capturing audio and video together, like a camera with microphone, then you WANT the two pins to be related. You want a filter-centric driver, where all the pins get filled together. That’s the only way to achieve lip sync.
I have no clear idea how much audio data should be filled for each StreamPointer
You’re simulating hardware here. If 16ms have elapsed since the last send, then you send 16ms worth of audio data. In your case, with 264,600 bytes per second, that would be 4,233 bytes.
You want a filter-centric driver, where all the pins get filled together. That’s the only way to achieve lip sync.
That’s really valuable information! Thanks a lot!
When processing video, I just need to fill the data for each frame. But for audio, I’m not sure if there’s a similar concept of a “frame”. So does that mean I should precisely calculate the elapsed time and then fill in the correct amount of data accordingly?
In my current practice, I just want to play back the entire audio file correctly. Can I simply fill the buffer sequentially with the full audio content without worrying about the elapsed time?
Sorry for asking so many questions — should I read any specific topics in the Hardware Developer Documentation to better understand this part?
Technically, a “frame” in the audio sense is one sample – 6 bytes for you. You just need to fill the buffers with whatever data you have. I thought the avssamp sample used the elapsed time to decide how much data to produce. The clock in your graphedt image means that the audio pin has been selected as the “master clock” for the graph, because audio tends to be continuous and well-regulated.
Audio pin works correctly now, thank you!
I originally thought the issue was related to synchronization or clock settings. But after your reminder that I just need to properly fill the buffer, I focused on checking the values of StreamHeader. In the end, I found that DataUsed wasn’t being set, because in video pin, it didn't do it in the process routine but is deferred to CompleteMappings.
