AVStream: avssamp question

I am experimenting with the avstream samples. I have compiled and installed both avshws and avssamp (video only use-case). Both work as expected in Graphedit , however I just cant figure out why -- avssamp refuses to load in any other application besides graphedit.

avshws on the other hand, does work as expected everywhere. I presume this must be a difference between "pin centric" vs "filter centric" processing, but I am confused what exactly is missing to make avssamp load in other apps.

Looking at DbgView, the IntersectHandler gets called, succeeds,... and then nothing happens. (whereas in Graphedit, DispatchSetFormat correctly gets called, followed by DispatchSetState, and video starts).

Any tips?