Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Network data transfers to/from user mode

Mark_HolidayMark_Holiday Member Posts: 27

I am a fairly experienced Windows device driver developer (NDIS, PCI, USB, WDM, KMDF) but I have been away from any heavy-duty work for a few years, and I am now involved in a new project. We are currently in the requirements/design phase and I wanted to run some things by this group - lest I start diverging down a not-so-useful path! - as this forum has proved invaluable in the past. Also, there may have been some advancements in NDIS & WDF in recent years of which I may not be aware, of which I may want to take advantage.

Currently there is a proprietary protocol/data controller software module (let's call this PDC) that interfaces directly with hardware for sending/receiving packets out on network. There is an antiquated GUI application that interfaces to this PDC via socket interface over TCP/IP. The need exists to extend the functionality by implementing a virtual NIC (VNIC) that can be used by that and other network applications, and that also provides the ability to bridge different network interfaces. Network packets from/to that VNIC would then be routed to/from the PDC.

The problem is that the PDC is currently a user-mode application, and we would like to keep that in user-mode as much as possible.

Looking at similar solutions for ideas, e.g. OpenVPN and TAP driver, it seems that one way that this is accomplished is for the user mode process to send down read & write requests that are "pended", and completed once kernel component can complete them. So when the VNIC has packets to transmit, it would copy these to previously-received buffers from the PDC to complete the send request. Similarly, when the PDC received data, it would need to indicate those to VNIC (somehow?) to be returned to GUI application.

Is this the most feasible scheme for this? It seems overly-complicated for what needs to be accomplished. In essence: what is the best way to move network-type data back and forth from kernel and user mode?

Thanks in advance!

«1

Comments

  • Mark_HolidayMark_Holiday Member Posts: 27

    I think I need to simplify my problem:

    Given a virtual network adapter, I am looking for the best (i.e. simplest) way to send network packets to a user-mode component for further processing.

    Should the user-mode component send down I/O requests (IRPs) that are pended for later processing?
    For receive packet processing, how does the user mode component then indicate the data back to the network adapter?

    Secondarily, is Windows Filter Platform (WFP) useful for this? (I am unfamiliar with using that.)

  • MBond2MBond2 Member Posts: 210

    Is the traffic latency or jitter sensitive? if not, then thunk to UM to your hearts content. The standard inverted call method using IOCP or the thread pool APIs will get the data into UM. and to get it back, define a separate IOCTL (or use ReadFile + WriteFile with the in box ones)

  • Pavel_APavel_A Member Posts: 2,746

    The simplest way to send network packets from usermode? Just open a socket and send your stuff?
    May be for a driver person this seems unusual... as the saying goes, everything looks like a nail to someone who's got a hammer.
    For a driver person, any I/O looks like IRPs or ioctl at best ;)
    -- pa

  • Mark_HolidayMark_Holiday Member Posts: 27

    Thanks Pavel. But I guess I was not clear in my description: the issue is for processing AFTER it has been received on the socket. It needs to go through a user-mode "protocol" module before sending out on h/w. Thus the potential need for pending requests from that user-space module.
    Thanks!

  • ThatsBerkanThatsBerkan Member Posts: 17
    edited November 17

    So you have two user-mode softwares, software A is sending data to the software B (the PCD), software B encodes and transfers the data to the hardware, then responds back to software A ? Is my understanding correct ? You're only talking about usermode components, but also mentionned IRPs, it is confusing.

    If your problem is about how you're going to design the way your application (soft. A) is going to wait for a reply from soft. B, we can't deduce that for you unfortunately. As you said, the PCD is proprietary code, we don't know how those requests have been implemented. Is it possible to open multiple sockets to the PCB ? If so, you could setup a queue and process everything in separate sockets (like each app that wants to communicate will have its own socket) ? You need a perfect understanding on how their GUI software works before going in.

    The problem is that the PDC is currently a user-mode application, and we would like to keep that in user-mode as much as possible.

    You can perfectly send requests over a socket from a kernel mode component to a usermode component. Don't you need a kernel mode component anyway, to setup a virtual network adapter ?

    After re-reading again... Is software A actually your kernel driver ?

    For receive packet processing, how does the user mode component then indicate the data back to the network adapter?

    Pipes ? Events ? Sockets ? I do not believe IRPs are designed for this.

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,695

    Do you have source for these applications? If so, could you not change one of them to use a different port number, and write a very simple this-to-that that listens on port A and writes to port B? That's about 10 lines of Python code.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Mark_HolidayMark_Holiday Member Posts: 27
    edited November 17

    Don't you need a kernel mode component anyway, to setup a virtual network adapter ?
    After re-reading again... Is software A actually your kernel driver ?

    Yes, the VNIC would be the kernel mode component. This would be necessary to allow for:

    • multiple applications to use socket interface
    • bridging disparate network interfaces

    So we have:
    Software A = GUI
    VNIC = new kernel component
    Software B = PDC (user-mode)

    The path I was envisioning was something along the lines of:

    Software A  (GUI)   <->   VNIC   <->  Software B (PDC) <-> h/w
    
    

    (That formatting may not be ideal, but hopefully it gets the picture across!)

    You can perfectly send requests over a socket from a kernel mode component to a usermode component.

    I had not thought of that! That seems like a much simpler approach than issuing & pending requests for buffer access.

    Are you talking about using Winsock Kernel (WSK)??

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158

    seems like a much simpler approach than issuing & pending requests for buffer access

    Really? I mean... that's just ordinary Windows user-mode talks to a driver stuff, isn't it??

    Sending stuff to/from KM via Sockets? You COULD... but, isn't that what ReadFile and WriteFile are for? I dunno... I probably shouldn't get involved in this thread. I haven't understood anything, including the first post. Your apps sends reads. Your driver puts those reads on a queue. When you have data, you use that data to satisfy the read. Heck, the students in my KMDF seminar do that by Thursday afternoon, never having written a Windows driver before. What am I missing?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_HolidayMark_Holiday Member Posts: 27

    Thank you for that comment, Peter.

    I think this thread became more complicated because - as you can see - fundamentally different suggestions have been offered, e.g. do not use inverted calls, but rather kernel sockets. Now after taking a look at that, I have to agree with you that maybe that is not a simpler approach after all.

    I am probably over-thinking this. Serving up pending requests (i.e. inverted call model) does seem like the way to go; also, it should provide the most flexibility - which certainly is not a bad thing!

    Agree?

  • ThatsBerkanThatsBerkan Member Posts: 17
    edited November 17

    But you are not able to modify the PDC code, do you ? From what I understood, that is specifically (aside from the 2 features you've mentionned) why you are doing all this (the constraints). You will have to open a socket to the PDC anyway ?

    How are you going to communicate between your VNIC and the PDC ?

  • Mark_HolidayMark_Holiday Member Posts: 27
    edited November 17

    But you are not able to modify the PDC code, do you ?

    For this iteration, I will not be modifying the PDC code, i.e. I need to maintain its socket interface and use it as a "block box".

    After posting my initial message to this thread, I came up with a scheme that I think will serve my needs. It involves adding a "shim" user-mode application that employs the inverted calls to exchange data with VNIC component. So now we would have:

    Software A (GUI) <-socket-> VNIC <-pending I/O requests-> Software C (shim) <-socket-> Software B (PDC) <--> h/w

    I think that allows me to develop iterations while preserving current interfaces.

    Thoughts on this??

    Sorry I was not clear earlier.

  • ThatsBerkanThatsBerkan Member Posts: 17
    edited November 17

    You're overcomplicating the issue. You simply want to "proxy" the communication between Soft. A and Soft. B (PDC). Adding a third layer will simply add more requirements and deployments issues.

    The inverted call model seems like a bad idea to me but it can be done. It's a matter of preferences I guess. I agree that Winsock is complicated if you just happen to learn about it. Needs multiple calls and IRP/Stack creation just to setup a simple socket object...

  • Mark_HolidayMark_Holiday Member Posts: 27
    edited November 17

    The inverted call model seems like a bad idea to me but it can be done. It's a matter of preferences I guess.

    Unless I misunderstood, I think that that is the method that Peter V. was advocating above.

    Eventually both Software A & B components will be modified. But for this iteration:

    • to demonstrate a VNIC for their use
    • while maintaining their current interfaces, and
    • not modifying them

    I think what I presented is one approach to get me there.

    I appreciate your feedback and suggestions here. Thank you!

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158
    edited November 17

    This has GOT to be the world's most confusing, over engineered, thread.

    If the idea is to "glue" the output of Software A (which sends data via Sockets) to Software B (which receives data via sockets)... and for some reason they can't talk directly to each other (hmmm... OK? They talk using different ports or something, I GUESS)... Then, write a user-mode sockets program that gets stuff from A and sends it to B, and your done.

    Like I said... I should so stay out of this thread. I just don't understand any of this.

    ETA:

    Gad! I'm not insane! I see that Mr. Roberts said exactly this, some days ago:

    write a very simple this-to-that that listens on port A and writes to port B? That's about 10 lines of Python code.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_HolidayMark_Holiday Member Posts: 27
    edited November 17

    Sorry - again - I was not clear. :(

    Software A & B do talk to each other, over a socket connection, that is true.

    I tried to explain in my initial post to this thread that there is a requirement to insert a VNIC between them while leaving them "as is".
    That is the impetus for this thread: how to facilitate that communication.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158

    a requirement to insert a VNIC between them

    A "VNIC" that does WHAT, exactly? How do YOU define a VNIC?

    How does the proverbial "10 line Python program" logically differ from your definition of a VNIC?

    As we so very often say here: "What larger problem are you trying to solve?" Does A send socket traffic to B, which is on a different, pre-defined, IP Address? So, you want this VNIC to fake the IP address?? What is the problem that needs solving here?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_HolidayMark_Holiday Member Posts: 27

    3rd attempt - who are my edits getting lost??

    A "VNIC" that does WHAT, exactly? How do YOU define a VNIC?

    VNIC = Virtual Network Adapter

    As we so very often say here: "What larger problem are you trying to solve?" Does A send socket traffic to B, which is on a different, pre-defined, IP Address? So, you want this VNIC to fake the IP address?? What is the problem that needs solving here?

    This is part of a larger government contract, so the less I say about "why" the better (if we can just leave it at that, it would be good...) ;)

    The requirement (from the powers that be) for the VNIC is to:

    • provide network interface for Software A
    • provide network interface for other future software applications
    • provide ability to bridge disparate network interfaces

    It is important to note that the "lower edge" of the VNIC does not interface directly with h/w; it is the user-mode component PDC ( Software B ) that interfaces with h/w devices that go out to their "cloud".

    Also, for this iteration I wanted to keep Software A & B "as is".

    I hope this helps to clarify. Sorry for my earlier vagueness. I think in being careful to not divulge too much I just made things more confusing.

  • MBond2MBond2 Member Posts: 210

    Presumably this is phase 1 in a larger project where you will make further changes? Possibly a new GUI control program is planned and this is some way of allowing both of them to talk to the controller simultaneously or interchangeably?

  • Mark_HolidayMark_Holiday Member Posts: 27

    I have a comment that is pending approval, but yes, this is the first phase of a larger implementation. I want to prototype the approach such that I can leave Software A & B unmodified - for now - and support the requirement (as it is) is to introduce a virtual network component between those 2 user-mode applications. That is the crux of the problem.

  • MBond2MBond2 Member Posts: 210

    to start, a network bridge. Presumably, with more functionality to come? but what kind? they already communicate via a network protocol, so it can't be remoting the access. that's why we are all confused i think

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,695

    This is part of a larger government contract, so the less I say about "why" the better...

    That's convenient, but remember that a professional engineer has a responsibility to point out when a solution is going the wrong direction.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Mark_HolidayMark_Holiday Member Posts: 27

    3rd attempt - who are my edits getting lost??

    A "VNIC" that does WHAT, exactly? How do YOU define a VNIC?

    VNIC = Virtual Network Adapter

    As we so very often say here: "What larger problem are you trying to solve?" Does A send socket traffic to B, which is on a different, pre-defined, IP Address? So, you want this VNIC to fake the IP address?? What is the problem that needs solving here?

    This is part of a larger government contract, so the less I say about "why" the better (if we can just leave it at that, it would be good...) ;)

    The requirement (from the powers that be) for the VNIC is to:

    • provide network interface for Software A
    • provide network interface for other future software applications
    • provide ability to bridge disparate network interfaces

    **It is important to note that the "lower edge" of the VNIC does not interface directly with h/w; it is the user-mode component PDC ( Software B ) that interfaces with h/w devices that go out to their "cloud".

    Also, for this iteration I wanted to keep Software A & B "as is".**

    I hope this helps to clarify. Sorry for my earlier vagueness. I think in being careful to not divulge too much I just made things more confusing.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158
    edited November 18

    Mr @Mark_Holiday ... This is frustrating, in many ways,

    First, do you see... at the top of the NTDEV list, where it says “ Please Read: Did You Post Something and It Did Not Appear? Edit a Post and Have it Disappear?”? You’ll see this message. Your repeated posts, all 7 of them, were marked as spam. And I had to clean them all up this morning.

    And, I’m sorry: “The less said the better” is a cop-out answer, You can be working on a TS/SCI project, and you can describe the engineering requirements in such a way as to not divulge any classified information.

    I’m about an inch away from locking this thread, just because it frustrates me and has little chance of providing you any helpful information... mostly because you’re doing nothing to “help us help you.”

    You wrote:

    The requirement (from the powers that be) for the VNIC is to:

    • provide network interface for Software A
    • provide network interface for other future software applications
    • provide ability to bridge disparate network interfaces

    I sincerely fail to understand how Mr. Roberts proverbial “ten line python program” does not adequately fulfill those requirements, if “provide a network interface for Software A” means “provide a connection accessible via sockets.”

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158

    And, you know, I think I probably knew what the initials VNIC stood for. So, thanks for spelling it out for me. /rolls eyes

    What I was asking is “What functionality, specifically, do you envision as part of this VNIC” and “what attributes of a NIC, specifically, require emulation”?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_HolidayMark_Holiday Member Posts: 27

    My apologies - and thank you for that well-deserved scolding! :s
    Please understand it is certainly not my intention here to create havoc or waste people's times. I sincerely appreciate the input from everyone here.

    I will go back and review what I have for requirements, and try formulate a more lucid request-for-comments here.

    Thank you!

  • anton_bassovanton_bassov Member MODERATED Posts: 5,190

    VNIC = Virtual Network Adapter

    Before we proceed any further, I have to remind you that, assuming that you speak about the full-fledged VNIC that network protocols may bind to, the above mentioned VNIC has to be implemented, on its upper edge, as a NDIS miniport driver. A driver of this kind is not meant to be directly accessed by anyone, apart from NDIS library......

    Serving up pending requests (i.e. inverted call model) does seem like the way to go; also,

    ...which means that it does not receive IRPs from the userland (in fact, from anyone, unless we are speaking about its special control device object that may be used for configuration and has to be the same for all instances of a VNIC, but this is not what we are speaking about here).

    Therefore, assuming that you really mean a VNIC, in your particular case you will have to implement it as a layered pair of two kernel modules. Driver A will deal with actually sending and receiving data (if I got it right, this part is done by means of some form of interaction with the userland app), and a driver B will actually implement a VNIC that is presented to the system. It will deal with a driver A on its lower edge and with NDIS library on its upper one, effectively presenting itself a VNIC that protocols may bind to. It is understandable that the inverted call part will be implemented in a driver A.

    This is what has to be done in your case if you really mean implementing a VNIC. However, I've got a weird feeling that this is a very obvious overkill in your case, and that your problem may be probably solved, indeed, simply by writing a dozen of lines in Python.....

    Anton Bassov

  • MBond2MBond2 Member Posts: 210

    government contracts resulting in overkill solutions dross and waste? impossible surely!

  • Mark_HolidayMark_Holiday Member Posts: 27

    Thank you for your comments.

    Indeed, I was noting that this was part of a government contract project not to "cop out" from providing requirements, but from simply letting you all know that by being sensitive to this, I probably did a poor job in describing the problem at hand.

    Also, I am trying to not over-engineer this and make it overly-complicated; this is why I am here in the first place. I am trying to ascertain the best (read: simplest) solution to provide the desired capabilities.

    That all being said, I would like to start again, not by describing what we have, but rather what we need.
    What we need is a new software "component" that will:

    • interface with 3rd party network applications, e.g. web browsers, at its upper edge, and existing user-mode proprietary protocol/data controller (call it PDC) at its lower edge. (Note: we do own and have full source code for this PDC module)
    • provide the ability to facilitate network bridging at TAP (i.e. Ethernet frame) level. For example, we need to allow a workstation to serve as a "gateway connection", with multiple PDC modules whose "upper edges" are bridged to a real physical NIC that connects to some other WAN/LAN.

    Ideally, this would be the same module. Therefore, one current thought is that a software-only virtual network adapter may solve both of these requirements, similar the TAP-Windows driver used by OpenVPN.

    I hope this helps to describe the larger problem(s) that we are trying to solve. If more information is required, please let me know.

    One last thing: I am asking for help and suggestions here. If you need to point out that something is not clear, or that more information is needed, I have to ask to please not do so in a condescending manner. Some of you may not realize how some of your comments come across. If that request of mine seems out of line, I apologize, but that is simply my opinion.

    Thank you in advance!

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,158

    So, PROTOCOLs at the upper edge (which are in turn accessed by modules that are accessed by 3rd party network applications)... and Some Special Software (your PDC) at its lower edge. If that meets your needs that an NDIS Virtual NIC driver, with NDIS at the upper edge and KMDF at the lower edge. This is a reasonably vanilla thing to do.

    Bridging as you describe it MIGHT be a bit more complicated, it seems to me. You could bridge among PDCs via your NDIS Virtual NIC driver as described above (taking from one PDC and giving to anther PDC)... you'll have protocol frames after all.

    But if your requirement is to bridge these PDCs to "real" LAN/WAN hardware... I'm not sure that's the same piece of software as we were just describing above (upper edge protocol, lower edge PDC) -- Because now you have software that has lower edge that talks to your PDC and also lower edge that talks to a REAL NIC. I'm not sure how you make that work, frankly.

    It has been a very, very, long time since I've done real work in the NDIS NIC area. All my experience is well before NDIS 6, even. So, I'll leave the more detailed comments about whether this is one or more than one piece of software to others who are more expert than I am.

    We have some really excellent folks from the NDIS team who occasionally stop by here... so mayhaps one of them will dip in to help.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Writing WDF Drivers 7 Dec 2020 LIVE ONLINE
Internals & Software Drivers 25 Jan 2021 LIVE ONLINE
Developing Minifilters 8 March 2021 LIVE ONLINE