Windows subsystem costomization

NTDEV
1

hello guys, I got an idea that I’d like to hear your insights about… windows NT kernel was a subsystem agnostic one in the past, it supported multiple subsystems (POSIX, OS/2, win32), but later the first 2 were removed and only win32 now exists..

but as far as I know NT kernel still has the logic to be subsystem agnostic, so can we make a custom subsystem for a windows that replaces win32 (some sort of a windows distro)?
I know I’ll need to interact with SMSS a lot and probably reverse engineer ALPC connections of SMSS and CSRSS that I should emulate
what recourses you suggest for me to study?
btw, the project is currently a private repo in github, but if I found interested people we can make a discord server and work on the project together!

2

(post deleted by author)

3

hello again, after some search here are my ideas about how I’d actually do so:

  • according to “WDDM architecture” article in MS learn (I can’t share links unfortunately), the display stack in windows is as:
    1- the kernel mode graphics driver, which is 3rd party supplied and differs from a company to another and even from a GPU to another
    2- dxgkrnl.sys, this is an abstraction layer for kernel mode graphics driver, it (I assume and wish) has a stable API to interact with instead of the changing kernel mode graphics driver
    3- win32k.sys, this is yet a wrapper for dxgkrnl.sys itself, it abstracts away the messy dxgkrnl.sys driver and shapes how the GUI works in windows, it handles windowing, menus, keyboard and mouse input, etc..
    4- win32GDI and gdi32.dll, these are the user-mode start, they interact with kernel mode drivers and I suppose they use syscalls to do so.

so, for a custom subsystem that needs TTY and perhaps GUI in the future, I need a way to draw i the screen directly!
here is what I thought of:

  • I replace win32k.sys driver with my own, which will interact with dxgkrnl.sys driver via its interface such as “DXGKRNL_INTERFACE” structure (search for it in MS learn)
    then I’ll make a DLL that interacts with my new driver, but I don’t know how to interact with it?
    I’ll not be able to use syscalls, so should I use IOCTL?