Yeah... We Lost "A Couple of Days"

My apologies to the membership.

In our ongoing attempt to clean things up as part of the Migration, last week we deleted a bit more than 1700 members. Now, MOST of those members were members who had email addresses that didn’t work at all (resulted in permanent failures when we sent email to them). But… a FEW were actual, legit, members. Like, ah… ME. And Scott.

Being the SQL Server animals that we are, the only way we could figure out to restore those few legit users was to restore the entire database from a backup made before the delete… that backup was made on 6 August at 2AM Eastern USA Time.

So, that’s what we did.

Any posts (or people who joined, or membership changes made) this week have therefore been “lost.”

This is entirely our fault (well, technically, it’s entirely MY fault… but I’ll use the “royal we” in this case to deflect part of the blame), and we regret the inconvenience.

May this be the only bump in the road that we suffer in the migration process,

Peter
OSR
@OSRDrivers

List Slave to the Stars

well recently after the moving thread was posted i made a complete dump of windbg in mbox format

it weighs about 67 mb containing 1.5 million lines counting 12000 posts in total

:\>ls -lh windbg
-rw-rw-rw- 1 XX 0 66M 2018-08-05 12:39 windbg

:\>grep -c “From -” windbg
12011

:\>find /C /v “” windbg

---------- WINDBG: 1548286

here is how
wget thunderbird
thunderbird -P
and set up a profile and point the local directory to a removable device (default directory is %appdata%)

setup nntp using the email registered with osrlists
subscribe to windbg and set to sync all posts for offline use

ntdev has around 200k posts wrt to 12k in windbg

The surprising part here is that you haven?t looked more into this. IMHO many important features in the NT kernel were developed expressly to meet need that the SQL server team has had. Thread scheduling, memory management, sync objects and even the network stack have all been affected. The storage stack goes without saying.

Probably the only other Microsoft application with as much influence on the OS is IIS ? which is in box with the OS since windows 2000? The scope of influence is on par with the GDI subsystem

Sent from Mailhttps: for Windows 10

________________________________
From: xxxxx@lists.osr.com on behalf of xxxxx@osr.com
Sent: Friday, August 10, 2018 10:37:20 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Yeah… We Lost “A Couple of Days”




NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:></https:>

On Fri, Aug 10, 2018 at 9:32 AM, xxxxx@osr.com wrote:
> My apologies to the membership.
>
> In our ongoing attempt to clean things up as part of the Migration, last week we deleted a bit more than 1700 members. Now, MOST of those members were members who had email addresses that didn’t work at all (resulted in permanent failures when we sent email to them). But… a FEW were actual, legit, members. Like, ah… ME. And Scott.

What are we even paying you for?

On Fri, Aug 10, 2018 at 2:35 PM, xxxxx@gmail.com
wrote:
> well recently after the moving thread was posted i made a complete dump of windbg in mbox format
>
> it weighs about 67 mb containing 1.5 million lines counting 12000 posts in total
>
> :&gt;ls -lh windbg
> -rw-rw-rw- 1 XX 0 66M 2018-08-05 12:39 windbg
>
> :&gt;grep -c “From -” windbg
> 12011
>
> :&gt;find /C /v “” windbg
>
> ---------- WINDBG: 1548286
>
> here is how
> wget thunderbird
> thunderbird -P
> and set up a profile and point the local directory to a removable device (default directory is %appdata%)
>
> setup nntp using the email registered with osrlists
> subscribe to windbg and set to sync all posts for offline use
>
>
> ntdev has around 200k posts wrt to 12k in windbg
>
> —

Wasn’t NNTP disabled? That’s the only bit confusing me.

On Mon, Aug 13, 2018 at 8:08 PM, xxxxx@hotmail.com
wrote:
> The surprising part here is that you haven’t looked more into this. IMHO
> many important features in the NT kernel were developed expressly to meet
> need that the SQL server team has had. Thread scheduling, memory
> management, sync objects and even the network stack have all been affected.
> The storage stack goes without saying.
>
>
>
> Probably the only other Microsoft application with as much influence on the
> OS is IIS – which is in box with the OS since windows 2000? The scope of
> influence is on par with the GDI subsystem
>

I can’t expect you to collate all of this - but are there any blog
posts, or a book, about the work you are referring to specifically?
What intrigues me about your comment is the possibility of a story,
not just a dump of facts about NT.

Cheers,
R0b0t1

Talk about thread drift…

Well, that’s a BIT of an over statement. The NT team had other “fish to fry” in the late 80’s when NT was initially being designed. It’s not like they sat down with the Sybase folks and said “Let’s make an OS that’s SQL friendly.”

That’s not to say that SQL wasn’t important, because it was. When NT V3.1 was released, MSFT moved SQLNT from OS/2 to Windows NT. It was undoubtedly the first large-scale app that run on Windows NT.

It’s all part of the story of how NT evolved from being “the next version of OS/2” to “the next version of Windows”

Now THAT is ENTIRELY true. At various times in the history of Windows, various parts of the OS have been changed and tuned to make them “better” for use with layered products such as SQL. The thread scheduling algorithms (which, most people don’t realize, have been constantly changed and tweaked), for example, have been altered at various times to specifically make SQL (and, one might argue, other large scale data intensive applications) run better.

Of course, in MY experience, the biggest thing that most database engine developers want from the OS is for the OS to “get out of the way” of the database system. The best OS, to the developer of a database engine, is the one in which THEY control EVERYTHING.

Hmmmm… I find that to be the oddest of statements: The GDI subsystem has always struck me as the bastard step-child of the OS. It has not had any overall lasting impact on the architecture of Windows that I know of. If you can cite some specific examples, I’d certainly be interested in hearing them.

Peter
OSR
@OSRDrivers

> GDI subsystem has always struck me as the bastard step-child of the OS.

It has not had any overall lasting impact on the architecture of Windows that I know of.

…apart from relatively tight integration between GUI and Windows scheduler - IIRC, the thread that currently has the window focus gets a temporary priority boost. This is why there is a specific field in ETHREAD that allows the kernel to make a distinction between GUI and non-GUI threads.

In fact, the very concept of kernel preemption is closely related to GUI and user interaction.
It does not really make sense for a server-grade system with overall efficiency as a primary concern to bother itself with kernel preemption, because it comes at a certain performance penalty and adds an extra complexity to the kernel. However, when it comes to the end user experience the kernel preemption is just of paramount importance - although a batch process may easily shrug off a delay of, say, 50 - 100 ms simply as a non-issue, an interactive user may find this delay pretty annoying.

Therefore, I would not be too much surprised to find out that NT kernel was designed with support for kernel-level preemption because NT was meant to be an interactive GUI-based system

Anton Bassov

>
t’s all part of the story of how NT evolved from being “the next version of OS/2” to “the next version of Windows”

This is really an exact rendition of the story :-).

“Cisco taking away the networking business, our OS/2 is not moving forward etc” are the talks still echos to my ears. On a 16MB/ 386 machine, it was noticeable….

-Pro

On Aug 14, 2018, at 8:05 AM, xxxxx@osr.com wrote:
>
> t’s all part of the story of how NT evolved from being “the next version of OS/2” to “the next version of Windows”

Well I?m glad that you might agree with somthing I said. You let me know which kind of fish they were and I?ll let you know if I agree with you on any of your points

Unfortunately I?m old.

nt from Mailhttps: for Windows 10

________________________________
From: xxxxx@lists.osr.com on behalf of xxxxx@osr.com
Sent: Tuesday, August 14, 2018 11:05:44 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Yeah… We Lost “A Couple of Days”

Talk about thread drift…



Well, that’s a BIT of an over statement. The NT team had other “fish to fry” in the late 80’s when NT was initially being designed. It’s not like they sat down with the Sybase folks and said “Let’s make an OS that’s SQL friendly.”

That’s not to say that SQL wasn’t important, because it was. When NT V3.1 was released, MSFT moved SQLNT from OS/2 to Windows NT. It was undoubtedly the first large-scale app that run on Windows NT.

It’s all part of the story of how NT evolved from being “the next version of OS/2” to “the next version of Windows”



Now THAT is ENTIRELY true. At various times in the history of Windows, various parts of the OS have been changed and tuned to make them “better” for use with layered products such as SQL. The thread scheduling algorithms (which, most people don’t realize, have been constantly changed and tweaked), for example, have been altered at various times to specifically make SQL (and, one might argue, other large scale data intensive applications) run better.

Of course, in MY experience, the biggest thing that most database engine developers want from the OS is for the OS to “get out of the way” of the database system. The best OS, to the developer of a database engine, is the one in which THEY control EVERYTHING.



Hmmmm… I find that to be the oddest of statements: The GDI subsystem has always struck me as the bastard step-child of the OS. It has not had any overall lasting impact on the architecture of Windows that I know of. If you can cite some specific examples, I’d certainly be interested in hearing them.

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:></https:>

Don?t propagate ridiculous nonsense.

Sent from Mailhttps: for Windows 10

________________________________
From: xxxxx@lists.osr.com on behalf of xxxxx@hotmail.com
Sent: Tuesday, August 14, 2018 12:38:31 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Yeah… We Lost “A Couple of Days”

> GDI subsystem has always struck me as the bastard step-child of the OS.
> It has not had any overall lasting impact on the architecture of Windows that I know of.

…apart from relatively tight integration between GUI and Windows scheduler - IIRC, the thread that currently has the window focus gets a temporary priority boost. This is why there is a specific field in ETHREAD that allows the kernel to make a distinction between GUI and non-GUI threads.

In fact, the very concept of kernel preemption is closely related to GUI and user interaction.
It does not really make sense for a server-grade system with overall efficiency as a primary concern to bother itself with kernel preemption, because it comes at a certain performance penalty and adds an extra complexity to the kernel. However, when it comes to the end user experience the kernel preemption is just of paramount importance - although a batch process may easily shrug off a delay of, say, 50 - 100 ms simply as a non-issue, an interactive user may find this delay pretty annoying.

Therefore, I would not be too much surprised to find out that NT kernel was designed with support for kernel-level preemption because NT was meant to be an interactive GUI-based system

Anton Bassov


NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:></https:>

> Don?t propagate ridiculous nonsense.

Cold you please be a bit more precise, and specify what exactly you find particularly “nonsensical” in my post so that we can try an argumented discussion…

Anton Bassov