rand at the kernel

Chuck,

I agree with you that Knuth already provided an wealth of information, and
there were lot of
hue and cry at Bell Labs about getting a good rand() functions, I was not
involved and definitely
was not qualified, but being the dev. lab we used get those internal memos,
and as per what I
recall they were pretty good after taking some of the concerns, and
obviously there are some
studies and reports about those early days compilers’ rand()…, even the
compactness of
algorithm, speed and others were analyzed in detail since it was a hot topic
then !!! Yes today,
it is a different story, since the adversaries also have astronomical
computing (enumeration ) power.

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chuck Batson
Sent: Monday, August 02, 2004 8:42 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] rand at the kernel

Typically, rand() in C runtime libraries is implemented as a linear
congruential generator. This is one of the worst methods to generate a
pseudo-random number sequence, as it suffers from many problems such as
a short period and an oscillating lower bit, to name a couple. This is
discussed by Knuth at length.

Chuck

----- Original Message -----
From: “Programmers Society Prokash Sinha”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, August 03, 2004 6:33 AM
Subject: RE: [ntdev] rand at the kernel

> Generating random sequence using compiler provided rand() or ddk
provided RtlRando() should be quite good already if seed are properly
selected. It’s been tested over and again, and some of the acid and
basic test(s) OP already know about, “Knuth’s semi-numerical algorithm
book”. In case someone wants to further, Jamey’s suggestion would be one
too …
>
> If MD5 has any pattern ( like monotone, bounded, periodic etc., etc)
then it would be predictible
>
> -pro


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Yep, in not to distant past, once we trashed md5 in favor of SHA, but again
these days analysis part is almost always out of style, just code and do it.

One thing to note is that when it comes to crypto/security 1+1 > 2, that
means
the protocols are much more important than just to worry about a fancy
algorithms,
a chain of protocols ( mainly auth ) mechanism is what makes it difficult to
break. And
that shows in your approach here, which is correct approach in my beliefs
… This is really
one fundamental idea I see being recommended again and again by experts:)

Once in a while I still pull out the Zen of Optimization by Micheal Abrash,
never found
a copy of “Zen of Assembly”. And for the size of instruction(s) as well as
instruction streams,
fixed size instruction’s bain-child is RISC technology. I still remember how
Dr Fredrick Brooks
was joking about their own innovation about CISC and FAT microcode to get
composite primitives. And
even in back early 80’s we used to simulate barrel-shifter using APL, of
course simulation is vital.

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of David J. Craig
Sent: Monday, August 02, 2004 9:22 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] rand at the kernel

Why not use one of the SHA, Secure Hash Algorithms, that come in various bit
sizes? Also why not use the time and maybe the RDTSC 64-bit value ORed or
XORed into the current time value? That would make reproducing the seed
much more difficult as finding both values much more unlikely. Since
GetSystemTime() returns 128 bits and RDTSC only returns 64 bits you might
want to merge the RDTSC value into GetSystemTime’s return value twice
treating it as two 64-bit values or if a 64-bit seed is needed just fold the
128 bit value together as two 64-bit pieces.

Anyone into optimization read Michael Abrash’s article in DDJ? Removing a
MUL from a instruction sequence actually slowing down the sequence sounds
like fun to encounter. I still remember the IBM PC where, except for a few
instructions, the size of the instruction stream was the controlling factor
for speed. How can you write an optimizing compiler if the only company
that has a simulator is Intel? Maybe that is why their compilers seem to
outperform almost all others. Of course, I suspect that Intel compilers
don’t do quite as well on AMD processors.

“Prokash Sinha” wrote in message news:xxxxx@ntdev…
> There you go, know with fairly good accuracy, that is guessing game, if
the
> algorithm scales, as
> I explained, there is no way but to know the alog. is scalling, so
MD5(Zw*()
> + fixed scale) going
> to make it really hard, then there are other hardening…
> Yes blind heuristic is basically enumeration, but there are some chance
> game, so even 5 try can knock the socks off. The question is how probable
is
> to come up with those 5 tries ???
>
> This is an exciting field of computer science and programming. I
personally
> think probablistic computing is an exciting, and as exciting as kernel
> programming, area.
>
> Kerboros uses MD5 too, an MIT brain-child. I’ve not looked at those from
> analytical point of view for a while, but once we used take limits on
> Integration over Big-Oh, just to comeup with some weired theorem(s)…
>
> This might be a real rant, and the OP must be yelling at us !
>
> Thanks for the excerpts !
>
> -pro
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Dmitriy Budko
> Sent: Monday, August 02, 2004 7:58 PM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] rand at the kernel
>
>
> The adversary could know the system time and the results of
> ZwQuerySytemTime() with high accuracy (about 1 sec).
>
> >From my reference:
>
> > Recently, one of my officemates (David Wagner <daw_at_cs.berkeley.edu>)
> > and I (Ian Goldberg <iang_at_cs.berkeley.edu>) finished the job
> > of seeing exactly how the encryption keys are picked.
> >
> > What we discovered is that, at least on the systems we checked (Solaris
> > and HP-UX), the seed value for the RNG was fairly trivial to guess by
> > someone with an account on the machine running netscape (so much so
> > that in this situation, it usually takes less than 1 minute to find
> > the key), and not too hard for people without accounts, either.
> > See below for details.
> >
> > Happy hacking,
> >
> > - Ian “who just saw Hackers today with some other Bay Area
cypherpunks,
> > and it put me in the mood”
> >
> > /* unssl.c - Last update: 950917
> >
> > Break netscape’s shoddy implementation of SSL on some platforms
> > (tested for netscape running RC4-40 on Solaris and HP-UX; other
> > Unices are probably similar; other crypt methods are unknown, but
> > it is likely that RC4-128 will have the same problems).
> >
> > The idea is this: netscape seeds the random number generator it uses
> > to produce challenge-data and master keys with a combination of the
> > time in seconds and microseconds, the pid and the ppid. Of these,
> > only the microseconds is hard to determine by someone who
> > (a) can watch your packets on the network and
> > (b) has access to any account on the system running netscape.
> >
> > Even if (b) is not satisfied, the time can often be obtained from
> > the time or daytime network daemons; an approximation to the pid can
> > sometimes be obtained from a mail daemon (the pid is part of most
> > Message-ID’s); the ppid will usually be not much smaller than the pid,
> > and has an higher than average chance of being 1. Clever guessing
> > of these values will in all likelihood cut the expected search space
> > down to less than brute-forcing a 40-bit key, and certainly is less
> > than brute-forcing a 128-bit key.
>
>
> Dmitriy Budko, VMware
>
>
>
> > -----Original Message-----
> > From: Prokash Sinha [mailto:xxxxx@garlic.com]
> > Sent: Monday, August 02, 2004 7:39 PM
> > To: Windows System Software Devs Interest List
> > Subject: RE: [ntdev] rand at the kernel
> >
> >
> > Sorry for the notationall problem it should if Pr( MD5(x) == MD5(y) ).
> >
> > Now if some one scales it even with a year, say we pick 13th aug of
> > (2004 - 1601)/2 + 1601 just about the middle year. Now I really need
> > someones help to even find the right one hour interval to try out.
> >
> > we can not enumerate over the whole ranges of one-hour !!!
> >
> >
> > -pro
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com]On Behalf Of Prokash Sinha
> > Sent: Monday, August 02, 2004 7:09 PM
> > To: Windows System Software Devs Interest List
> > Subject: RE: [ntdev] rand at the kernel
> >
> >
> > Okay, assuming the digest is 128bit, and it is one-way it is highly
> > improbable that
> > x<>y in the domain would match the hash, we could always
> > compute this…, I
> > mean the
> > Pr(Md5(x) = Pr(Md5(y), where x <> y.
> >
> > Given that it is all but enumeration to be tried out for
> > solving or finding
> > x for Md5(x) !
> >
> > If the intervals are 100 nano-sec, means that 10,000,000
> > interval points in
> > a second, now even if I try for
> > any onehour interval for trying out the enumeration then
> > 606010,000,000
> > enumeration, that is already quite big ???
> >
> > Now think of some other random sequence genrator, they might
> > not be far
> > better than this for enumeration(s)???
> >
> > -pro
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com]On Behalf Of Dmitriy Budko
> > Sent: Monday, August 02, 2004 6:00 PM
> > To: Windows System Software Devs Interest List
> > Subject: RE: [ntdev] rand at the kernel
> >
> >
> > > but MD5(x), MD5(x++) … could very well be
> > > a nice random sequence, …
> > >
> > > -pro
> >
> > It is not true if x is not random. Results of ZwQuerySystemTime()
> > (the number of 100-nanosecond intervals since January 1,
> > 1601) could be
> > easily guessed. It should take only 1.e7 attempts if the
> > system’s time is
> > accurate to
> > 1 sec.
> >
> > For example, see this classic SSL (Netscape) hole on some platforms:
> > http://seclists.org/bugtraq/1995/Sep/0064.html
> >
> >
> > Dmitriy Budko, VMware
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@garlic.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@garlic.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@vmware.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@garlic.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com</iang_at_cs.berkeley.edu></daw_at_cs.berkeley.edu>

> the protocols are much more important than just to worry about a fancy

algorithms,
a chain of protocols ( mainly auth ) mechanism is what makes it
difficult to
break.

This is a common misconception, and would fall under the category of
“security via obscurity.” A secure message should be able to withstand
attack even when an interceptor knows the exact procedures used to
encode and decode the message. What makes modern cryptographic systems
“difficult to break” is the fact that it’s (currently) easier to
generate products of large prime numbers than it is to factor an
arbitrarily large integer. There is no correlation to the number of
protocols involved.

Chuck

Most, if not all, random number generators fall into one of the following
categories:

  • Linear Congruential Generators: N -> Mod[a n + b, m]
  • Linear Feedback Shift Registers: PolynomialMod[x^t, {1 + x + x^n,
    2}]
  • Generalized Fibonacci Generators: Mod[f[n - p] + f[n - q], 2^k]
  • Cryptographic Generators: Stream ciphers (DES for example)
  • Quadratic Congruential Generators: N -> Mod[n^2, m]

* Cellular Automation Generators:

I do not know much about these CA generators; yet. I think some of the CA
rules can have repeat periods of 2^0.63n. Again, I do not remember the
details of these random number generators.

There is a great chapter on Randomness in Wolfram’s book. If you are at all
interested in musing over randomness and random number generators, you will
enjoy this chapter and maybe even the rest of the book.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Monday, August 02, 2004 7:09 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] rand at the kernel

Okay, assuming the digest is 128bit, and it is one-way it is highly
improbable that
x<>y in the domain would match the hash, we could always compute this…, I
mean the
Pr(Md5(x) = Pr(Md5(y), where x <> y.

Given that it is all but enumeration to be tried out for solving or finding
x for Md5(x) !

If the intervals are 100 nano-sec, means that 10,000,000 interval points in
a second, now even if I try for
any onehour interval for trying out the enumeration then 60*60*10,000,000
enumeration, that is already quite big ???

Now think of some other random sequence genrator, they might not be far
better than this for enumeration(s)???

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dmitriy Budko
Sent: Monday, August 02, 2004 6:00 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] rand at the kernel

but MD5(x), MD5(x++) … could very well be
a nice random sequence, …

-pro

It is not true if x is not random. Results of ZwQuerySystemTime()
(the number of 100-nanosecond intervals since January 1, 1601) could be
easily guessed. It should take only 1.e7 attempts if the system’s time is
accurate to
1 sec.

For example, see this classic SSL (Netscape) hole on some platforms:
http://seclists.org/bugtraq/1995/Sep/0064.html

Dmitriy Budko, VMware


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Seems like you r very current on this. But the fact of the matter I wanted
to
point is not what your driving direction here…

After weighing, those algorithms performance ( yes it perf ), it has been
found
that a combination of those algorithms are much better. And protocols are at
least
50% of the game.

For the last point about factoring, could pls point me to the literature(s).

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chuck Batson
Sent: Tuesday, August 03, 2004 12:09 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] rand at the kernel

the protocols are much more important than just to worry about a fancy
algorithms,
a chain of protocols ( mainly auth ) mechanism is what makes it
difficult to
break.

This is a common misconception, and would fall under the category of
“security via obscurity.” A secure message should be able to withstand
attack even when an interceptor knows the exact procedures used to
encode and decode the message. What makes modern cryptographic systems
“difficult to break” is the fact that it’s (currently) easier to
generate products of large prime numbers than it is to factor an
arbitrarily large integer. There is no correlation to the number of
protocols involved.

Chuck


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Guess what, Jemmy !. I looked at the online chapter summary on amazon, and
ordered yesterday an used copy :).
-pro
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Jamey Kirby
Sent: Tuesday, August 03, 2004 2:06 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] rand at the kernel

Most, if not all, random number generators fall into one of the following
categories:

  • Linear Congruential Generators: N -> Mod[a n + b, m]
  • Linear Feedback Shift Registers: PolynomialMod[x^t, {1 + x + x^n,
    2}]
  • Generalized Fibonacci Generators: Mod[f[n - p] + f[n - q], 2^k]
  • Cryptographic Generators: Stream ciphers (DES for example)
  • Quadratic Congruential Generators: N -> Mod[n^2, m]

* Cellular Automation Generators:

I do not know much about these CA generators; yet. I think some of the CA
rules can have repeat periods of 2^0.63n. Again, I do not remember the
details of these random number generators.

There is a great chapter on Randomness in Wolfram’s book. If you are at all
interested in musing over randomness and random number generators, you will
enjoy this chapter and maybe even the rest of the book.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Monday, August 02, 2004 7:09 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] rand at the kernel

Okay, assuming the digest is 128bit, and it is one-way it is highly
improbable that
x<>y in the domain would match the hash, we could always compute this…, I
mean the
Pr(Md5(x) = Pr(Md5(y), where x <> y.

Given that it is all but enumeration to be tried out for solving or finding
x for Md5(x) !

If the intervals are 100 nano-sec, means that 10,000,000 interval points in
a second, now even if I try for
any onehour interval for trying out the enumeration then 60*60*10,000,000
enumeration, that is already quite big ???

Now think of some other random sequence genrator, they might not be far
better than this for enumeration(s)???

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dmitriy Budko
Sent: Monday, August 02, 2004 6:00 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] rand at the kernel

but MD5(x), MD5(x++) … could very well be
a nice random sequence, …

-pro

It is not true if x is not random. Results of ZwQuerySystemTime()
(the number of 100-nanosecond intervals since January 1, 1601) could be
easily guessed. It should take only 1.e7 attempts if the system’s time is
accurate to
1 sec.

For example, see this classic SSL (Netscape) hole on some platforms:
http://seclists.org/bugtraq/1995/Sep/0064.html

Dmitriy Budko, VMware


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

> for speed. How can you write an optimizing compiler if the only company

that has a simulator is Intel? Maybe that is why their compilers seem to
outperform almost all others.

Surely.

From what I know, Intel slowly but surely shares the information to MS compiler
team, and leak some information to GCC community.

The rate of this sharing/leaking is slow enough to preserve the Intel’s
compiler (now maintained by the Intel’s research center in Russia) as a leader,
and is fast enough to avoid MS and the open source community from switching to
AMD due to better code optimization.

Dunno about ARM compiler used in eVC++ and WinCE Platform Builder. The same
story possibly.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com