This article aims to simply explore some principles and characteristics of linear congruence generators. Many ideas are borrowed from TAOCP . If you want to explore this knowledge in depth, it is recommended to read the original work directly.
1. Formal Definition and Periods of Linear Congruential Sequences
In discrete data and its applications , if
Then, when a modulo m is said to be congruent to b (or when a modulo m is said, a is equivalent to b), it can be written as
And the linear congruence can be expressed like this:
The linear congruence generator is somewhat related to the linear congruence above.
2.1 Formal Definition
According to the idea of 3.21. The Linear Congruential Method in The Art of Computer Programming, Volume 2 [1] , Linear Congruential Generators (LCG: Linear Congruential Generators ) can be defined as follows:
in:
The modulus m and the coefficient a are the most important parameters in this formula. How to choose these two parameters reasonably determines the linear congruential sequence (LCS: Linear Congruential Scquence): the quality of <X> (<X> >=X 1 ,X 2 ,X 3 ..X n ...).
The constant c may or may not be 0. Usually, if c=0, then (2) is also called a multiplicative linear congruential generator (MCG: Multiplicative Congruential Generator), if c is not 0, (2) is called a mixed linear congruential generator (Mixed Congruential Generator). Linear Congruential Generator ).
X 0 is called the initial value, which is the so-called seed seed.
2.2 Periods of Linear Congruential Sequences
The linear congruence sequence generated by the above linear congruence generator must have a period P. In TAOCP , the author poses this problem as an exercise ( exercise 3.1-6 ). This question is explained below through a straightforward but imprecise reasoning.
The above linear congruence formula is abstracted as a function f (map X n to X n+1 ), which has self-closing properties. It is not difficult to find that the following known conditions actually exist:
Define two sets: S and T. In the initial state, the set S contains all m elements from 0 to m-1, and the set T is an empty set.
Now simulate the process of generating LCS, take any value X0 as a parameter, and generate the first pseudo-random number X 1 . Its value must belong to set S, and X1 is moved from set S to set T at this time.
Taking X 1 as a parameter, a second pseudo-random number X 2 =f(X 1 ) is generated. At this time, X 2 may belong to the set S or the set T.
1) If X 2 belongs to the set S, then a cycle has not been generated at this time;
2) If X 2 belongs to the set T, which is exactly equal to X 1 , then a cycle is generated at this time, and the cycle P=1.
More generally, assuming that in the process of generating X 1 to X i-1 , each number is found in set S, then each number is moved from set S to set T, and the state of the two sets at this time is :
Then when X i is generated ,
1) If X i =f(X i-1 ) is in the set S, it fails to generate a cycle;
2) If X i =f(X i-1 ) is in the set T, then a cycle is generated, and the cycle P<=i-1 at this time.
Of course, the period P may also be equal to m, that is, the set S is eventually an empty set, the set T contains all elements from 0 to m-1, and f(X m )=X 1 .
Therefore, it is not difficult to know from the above reasoning that there must be a period P in LCS, and P<=m.
It is not difficult to infer that:
1) If the period P of the random sequence generated by a certain LCG is less than m, the LCS generated by selecting different initial values X 0 may have different periods.
2) If its period P=m, even if different X 0 are chosen , these LCSs generated have the same period and must be P.
For example, for the following generator:
If the random sequence is generated with the initial value X 0 =12:
7,6,9,0,7,6,9,0,7,6,9,0,7,6,9,0......
If the random sequence is generated with the initial value X 0 =13, it is:
8,3,8,3,8,3,8,3,8,3,8,3,8,3......
However, for a generator like:
No matter what value is chosen for the random sequence generated by the seed, its period is 16.
2. About parameter selection
It is not easy to construct a linear congruential generator with good performance, not only considering the period and random distribution characteristics of the random sequence it generates, but also the computational efficiency.
In terms of parameter selection, the most critical is the choice of modulus and multiplier.
3.1 modulus selection
The modulus should be as large as possible, so that it is possible to generate a longer period. If the word length of the computer is w bits, it is recommended in TAOCP that m=2^w, or m=2^w+1, or m=2^w-1, or the largest prime number less than 2^w can be taken , this m-value selection method is adopted in the paper TABLES OF LINEAR CONGRUENTIAL GENERATORS OF DIFFERENT SIZES AND GOOD LATTICE STRUCTURE [2] .
Generally speaking, if m=2^w, the use of bit operations tends to make the calculation process more convenient and efficient, but there is a problem: the randomness of the low-order bits of each element in the generated random sequence is not very good .
The simple explanation is that when m=2^w, for an integer Z of s bits, the result of Z modulo m is actually the result of the right sw bit in the bits of Z.
The author uses a more formal description in the original text to illustrate this problem:
Assuming that d is a factor of m and q is an integer, let Y n satisfy the following relation:
Then do the following transformation:
It is not difficult to find that the formula 3-1-1 is actually a linear congruence generator, and the generated random sequence also has a period, but its period is less than or equal to d.
The <Y> sequence here actually corresponds to the low-order byte of the original linear congruence sequence <X>, and the sequence <Y> can be understood as a sequence composed of the low-order bits of <X> extracted separately. For example, if d=2^4, the maximum period of the sequence <Y> is 16, and the maximum period corresponding to the lower 4 bits of each element in the sequence <X> is 16. Obviously, the randomness of the lower bits is not very good. . It is for this reason that the implementation of some platforms will discard these low-order bits with poor randomness, and truncate the high-order bits to obtain a better randomness effect. In most application scenarios, the low bits will not affect the final use, so choosing m=2^w can basically meet the requirements. In fact, many platforms do take m=2^w.
If m takes 2^w+1 or m=2^w-1, the above problem will not occur.
3.2 The choice of multiplier
The selection reasoning process of multiplier is more complicated. Generally speaking, the period of LCS should be as long as possible (the longest is m), and then only use elements in one period, but the sequence with long period may not have good randomness.
For example, the following linear congruence generator:
The result of generating a random sequence with an initial value of X 0 =3:
4,5,6,7,0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7,0,1,2,3......
It can be seen that the randomness of the above sequence is very bad.
The following theorem is proved in TAOCP . The coefficient a is chosen in the following way to produce an LCS with a maximum period of m.
The above theorem shows that when c is not equal to 0 (of course, it is impossible for c and m to be relatively prime to be equal to 0), it is possible to generate LCS with period m.
On the other hand, when c=0, that is:
Is it also possible to generate LCS with period m? The answer is absolutely impossible.
A simple proof by contradiction:
If c=0, an LCS with period m is generated, then 0 must be in this sequence, but if 0 is in the sequence, it will inevitably cause LCS to degenerate into a sequence of all 0s, so the original proposition must not hold.
Consider from another angle:
Consider that d is a factor of m, and X n is a multiple of d, that is, X n =kd, where k is some integer. So there is the following derivation:
It can be seen from this that X n+1 is also a multiple of d. Similarly, the following X n+2 and X n+2 are also multiples of d, and this sequence is not a sequence of numbers. Therefore, if each element in the sequence is required to form a coprime relationship with m, the number of elements in such a sequence is actually the Euler function to evaluate m, that is:
A few simple points are:
1) The modulus m should be as large as possible, usually at least greater than 2^30. For computational efficiency, the value of m is usually selected in combination with the word length of the computer.
2) If m is selected as a power of 2, that is, m=2^w, the selected a should usually satisfy that a modulo 8 is equal to 5.
3) When the selection of parameters m and a is reasonable, the choice of c is not very restrictive, but it is necessary to ensure that c and m are relatively prime. For example, c can choose 1 or 11.
4) The seed seed should be randomly selected, and the timestamp can be used as the seed.
Because the randomness of the least significant bits of X n is not very good, in application scenarios that are sensitive to the random characteristics of X n , the high bits should be used as much as possible. In fact, it is better to think of the value Xn / m as a uniform distribution between 0 and 1, rather than directly thinking of Xn as a uniform distribution between 0 and m-1. Therefore, if you want to generate uniformly distributed pseudo-random numbers between 0 and k-1, you should use the method of kX n /M. When constructing an LCG specifically, it is not necessary to refer to the table in TABLES OF LINEAR CONGRUENTIAL GENERATORS OF DIFFERENT SIZES AND GOOD LATTICE STRUCTURE [2] . This paper gives a number of optional parameters m and a for MLCG and LCG. Value: For MLCG, m takes the largest prime number less than 2^w; for LCG, m takes 2^w.
3. Pseudo-random number generation in JDK
Many platforms use the above linear congruence generator to implement the pseudo-random number generation mechanism. For example, the following are the corresponding parameters in common platform implementations ( from wiki ):
The code demo is as follows ( from rosettacode ):
package com.demo; import java.util.stream.IntStream; import static java.util.stream.IntStream.iterate; public class LinearCongruentialGenerator { final static int mask = (1 << 31) - 1; static IntStream randBSD(int seed) { return iterate(seed, s -> (s * 1_103_515_245 + 12_345) & mask).skip(1); } static IntStream randMS(int seed) { return iterate(seed, s -> (s * 214_013 + 2_531_011) & mask).skip(1) .map(i -> i >> 16); } public static void main(String[] args) { System.out.println("BSD:"); randBSD(0).limit(10).forEach(System.out::println); System.out.println("\nMS:"); randMS(0).limit(10).forEach(System.out::println); } }
In the implementation of the java.util.Random class in java, the key code of the generator is as follows:
private static final long multiplier = 0x5DEECE66DL; private static final long addend = 0xBL; private static final long mask = (1L << 48) - 1; protected int next(int bits) { long oldseed, nextseed; AtomicLong seed = this.seed; do { oldseed = seed.get(); nextseed = (oldseed * multiplier + addend) & mask; } while (! seed.compareAndSet(oldseed, nextseed)); return ( int )(nextseed >>> (48 - bits));//Discard low bits, keep high bits } public int nextInt() { return next(32); }
At the same time, it can be seen that the bits are deliberately truncated in the full implementation, the low bits are discarded, and the high bits are reserved.
四、Reference
1、Donald Knuth,The Art of Computer Programming, Volume 2
2、TABLES OF LINEAR CONGRUENTIAL GENERATORS OF DIFFERENT SIZES AND GOOD LATTICE STRUCTURE
3、https://en.wikipedia.org/wiki/Linear_congruential_generator