ENIGMA 2000 Newsletter - Issue 22

May 2004
Articles, newsreports and Items of interest :enigma2000-owner@yahoogroups.com

ENIGMA 2000 article - Just How Random is Random ?

Two offerings this time, first the excellent piece from IW who has kindly supplied software [available from the ENIGMA 2000 Numbers Group]:

Just How Random is Random ? - By Ian Wraith

[NOTE: The program is only available from ENIGMA 2000 Group].

It’s funny that while ENIGMA 2000's members frequently study most aspects of numbers stations transmissions such as the frequencies used and the code number the message is addressed to the actual traffic itself is largely ignored.

There is a good reason for this of course as it is rumoured that the majority of numbers stations are sending messages using the unbreakable one time pad encryption system. Plus of course many of messages sent by numbers stations probably contain no message and consist of nothing more than random numbers.

The purpose of these messages is just to confuse the other countries SIGINT organisations by preventing traffic analysis.

However these aren't the only kind of messages sent by the stations that interest ENIGMA 2000 members.

For instance station M21 sends Russian air defence information but there are also probably other stations sending machine encryption message and probably even tactical and training nets sending messages using crude hand encryption methods. I always thought it would be nice if there was Some way of knowing which messages were random and which weren't but I had no idea how this could be done.

But then in December last year I was talking to a friend of mine who is a medical researcher. Part of his work includes looking at the medical data from thousands of different patients. He then has to select, at random, just a few of these patients for more advanced tests.

However to prevent him from just selecting the patients he knows will prove his theories he explained that his papers have to mathematically prove the patients were picked at random. He told me that that this was done with something called a "Runs Test" which measures just how random a series of numbers are. When he said this to me I realised that there may be a way of knowing what kind of traffic a numbers station is sending.

But before we get into the practical applications of the Runs Test let me explain the basics of the theory behind it. The test looks at the number of "runs" in a sequence of numbers. A run is a series of numbers where each number increases. So this ..

7,11,36,37,49

is an upward run while ..

2221,2077,68

is a downward run. The runs test looks at the number of runs in a sequence of numbers and then calculates the odds of that number of runs occuring in that size of message. As you can imagine calculating the run test result from a series of say 200 numbers by hand would be time consuming so I decided to write a simple computer program that did this instead.

Thanks to there being lots of examples of how to program runs tests on the Internet this only took a few hours and I was ready to test the program.

For my first test I decided to give the program a sequence of numbers That definately weren't random so I created a text file which contains the numbers 1 through 49 in numerical order. When I ran it through the program it told me this sequence of numbers has a runs test number of -10.818. Next I programmed my computer to create a text file containing 30,000 random numbers.

When I ran this through the program it said it had a runs test number of -0.388.

So in other words the larger the number the more random it is. Now it is well known that computers are very poor at generating random numbers because if you think about it a computer can only do what it is told and randomness isn't something they can understand. So computers have to use something called a Pseudo Random Number Generator (PRNG) this is a formula that generates numbers which look random but aren't really. Given enough output from a PRNG any good cryptographer will be able to spot a pattern in the data.

For my next test I decided to give the program some real numbers traffic which was sent by E17 on 7/8/2000 and this gave a runs test value of +0.499. So no suprises here as it seems to prove that E17 is sending what appears to be random numbers. Of course this could be a one time pad encrypted message or it could just be random numbers to prevent traffic analysis.

Next I decided to look at several messages from the same station.

Due to the way it is sent there is more traffic around for XP than any other so thanks to the kindness of various Enigma 2000 members I used that.

When this was put through my program these were the results:

Date 18/04/2003 runs test value +0.925 Date 22/04/2003 runs test value 0.000 Date 30/05/2003 runs test value +1.261 Date 22/08/2003 runs test value -0.225 Date 05/09/2003 runs test value -0.372 Date 16/09/2003 runs test value +0.054 Date 19/09/2003 runs test value +0.277 Date 24/10/2003 runs test value +1.034 Date 21/12/2003 runs test value +1.273
Date 06/01/2004 runs test value -2.129 (*)
Date 08/01/2004 runs test value -1.527 (*)
Date 12/01/2004 runs test value +0.836 Date 13/01/2004 runs test value -0.458

Now the majority of these results didn't surprise me as they indicate the traffic sent by XP is random but the days marked with a * didn’t.

These much lower values indicate that the traffic sent on those days was less random than on the other days and I can only think of the following reasons for this ..

Perhaps the traffic sent on these days uses machine encryption rather than one time pads.
Perhaps the traffic sent on the other days was just random dummy traffic while these days contained real traffic.
Or perhaps there is something in the structure of the XP traffic and the way it is sent that has caused this.

It is hard for me to say which of these is the reason for my results.

I think what we need is one of the groups dedicated XP monitors to put every days’ XP traffic through the program and to look for patterns in the days on which the seemingly non random traffic was sent.

As another test I decided to put some Cuban numbers traffic through the program. I only had 2 sets of messages but these produced the following results:

V02A Date 14/08/2000 runs test value +0.703 V02 Date 25/10/2000 runs test value +1.192

So it looks as though both of these messages contained random numbers.

Now the runs test isn't the only test of randomness so decided to include a couple of other tests in my program. The first of these counts which digits appear in the message numbers as in a truly random message you would expect the digit 0 to make up 10% of the total digits, the digit 1 to make up another 10% and so on.

But if you see messages where one digit appears a lot more or a lot less than the others it strongly suggests non random numbers.

This test produced some interesting results when I looked at some traffic from stations G04 and M29 when the majority of messages from these stations contained double the number of 5 digits than would be expected. Plus it confirmed that from 2000 onwards the digit 9 never appeared in any G04 messages.

When I looked at some messages from station M10 it appears that some messages contain double or treble the number of 3 digits than would be expected. This suggests to me that the G04 , M04 and M29 stations are not sending one time pad or random dummy messages but something else.

The other test of randomness I have included in the program is a count of the times in which numbers in the traffic are repeated. Traffic with only a few number groups shouldn't contain many repeats numbers if they are random. The stations G04 and M29 contain a suspiciously large number of repeat groups which adds further weight to the suspicion these stations aren't sending random traffic.

The little program I have written can be downloaded from the groups Files area.

To install it to your PC create a folder for it on your hard drive (you can call that anything you want) then use an unzip program, such as Winzip, to uncompress the file. Then put all the files from it into the folder you have just created.

To run the program use "My Computer" to look in this new folder then double click on the file "Random_Test".

When you do this a small dialog box should appear which contains 2 buttons.

The lower button "Exit" will shutdown the program if you click on it.

If you click on the top button a file selection dialog box will appear which asks you to select a file containing the traffic you want the program to look at. I have included a number of sample traffic and test files for you to test the program with.

For instance the file UP.DAT contains the numbers 1 to 49 in numerical order while RANDOM.DAT contains 30,000 pseudo random numbers generated by my PC.

Once you select a file then click on the OK button and shortly afterwards a message box will appear telling you the runs test number of the data , the percentage makeup of its digits and the number of repeats. Usually it only takes a PC a couple of seconds to calculate this but if your PC is slow or if there is a lot of data (such as in the RANDOM.DAT file) then it will take longer.

Having done that you will probably want the program to look at some traffic which you have monitored. Creating a file for the Randomness Tester to read is easy. Open up the Windows Notepad program (or any text editor) and enter the numbers. These numbers can be anything between 1 and 5 digits long and there must be one or more non number character(s) between them. So for instance:
39384 14022 52325 54455 56441 43738 53140 00434 27153 92424 12541 75631would be OK, as would ..
39384 14022 52325 54455 56441 43738 53140 00434 27153 92424 12541 75631

However this ..
393841402252325544555644143738 531400043427153924241254175631
couldn't be read by the program.

Once you have created the file then save it with a name that ends in .DAT then my program can look at it.

Incidentally I had the thought that my program might not only be useful at looking at numbers station traffic but may also be useful in analysing the patterns of numbers that are used as the address of the person the message is intended for. If anyone wants to try it then it may be worth noting down the address numbers that messages are sent to by one particular station over a period of several months. Then enter these address numbers into a .DAT file and have the program look at them. It would be very interesting to know if these numbers are indeed random.

Please note that this article and my program are only a very simple and basic introduction to this complex subject but I hope it provides a useful introduction for members who have never thought about this before. It should, I hope, indicate to users when unusual traffic has been monitored or if the traffic might not be random and could be worth further study.

However, please don't draw any quick conclusions about the nature of traffic from a particular station on the basis of just a few messages. You really need to capture a lot of messages over a regular period of time from a single station before you can decide what kind of traffic you think it is carrying.

If you have any views on this program or ideas for possible improvements please email the ENIGMA 2000 mailing list. I don't claim to be an expert in the field (I had never even thought about it before last month) so I am pretty sure I have made plenty of mistakes but I hope this article has been an interesting read.

In closing I offer my thanks to all those members who offered their archived messages and advice on this matter.