Enigma - Historical Lessons in Cryptography

A Brief Note About Bletchley Park Today

At the time of writing this article (March / April 2020), Bletchley Park is closed to visitors and all staff have been furloughed due to the COVID-19 pandemic. The response and public reaction to the pandemic in the UK has been compared in some aspects to the public reaction during WWII. During the global crisis that was WWII, Bletchley Park was at its height, during this one it stands quiet.

While there are many sadnesses at this time and the closure of a museum is but a trivial one. My thoughts are with those who are out of work as once again Bletchley Park and other sites play their part in a global crises, this time by staying closed and preventing the unnecessary spread of the virus.

I look forward to the day when we re-open the doors of the Park as a true sign that this crisis has passed. Fittingly rather than the doors closing and people leaving Bletchley Park to be quiet and empty at the end of one crisis, this time the doors will opened and people flooding in will signify the end of another.

This is dedicated to all those on furlough, especially those from Bletchley Park.

Introduction

There are many documents, books and references available which describe how the Enigma Machine worked so why write another one?

The answer comes in several parts. Firstly I personally wanted to really understand how the Enigma Machine was operated and I felt that writing it all down would both help me get to grips with it and provide a nice reference. In the first instance for my to indulge and satisfy my own interest, if others should gain value from it then that is a bonus.

Secondly I wanted to write an article that discusses the machine from first principles so that anyone can understand it.

Thirdly I wanted to looks at the Enigma Machine both from how it works technically and how it was operated and discuss some interesting aspects which lead the codebreakers being able to recovery the keys and read the German radio traffic.

Finally I want to use the Enigma machine as a model for exploring some principles in cryptography that are valid even today.

With all this in mind, I’d like this article to appeal to those who have a purely historical interest in Enigma and how it was broken as well as to serve as starting point for a journey into the world of cryptography as it pretty much was for me.

One thing that isn’t discussed in as great detail as possible at this point is the breaking of Enigma by the Polish code breakers and at Bletchley Park. Although some detail is given on some of the codebreaking techniques, these are not discussed in deep technical detail. For now I want to largely focus on the how the Enigma Machine worked and how some of operational mistakes led to weaknesses would be exploited by the codebreakers. This will be the subject of other articles to follow.

Mechanised Secrecy

From the early 1920’s following World War I, anyone with an inclination to do so could go and buy an Enigma machine from its manufacturers Scherbius & Ritter or later Chiffriermaschinen Aktien-Gesellschaft corporation. Many government agencies, militaries and other organisation outside of Germany did look at Enigma as an option for mechanising the security of their communications.

As armies modernised yet again, highly mobile armed forces requiring the wide spread and rapid use of wireless communications required rapid and mechanised secrecy. And hence machines like Enigma were sought after. One would assume then, that with the machine being commercially available the characteristics of the Enigma were widely known to all those who cared about such things, and in fact, they were. Why then was the Enigma such a big problem for codebreakers a decade later? They already knew how the machine worked and some even had access to a commercial version of one. The answer may or may not be obvious but it is so fundamental to the way cryptography is done to this day it is worth spelling out.

Cipher System Secrecy

Even though it was possible to buy or later capture an Enigma Machine and understand it fully you would still not be able to break messages enciphered on it. This is because the Enigma Machine doesn’t just produce a single repeatable encipherment that is easily known once you see the machine. It is capable of producing multiple trillions of different encipherments based on the many unique ways you could configure the machine. You can think of a single Enigma Machine as being able to act as trillions of different machines each uniquely defined via the changeable wiring.

It is the state of the wiring as created by a particular machine setting and used to encipher a message that needs to be secret and not the details of the machine itself. Not only can you not use your captured machine to instantly decipher all other messages, but because there are so many combinations of wirings you can’t even use the machine to methodically test (also called brute force test) all the possible configurations of settings. At least not before the heat death of the universe, let alone in the 24 hours before the configuration is changed again and not using 1940’s technology. Therefore the Enigma Machine provided a sound and secure way of mechanically encrypting messages.

Knowing every millimetre of an Enigma Machine (either commercial or military) doesn’t get you anywhere near being able to decipher messages encrypted on one. It is true that knowing how the machine works is part of it, but if a military or government were to rely on that fact alone, their secrets would only be safe until a single machine was captured.

During the war the Germans expected the machines to fall into enemy hands and they even expected the wiring of the military machine components to be discovered but they rightly expected to continue using the machines securely even after all this was known. The lack of knowledge of the internal wiring of the machines components would frustrate codebreakers for a period for sure. But when they eventually did (very cleverly) figure out the wiring of the components of the Enigma Machines used by the military, this knowledge alone wasn’t enough to for them to be able break messages on a continuous basis.

This is an extremely important feature of every cipher system to this day: The security of any cipher system should never rely on the secrecy of the underlying mechanics of the system itself. In fact today the security of modern cipher systems is greatly improved by making them open and available to cryptanalyst to test and validate. Such that if the is a product purporting to have a new and improved proprietary encryption system the details of which are secret, it should avoid it like the plague because at best it hasn’t been properly tested and is most likely insecure and at worst it has backdoors in it.

In the next section the details of the Germany Military Enigma Machine will be discussed in detail in order to get a technical understanding of the machine and how it functioned.

The Enigma Machine

As mentioned above the Enigma Machine could vary its internal wiring state to produce multiple trillions of unique encipherment settings or keys. This is called the key space

The actual number is 103,325,660,891,587,134,000,000 or 103 sextillion (See how that number arises in the Maths section below) possible key settings and thus outputs. Which is a number so astronomically large it would take over 200,000 times the age of the universe to check each setting if it took one second to test each one. This clearly demonstrates that a head on attack of key space is impossible and this is why the Germans were so confident Enigma could not be broken.

To explain how the Enigma Machine was configured and how its internal wiring state would vary, a military Enigma Machine will be used as the model. The military Enigma Machines were similar to the commercial versions, however they were never sold publicly and the German army (who just before the war secured the exclusive rights to Enigma) added addition features such as the plug board, alphabet rings, and new rotor wirings which made them more secure and more portable.

An Enigma machine consists of a main body of the machine which has four key fixed components: a keyboard, the lamp board, the input rotor and the reflector. The machine has two operator changeable components: The rotors and the plug board (steckerbrett in German) wires. The plug board was introduced for the military Enigma machines and it is the most obvious way the military machines differ from the commercial machines.

Each of these elements is discussed below.

The Keyboard

The keyboard is a standard German QWERTZ configuration with electromechanical key switches which are wired electrically to the plugboard. When a key is pressed it mechanically rotates one or more rotors depending on where the rollover notches (see below) are located in the current state and starts the electrical current going through the plugboard.

An important note here is that the action of pressing a key mechanically rotates the rotor (or rotors depending on the turnover positions) before the electrical current enters the rotors. This means the initial setting is never actually used to encipher a letter.

A Detour into The Enigma Cipher System

At this point it is worth looking at how the cipher system the Enigma Machine implemented.

If the machine state didn’t changed during operation, the Enigma Machine would implement simple substitution cipher. This is where one letter of the alphabet is substituted for another based on a cipher alphabet or mapping between the input text (plaintext) and the output text (cipher text). The Enigma Machine’s wiring provides that mapping. However because one or more rotors moves on each keypress, the machine’s wiring state changes for every letter that is input from the plaintext. This means a new substitution cipher alphabet is generated each time a key is pressed. If you were to press the ‘A’ key multiple times in a row, the resulting cipher-text output for that ‘A’ would always be different a different letter. In this way, the Enigma Machine implements a poly-alphabetic (many alphabets) substitution cipher. In fact it can generate 103 sextillion different cipher alphabets as mentioned above.

If the wiring did not change and the Enigma Machine just output the same encipherment for the letter ‘A’ each time it was used in the plain text. i.e. the machine generates a single cipher alphabet based on a fixed key. Then the cipher would have been trivial to break. This is because a simple substitution cipher system like this does not alter the frequency of the occurrence of a letter. For example if the letter ‘A’ appears 100 times in a message and is enciphered to ‘X’ then ‘X’ would appear 100 times in the cipher text and codebreakers could use a technique called frequency analysis to break the cipher.

Frequency analysis is simply counting the occurrence of each letter in a cipher text then by looking at the frequencies that letters occur in a language its very easy to work out which plaintext letters map to which cipher-text letters. For example, in English the most commonly occurring letter is the letter ‘E’. If you were to tally the occurrences of all the letters in a piece of English substitution cipher text, the enciphered letter that occurs the most is likely to represent ‘E’. Once one letter is broken it then gets progressively easier to find the others, either by further frequency analysis or simply by the fact that words start to emerge and you can map letters by filling in the gaps in words.

The example below demonstrates visually how easily substitution ciphers can be broken by simple frequency analysis.

Removing letter frequency patterns is a critical feature of any cipher system. In fact removing any patterns in the cipher text is a critical feature of a cipher system. Cipher text output should look like random noise where all letter frequencies are roughly the same.

Below is a frequency chart for a text encoded with Enigma. This shows the effect of the poly alphabetic cipher on letter frequencies.

The example above was generated using the same large input text as for the simple substitution cipher example for comparison purposes. In reality Enigma messages were kept at ~250 characters to avoid effects like can be seen above and also to limit the rotors cycling which the Germans knew would make it easier to analyse.

The Enigma Machine certainly resisted frequency analysis, but the cipher text it produced was not devoid of patterns, as the codebreakers would discover.

The Plugboard

The plugboard consists of a pair of plug sockets for each letter on the keyboard. One socket is for the positive wire and the other is for the negative wire. A wire with a plug at each end is used to connect any letter on the plugboard to any other letter on the plugboard. This has the effect of performing an initial substitution even before the signal hits the rotors. It is possible to cable together up to 13 letters, however the standard was to use 10 cables. In fact patching all 13 letters is significantly weaker than using only 10. This is because there are fewer permutations of patch configurations when every letter is patched as opposed to the a combination on patched and un-patched letters.

When a plug is inserted it pushes a back a bar that normally shorts the plug socket and routes the circuit via the patch wire to the patched letter and then onto the rotors. When a letter is left un-patched the shorting bar is left in place and the letter on the keyboard is electrically connected directly to the rotors with no substitution at the plugboard taking place.

The plugboard adds a huge amount of possible key combinations. See the Maths section below.

The Input Rotor

The input rotor is a fixed non-rotating rotor which connects the plugboard to the rotors. The right most rotating rotor contacts the output pins on the input rotor. The current enters from the plugboard to the input rotor and then on through the movable rotors.

Initially the codebreakers assumed the link between the plugboard and the input rotor might be some random order they would have to work out, however it was discovered that on the military Enigma the wiring between the plugboard and the input rotor was in alphabetical order. i.e. Plugboard letter A connected to input pin 1 on the input rotor.

The Rotors

The rotors are cylinders with a notched thumb plate on them to allow them to be rotated by hand by the operator once inserted.

They have 26 spring loaded pins on one side that press against the 26 input pads on the rotor to its left. The pins and pads can be seen on the images below.

Each rotor also had a notch on its output side that determines where a turnover of the rotor to its left would occur. When the turnover notch (shown in the annotated picture below) rotates round to the the movement mechanism (a set of 3 pawls that push the rotors around), it allows the mechanism to engage the rotor to the left of the moving rotor which it turn will move the rotor to the left on one position. In this way the Enigma Machine operates like a car odometer.

On an army Enigma, three rotors are mounted on a rod and all three are inserted into the machine.

The army Enigma could house three rotors from a set of five available to choose from. The number of rotors the machine could house and the choices changed in some cases during the war, but for now we will focus on the original army Enigma in which there were three rotors chosen from a set of five.

The rotors could be put into the machine in any order. For example you could put any of the five rotors in the left, middle or right position in the machine any of the remaining four rotors in one of the remaining empty positions and any of the remaining three rotors in the last empty position. Leaving two rotors unused for a the particular configuration.

Not only would you choose which position to put the rotor in, you could also vary how the output terminals were labeled on the rotor itself by moving a ring (letter or alphabet ring) on the rotor. The letter ring showed either numbers or letters on it and it could be moved around to any desired starting position before inserting it in its position within the machine. These numbers or letters would be visible through apertures in the top of the rotor cover to show the operator the position of the rotors once they were inserted into the machine. Because the letter rings could be moved around on the rotors, it wasn’t enough just to know what indicators were at the start. You also need to know how they related to the input and output terminals on the rotor which was varied by moving the ring. Once inserted the operator would rotate each rotor to a chosen initial position.

When in place the rotor (or rotors depending on the state) rotate as the operator uses the keyboard. The right-most rotor would rotate on every keypress, the others would only rotate when the turnover notches allowed them to do so.

Each rotor has a unique path (internal wiring) through which the electrical current would travel. Entering at one point and exiting at another into the adjacent rotor or the reflector. Each combinations of rotors would produce a distinct electrical path.

The Reflector

The reflector is another static rotor component. It is located at the left of the movable rotors with the leftmost rotor contacting its input/output pins. Each pin on the reflector can either be an output or input because the job of the reflector is to take the electrical input and reflect it back out through the rotors on a different path. That is to say, a letter is transformed by the rotors twice via a two distinct wiring paths.

Another feature of the reflector is to ensure the Enigma machine is self-reciprocal meaning that if for example ‘Q’ is enciphered to ‘A’ using a given key, then ‘A’ is enciphered to ‘Q’ when using the same key. This means you can feed plain text (un-enciphered clear text) into the machine and get cipher text (enciphered text) out. But you can also feed in cipher text and get plain text out. In this way an Enigma machine can both encipher and decipher using the same key without needing to be reconfigured for each operation.

This property is a huge benefit operationally. However the way it is implemented in an Enigma machine introduces a major weakness and its worth taking a minute to discuss this here in context.

The way the reflector works also ensures that no plain text letter can be enciphered as itself. i.e. Pressing ‘A’ on the keyboard will never result in an ‘A’ being output at the lamp board. At first glance this may seem like a beneficial property because every letter is always seemingly obscured. But in fact this is a major weakness that was exploited by the codebreakers.

Knowing that no letter in the cipher text can represent itself in the plain text means an Enigma cipher text leaks information about the plain text and therefore the key. This is a very bad property of any cipher system. Information about the plain text or the key should never be able to be gained by analysis of a cipher text in any secure cipher system.

Side Note: The Enigma in the picture must have been built after the 1st of November 1937. We can tell this because the Reflector is labelled with a ‘B’. The B reflector was introduced on the 1st of November 1937.

The Lamp board

The lamp board is a simple field of 26 bulbs, 1 for each letter of the alphabet. There is a cover which sits over the lamps and rotors, each bulb sits beneath a translucent window on the cover. Each window has a letter is printed on it. The layout mirrors the keyboard. The lamp board is connected to the plugboard and when the electrical current returns from its path from the keyboard through the plugboard and rotors then back through the rotors and plugboard again, it illuminates a bulb which shines through its window indicating the enciphered letter.

Note again that the current passes through the encrypting mechanisms of the machine twice for each keypress as implemented by the reflector.

The Maths

It was stated above that there are 103 sextillion possible settings for the Enigma machine. Without diving into too much complicated maths, it can be shown how this number comes about and how the components discussed above contribute.

The Rotor Maths

With the choice of five rotors, there are 5 x 4 x 3 = 60 ways to choose the 3 rotors. There are 26 starting positions for each rotor. This means there are 26 x 26 x 26 = 17,576 possible starting states that can be generated using the rotors.

17,576 is the correct number if all rotors rotated through 26 positions. However there was a mechanical effect that would cause the middle rotor to double step this meant in reality the rotor period was 26 x 25 x 26 = 16,900

Finally the turnover notch positions with regard to the alphabet rings need to be taken into account. The notch can be at 1 of 26 positions relative to the alphabet ring. As discussed above the notch affects the rotor to its immediate left. Because the right most rotor is not the the left of another rotor there is no notch effect on it. This means only the notches in the middle and left positions matter. Therefore there are 26 x 26 = 676 possible notch positions.

The Plugboard Maths

If 20 letters are plugged (10 wires) it creates the 10 plugged letter pairs and leaves 6 letters unplugged. This means there are 26!/6! ways to choose those 20 letters. This makes the 10 plugged pairs, but the order of the pairs doesn’t matter (i.e. A plugged to Z is the same as Z plugged to A). So we need to divide in 2 for each pair. Which gives us: 26!/6!*2^10*10! = 150,738,274,937,250

It can be seen how much of an impact the plugboard has on the numbers.

The Result

* 16,900 - Possible rotor start positions
* 60 - Possible ways to choose 3 out of 5 rotors
* 676 - Possible locations for the turnover notch
* 150,738,274,937,250 - Possible combinations for wires on the plugboard

Multiply all these factors together and the answer is:

103,325,660,891,587,000,000,000 (103 Sextillion) possible settings.

Operation

Setting up the Enigma Machine

To set up an Enigma Machine for use, an operator has to configure the machine by following a set of instructions for configuring the Key. The key is the secret setting used to establish the unique configuration used to encrypt all messages send over a defined period. In most cases a key was set daily.

The next few sections below detail what the key consists of and how it was setup on the machine.

Components of the Key

On looking at an Enigma machine it might be tempting to think that the key is simply what is shown in the rotor windows.

However the daily key is not a single element but consists of carefully setting up the Enigma machine in a specific way and involves all of the components that were discussed above.

An Enigma key consists of the following elements:

• The particular rotors to use - Which 3 out for the 5 available
• The order of the rotors - Which one are left, middle and right
• The ring settings - Where the alphabet rings on the rotors should be rotated to
• The starting position of rotors in the machine - What letter / number should be displayed in the rotor windows initially
• The plug board wiring - Which letters should be wired to which other letters on the plug board

These settings were listed on a monthly key sheet which shows the setting for each of these elements for every day of the month. The key sheet or sheets or code books were the things the Germans needed to protect the most, should these fall into enemy hands then communications for that entire month with be breakable and new sheets would need to be generated. On occasion these sheets were captured and there are some heroic tales about how some of these captures or pinches, as they were called, happened. One of the most famous is the ‘Petard Pinch’ which was the capture of the codebooks for the Naval M4 enigma from U-boat U-559 which was pulled off by members of the crew of the HMS Petard. The story of the Petard Pinch can be seen at Bletchley Park on video in Hut 8

A typical key sheet would have looked like this:

From: http://users.telenet.be/d.rijmenants/en/enigmaproc.html

Note that the days a numbered from 31 at the top down to 1 at the bottom. This is because the key sheet would typically be kept in a safe accessible only to a senior officer. Each day the officer would tear off the bottom row of the key sheet and give it to his Enigma operator to enable him to setup the machine for the day. This helped to mitigate the risk of an entire months worth of keys getting captured if the Enigma machine and operators were targeted.

Also note the kenngruppen or key identifier groups. These were not strictly part of the key but were used to identify which key was in use for a particular message. See below in the Operating the Enigma Machine section for more details on this.

Configuring the Enigma Machine with the Key

Each day, the Enigma Operator would setup the machine with the daily key as specified by the details from the key sheet / code book. The machine would then be setup as follows:

• Adjust the alphabet rings on the required rotors
• Put the rotors on the rotor shaft in the specified order and insert them into the machine
• Wire the plugboard according to the letter pairings shown on the key sheet
• Destroy or secure the key sheet strip

Note that the rotation of the rotors in the machine is not part of the daily key setting. There was no explicit instruction for the operator to rotate the rotors. This is of interest because this was one of many operator or operational weaknesses the codebreakers exploited. It is worth mentioning that the majority of the flaws that the codebreakers used to break enigma were weaknesses introduced by the operation of the machine and not weaknesses of the machine itself. Operational weakness will be discussed briefly below and further in a later codebreaking article.

Once the machine was setup it was ready for use to send messages for a 24 hour period.

Note that there was not a single codebook for the entirety of the German WWII armed forces. The army, navy and airforce all had separate sets of codebooks and even different machines. Even within each branch of the military there were multiple codebooks in use. For example, army had serval codebooks in use across the various deployments and theatres. These formed networks. Messages destined for another network would have to be relayed via a hub or relay point which would receive on one Enigma Key and resend on another. Therefore the codebreakers at Bletchley Park not only had to break a single Enigma key every day, but break multiple keys for the multiple Enigma networks.

Over and above knowing what the content of the messages were, Bletchley Park and the codebreakers were also interested in the make-up of the networks. A lot of valuable intelligence could be gained by simply understanding how message traffic was flowing. This lead to the birth of traffic analysis. This was Pioneered by the architect of the Bletchley Park system, Gordon Welchman

Side Note - Traffic Analysis

Traffic analysis is another key tool in intelligence gathering. It is the process of gathering information such as which entities were sending and receiving messages, where they were sent from , what time they were sent and traffic volumes amongst other metadata. Bletchley Park got this information from radio direction finding and by practises such as identifying individual radio operators signatures or fists as they were called. Each operator had a unique way of sending the morse code messages and a skilled intercept operator could not only tell them apart via their fist but she (as they were more often than not women) could also identify individual senders.

Traffic analysis was done without decrypting the message itself and huge amounts of valuable intelligence was gained by watching the traffic alone. Troop movements, reassignments, reactions to other messages and other operational information could be obtained without having to read a single word of the actual message. In this way, maps of the various Enigma networks could be built. Knowing the size of each network and the interconnects between them was extremely valuable in and of itself.

Traffic analysis could also give clues as to the content of messages which could in turn be used for cracking a key. For example if a unit moved from one position to another, code breakers could infer that messages sent prior to the move were the movement orders. Knowing when a message was relayed was also important. Knowing when a message from a network where the key was broken was relayed to a network where the key wasn’t broken would give codebreakers an in on the unbroken key.

Traffic analysis is still a vital tool used by intelligence agencies and military signal intelligence units today. A simple example used by law enforcement regularly is requesting mobile phone metadata on a suspect. They need not know the detail of the calls, but know who a suspect contacted and when is critical information.

The sections below cover how the machine was operated and used to encipher and decipher messages.

Operating the Enigma Machine

Once setup with the daily key an Enigma Machine is ready to use. The operator would now not alter the rotor’s or plugboard until the following day and it was time to change the key again. The only element an operator could change between key changes was the starting point of the rotor wheels.

The first thing an operator would do before starting to encipher a message is randomly rotate each rotor to a new starting position. This was called the Grundstellung. The Grundstellung was attached to the message in the clear i.e. not part of the cipher text. The operator would then pick a second set of three letters at random to use for the Message Key. The chosen message key is then entered via the keyboard using the Grundstellung setting and the output is recorded.

Side Note - The Herival Tip

It was sometimes the case that for the first message of the day the operator would simply use whatever letters or numbers were showing after the machine had been setup for the day as the Grundstellung, rather than rotating the rotors to a new position. This seems like a safe thing to do, however a clever codebreaker by the name of John Herivel reasoned that the operator, when setting up the machine, would rotate the alphabet ring to its key position then insert the rotors into the machine at or near the position of the ring settings for each rotor. If the operator then didn’t rotate the rotor for the first message of the day, the the Grundstellung and message key would provide a tip for the ring settings to the codebreakers. Hence this became known as the Herival Tip. A way for the codebreakers to find the ring settings of the rotors for a particular key. This is one example of a technique developed at Bletchley Park that exploited operator error. Its also an amazing insight into how the codebreakers were able to think outside the box and make logical leaps which allowed them to shortcut their way into breaking Enigma. More of these will be discussed in the codebreaking article to come.

Message Keys

A message key and Grundstellung are important because it ensures that each message is sent with a different configuration as defined by the rotation of the rotors. If the same key was used for every message then this would allow codebreakers to analyse the ciphertext of a set of messages and begin to work out the key via logical operations. This is another core principle in cryptography even today. You should never use the same key for multiple messages. In modern ciphers instead of Grundstellung and message key, an initialisation vector used. This is a randomly generated chunk of data built by the cipher algorithm which ensures each message has a unique component to the key. This is why cryptographically secure random number generators are important for modern cryptography. Having a bad random number generator in a modern cipher is analogous to having a lazy Enigma Operator.

Enigma Message keys and the Grundstellung are quite a good way of understanding initialisation vectors, which are sometimes hard to get to grips with. How they are used is discussed further below.

Sending

To recap:

• The operator would choose 3 random letters to use for the starting position (Grundstellung). This would be recorded on the cipher sheet in the clear to be sent as part of the message metadata (data about the message but not part of the message itself) to the receiving end

• He would choose a second set of three letters to use as the message key

• He would rotate the rotors so the letters of the Grundstellung are shown in the rotor indicator windows and encode the message key. The encoded message key would then be sent as part of the encrypted message’s metadata with the Grundstellung

• He would rotate the rotors to the message key positions ready to encrypt the body of the message

  Note: Before 1940, the message key was encode twice and both encodings sent with the message in order to eliminate errors. This was a critical weakness that was used by the Polish and British to break Enigma early on. The Germans realised this was a weakness and stopped the practice. This significantly slowed down codebreaking early on.

• The final piece of metadata was to include one of the Kenngruppen letter groups for the days key. This was prefixed with another 2 randomly selected letters to make up a 5 letter group. This was to assist the receiving end with identifying and validating which key was used to encode the message

  Note: Individual Enigma messages were limited to 250-350 letters because the German cryptanalysts knew longer messages would be easier to break because cycles would start to appear as the rotors went round full rotations. Therefore if a message to be sent was too long it was be broken up. New message keys would be used and a second group from the kenngruppen for the days key would be selected. In this way a single message would be broken up into several Enigma encoding’s and sent as a multi-part message.

Before any encrypted message was generated and sent these additional pieces of information (metadata) were included on the cipher sheet.

An Example

The rotors are set to three randomly chosen letters for the Grundstellung: K V H. K V H -> Written on the Cipher sheet as the Grundstellung

Three randomly chosen letters for the Message Key and are encoded with the Grundstellung setting: Q H M Chosen and encoded to L V A. L V A -> Written on the cipher sheet as the message key.

One of the three letter Kenngruppen letter sets is prefixed with two randomly chosen letters: C X chosen. C X O O L -> Written to the cipher sheet as the Kenngruppen

The rotors are rotated to the message key setting: Q H M in this example.

The cipher clerk is now ready to start encoding the body of the message and all the data is provided for the receiving cipher clerk to be able to decode the message using the daily key.

From the example it can be seen how an element of randomness is added to each message to ensure each message has a unique key setting ensuring the codebreakers don’t have a large body of text using the exact same key.

Following generating the message keys and recording the relevant data on the cipher sheet, the cipher clerk would enter the message to be sent into the Enigma machine and record each encoded letter on the cipher sheet. Encoded messages were broken into blocks of five letters.

A typical message based on the example above might look like: Based on the sample from: http://users.telenet.be/d.rijmenants/en/enigmaproc.html

Side Note - Depth

As mentioned, using a random starting position (Grundstellung) and random message key ensured each message had a random component to the key which in turn meant each message was unique. Even if an operator were to send the exact same message body, he would use a different Grundstellung and message key ensuring that even an identical message sent twice looked completely different. Sometimes though, operators would get lazy and and send two messages using the same Grundstellung and message key. When this happens the messages are said to be in Depth. At Bletchley Park they called this a depth. Codebreakers would hunt for depths by collecting enough ciphertext and using clever techniques, look for depths generated by the machine to unravel a key (more on this in a later article). This took a lot of time and effort, but a depth generated through operator error or laziness would allow them to significantly shorten the time to break a key.

When a depth was discovered, either my analysis of a large enough amount of cipher text or by intercepting two messages using the same key, the codebreakers could see how the key was affecting transformations over plaintext’s and infer the key. Depths were one of the ways of breaking Enigma and codebreakers used them extensively.

The same is true of depths today. Sending two messages with an identical key severely weakens the cryptographic security of the enciphered message. However as commented on above, modern ciphers handle the message key generation internally via a randomly generated initialisation vector, allowing the sender to only have to provide an initial key. Commonly in modern ciphers this is a password or passphrase.

Receiving

At the receiving end, the Enigma Operator would have the machine setup with the same daily key. He would then validate which key was in use and that it corresponded to his current daily key by looking at the Kengruppen (OOL in the example above).

Once confirmed he would set the rotors to the starting position (Grundstellung) as indicated by the message metadata (QHM in the example) then type the Message Key (LVA in the example) into the machine and read off the letters that light up for each input. He would then set the rotors in his machine to the letters that resulted from typing in the message key. He is now ready to decipher the message body. Entering the letters from the message body one by one and recording the resulting output he would reassemble the plaintext message.

This is how a message was encrypted and decrypted using an Enigma Machine during World War II. As can been seen, the procedure involved a lot of different components and security features. The keys are more complicated the first meets the eye and the operating procedure is not as simple as just entering messages based on a fixed key.

Summary

The detail above is intended to give a full breakdown of how an Enigma machine and the Enigma cipher works as well as how the machine was operated in practise. Some side notes and technical detail on code breaking and the practices employed at Bletchley Park are included in context to provide a flavour for the problem of Enigma and how it was approached by the code breakers.

Enigma is a relatively simple machine on the face of it, but the actual set-up and operating of it are not as simple as might first meet they eye. The intention of this article is to detail how Enigma was used in the field to send secure messages and to hint at what led to it being broken despite it being a relatively secure cipher system for its time.

As mentioned in the introduction above in subsequent articles I hope to do a deeper dive into more of the technical aspects of the Enigma Machine from a code breaking point of view and look at the machines used by the Polish and British codebreakers to break it.

Notes, Credits and References

Acknowledgements

While extensive reading and research was done for this article there will undoubtable be mistakes. These mistakes are all mine and mine alone.

The Enigma machine and rotors shown in this article were taken by me at Bletchley Park. The machine photographed is one of the machines used by the Bletchley Park Trust’s Education Department. As a volunteer steward and guide I was lucky enough to be able to use and familiarise myself with a real Enigma machine which helped greatly with this article.

I highly encourage everyone to visit Bletchley Park. If you are at all interested in the history of codebreaking during WWII and the Enigma. There is far more to learn there than you can read here. If you happen to see me there on your visit, please come and say hello!

Code

Some custom tools were used to generate the encoded messages and perform the frequency analysis. Including a full emulation of an Enigma machine. These are all publicly available on my GitHub site:

• https://github.com/andrewsjg/crypto-tools - Simple substitution cipher and frequency analysis tool
• https://github.com/andrewsjg/go-enigma - A Golang implementation of an Enigma Machine

Books

The following books are fantastic references on Bletchley Park, Codes and Ciphers, Codebreaking and the Enigma Machine itself:

The Code Book: The Secret History of Codes and Code-breaking by Simon Singh

This is my No.1 recommendation for anyone interested in codes and ciphers and wants a compelling read that presents all the technical detail without overwhelming the reader. For me Simon Singh is one of, if not the best popular science writer working today. Every book I have read by Simon Singh has been a literal page turner. He has a compelling writing style that engages the reader with the topic at hand in such a gripping way. The Code Book is no exception.

Simon himself owns an Enigma Machine which he uses for demonstrations, lectures and learning.

The Hut Six Story: Breaking the Enigma Codes by Gordon Welchman

Written by the code breaker and head of Hut 6, Gordon Welchman. The man who in addition to being a talented codebreaker, created a large amount of the processes and procedures used at Bletchley Park to turn it into a codebreaking factory. Welchman paid a big price for writing this book as he broke the silence on what was done during the war. He was ostracised by the intelligence agencies he worked for. He was compelled to write the book in order to set the record straight as some distorted half accounts were starting to appear. He felt it was time the codebreakers got the credit they deserved and ensure the lessons of the past were not lost. The book goes into great technical detail and is the first hand account of the time that I would recommend everyone interested in the technical aspects and mechanics codebreaking at Bletchley Park during World War II

XY&Z: The Real Story of How Enigma was Broken by Dermot Turing

Sporting famous surname in the history of Bletchley Park, AI, computation and mathematics. Dermot Turing is the nephew of Alan Turing and in this book he details how Enigma was broken not just by Bletchley Park but in collaboration with the French secret services and the Polish codebreakers.

In the past there has been a lot of misinformation around the massive part the Polish codebreakers played in breaking Enigma. This is something I would like to write about one day also, but in XY&Z Dermot Turing does a fantastic job of telling this amazing story. Its a very well written book and an enjoyable read especially for people with an interest in the history of codebreaking during WWII

The Codebreakers: The Comprehensive History of Secret Communication from Ancient Times to the Internet by David Khan

Often called the definitive work on the use and breaking of codes and ciphers, The Codebreakers, is a weighty reference that covers everything to do with codes and ciphers throughout history. This book is an essential reference for all people with an interest in the subject.

There are countless other amazing books on Codes and Ciphers, Codebreakers and the Enigma Story. To many to list here. If you have any recommendations or questions feel free to reach me via Twitter (@jg_andrews)

Web Resources

For the descriptions of the operating procedures and pictures of messages, Dirk Rijmenants enigma procedures page, was a critical reference.

The late Tony Sales very detailed Codes and Ciphers website provided huge amounts of technical detail

The Enigma Wikipedia pages of course provide a lot of good information which was used in researching some of the detail above

The Complexity of the Enigma page on Graham Ellsbury’s website helped me to validate the numbers in the Math section.

Both Computerphile and Numberphile have excellent video’s on Enigma which are worth checking out. Note that the Numberphile video over simplifies the number of settings by omitting the notch positions from the calculations. They arrive at the number of settings that was used for many years prior to the 103 sextillion figure.