Signals intelligence in the Second World War

From Citizendium
Jump to navigation Jump to search
This article is developing and not approved.
Main Article
Discussion
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article is under development and subject to a disclaimer.

In a true world war, signals intelligence (SIGINT) still tended to be separate in the various theaters of the Second World War. Communications security, on the part of the Allies, was more centralized. Given that there were three major Axis powers, each primarily operating in a subset of the theaters, it is convenient to look at SIGINT from a primarily theater standpoint. From the Allied perspective, the critical theater-level perspectives were the ULTRA SIGINT against the Germans in the European theater (including the Battle of the Atlantic, the Mediterranean Theater, and MAGIC against the Japanese in the Pacific Theater and the China-Burma-India theater. Germany enjoyed some SIGINT success against the Allies, especially with the Merchant Code and, early in the war, reading American attaché traffic. Japan was the least effective of the major powers in SIGINT.

In addition to the official Allies and Axis battle of signals, there was a growing interest in Soviet espionage communications, which continued after the war.

See SIGINT from 1945 to 1989 for the history of SIGINT after the end of World War II, through the end of the Cold War and the rise of regional and nonstate concerns. SIGINT from 1990 to the present deals with the next historical period.

Allied European Theater

The use of SIGINT had even greater implications during World War II. The combined effort of intercepts and cryptanalysis for the whole of the British forces in World War II came under the code name "ULTRA" managed from the former Government Code and Cypher School (Bletchley Park).

(British) Royal Navy

Early on, Admiralty dismissal of SIGINT information (also traffic analysis in this instance) contributed to the loss of HMS Glorious in 1940.

Perhaps the most important role SIGINT played for the Royal Navy, and the merchant ships it protected, was in the Battle of the Atlantic. By comparison with the close and garrulous radio communication between the U-boat submarine high command, BdU, and German submarines in the Atlantic, US submarines in the Pacific enjoyed the freedom of fish. While ULTRA cryptanalysis certainly played a role in dealing with German submarines, HF/DF and traffic analysis were complementary.

It is unclear why the German submarine command believed that frequent radio communications were not a hazard to their boats, although they seemed confident in the security of their Enigma ciphers, both in the initial three-rotor and subsequent four-rotor versions (known as Triton to the Germans and Shark to the Allies). There was an apparent, mutually reinforcing belief that wolf pack attacks by groups of submarines were much more deadly than individual operations, and confidence the communications were secure. Arguably, the Germans underestimated HF/DF even more than they did British cryptanalysis [1]. Apparently, the Germans did not realize that the Allies were not limited to slow, manually operated direction finders, and also underestimated the number of direction finders at sea.

Battle of Britain

ELINT and electronic warfare became critical parts of the Battle of Britain. R.V. Jones was a key scientist in the "Battle of the Beams", defeating Nazi radio navigation systems (e.g., Knickebein). While the ULTRA COMINT successes against the Germans were not declassified until 1975, Winston Churchill paid homage to electronic warfare, and its companion ELINT, in his series on the Second World War: "During the human struggle between the British and the German Air Forces, between pilot and pilot, between AAA batteries and aircraft, between ruthless bombing and fortitude of the British people, another conflict was going on, step by step, month by month. This was a secret war, whose battles were lost or won unknown to the public, and only with difficulty comprehended, even now, to those outside the small scientific circles concerned. Unless British science had proven superior to German, and unless its strange, sinister resources had been brought to bear in the struggle for survival, we might well have been defeated, and defeated, destroyed."[2]. In modern terms, of course, MASINT was as important as SIGINT in defeating Nazi navigational systems, with radar control of the defenses a key part of the Battle of Britain.

French Resistance and Free French

France consolidated a number of general intelligence and SIGINT units in World War II, producing the wartime Directorate of Studies and Research (DGER) by November 1944. As the Cold War heated, France was concerned with the presence of Communist networks among these units, so, in 1946, created the External Documentation and Counterespionage Service (SDECE) subordinated to the prime minister.[3]

Efforts at US coordination during World War II

During the Second World War, the US Army and US Navy ran independent SIGINT organizations, with limited coordination, first on a pure personal basis, and then through committees. Perhaps the strongest outside effect, prior to and during WWII, was the United States Department of State and the White House, the only consumers of intelligence outside the military, especially since both the Army and Navy wanted to have the prestige of providing them with diplomatic COMINT. Note that while the Office of Strategic Services was a fairly autonomous WWII agency, it still, technically, reported to the Joint Chiefs of Staff and received COMINT through military channels. [4]

During the war, the military departments became concerned with the creation of new cryptanalytic units in the US government, including the Federal Bureau of Investigation (FBI), Federal Communications Commission (FCC) and Department of State. The military finally formalized the sharing of targets in 1944, but that did not cover the non-military organizations. They established a Joint Army-Navy Radio Intelligence Coordinating Committee, which soon changed its name to the Joint Army-Navy Communications Intelligence Coordinating Committee.

US Army

After the Normandy landings, Army SIGINT units accompanied major units, with traffic analysis as or more important than the tightly compartmented cryptanalytic information. Bradley's Army Group, created on August 1, 1944, had SIGINT including access to ULTRA. Patton's subordinate Third Army had a double-sized Signal Radio Intelligence Company attached to his headquarters, and two regular companies were assigned to the XV and VIII Corps.

The 3250th Signal Radio Intelligence Company, attached to V Corps, moved 10 times in June and July 1944, and suffered nearly 20 percent casualties during the Battle of the Bulge, including four killed in action. [5]

US Navy

In World War II ASW, shore or ship-based SIGINT often vectored long-range patrol aircraft to U-boats, which they might detect visually or by airborne radar if the submarine was surfaced, or by early sonobuoys used from 1944 on, which could cue dropping depth charges or very early homing torpedoes. The Army demonstrated feasibility of the AN/CRT-1 sonobuoy, and, by 1944, the Navy had ordered almost 60,000 Cite error: Invalid <ref> tag; invalid names, e.g. too many

A daring US Navy feat that received very mixed reviews was the capture of the Unterseeboot 505 (U-505) by CAPT Daniel Gallery's escort carrier group. While useful cryptomaterial, including an [[ENIGMA machine], was taken from the boat, Gallery and his immediate chain of command were unaware of the ULTRA successes against German submarines. There was considerable concern at Bletchley Park that if the Germans realized a U-boat, and presumably its Enigma had been captured, the Germans might change cryptosystems. The notoriously hot-tempered Chief of Naval Operations, FADM Ernest J. King considered court-martialing Gallery, but relented and authorized the award of a Distinguished Service Medal with a classified citation.

Appropriately, however, the first US sailor, LT Albert David to go down the hatch of submarine, which might have scuttling charges about to detonate or have water rushing in, received the Medal of Honor. The two sailors behind him received the Navy Cross.

Axis European Theater

The entire Nazi system suffered from Hitler's deliberate fragmenting of authority, with Party, State, and military organizations competing for power, with only Hitler really pulling the strings. Hermann Goering also sought power for its own sake, but was much less effective as the war went on and he became more focused on personal status and pleasure.

German air intelligence, during the Battle of Britain, suffered from the structural problem that subordinated intelligence to operations. Operations officers often made conclusions that best fit their plans, rather than fitting conclusions to information[6].

In contrast, British air intelligence was systematic, from the highest-level, most sensitive ULTRA to significant intelligence product from traffic analysis and cryptanalysis of low-level systems. Fortunately for the British, German aircraft communications discipline was poor, and the Germans rarely changed call signs, allowing the British to draw accurate inferences about the air order of battle.[7] A 1939 German intelligence study [6] discounted British radar and ground-controlled interception, and believed the only serious defenses were in the London area. Goering was not receptive to dissenting views that key targets were out of bomber range, and significantly out of the range of escort fighters.

Allied Pacific Theaters

Several theaters were involved in this part of World War II: CINCPAC/CINCPOA, CINCSWPAC, CINCCBI.

Allied cooperation in the Pacific Theater included the joint RAN/USN Fleet Radio Unit, Melbourne (FRUMEL), and the Central Bureau which was attached to the HQ of the Allied Commander of the South-West Pacific area.

Australian Army

After consultations between Australian and US signal and communications senior staff, MacArthur ordered Central Bureau to be created, partially to avoid his being dependent on Navy SIGINT[8]. Central Bureau was made up of:

  • The intelligence section of the former No. 4 Australian Special Wireless Section
  • Australian Military personnel
  • RAAF personnel
  • US Army intelligence personnel who had escaped from the Philippines
  • US Army intelligence personnel from USA (6 officers and 8 men of the 837 Signals Service Detachment)
  • British intelligence staff from Singapore

At first, Central Bureau was made up of 50% American, 25% Australian Army and 25% Royal Australian Air Force (RAAF) personnel, but additional Australian staff joined. In addition, RAAF operators, trained in Townville in intercepting Japanese telegraphic katakana were integrated into the new Central Bureau. Other components of Central Bureau included:

  • the Geographical Section which produced maps and geographical data about the SWPA
  • the Allied Translator and Interpreter Section (ATIS) which interpreted millions of captured documents, intercepted messages and interrogated thousands of Japanese POW's
  • the Australian Coast Watching Service
  • a POW interrogation center.

Central Bureau broke into two significant Japanese Army cryptosystems in mid-1943.

Australian Navy

FRUMEL was the joint US-Australian naval SIGINT unit. Commander, later Captain, Eric Nave did not stay long with FRUMEL, which was put under U.S. Navy control in mid-1942. He was sent to Central Bureau in mid-1942, but it has been suggested he dealt only with lesser Japanese systems, although he had both Japanese language skill and experience with their cryptosystems. The major systems were the target of US Col. Abraham Sinkov.[9]

Until Central Bureau received replacement data processing equipment for that which was lost in the Philippines, as of January 1942, U.S. Navy stations in Hawaii (Hypo), Corregidor (Cast) and OP-20-G (Washington) decrypted Japanese traffic well before the U.S. Army or Central Bureau in Australia. Cast, of course, closed with the evacuation of SIGINT personnel from the Philippines.

US Navy

US strategic stations targeted against Japanese sources included Station HYPO in Hawaii, Station CAST in the Philippines, station BAKER on Guam, and other locations including Puget Sound, and Bainbridge Island.

US COMINT recognized the growing threat before the Pearl Harbor attack, but a series of errors, as well as priorities that were incorrect in hindsight, prevented any operational preparation against the attack. Nevertheless, that attack gave much higher priority to COMINT, both in Washington DC and at the Pacific Fleet Headquarters in Honolulu. Organizational tuning corrected many prewar competitions between the Army and Navy.

Perhaps most dramatically, intercepts of Japanese naval communications [10] yielded information that gave Admiral Nimitz the upper hand in the ambush that resulted in the Japanese Navy's defeat at the Battle of Midway, six months after the Pearl Harbor attack.

US Army

The US Army had shared, with the Navy, the Purple attack on Japanese diplomatic cryptosystems. Many histories assume Purple included Japanese military cryptanalysis, but those were separate projects, although generally under the same organizations.

After creation of the Army Signal Security Agency, the cryptographic school at Vint Hill Farms, Warrenton, VA, trained analysts. As a real-world training exercise, the new analysts first solved the message center identifier system for the Japanese Army. Until Japanese Army cryptosystems were broken later in 1943, the order of battle and movement information on the Japanese came purely from direction finding and traffic analysis.

Traffic analysts began tracking Japanese units in near real time. A critical result was the identification of the movement, by sea, of two Japanese infantry divisions from Shanghai to New Guinea. Their convoy was intercepted by US submarines, causing almost complete destruction of these units. [5]

Army units in the Pacific included the US 978th Signal Company. based at the Allied Intelligence Bureau's secret "Camp X", near Beaudesert south of Brisbane in southern Queensland. [11]. This unit was a key part of operations behind Japanese lines, including communicating with guerrillas and the Coastwatcher organization. It also sent radio operators to the guerrillas, and then moved with the forces invading the Philippines.

US Army Air Force

Even as the planes burned at Clark Field, hours after the Pearl Harbor attack, LT Howard Brown, of the 2nd Signal Service Company in Manila, ordered the unit to change its intercept targeting from Japanese diplomatic to air force communications. The unit soon was analyzing Japanese tactical networks and developing order of battle intelligence. He moved from Manila to Corregidor on Christmas Eve.

They learned the Japanese air-to-ground network was Sama, Hainan Island, with one station in Indo-China, one station near Hong Kong, and the other 12 unlocated.[5] Two Japanese naval stations were in the Army net, and it handled both operations and ferrying of aircraft for staging new operations. Traffic analysis of still-encrypted traffic helped MacArthur predict Japanese moves as the Fil-American forces retreated in Bataan.

Evacuated, as were most SIGINT people, from the Philippines, Brown, helped build the Australian-American intercept station, and 126th Radio Intelligence Company, at Townsville, Queensland. He later trained the Air Force SIGINT staff. US Air Force Far East, and its subordinate 5th Air Force, took control of the 126th in June 1943. The 126th was eventually placed under operational control of U.S. Air Force Far East in June 1943 to support 5th Air Force. Interception and traffic analysis from the company supported the attack into Dutch New Guinea in 1944.[5]

The US began airborne ELINT against Japanese radar in the Aleutians, using a modified B-24 aircraft in January 1943. ELINT was much less significant in the early Pacific War than in the European Theater, probably because strategic bombing using electronic navigation aids was not a critical issue.

US Marine Corps

In 1943, the US Marines organized the 2nd Radio Research Platoon, which was the original unit in a chain of tactical SIGINT units that also made strategic contributions[12].

Japanese SIGINT

Japan had been fighting in China and Manchuria since the 1930s. They were overconfident in their communications security. [13]

Indochina

In September 1940, the Japanese moved into the Haiphong area of French Indochina, claiming they wanted to disrupt supply lines to their war in China. In June 1941, they expanded their occupation to all of the colony, to which the US responded with embargoes that the Japanese regarded as a casus belli for the Attack on Pearl Harbor. US Army and Navy cryptanalysts were able to follow events, initially through their penetration of the RED cryptomachine, and then the PURPLE system, introduced in 1939 and broken in 1940.

Principally to track shipping, the US monitored Japanese, and eventually French colonial administration, traffic, through WWII. In general, the area was not of strong operational interest to the Allies, except for planning submarine attacks on shipping, and occasional air raids on transportation infrastructure. On the strategic level, however, the US began to learn more about the resistance groups in Indochina. These groups, especially the Viet Minh, fought the Japanese, but would later fight the French administration, and eventually the Republic of Vietnam (RVN; South Vietnam).

In March 1945, the Japanese, through their own COMINT, were alerted of a potential French coup against the Japanese occupation. Within 48 hours, all the French administrators and troops were captured, except for about 4,000 troops who fled into China. [14]

September 1945 found an emboldened Viet Minh, under Ho Chi Minh and assisted by a US OSS team under MAJ Archimedes Patti, declare the independence of the Democratic Republic of Vietnam (DRV; North Vietnam).[15] The Allies, however, did not recognize Ho's government, staying loyal to the French.

While the French claimed Ho's movement was Communist, US State Department analytic reports in 1947 and 1948, written from all-source intelligence including COMINT, gave no indication that the Vietnamese Communist Party was controlled by Moscow.[14]

Western counterespionage

From 1943 to 1980, the VENONA project, principally a US activity with support from Australia and the UK, recovered information, some tantalizingly only in part, from Soviet espionage traffic. While the Soviets had originally used theoretically unbreakable one-time pads for the traffic, some of their operations violated communications security rules and reused some of the pads. This reuse caused the vulnerability that was exploited. VENONA gave substantial information on the scope of Soviet espionage against the West, but critics claim some messages have been interpreted incorrectly, or are even false. Part of the problem is that certain persons, even in the encrypted traffic, were identified only by code names such as "Quantum". Quantum was a source on US nuclear weapons, and is often considered to be Julius Rosenberg. The name, however, could refer to any of a number of spies.

References

  1. Barratt, John (2002), "Enigma and Ultra - the Cypher War", Military History Online.com, Barratt 2002
  2. Winston Churchill (2005). The Second World War, Volume 2: Their Finest Hour. Penguin Books Ltd. ISBN 0141441739. 
  3. John, Pike, DGSE - General Directorate for External Security (Direction Generale de la Securite Exterieure).
  4. Thomas L. Burns (1990), The Origins of the National Security Agency, 1940-1952, National Security Agency
  5. 5.0 5.1 5.2 5.3 Joseph, Browne (2006), "Radio-traffic analysis' contributions", Army Communicator Cite error: Invalid <ref> tag; name "Browne" defined multiple times with different content Cite error: Invalid <ref> tag; name "Browne" defined multiple times with different content
  6. 6.0 6.1 Lund, Earle. The Battle of Britain: A German Perspective; Addendum, Luftwaffe Air Intelligence During the Battle of Britain. Cite error: Invalid <ref> tag; name "Earle" defined multiple times with different content
  7. Jones, Reginald V. ((1978)), The wizard war: British scientific intelligence, 1939-1945, Coward, McCann & Geoghegan
  8. Dunn, Peter (9 April 2000), Central Bureau in Australia during World War II: A Research and Control Centre for the Interception and cryptanalyzing of Japanese intelligence
  9. Dunn, Peter (14 November 2000), RAN/USN Fleet Radio Unit, Melbourne - FRUMEL
  10. National Security Agency, Battle of Midway, NSA Midway
  11. Dunn, Peter (2003). 978th Signal Service Company Based at Camp Tabragalba, near Beaudesert, QLB during World War II.
  12. USMC, 1st Radio Battalion, Vietnam Veterans. History - 1st Radio Battalion 1943 - 1973.
  13. David Kahn (1996). The Codebreakers - The Story of Secret Writing. Scribners. ISBN 0684831309. 
  14. 14.0 14.1 Hanyok, Robert J. (2002), Prelude: Indochina Before 1950, Spartans in Darkness: American SIGINT and the Indochina War, 1945-1975, Center for Cryptologic History, National Security Agency
  15. Patti, Archimedes (1982). Why Vietnam? Prelude to America's Albatross. University of California Press.