Talk:ULTRA

Changing meaning

I don't have the references right here at hand, but my recollection is that the meaning of "Ultra" changed over time. I think it initially referred to just the Enigma system(s) (both Army, GAF and Navy), but eventually came to cover all high-level cryptanalytic intel. I think the same was true of the US term MAGIC, which came to cover all HLCI. If I have time, I'll try and research this issue further. J. Noel Chiappa 20:05, 24 October 2008 (UTC)

While I don't have the references at hand, I did once spend some research time with the original documents. There were a variety of names, both internal to the COMINT community and outside it. At Churchill's level, the material usually was code-worded BONIFACE or BLACK JUMBO, and even among the very few nontechnical people that saw it, the conscious impression was given that it came from a human source. When the SLU system started distributing it to senior commanders, they had a need to know that it was COMINT. Some SLU-related material suggests there may have been specific distribution lists, which the U.S. now calls BIGOT lists, but there was no special marking that differentiated the Mediterranean BIGOT list from the Channel list.

Eventually, there were three levels of British COMINT: ULTRA RABID, ULTRA DEXTER and ULTRA...ummm...I'lll think of it. It's in Kahn's Codebreakers in the other room. The lowest level was mostly non-cryptanalytic: traffic analysis, especially sensitive direction finding, and what we'd now call a radiofrequency MASINT technique of using recorders and time-domain oscilloscopes to recognize individual variations in Morse keying. Of course, an experienced operator could hear those variationsm, but that takes years to train. There were other markings for tactical direction finding.

I never saw a MAGIC document with any supplemental codewords. In Dan Gallery's book on the capture of the U-505, he talked about a "special TOP SECRET marking", but I went through the logs of his task group at the Naval Operational Archives, and they were just TS, not even MAGIC.

There was a little-known U.S. organization called Joint Security Control (JSC), which was acknowledged to be the steward of agreements on markings. For example, there was a Joint Security Classification Agreement in 1942, in which the British agreed to change from MOST SECRET to TOP SECRET. BIGOT was also established as a code word for TORCH.

What did not come out until the late seventies was that JSC was the U.S. counterpart to the utterly black London Controlling Section (LCS), the strategic deception organization that ran what was first Plan JAEL, and then Operation Bodyguard (Churchill: "In wartime, truth is so precious that she must be protected by a bodyguard of lies.")

I am not clear if the Double-Cross system had any special markings; I've never seen originals of their documents. Howard C. Berkowitz 20:40, 24 October 2008 (UTC)

I've mostly relied on secondary sources (with limited exceptions such as memoirs, which of course have their own issues, unless carefully researched).

Was BIGOT used on TORCH too? I thought it was an Overlord compartmented code word.

I've been working on a bibliography for Ultra, much of which applies to Enigma. I should add a few Colossus-related books, which are of course specific to FISH; I'd been (incorrectly) thinking 'Enigma' when I saw 'Ultra'. (Blasted code names - I get them all mixed up, and the tendency to drift in meaning a bit over time on the part of some, like 'Ultra', doesn't help!)

Speaking of which, do we have an Enigma article yet? I assume as some point we should have separate article on Enigma, and the breaking thereof. I seem to recall WP had a good article about Enigma, we should probably snarf that. (I see they have a separate article on the attack on it.)

Actually, WP has a number of good article in this area we could snarf - Colossus and Sturgeon - the Fish and Tunny ones aren't so good - although we could use them for source, and re-write. J. Noel Chiappa 23:34, 24 October 2008 (UTC)

BIGOT, as you probably know, originated when the TORCH planners going to Gibraltar had their orders stamped TO GIB; BIGOT spelled it backwards. We get sort of nuanced here; it probably started being used after TORCH, but before Overlord was even coined; it was probably somewhere in the Bolero-Roundup-Sledgehammer stage.

Incidentally, have you been to the National Cryptologic Museum? I was absolutely amazed when I got to play with an Enigma; the thing looked and felt like a badly maintained 1930-ish manual typewriter. The U.S. SIGABA, which was behind glass, looked like Body by Porsche.

My interest in the primary sources was more in the history of classification, in the mid-seventies. I think I started before the big declassifications, but I had some sponsors that got me a lot of access. While Kahn discusses BIGOT, I first had it explained in an on-the-record interview with the director of security at CIA. BIGOT lists remain a standard procedure for compartmented control systems, although their use is up to the compartment manager. Howard C. Berkowitz 00:28, 25 October 2008 (UTC)

Yes, I think there needs to be something on changes over time, a link to MAGIC, ... I considered rewriting the intro to do that, but decided I didn't know enough to do it well.

More generally, though, the article just needs to be written up beyond stub. Also, related articles like Enigma machine and Colossus need to be written and others like Alan Turing expanded. I think the changes issue will be dealt with in that process. Sandy Harris 03:53, 25 October 2008 (UTC)

OK, here's some data on changing meaning of "Ultra". Welchman, "Birth of Ultra", says (p. 77):

The term 'Ultra', although originally coined by Winterbotham to describe intelligence derived from Hut 6 decodes, has been applied to information from other cryptanalytic sources..

At that point, I'm pretty sure Hut 6 worked on just German Army/AirForce Enigma - see, e.g. Welchman, "Hut Six Story", p. 58. The other bit I have is from M. Smith, "The Emperor's Codes" (a great book all cryptophiles should have, BTW - it covers the little-known but significant British work on Japanese codes), where someone stationed in the Phillipines working on Japanese codes says (p. 264):

The production of Ultra by [our unit]

which clearly indicates that late in the war the term Ultra had come to mean all cryptanalytic intelligence. J. Noel Chiappa 18:07, 30 October 2008 (UTC)

Here's something on the origins of the term "Magic" (put here since we don't yet seem to have a matching "Magic" page). This is from Parrish, "Ultra Americans" (p. 56):

So wondrous were the results of the efforts against the Red and the Purple machines .. that General Maubourgne developed the agreeable habit of referring to his [Army SIS] cryptanalytic team as "magicians" - from which came the U.S. designation of intelligence produced by cryptanalysis as "Magic"

A footnote records that this is from Rowlett; I have yet to consult his autobiography and see if it gives more details. Still, this does seem to confirm an early usage of the term for referring to Japanese diplomatic traffic. J. Noel Chiappa 22:23, 2 November 2008 (UTC)

Article coverage

I'm still pondering exactly what the above should mean in terms of our coverage. By the end of the war, US and UK cryptanalysts were working so closely together on all sorts of cryptologic systems (e.g. German Shark, and Japanese JN25) that we can't really divide things up into 'system the UK worked on' and 'systems the US worked on' because there's way too much overlap. Perhaps Ultra should just be a short article covering the history of the term, with most of the coverage in something like Cryptanalytic intelligence in World War II, which could link to articles on each major system. J. Noel Chiappa 18:07, 30 October 2008 (UTC)

While the title should be updated, SIGINT before the Second World War and SIGINT in the Second World War should be starting points. Note that these cover COMINT, ELINT, and some MASINT. Well, OK--no ELINT before WWII. It's not always clear, for example, that the GCHQ program had elements from all three disciplines, and, for that matter, COMSEC/CRYPTO was yet a different set of compartmented control systems.Howard C. Berkowitz 20:13, 30 October 2008 (UTC)

The existing SIGINT in the Second World War article is the start of a good overview of the entire field, including ELINT. (BTW, there was a tiny bit of ELINT before WWII - the Germans used the Graf Zeppelin to try and pick up British radar pulses, see this and this, which give some details - I have both the refs in the latter if you want me to look it up there. I'm not sure if the British tried to do anything with German radar before the war - I have the complete ref works on British radar work, so I can check if you need. I should create a biblio for the WWII article, too - although now that I think about it, only some of the books are about ELINT, many are about Allied offensive radar work.)

However, getting back to the topic, I was thinking of something considerably more focussed than that article, looking just at cryptanalysis, not the other elements of COMINT, such as traffic analysis (which IIRC was done by MI8 as well as Bletchely for the UK), D/F (which was an extensive program), etc. (BTW, is there a specific term for cryptanalytic intel?) Although of course the cryptanalysts drew on the rest of COMINT to help with their breaks.

Oh, BTW, I should have said Allied cryptanalytic intelligence in the Second World War, since I was thinking mostly of the (extensive) allied efforts. Although I suppose it really should be Cryptanalytic intelligence in the Second World War, or perhaps just Cryptanalysis in the Second World War, covering both sides, with links to articles on specific topics (systems, organizations, etc). J. Noel Chiappa 20:56, 30 October 2008 (UTC)

There may be more articles here than you realize, but I probably need to do some systematization of titles, and some related articles sections so you can see what is there. There is electronic intelligence, for example, independent of the SIGINT history series. There are SIGINT platforms by location (ground, air, sea, submarine, space), SIGINT by nation and alliance, etc. There is traffic analysis. Radiofrequency MASINT covers some areas; the British Y-service, which did DF and, IIRC, T/A, also recorded Morse signals and analyzed them in the time domain to find operator variations. Howard C. Berkowitz 21:26, 30 October 2008 (UTC)

"may be more articles here than you realize" as in 'there are more done than you realize', or 'there are probably more left to do than it seems from your comments above'? I looked at SIGINT in the Second World War/Related Articles and didn't see much.

BTW, I have 3 books on the British Y service, so we have good reference works there. J. Noel Chiappa 21:55, 30 October 2008 (UTC)

More on aspects of SIGINT generally, not WWII or cryptanalytic specific. Incidentally, there is a standalone Battle of Britain that deals more with the military operations; Battle of the Beams actually is the more relevant part. I had hoped the basic BoB article could move toward approval, but I may be the only active Military or History editor. That being said, there perhaps could be some links from BoB to such things as ULTRA giving the confirmation of the cancellation of Operation Sea Lion.

In general, however, when I ported, then split, the SIGINT articles, I didn't always spend a lot of time on doing Related Articles. Howard C. Berkowitz 22:07, 30 October 2008 (UTC)

Ah, got it. Well, I don't think that will be an issue - I'm particularly interested in WWII cryptanalytic stuff, so if the existing stuff you've put in is more general, then we probably have a bit of a lack in that area of specifics.

I would appreciate it if you could do a bit on the 'Related Articles' front, especially for SIGINT, so I can see what we have - trawling through articles and clicking on links is somewhat inefficient... J. Noel Chiappa 15:14, 31 October 2008 (UTC)

What is cryptanalytic intelligence?

So, is there a specific term for cryptanalytic intelligence? J. Noel Chiappa 15:14, 31 October 2008 (UTC)

No general term of which I'm aware. I can think of Cold War multiple-level subcompartments, where something would be very tightly held due to subject sensitivity, but a couple come to mind where one was cryptanalytic and another was a specialized intercept technique.

I'm not even sure that, other than "cryptanalytic intelligence", such a term is even desirable. Various examples in -- well, let's say Europe and Pacific rather than ULTRA and MAGIC -- where there was only a partial recovery, and that needed correlation with other methods before meaningful intelligence inferences could be drawn. Perhaps the picture could be constructed passively, combining the partial decrypt with DF'ing the transmitter and perhaps some HUMINT or IMINT indicating a particular unit or commander was plausible as the subject of the garble.

Without even beginning to get into deception, is a situation such as recovering, from JN25, the plan for an attack on "AF", which could have been at least three places, cryptanalytic intelligence? In other words, I think of "intelligence" as at least somewhat finished. In this case, IIRC, it was Jasper Holmes that came up with the idea of securely telling Midway to send a "barium meal" in a cryptosystem we knew the Japanese had compromised (but they didn't know that we knew), and then recovering the "AF is short of fresh water" from a JN25 decrypt. Was each step, some steps, or the set of steps there "cryptanalytic intelligence?" Howard C. Berkowitz 15:26, 31 October 2008 (UTC)

It seems to me that along the path from original information sources, and the 'producers' who gather that data and insert it into the system, to 'consumers' (military/civilian decision-makers), intelligence changes from the original individual items of data, into complete 'pictures'; to the consumers, they don't care that much as to where the pieces of the picture came from, be it from reconnaissance or SIGINT or whatever (except if they want to evaluate the reliability of the picture which is being presented to them).

So, yeah, I don't think these distinctions are too useful to the consumers, but in writing about intelligence, and the creation of it, I think it's useful to classify the various sources of original data that go into the picture. Does this make sense? J. Noel Chiappa 16:32, 2 November 2008 (UTC)

The difficult part is not to the consumers, but in what comes before anything is ready to go to a policymaker. There is a spectrum from "raw" collection, to what I will loosely call processing, to what tends to be called, these days "intelligence fusion". Some discipline-specific fused techniques include geospatial intelligence, wherein imagery (of various sorts) is precisely correlated with geodetic informatin. Another is multispectral imagery, such as learning that the apparent forest in the visual spectrum photographs actually shows an infrared spectrum of paint, not chlorophyll. An apparent factory, even in multiple light spectra, might get scanned with imaging radar, which doesn't have great resolution but confirms there are no large metal objects inside.

More in the areas being discussed, cryptanalysis is neither a collection nor a fusion technique. Let's say there is a set of receiving antennas in a general-purpose frequency band such as UHF or X-band. Assume they are both sensitive and directional. The first level of processing might be direction finding. Next, analysis of modulation and other electronic characteristics can make a rough separation into deliberate communications radiation, inadvertent communications related radiation, deliberate noncommunications radiation, and inadvertent noncommunications radiation. It's still an at least two-stage process, but one can think of this as "pure" or "undifferentiated" SIGINT, with common techniques of DF'ing, spectral analysis, modulation and pulse measurement, etc.

On the deliberate communications radiation, once I can identify even encrypted communications, I can start doing traffic analysis, which is now COMINT. Cryptanalysis is a second stage in the COMINT path. I'd argue that the output of cryptanalysis is an intermediate result from a mathematically based technique. Even in WWII, it wasn't necessarily acceptable even for the people who are trying to figure out enemy intentions and tell consumers what they are. Let's say the particular messages are in an enemy espionage cryptosystem.

Now, if I have legitimate espionage traffic, there's predictive value to knowing what the enemy commander wants to find out, and, assuming it's a real agent, it's important for us to know what they know. But what, as someone in Hut Whatever at Bletchley is cheerfully breaking the message from the spy reporting on Portsmouth Dockyard, a HF/DF sort of fellow wanders in and points out that the "agent" transmitter is in France and the "control" transmitter is in Germany. Suddenly, while the mathematical cryptanalysis doesn't change, the text result of the cryptanalysis has to be considered in a completely different context.

Ignoring that there may be subplots, I can make the reasonable inference:

• The Abwehr, or whoever, thinks we can read that particular cipher. Why do they believe that? Was it deliberately weak, or do they have some penetration into GCHQ and know it was broken?
• They want us to believe some synthetic exchange beween an ostensible agent and ostensible controller. Why? There are lots of potential reasons, not mutually exclusive. They may be feeding deception about their interests, asking about detailed targets in Portsmouth so we will shift air defense from the real target in Dover. They may want us to try to find the spy in Portsmouth, so the counterespionage people are busy there and not aware of the really dangerous spies in Bletchley and Birmingham.
• Once someone suspects the deception, they might compare recordings of the Morse telegraphy timing, which is highly individual, and recognize that the operator ostensibly in France has previously been associated with an SD radio deception unit that used to be in Belgium. Did they move it, or the operator? If the unit moved, why? Howard C. Berkowitz 17:17, 2 November 2008 (UTC)

The more I think about this, perhaps there is an article that might shake out of the much longer general intelligence analysis (including processing, and the...well, a barium meal in tradecraft). "Article" is less, perhaps, what I have in mind than "case study". Subpage?

As you may have noticed in forum discussions with Chris and Aleta, there's a sense of needing more constructs, things that may not justify full clusters, but either are not definitions, or are a shim that lets you link discussions. Since Sandy can't get to the Forum, I've set up a temporary page for discussion that should move out of userspace: CZ talk: usability Howard C. Berkowitz 18:51, 1 November 2008 (UTC)

I'll try and check it out. J. Noel Chiappa 16:32, 2 November 2008 (UTC)

Apropos Related Articles

As r-templates aren't very good for subsections, I'm wondering about the most efficient way to get the appropriate cross-indexing. Here's an example. See SIGINT before the Second World War#Naval direction finding and message interception. I'm reminded of that because the Chatham station is, I'm told, within a couple of miles of my desk, and I think there are physical remnants.

The larger section in which that is contained, just U.S. intercept station coverage, is important -- can't cryptanalyze what you don't receive. Do I need to pull out a whole new article on "intercept facilities" (not sure how best to name it to tie it to the context), so I can have a related article? Howard C. Berkowitz 15:34, 31 October 2008 (UTC)

I may go try to find the intercept station, or what's left of it. A local isn't sure if it's the right site, but there is a place, certainly less than two miles away, where there are the remnants of antenna mounts. The ASW school is closer, and its airstrip, which is in use today, is about three blocks down the street. Howard C. Berkowitz 01:56, 1 November 2008 (UTC)

You do know that {{r|SIGINT before the Second World War|Early SIGINT}} produces:

• Early SIGINT [r]: Add brief definition or description

right?

But that's probably not what you want to do anyway, because the whole point of {{r}} is to use canned definitions in [[{foo}/Definition]]. You may at some point produce a definition to go with SIGINT before the Second World War, but there's no way to do per-section definitions.

So I don't have a good immediate answer, but let me think about it. J. Noel Chiappa 18:19, 1 November 2008 (UTC)

Howard you should check out the supernova example at Template:R#Redirects. {{R|Supernova}} points to a subsection of Nova (astronomy) but shows the definition at Supernova/Definition. The redirect that exists at Supernova points to the specific subsection in Nova. I think this is the way you are hoping to use the subsection and definition in tandem. Chris Day 22:06, 1 November 2008 (UTC)

How clever! I'd forgotten about that. Yes, that's definitely the way I'd go about it. J. Noel Chiappa 16:12, 2 November 2008 (UTC)

Spanish Enigmas

The Spanish Army had 26 machines stashed way. [1] Sandy Harris 03:55, 25 October 2008 (UTC)

An oddity

Cryptome has an NSA report on The Last Days of the Enigma. Apparently they were still in use in East Germany well into the 50s, with Americans running bombes and reading the traffic. Sandy Harris 11:36, 11 December 2010 (UTC)