Image credit: X (Twitter)
IBM built a secret computer for the US National Security Agency that began operating in 1962 and remained in service for 14 years, processing some of the agency’s most sensitive cryptographic workloads during the Cold War. Known as Harvest, the machine could run certain jobs up to 200 times faster than other computers of its time, according to accounts of its capabilities. Harvest finally stopped running on February 27, 1976, after a mechanical component in its automated tape library wore out, according to a report by IEEE Spectrum. By then, the machine had operated through the Cuban Missile Crisis, the Vietnam War, and the period leading up to the 1975 Helsinki Accords. Harvest was officially designated the IBM 7950 and was built around IBM’s 7030 Stretch mainframe. After two unsuccessful proposals, IBM secured the NSA contract in 1958. One documented example described the machine searching 3.5 billion characters of text for any of 7,000 target terms in just under four hours. NSA analyst Robert Looney said in 1972 that the job took about 10 hours, although he did not disclose the purpose of the search. As a complete system, Harvest could stream one byte every 0.3 microseconds and had about 800 kilobytes of addressable memory.
The processor used overlapping stages, or a pipeline, so that while one pair of data bytes was being processed, the next could already be fetched from memory.
Another job, referred to as “Moretown,” involved searching 11 million messages covering 16 years of intercepted traffic against about 8,000 search terms. “It was quite an amazing computer,” Allen said in a 2001 oral history. The manufacturer of the part was no longer in business, leaving the NSA without a practical way to replace it. IBM developed Stretch for scientific computing, but the NSA needed a system that could continuously process large volumes of intercepted communications without requiring operators to manually stage each batch of tape. The resulting system combined Stretch with a custom streaming processor and an automated tape-storage system called Tractor. Unlike a general-purpose computer, Harvest’s custom processor was designed to repeatedly apply a fixed sequence of operations to data streams. It could take two streams, known as P and Q, from memory, process them and return the result as a third stream, R. Harvest shared memory with Stretch and could address individual bits rather than being restricted to complete bytes or words. Its processing units included a logic unit for bitwise operations and a table-lookup unit that could use incoming data to address information in memory. The NSA used Harvest for cryptanalytic and large-scale text-processing jobs. IBM engineer Frances Allen, who worked on both Stretch and Harvest, later recalled the type of operations the machine could perform. “One instruction, for example, could do sorts, and do statistical analysis of the data that was streaming by it.… Everything we were doing at that time was on the cutting edge. There was no question about it.”
Some officials reportedly looked at the machine during employee tours and said, “It’s beautiful, but it doesn’t work, according to James Bamford’s 2001 history of the agency. Each Tractor cassette weighed about 6 to 7 kilograms and contained roughly 120 megabytes of data on a 550-metre reel of tape. Each storage unit could hold up to 160 cassettes. When Harvest began operating in 1962, three automatic cartridge units were connected to it, each serving two tape drives. Together, they provided about 44 gigabytes of online storage. The mechanism could move at about 6 metres per second, while software running on Stretch controlled cassette movement and data reading and writing. Moving from one completed tape to the next could take about 18 seconds, once the next cassette had been retrieved and was ready to mount. On February 27, 1976, operators shut Harvest down for the final time. The NSA eventually replaced Harvest with the Cray-1 supercomputer. Unlike Harvest, the Cray-1 used newer integrated circuits and was designed primarily for numerical and scientific computing, but its processing capabilities also made it suitable for workloads that had previously required Harvest’s specialised hardware.
IBM therefore built Tractor, an automated magnetic-tape library officially designated the IBM 7955. Tractor therefore functioned as both long-term storage and a source of data for Harvest’s processing jobs.
A “scab,” represented by a question mark, indicated a character that was known to exist but whose identity was unknown. A blank space represented a “pad,” or null character. “I spent the good part of a summer on that,” she recalled in 2001. Allen said the NSA was not widely known outside classified intelligence circles when she began working on the machine, so she initially believed she was working on a naval project. I was probably the largest, fastest, and most technically advanced computer system in the world,” it said Although not widely known.
Harvest’s specialised design also created challenges within the NSA. The system nonetheless continued to handle large jobs that general-purpose machines were not well suited to process. Harvest’s processing capacity required a storage system that could supply large datasets without constant human intervention. The automated system used a motor-driven arm to retrieve cassettes from storage racks and deliver them to tape drives. Retrieving and preparing a cassette did not require manual handling. Conventional reel-to-reel drives connected to Stretch were also used when large datasets had to be transferred into or out of the system. IBM and the NSA also developed a programming language specifically for Harvest. Called Alpha, short for Advanced Language for Programming Harvest, it was designed around the machine’s streaming architecture and the needs of cryptanalysts. Alpha allowed programmers to define the alphabet used for cryptographic data and included symbols for representing uncertain or null characters. The language also provided ways to combine strings into larger structures, giving programmers a way to describe complex intercepted messages and cryptanalytic datasets. Allen worked on the Harvest software and later recalled that the technical report she produced for her part of the project disappeared into the NSA’s classified system after she completed it. “And it just disappeared into Fort Meade somewhere. Harvest’s secrecy also shaped how some IBM employees understood the project. The budget code name for the NSA was “Bureau of Ships. A mechanical component in the Tractor tape library had worn out, and the company that manufactured the part was no longer operating. Replacing the system’s ageing hardware was no longer practical. The NSA marked the occasion with a mock telegram written in Harvest’s name. “I first began operations at NSA. “And now, fourteen years later, the time to retire has come. The cost of my upkeep and operation has been overtaken by more modern equipment and the newer technologies.
By 1971, Harvest was operating about 115 hours a week, according to Looney, representing more than two-thirds of its available operating time. At the same time, the number of jobs it handled had been declining since 1967 as newer general-purpose computers took over ordinary data-processing work. At the system’s 10th anniversary in 1972, Looney described it as a machine “conceived in the fifties, born in the sixties, and irreplaceable in the seventies. The Cray-1 was also commercially available to multiple customers, unlike Harvest, which had been built specifically for the NSA.
Because its architecture was built around the NSA’s particular requirements, harvest did not have a direct line of successors, partly. Harvest was increasingly reserved for large, specialised workloads that other machines could not handle as efficiently. The description did not last much longer. As newer general-purpose and scientific systems improved, maintaining a custom machine for one specialised workload became less practical. Several of its concepts, however, appeared later in other areas of computing. Tractor’s automated tape library anticipated robotic storage systems later used in large data centres. Harvest’s pipeline architecture also resembled approaches that became part of dataflow computing, while its continuous pattern-matching operations have parallels in later systems designed to inspect network traffic. Harvest’s operational life ended with a worn mechanical part and an unavailable manufacturer, but the ideas used to build the system continued to appear in computing systems developed after it. Get the latest technology news and updates. Download the TOI App.