Government studies, news reports, and interviews with a score of experts on national security and computer security issues indicate that the NSA monitors and hacks into the Internet and the World Wide Web. While the NSA is generally forbidden to spy on U.S. citizens without probable cause and a court order, there are numerous loopholes in surveillance law that, especially since Sept. 11, allow the NSA to track U.S. citizens in cyberspace at will.

In the aftermath of the terrorist attacks, Congress passed laws and budgets that expanded the NSA's surveillance powers. The so-called Patriot Act (an acronym for Providing Appropriate Tools Required to Intercept and Obstruct Terrorism) increases the government's authority to intercept electronic communications on the Internet for use in court as evidence against terrorists, criminals, and political activists of all stripes. It also shreds constitutional protections against unreasonable search and seizure to allow intelligence agencies, such as the NSA, to throw wide nets into cyberspace to capture intelligence for purely military, not prosecutorial, purposes.

In its search for terrorists, the NSA has a stunning array of tools at its disposal. It can intercept and analyze millions of electronic messages a day at a handful of Internet crossroads (including one in San Francisco), made possible by technology supplied in part by Bay Area companies. It can also narrowly target individuals and groups -- hiding on their computer hard drives, spying on their e-mails and Internet travels, and electronically following anyone who visits certain red-flagged Web sites.

Yet those broadened powers, and the airtight secrecy surrounding the agency, have also raised questions about what precedents are being set for spying on American citizens once the current crisis has passed.

The internal legal guidelines the NSA uses to decide under what conditions it is allowed to track "U.S. persons" (citizens, permanent aliens, and businesses) are largely classified, says Michael Vatis, director of the Institute for Security and Technology Studies at Dartmouth College in New Hampshire. Vatis also served as the first director of the National Infrastructure Protection Center, a joint intelligence and law enforcement operation of several federal departments, including the FBI and NSA, that coordinates the government's daily observation of the Internet and private-sector databases and advises computer experts how to plug security holes. In international cyberspace, however, it is very difficult to separate U.S. persons from foreigners, who are generally fair game for surveillance, Vatis observes. "I am certain that the NSA monitors open source [i.e., public material in cyberspace]," Vatis says. "I am confident that the government is doing all it can with regard to the Internet and using existing legal authorities to get access to the communications of terrorists (or anyone they label as such)."

Other experts are fairly certain the NSA is breaking their legal authorities.

In 1982, investigative journalist James Bamford wrote The Puzzle Palace, an exhaustively documented history of the National Security Agency. Bamford, who until recently was Washington investigative producer for ABC's World News Tonight With Peter Jennings, brought The Puzzle Palace up to date this year with Body of Secrets. He is widely acknowledged as an authority on the NSA.

In an interview from his home in Washington, D.C., Bamford said the Patriot Act dilutes the burden of proof required to show probable cause for targeting a U.S. person and that the NSA is "probably pushing the [internal] guidelines to the limit" to allow the agency to track terrorist suspects throughout the Internet and the World Wide Web.

"It's not easy to penetrate what the NSA is doing, but the climate has changed entirely since Sept. 11. The emphasis is on aggressive pursuit."

The NSA has long had the capability and the motivation to spy on U.S. citizens on the Internet without restriction, Bamford observed. The passage of the Patriot Act gave it the political go-ahead to do that under certain conditions.

"Just as mighty navies once ruled the high seas," Bamford wrote in Body of Secrets, "the [NSA's] goal is to rule cyberspace."

---

The NSA reportedly intercepts 3 billion messages a day from all the mediums it watches. Modern communications -- and the Internet specifically -- have dramatically changed the nature of national security and intelligence-gathering. The battle for control of the world's information infrastructure has even been given a name: Netwar. Under these new rules of engagement, America's enemies are no longer well-defined, monolithic opponents like the Soviet Union, say experts like John Arquilla, who is an associate professor at the Naval Postgraduate School in Monterey, which trains Navy officers in intelligence techniques and special operations.

Last month, Arquilla gave a talk on Netwar at a meeting of the Council on Foreign Relations in San Francisco. Arquilla defines America's enemies today as loosely organized networks of terrorists, street gangs, international syndicates of criminals, and even anti-World Trade Organization protesters. These anti-hierarchical, leaderless networks tend to operate in the shadows until a trigger moment, Arquilla says, when they suddenly "swarm" a target, such as the World Trade Center towers or the 1999 WTO meeting in downtown Seattle. The Internet enables the underground networks to flourish by providing rapid communication channels, information on how to design weapons, and, most important, an international forum to air grievances and promote political causes.

The NSA is fighting back, Arquilla says, by tracking these enemies with "intelligent software agents" -- bots.

"The NSA is engaged in clandestine technology intelligence -- ECHELON, Semantic Forests -- there are other names that I am not allowed to tell you. Clandestine technology intelligence allows us to track dark networks; if you go to certain Web sites a bot will follow you. Bots empower human agents."

Simply put, bots are strings of computer code that roam the Internet like fish in the sea, entering people's computers and performing automated tasks on their own. They are everywhere on the Web. Businesses commonly send out bots on the Internet to build customer profiles, search for mentions of the corporate name, or mine Web sites for demographic data to use in market research. Bots can piggyback into your computer every time you visit a Web page, open an e-mail, or download music online. Search engine bots travel the Web, creating indexes for Google and Yahoo!. Once inside a personal computer, bots can, quite legally, connect to the Internet without the computer owner's knowledge, sending out streams of data, including personal financial information sucked out of money-management programs.

In the hands of the NSA, however, bots become a different animal. "[The NSA's] bots are like corporate-marketing and data-mining bots on steroids," Arquilla says.

After cautioning the interviewer that he is "approaching a no-go zone," Arquilla says, "We can use bots to see who is accessing information of dual use to science and terrorism." For example, he says, Web sites that display the human genome can be used to create biological weapons. Spybots inserted into the operating system code at a genome Web server can be instructed to follow any visitor and then report back to the NSA.

Two years ago, the European Parliament issued an investigative report that offered an unprecedented glimpse into the NSA's ECHELON program, which covertly intercepts the world's military, diplomatic, commercial, and civilian communications as they are transmitted through the air or by wire. (ECHELON is a joint effort of the English-speaking countries. The Parliament's investigation was inspired by allegations that the NSA was using those intercepts to give hot tips to American businesses at the expense of their European competitors.) The ECHELON report said the NSA uses custom-designed bots to ferret out information on Web pages, Usenet, and open databases. These spybots are many times more powerful than common search engines, which skim the surface of the Web as they follow a series of hypertext links from one Web page to another.

One South Dakota company designs bots for the government that access 500 times more data than normal searchbots. Gerard Tardif, vice president of BrightPlanet, says his bots, unlike ordinary search engines, mine the vast collection of databases underlying public Web pages. Tardif's quasi-intelligent bots do not just blindly follow hypertext links, they enter into a dynamic relationship with a database, querying it for deep content. For example, "LexiBot" can visit the Web site of a nonprofit organization that raises money to educate Palestinian schoolchildren and retrieve its latest financial data, if that data is connected to the Web page server. A normal bot would just return the URL for the Web page.

"Some of our customers are using our products for intelligence-gathering in support of national defense," says Tardif. "We are permitted to mention some customer names, such as NATO and Lawrence Livermore Laboratories. But the others must remain anonymous."

The clever searchbots are dumb beasts, however, compared to the NSA's "Trojan horses," a generic name for a particularly malicious type of software, or "malware." Trojan horse bots can be written to carry out a series of complex tasks, such as finding the password to your online bank account and using it to transfer your terrorist trust fund, penny by penny, to thousands of randomly selected accounts, where the tiny deposits will not be noticed.

Intelligent software agents, such as Trojan horses, are self-contained miniprograms that act on their own initiative after being set free to hunt pre-parametered prey in the cyberjungle. Aggressive malware can take control of a computer, without the user being aware of its presence, by seizing on flaws in the computer's operating system, such as the widespread use of the practically insecurable ActiveX programming language used in many Microsoft applications.

Government malware is analogous to commercial applications, such as Symantec Corp.'s pcAnywhere, and powerful hacker tools, such as Back Orifice. These sophisticated bots can be quietly installed on the hard drives of computers that are connected to the Internet (or by real-life government burglars trained to break and enter the old-fashioned way). Once hidden inside the millions of lines of code that are the life force of a computer, a malicious bot can copy logs of the Web sites a suspect has visited, steal his credit card numbers, or purloin the embarrassing love poem he thought he had trashed and send it all back in a bundle to the bot's master by way of an untraceable route.

A nasty "warbot" can mine the suspect's data for information on the whereabouts of the other members of his terrorist cell -- and then wipe his hard drive clean. A "worm" or "logic bomb" can attach itself to his e-mails and the e-mails of the people he sends e-mail to, and their e-mail lists, ad infinitum. On a certain date, thousands of self-replicated copies of this badbot, nesting inside hundreds of innocent computers, can send cascades of 1,000-page e-mail files to the server hosting the Web site of the front group for the suspect's terrorist organization, crashing it. On the other hand, a low-profile snitchbot can just sit quietly inside a font file and rat him out to the NSA every time he goes online.

Bruce Schneier, founder and chief technical officer of Counterpane Internet Security Inc. in Cupertino, has worked with the National Security Agency. "The NSA would be foolish not to make attacks using malware," says Schneier. "It would not be doing its job if it didn't." Indeed, the NSA's mandate to protect and defend the country's cyber-infrastructure necessitates that it engage in comprehensive surveillance and "defensive" hacking.

Federal law does not criminalize surveillance or hacking unless $5,000 worth of damage is done. Aside from that threshold, there is almost no case law to guide plaintiffs who object to being monitored by bots, be they taxpayer-financed bots or private-sector bots. While the Fourth Amendment to the Constitution generally forbids the government to search and seize private property without a court order, it does not define the boundaries in cyberspace at which a bot becomes an unauthorized intruder, by, for instance, crossing from sniffing around inside a public Web site to peeking into a private database. Schneier points out that in the United States people basically do not own their personal data, which can be sold by others for profit.

Christopher O'Ferrell is the director of ethical hacking for NETSEC, a computer security company founded by two ex-NSA officials. The Virginia-based company has several contracts with federal intelligence agencies to deepen the security of government computer networks and to surveil the Internet in real time.

O'Ferrell, who used to work for the FBI and the Secret Service, says, "Oh sure, definitely, without question government [intelligence] agencies use bots. The terrorists attack us with worms, so, of course, we use worms against them." O'Ferrell says the NSA conducts "black projects" -- covert operations -- in cyberspace.

"Of course they do that stuff [hacking]," he says. "They'd be crazy not to." O'Ferrell notes that the military establishment and the law enforcement and intelligence agencies need to "think outside the box."

"If we stay within legal bounds," O'Ferrell says, "we have lost the game."

---

Besides targeting suspect individuals or groups with bots that burrow and tools that hack, the NSA eavesdrops generally on cyberspace. The nationwide paranoia after the horror of September's terror attacks has lent popular approval to this practice. A bill to increase the NSA's budget by adding several billion dollars to the approximately $30 billion a year we spend on foreign intelligence-gathering is working its way through Congress. The bill specifically funds the NSA to change its current focus from intercepting messages transmitted by satellite and microwave dish to intercepting electronic traffic, particularly Internet traffic, that speeds through the land and sea networks of fiber-optic cables, which transmit voice and data communications.

A few years ago, Lt. Gen. Kenneth A. Minihan, then-director of the NSA, wrote an article revealing that the NSA defends the security of the Internet by spying on it. Stripped of bureaucratic jargon, what Minihan said was that the NSA attaches "sensors" on the Internet backbone and "in the underlying telecommunications infrastructure itself" to detect potential "threats" from nations, terrorists, and radical groups.

Contrary to popular conspiracy theories, the NSA can't monitor every man-made electron orbiting the Earth and pick out keywords, such as "anthrax" or "bribe," according to the European Parliament's ECHELON report. For one thing, trying to analyze huge volumes of phone calls by keywords is beyond the agency's capabilities because spoken language contains too many variables. The NSA can, however, analyze tremendous amounts of nonvoice data using keywords. Still, experts say that while it is theoretically possible for the NSA to monitor cyberspace in real time, the $4-billion-a-year spy agency, which is reported to employ more hackers and mathematicians than any other organization in the world, is not yet able to trap and analyze the unbelievably mammoth content of the Internet slipstream as it passes through the government's interception devices. Clearly, though, the NSA is working hard to do so.

There is no single physical point of connection through which all traffic passes, says security scientist Schneier. Instead, the NSA can connect "sniffers" -- Internet wiretap devices -- on overseas cables and at nine connection points in the U.S. (including in the Pacific Bell headquarters building in San Francisco). The problem with analyzing intercepted data, Schneier remarks, is knowing what information to ignore. It's a question of time. If it takes more than one second to analyze a second's worth of data, you fall behind in a fatal spiral, says Schneier, never catching up.

The trick is to narrow the focus of interception as much as possible -- to selected regions of cyberspace, certain chat rooms, Web sites, groups, and individuals.

The NSA's biggest challenge appears to be buying or inventing programs capable of analyzing the billions of messages it captures every day. To that end, the NSA openly partners with and makes substantial investments in a wide range of technology companies, such as Northrop Grumman Corp. and Verizon Communications, that manufacture hardware and software capable of scouring the microwave spectrum and tapping into fiber-optic pipelines to look for targeted content.

According to the ECHELON report, an array of private companies, several owned and operated by ex- NSA officials, has contracts worth hundreds of millions of dollars with the NSA. The report singles out Applied Signal Technology Inc., which is headquartered in Sunnyvale. John P. Devine, a member of Applied Signal's board of directors, was a deputy director of the NSA in 1995 when he left to join Applied Signal, which is described by the report as a "one-stop ECHELON shop."

Gary Yancey, founder and president of Applied Signal, says that his company has contracts with the NSA. "I know the ECHELON report well," he remarks. "But I can't comment on anything to do with it because of security clearances, and I would be excommunicated by the NSA if I did."

According to the ECHELON report, Applied Signal's devices intercept real-time data from high-speed Internet backbone links, then separate the raw intercept into tens of thousands of individual channels, each carrying a digitized telephone, fax, or modem "conversation." Although the rate of capture is quite impressive, it takes a relatively long time to interpret the captive data.

That is one reason the NSA apparently failed to analyze its raw intercepts on al Qaeda's U.S. branch prior to Sept. 11. Separating an unfathomable number of intercepted bits into intellectually analyzable categories is a laborious process. While the NSA divulges almost nothing about its techniques, it is possible to glimpse how it deals with intercepted data by looking at commercial products sold by firms that are close to the NSA.

Paracel, a subsidiary of Celera Corp. (famous for sequencing the human genome), describes its $100,000-plus TextFinder supercomputing processor as "designed to filter, search, categorize, and disseminate massive quantities of information for the Department of Defense." The chip can run hundreds of query searches on 50,000 pages of data per second (which is only a tiny fraction of the Internet's data flow), Paracel officials say. It can scan data in all languages simultaneously, while sorting it into patterns, according to Andrew Basile, TextFinder's project manager. Basile would neither confirm nor deny that the NSA uses TextFinder. The NSA does hold a patent on a software program, Semantic Forests, that has related capabilities and, according to Schneier and other experts, is designed for use on the Internet.

Eventually, intercepted data is threshed and winnowed down to the PC level, where it can be manipulated by human brains. For example, Applied Signal's Pager Identification and Message Extraction device, as viewed on the company's Web site, inputs intercepted data, such as telephone numbers, directly into an Excel spreadsheet. Shelves of commercially available analytical software, such as Xanalysis Inc.'s Watson and i2 Inc.'s Analyst's Notebook, help human beings connect people, places, things, money, weapons, and credit card charges into meaningful patterns. Hard information can be cross-referenced by NSA agents against behemoth databanks maintained by the government and corporations, such as Microsoft, ChoicePoint, DoubleClick, and other data-mining and financial-service companies that openly do business with the NSA and its sister agencies.

---

The Patriot Act's expansion of the NSA's spying authority is the latest political development in a decades-long struggle between NSA hard-liners and civil liberties advocates. The NSA was created shortly after World War II to advance encryption and decryption techniques, to defend the nation's telecommunications system from attack, and to covertly intercept messages and sort them into useful categories for intelligence purposes. For 30 years, its very existence was a state secret. In August 1975, liberal members of the Senate Intelligence Committee launched an investigation which revealed that, on a daily basis, the major telecommunications companies delivered to the NSA copies of all international telegrams and telexes sent to and from the United States by citizens and noncitizens alike. The NSA also used FBI and CIA watch lists to target the communications of more than 1,600 Americans, mostly critics of the Vietnam War. (Bamford reported in Body of Secrets that the NSA receives Internet watch lists from the CIA, State Department, and other bodies, containing topical keywords, names, phrases, and telephone and fax numbers.)

The Senate committee's revelation that the NSA was spying domestically, without court authorizations, resulted in the passing of new laws, most notably the Federal Intelligence Surveillance Act of 1978, which were intended to restrict the NSA and the federal government's 12 other intelligence agencies from spying on U.S. persons without a warrant. FISA requires the NSA to obtain a court order from a secret panel of federal judges when targeting U.S. persons in the United States for surveillance.

The NSA rarely applies to the FISA court, however. David L. Sobel, general counsel for the Electronic Privacy Information Center in Washington, D.C., says, "It is NSA policy to err on the side of assuming that collected communications are not of U.S. persons." Sobel believes that this presumption should be reversed, since non-U.S. persons have little or no protection against unreasonable search and seizure of their e-mail.

Instead, the NSA operates according to partially declassified internal guidelines that allow its director to authorize domestic intelligence-gathering. Severely redacted copies of two of these internal guides have been obtained by the Federation of American Scientists and posted on its Web site. According to these documents, the NSA director is allowed to order the collection and dissemination of information about a U.S. person according to a whole catalog of reasons, including for law enforcement purposes; to protect classified secrets; to protect the country's electronic communications system; or in the case of significant importance to foreign intelligence-gathering. The NSA is allowed to share its information on U.S. persons with the FBI and other government agencies under a wide variety of circumstances and without court approval. It builds databases on U.S. persons that include personal information such as "criminal, educational, financial, and medical histories associated with a specific name or other personal identifier (such as Social Security Account Number, passport number, or bank account number)."

The ability of American intelligence agencies to gather such information was given a huge technical boost in 1994, when Congress passed the Communications Assistance for Law Enforcement Act, which required telecommunications carriers to modify their equipment so that law enforcement and intelligence operatives can, in essence, double click on an icon to turn on a telephone or high-speed modem line wiretap.

The Patriot Act enables the NSA, and other intelligence and law enforcement agencies, to minimize the red tape in the system of checks and balances previously set up by Congress to protect U.S. persons from having their communications eavesdropped on and analyzed. It allows the NSA to install wiretaps and to access stored electronic communications, such as voice mail, without obtaining a specific court order for a specific location. It changes federal law to allow the NSA to follow the electronic trail of suspects as they move across the country using multiple telephones and computers. Privacy advocates are concerned that wiretapping without a court order based on a showing of probable cause that the suspect is a terrorist violates the Fourth Amendment of the Constitution. Most important, the Patriot Act broadens the definition of who can be defined as a "terrorist." Some civil liberties groups claim the language is vague enough that it could include U.S. citizens who take part in acts of civil disobedience.

But even before Sept. 11, the government's definition of terrorism was quite broad. Last year, FBI Director Louis J. Freeh testified before the Senate Committee on Appropriations about cyberthreats to national security. Before Sept. 11, the FBI considered domestic terrorism to be the main threat to America. The "domestic terrorist threat comes from right-wing extremist groups, left-wing and Puerto Rican extremist groups, and special interest extremists," Freeh said, "including pro-life, environmental, and anti-nuclear [groups] ... the Animal Liberation Front, the Earth Liberation Front ... and anarchists, operating individually and in groups, [that] caused much of the damage during the 1999 World Trade Organization ministerial meeting in Seattle."

Clearly, the NSA is searching for these perceived enemies by monitoring and hacking certain regions of cyberspace. The agency is being mandated by Congress to strengthen its grip on the Internet. The legislative trend, since Sept. 11 and for the foreseeable future, is to allow the NSA its way in cyberspace.By Peter Byrne