In his nationally televised speech about Syria on 10 September, Obama laid the blame for the nerve gas attack on the rebel-held suburb of Eastern Ghouta firmly on Assad’s government, and made it clear he was prepared to back up his earlier public warnings that any use of chemical weapons would cross a ‘red line’: ‘Assad’s government gassed to death over a thousand people,’ he said. ‘We know the Assad regime was responsible … And that is why, after careful deliberation, I determined that it is in the national security interests of the United States to respond to the Assad regime’s use of chemical weapons through a targeted military strike.’ Obama was going to war to back up a public threat, but he was doing so without knowing for sure who did what in the early morning of 21 August.

He cited a list of what appeared to be hard-won evidence of Assad’s culpability: ‘In the days leading up to August 21st, we know that Assad’s chemical weapons personnel prepared for an attack near an area where they mix sarin gas. They distributed gas masks to their troops. Then they fired rockets from a regime-controlled area into 11 neighbourhoods that the regime has been trying to wipe clear of opposition forces.’ Obama’s certainty was echoed at the time by Denis McDonough, his chief of staff, who told the New York Times: ‘No one with whom I’ve spoken doubts the intelligence’ directly linking Assad and his regime to the sarin attacks.

But in recent interviews with intelligence and military officers and consultants past and present, I found intense concern, and on occasion anger, over what was repeatedly seen as the deliberate manipulation of intelligence. One high-level intelligence officer, in an email to a colleague, called the administration’s assurances of Assad’s responsibility a ‘ruse’. The attack ‘was not the result of the current regime’, he wrote. A former senior intelligence official told me that the Obama administration had altered the available information – in terms of its timing and sequence – to enable the president and his advisers to make intelligence retrieved days after the attack look as if it had been picked up and analysed in real time, as the attack was happening. The distortion, he said, reminded him of the 1964 Gulf of Tonkin incident, when the Johnson administration reversed the sequence of National Security Agency intercepts to justify one of the early bombings of North Vietnam. The same official said there was immense frustration inside the military and intelligence bureaucracy: ‘The guys are throwing their hands in the air and saying, “How can we help this guy” – Obama – “when he and his cronies in the White House make up the intelligence as they go along?”’

The complaints focus on what Washington did not have: any advance warning from the assumed source of the attack. The military intelligence community has for years produced a highly classified early morning intelligence summary, known as the Morning Report, for the secretary of defence and the chairman of the Joint Chiefs of Staff; a copy also goes to the national security adviser and the director of national intelligence. The Morning Report includes no political or economic information, but provides a summary of important military events around the world, with all available intelligence about them. A senior intelligence consultant told me that some time after the attack he reviewed the reports for 20 August through 23 August. For two days – 20 and 21 August – there was no mention of Syria. On 22 August the lead item in the Morning Report dealt with Egypt; a subsequent item discussed an internal change in the command structure of one of the rebel groups in Syria. Nothing was noted about the use of nerve gas in Damascus that day. It was not until 23 August that the use of sarin became a dominant issue, although hundreds of photographs and videos of the massacre had gone viral within hours on YouTube, Facebook and other social media sites. At this point, the administration knew no more than the public.

Obama left Washington early on 21 August for a hectic two-day speaking tour in New York and Pennsylvania; according to the White House press office, he was briefed later that day on the attack, and the growing public and media furore. The lack of any immediate inside intelligence was made clear on 22 August, when Jen Psaki, a spokesperson for the State Department, told reporters: ‘We are unable to conclusively determine [chemical weapons] use. But we are focused every minute of every day since these events happened … on doing everything possible within our power to nail down the facts.’ The administration’s tone had hardened by 27 August, when Jay Carney, Obama’s press secretary, told reporters – without providing any specific information – that any suggestions that the Syrian government was not responsible ‘are as preposterous as suggestions that the attack itself didn’t occur’.

The absence of immediate alarm inside the American intelligence community demonstrates that there was no intelligence about Syrian intentions in the days before the attack. And there are at least two ways the US could have known about it in advance: both were touched on in one of the top secret American intelligence documents that have been made public in recent months by Edward Snowden, the former NSA contractor.

On 29 August, the Washington Post published excerpts from the annual budget for all national intelligence programmes, agency by agency, provided by Snowden. In consultation with the Obama administration, the newspaper chose to publish only a slim portion of the 178-page document, which has a classification higher than top secret, but it summarised and published a section dealing with problem areas. One problem area was the gap in coverage targeting Assad’s office. The document said that the NSA’s worldwide electronic eavesdropping facilities had been ‘able to monitor unencrypted communications among senior military officials at the outset of the civil war there’. But it was ‘a vulnerability that President Bashar al-Assad’s forces apparently later recognised’. In other words, the NSA no longer had access to the conversations of the top military leadership in Syria, which would have included crucial communications from Assad, such as orders for a nerve gas attack. (In its public statements since 21 August, the Obama administration has never claimed to have specific information connecting Assad himself to the attack.)

The Post report also provided the first indication of a secret sensor system inside Syria, designed to provide early warning of any change in status of the regime’s chemical weapons arsenal. The sensors are monitored by the National Reconnaissance Office, the agency that controls all US intelligence satellites in orbit. According to the Post summary, the NRO is also assigned ‘to extract data from sensors placed on the ground’ inside Syria. The former senior intelligence official, who had direct knowledge of the programme, told me that NRO sensors have been implanted near all known chemical warfare sites in Syria. They are designed to provide constant monitoring of the movement of chemical warheads stored by the military. But far more important, in terms of early warning, is the sensors’ ability to alert US and Israeli intelligence when warheads are being loaded with sarin. (As a neighbouring country, Israel has always been on the alert for changes in the Syrian chemical arsenal, and works closely with American intelligence on early warnings.) A chemical warhead, once loaded with sarin, has a shelf life of a few days or less – the nerve agent begins eroding the rocket almost immediately: it’s a use-it-or-lose-it mass killer. ‘The Syrian army doesn’t have three days to prepare for a chemical attack,’ the former senior intelligence official told me. ‘We created the sensor system for immediate reaction, like an air raid warning or a fire alarm. You can’t have a warning over three days because everyone involved would be dead. It is either right now or you’re history. You do not spend three days getting ready to fire nerve gas.’ The sensors detected no movement in the months and days before 21 August, the former official said. It is of course possible that sarin had been supplied to the Syrian army by other means, but the lack of warning meant that Washington was unable to monitor the events in Eastern Ghouta as they unfolded.

The sensors had worked in the past, as the Syrian leadership knew all too well. Last December the sensor system picked up signs of what seemed to be sarin production at a chemical weapons depot. It was not immediately clear whether the Syrian army was simulating sarin production as part of an exercise (all militaries constantly carry out such exercises) or actually preparing an attack. At the time, Obama publicly warned Syria that using sarin was ‘totally unacceptable’; a similar message was also passed by diplomatic means. The event was later determined to be part of a series of exercises, according to the former senior intelligence official: ‘If what the sensors saw last December was so important that the president had to call and say, “Knock it off,” why didn’t the president issue the same warning three days before the gas attack in August?’

The NSA would of course monitor Assad’s office around the clock if it could, the former official said. Other communications – from various army units in combat throughout Syria – would be far less important, and not analysed in real time. ‘There are literally thousands of tactical radio frequencies used by field units in Syria for mundane routine communications,’ he said, ‘and it would take a huge number of NSA cryptological technicians to listen in – and the useful return would be zilch.’ But the ‘chatter’ is routinely stored on computers. Once the scale of events on 21 August was understood, the NSA mounted a comprehensive effort to search for any links to the attack, sorting through the full archive of stored communications. A keyword or two would be selected and a filter would be employed to find relevant conversations. ‘What happened here is that the NSA intelligence weenies started with an event – the use of sarin – and reached to find chatter that might relate,’ the former official said. ‘This does not lead to a high confidence assessment, unless you start with high confidence that Bashar Assad ordered it, and began looking for anything that supports that belief.’ The cherry-picking was similar to the process used to justify the Iraq war.

The White House needed nine days to assemble its case against the Syrian government. On 30 August it invited a select group of Washington journalists (at least one often critical reporter, Jonathan Landay, the national security correspondent for McClatchy Newspapers, was not invited), and handed them a document carefully labelled as a ‘government assessment’, rather than as an assessment by the intelligence community. The document laid out what was essentially a political argument to bolster the administration’s case against the Assad government. It was, however, more specific than Obama would be later, in his speech on 10 September: American intelligence, it stated, knew that Syria had begun ‘preparing chemical munitions’ three days before the attack. In an aggressive speech later that day, John Kerry provided more details. He said that Syria’s ‘chemical weapons personnel were on the ground, in the area, making preparations’ by 18 August. ‘We know that the Syrian regime elements were told to prepare for the attack by putting on gas masks and taking precautions associated with chemical weapons.’ The government assessment and Kerry’s comments made it seem as if the administration had been tracking the sarin attack as it happened. It is this version of events, untrue but unchallenged, that was widely reported at the time.

An unforseen reaction came in the form of complaints from the Free Syrian Army’s leadership and others about the lack of warning. ‘It’s unbelievable they did nothing to warn people or try to stop the regime before the crime,’ Razan Zaitouneh, an opposition member who lived in one of the towns struck by sarin, told Foreign Policy. The Daily Mail was more blunt: ‘Intelligence report says US officials knew about nerve-gas attack in Syria three days before it killed over 1400 people – including more than 400 children.’ (The number of deaths attributable to the attack varied widely, from at least 1429, as initially claimed by the Obama administration, to many fewer. A Syrian human rights group reported 502 deaths; Médicins sans Frontières put it at 355; and a French report listed 281 known fatalities. The strikingly precise US total was later reported by the Wall Street Journal to have been based not on an actual body count, but on an extrapolation by CIA analysts, who scanned more than a hundred YouTube videos from Eastern Ghouta into a computer system and looked for images of the dead. In other words, it was little more than a guess.)

Five days later, a spokesman for the Office of the Director of National Intelligence responded to the complaints. A statement to the Associated Press said that the intelligence behind the earlier administration assertions was not known at the time of the attack, but recovered only subsequently: ‘Let’s be clear, the United States did not watch, in real time, as this horrible attack took place. The intelligence community was able to gather and analyse information after the fact and determine that elements of the Assad regime had in fact taken steps to prepare prior to using chemical weapons.’ But since the American press corps had their story, the retraction received scant attention. On 31 August the Washington Post, relying on the government assessment, had vividly reported on its front page that American intelligence was able to record ‘each step’ of the Syrian army attack in real time, ‘from the extensive preparations to the launching of rockets to the after-action assessments by Syrian officials’. It did not publish the AP corrective, and the White House maintained control of the narrative.

So when Obama said on 10 September that his administration knew Assad’s chemical weapons personnel had prepared the attack in advance, he was basing the statement not on an intercept caught as it happened, but on communications analysed days after 21 August. The former senior intelligence official explained that the hunt for relevant chatter went back to the exercise detected the previous December, in which, as Obama later said to the public, the Syrian army mobilised chemical weapons personnel and distributed gas masks to its troops. The White House’s government assessment and Obama’s speech were not descriptions of the specific events leading up to the 21 August attack, but an account of the sequence the Syrian military would have followed for any chemical attack. ‘They put together a back story,’ the former official said, ‘and there are lots of different pieces and parts. The template they used was the template that goes back to December.’ It is possible, of course, that Obama was unaware that this account was obtained from an analysis of Syrian army protocol for conducting a gas attack, rather than from direct evidence. Either way he had come to a hasty judgment.

The press would follow suit. The UN report on 16 September confirming the use of sarin was careful to note that its investigators’ access to the attack sites, which came five days after the gassing, had been controlled by rebel forces. ‘As with other sites,’ the report warned, ‘the locations have been well travelled by other individuals prior to the arrival of the mission … During the time spent at these locations, individuals arrived carrying other suspected munitions indicating that such potential evidence is being moved and possibly manipulated.’ Still, the New York Times seized on the report, as did American and British officials, and claimed that it provided crucial evidence backing up the administration’s assertions. An annex to the UN report reproduced YouTube photographs of some recovered munitions, including a rocket that ‘indicatively matches’ the specifics of a 330mm calibre artillery rocket. The New York Times wrote that the existence of the rockets essentially proved that the Syrian government was responsible for the attack ‘because the weapons in question had not been previously documented or reported to be in possession of the insurgency’.

Theodore Postol, a professor of technology and national security at MIT, reviewed the UN photos with a group of his colleagues and concluded that the large calibre rocket was an improvised munition that was very likely manufactured locally. He told me that it was ‘something you could produce in a modestly capable machine shop’. The rocket in the photos, he added, fails to match the specifications of a similar but smaller rocket known to be in the Syrian arsenal. The New York Times, again relying on data in the UN report, also analysed the flight path of two of the spent rockets that were believed to have carried sarin, and concluded that the angle of descent ‘pointed directly’ to their being fired from a Syrian army base more than nine kilometres from the landing zone. Postol, who has served as the scientific adviser to the chief of naval operations in the Pentagon, said that the assertions in the Times and elsewhere ‘were not based on actual observations’. He concluded that the flight path analyses in particular were, as he put it in an email, ‘totally nuts’ because a thorough study demonstrated that the range of the improvised rockets was ‘unlikely’ to be more than two kilometres. Postol and a colleague, Richard M. Lloyd, published an analysis two weeks after 21 August in which they correctly assessed that the rockets involved carried a far greater payload of sarin than previously estimated. The Times reported on that analysis at length, describing Postol and Lloyd as ‘leading weapons experts’. The pair’s later study about the rockets’ flight paths and range, which contradicted previous Times reporting, was emailed to the newspaper last week; it has so far gone unreported.

The White House’s misrepresentation of what it knew about the attack, and when, was matched by its readiness to ignore intelligence that could undermine the narrative. That information concerned al-Nusra, the Islamist rebel group designated by the US and the UN as a terrorist organisation. Al-Nusra is known to have carried out scores of suicide bombings against Christians and other non-Sunni Muslim sects inside Syria, and to have attacked its nominal ally in the civil war, the secular Free Syrian Army (FSA). Its stated goal is to overthrow the Assad regime and establish sharia law. (On 25 September al-Nusra joined several other Islamist rebel groups in repudiating the FSA and another secular faction, the Syrian National Coalition.)

The flurry of American interest in al-Nusra and sarin stemmed from a series of small-scale chemical weapons attacks in March and April; at the time, the Syrian government and the rebels each insisted the other was responsible. The UN eventually concluded that four chemical attacks had been carried out, but did not assign responsibility. A White House official told the press in late April that the intelligence community had assessed ‘with varying degrees of confidence’ that the Syrian government was responsible for the attacks. Assad had crossed Obama’s ‘red line’. The April assessment made headlines, but some significant caveats were lost in translation. The unnamed official conducting the briefing acknowledged that intelligence community assessments ‘are not alone sufficient’. ‘We want,’ he said, ‘to investigate above and beyond those intelligence assessments to gather facts so that we can establish a credible and corroborated set of information that can then inform our decision-making.’ In other words, the White House had no direct evidence of Syrian army or government involvement, a fact that was only occasionally noted in the press coverage. Obama’s tough talk played well with the public and Congress, who view Assad as a ruthless murderer.

Two months later, a White House statement announced a change in the assessment of Syrian culpability and declared that the intelligence community now had ‘high confidence’ that the Assad government was responsible for as many as 150 deaths from attacks with sarin. More headlines were generated and the press was told that Obama, in response to the new intelligence, had ordered an increase in non-lethal aid to the Syrian opposition. But once again there were significant caveats. The new intelligence included a report that Syrian officials had planned and executed the attacks. No specifics were provided, nor were those who provided the reports identified. The White House statement said that laboratory analysis had confirmed the use of sarin, but also that a positive finding of the nerve agent ‘does not tell us how or where the individuals were exposed or who was responsible for the dissemination’. The White House further declared: ‘We have no reliable corroborated reporting to indicate that the opposition in Syria has acquired or used chemical weapons.’ The statement contradicted evidence that at the time was streaming into US intelligence agencies.

Already by late May, the senior intelligence consultant told me, the CIA had briefed the Obama administration on al-Nusra and its work with sarin, and had sent alarming reports that another Sunni fundamentalist group active in Syria, al-Qaida in Iraq (AQI), also understood the science of producing sarin. At the time, al-Nusra was operating in areas close to Damascus, including Eastern Ghouta. An intelligence document issued in mid-summer dealt extensively with Ziyaad Tariq Ahmed, a chemical weapons expert formerly of the Iraqi military, who was said to have moved into Syria and to be operating in Eastern Ghouta. The consultant told me that Tariq had been identified ‘as an al-Nusra guy with a track record of making mustard gas in Iraq and someone who is implicated in making and using sarin’. He is regarded as a high-profile target by the American military.

On 20 June a four-page top secret cable summarising what had been learned about al-Nusra’s nerve gas capabilities was forwarded to David R. Shedd, deputy director of the Defense Intelligence Agency. ‘What Shedd was briefed on was extensive and comprehensive,’ the consultant said. ‘It was not a bunch of “we believes”.’ He told me that the cable made no assessment as to whether the rebels or the Syrian army had initiated the attacks in March and April, but it did confirm previous reports that al-Nusra had the ability to acquire and use sarin. A sample of the sarin that had been used was also recovered – with the help of an Israeli agent – but, according to the consultant, no further reporting about the sample showed up in cable traffic.

Independently of these assessments, the Joint Chiefs of Staff, assuming that US troops might be ordered into Syria to seize the government’s stockpile of chemical agents, called for an all-source analysis of the potential threat. ‘The Op Order provides the basis of execution of a military mission, if so ordered,’ the former senior intelligence official explained. ‘This includes the possible need to send American soldiers to a Syrian chemical site to defend it against rebel seizure. If the jihadist rebels were going to overrun the site, the assumption is that Assad would not fight us because we were protecting the chemical from the rebels. All Op Orders contain an intelligence threat component. We had technical analysts from the Central Intelligence Agency, the Defense Intelligence Agency, weapons people, and I & W [indications and warnings] people working on the problem … They concluded that the rebel forces were capable of attacking an American force with sarin because they were able to produce the lethal gas. The examination relied on signals and human intelligence, as well as the expressed intention and technical capability of the rebels.’

There is evidence that during the summer some members of the Joint Chiefs of Staff were troubled by the prospect of a ground invasion of Syria as well as by Obama’s professed desire to give rebel factions non-lethal support. In July, General Martin Dempsey, chairman of the Joint Chiefs, provided a gloomy assessment, telling the Senate Armed Services Committee in public testimony that ‘thousands of special operations forces and other ground forces’ would be needed to seize Syria’s widely dispersed chemical warfare arsenal, along with ‘hundreds of aircraft, ships, submarines and other enablers’. Pentagon estimates put the number of troops at seventy thousand, in part because US forces would also have to guard the Syrian rocket fleet: accessing large volumes of the chemicals that create sarin without the means to deliver it would be of little value to a rebel force. In a letter to Senator Carl Levin, Dempsey cautioned that a decision to grab the Syrian arsenal could have unintended consequences: ‘We have learned from the past ten years, however, that it is not enough to simply alter the balance of military power without careful consideration of what is necessary in order to preserve a functioning state … Should the regime’s institutions collapse in the absence of a viable opposition, we could inadvertently empower extremists or unleash the very chemical weapons we seek to control.’

The CIA declined to comment for this article. Spokesmen for the DIA and Office of the Director of National Intelligence said they were not aware of the report to Shedd and, when provided with specific cable markings for the document, said they were unable to find it. Shawn Turner, head of public affairs for the ODNI, said that no American intelligence agency, including the DIA, ‘assesses that the al-Nusra Front has succeeded in developing a capacity to manufacture sarin’.

The administration’s public affairs officials are not as concerned about al-Nusra’s military potential as Shedd has been in his public statements. In late July, he gave an alarming account of al-Nusra’s strength at the annual Aspen Security Forum in Colorado. ‘I count no less than 1200 disparate groups in the opposition,’ Shedd said, according to a recording of his presentation. ‘And within the opposition, the al-Nusra Front is … most effective and is gaining in strength.’ This, he said, ‘is of serious concern to us. If left unchecked, I am very concerned that the most radical elements’ – he also cited al-Qaida in Iraq – ‘will take over.’ The civil war, he went on, ‘will only grow worse over time … Unfathomable violence is yet to come.’ Shedd made no mention of chemical weapons in his talk, but he was not allowed to: the reports his office received were highly classified.

A series of secret dispatches from Syria over the summer reported that members of the FSA were complaining to American intelligence operatives about repeated attacks on their forces by al-Nusra and al-Qaida fighters. The reports, according to the senior intelligence consultant who read them, provided evidence that the FSA is ‘more worried about the crazies than it is about Assad’. The FSA is largely composed of defectors from the Syrian army. The Obama administration, committed to the end of the Assad regime and continued support for the rebels, has sought in its public statements since the attack to downplay the influence of Salafist and Wahhabist factions. In early September, John Kerry dumbfounded a Congressional hearing with a sudden claim that al-Nusra and other Islamist groups were minority players in the Syrian opposition. He later withdrew the claim.

In both its public and private briefings after 21 August, the administration disregarded the available intelligence about al-Nusra’s potential access to sarin and continued to claim that the Assad government was in sole possession of chemical weapons. This was the message conveyed in the various secret briefings that members of Congress received in the days after the attack, when Obama was seeking support for his planned missile offensive against Syrian military installations. One legislator with more than two decades of experience in military affairs told me that he came away from one such briefing persuaded that ‘only the Assad government had sarin and the rebels did not.’ Similarly, following the release of the UN report on 16 September confirming that sarin was used on 21 August, Samantha Power, the US ambassador to the UN, told a press conference: ‘It’s very important to note that only the [Assad] regime possesses sarin, and we have no evidence that the opposition possesses sarin.’

It is not known whether the highly classified reporting on al-Nusra was made available to Power’s office, but her comment was a reflection of the attitude that swept through the administration. ‘The immediate assumption was that Assad had done it,’ the former senior intelligence official told me. ‘The new director of the CIA, [John] Brennan, jumped to that conclusion … drives to the White House and says: “Look at what I’ve got!” It was all verbal; they just waved the bloody shirt. There was a lot of political pressure to bring Obama to the table to help the rebels, and there was wishful thinking that this [tying Assad to the sarin attack] would force Obama’s hand: “This is the Zimmermann telegram of the Syrian rebellion and now Obama can react.” Wishful thinking by the Samantha Power wing within the administration. Unfortunately, some members of the Joint Chiefs who were alerted that he was going to attack weren’t so sure it was a good thing.’

The proposed American missile attack on Syria never won public support and Obama turned quickly to the UN and the Russian proposal for dismantling the Syrian chemical warfare complex. Any possibility of military action was definitively averted on 26 September when the administration joined Russia in approving a draft UN resolution calling on the Assad government to get rid of its chemical arsenal. Obama’s retreat brought relief to many senior military officers. (One high-level special operations adviser told me that the ill-conceived American missile attack on Syrian military airfields and missile emplacements, as initially envisaged by the White House, would have been ‘like providing close air support for al-Nusra’.)

The administration’s distortion of the facts surrounding the sarin attack raises an unavoidable question: do we have the whole story of Obama’s willingness to walk away from his ‘red line’ threat to bomb Syria? He had claimed to have an iron-clad case but suddenly agreed to take the issue to Congress, and later to accept Assad’s offer to relinquish his chemical weapons. It appears possible that at some point he was directly confronted with contradictory information: evidence strong enough to persuade him to cancel his attack plan, and take the criticism sure to come from Republicans.

The UN resolution, which was adopted on 27 September by the Security Council, dealt indirectly with the notion that rebel forces such as al-Nusra would also be obliged to disarm: ‘no party in Syria should use, develop, produce, acquire, stockpile, retain or transfer [chemical] weapons.’ The resolution also calls for the immediate notification of the Security Council in the event that any ‘non-state actors’ acquire chemical weapons. No group was cited by name. While the Syrian regime continues the process of eliminating its chemical arsenal, the irony is that, after Assad’s stockpile of precursor agents is destroyed, al-Nusra and its Islamist allies could end up as the only faction inside Syria with access to the ingredients that can create sarin, a strategic weapon that would be unlike any other in the war zone. There may be more to negotiate.