User-agent: * ### don't allow member spam profiles to get indexed: Disallow: /member/* ### googlebot somehow got into a loop with photo pagination: Disallow: /photos/archive/Y*/P*/P ### more trouble with photo pagination Disallow: photos/archive/Y20099*/P* Disallow: photos/archive/Y20098*/P* ### googlebot following more bad pagination links Disallow: /photos/*/%E2%80%9Dhttp:/* Disallow: /forms/licensing/gl/ Disallow: /forms/licensing/radio/ Disallow: /forms/licensing/newmedia/ ### Let's not index the word version of our press releases Disallow: /press/releases/*.doc$ Disallow: /press/releases/*.docx$ ### Nov 2013 accidentally published a link to 'test' instead of 'tag' Disallow: /video/test/ ### don't index old flash slideshow xml files Disallow: /images/*.xml ### do not index site search results Disallow: /search/results* Disallow: /news/search_results* Disallow: /photos/search_results* Disallow: /video/search_results* Disallow: /events/search_results* Disallow: /press/search_results* ### do not index JSON feeds/AJAX Disallow: /api* Disallow: /ssi/* Disallow: /leadgen/* ### do follow calendar into the distant future or past Disallow: /events/calendar/204* Disallow: /events/calendar/205* Disallow: /events/calendar/19*