User-agent: * ### don't allow member spam profiles to get indexed: Disallow: /member/* ### don't index the courtesy galleries Disallow: /images/special/ ### googlebot somehow got into a loop with photo pagination: Disallow: /photos/archive/Y*/P*/P ### more trouble with photo pagination Disallow: photos/archive/Y20099*/P* Disallow: photos/archive/Y20098*/P* ### googlebot following more bad pagination links Disallow: /photos/*/%E2%80%9Dhttp:/* Disallow: /*_cardinal Disallow: /newpress/ Disallow: /forms/licensing/gl/ Disallow: /forms/licensing/radio/ Disallow: /forms/licensing/newmedia/ Disallow: /licensing/entry/musicservicee* ### Let's not index the word version of our press releases Disallow: /press/releases/*.doc$ Disallow: /press/releases/*.docx$ ### Nov 2013 accidentally published a link to 'test' instead of 'tag' Disallow: /video/test/ ### don't index old flash slideshow xml files Disallow: /images/*.xml