User-agent: * # how often to fetch a page (in seconds) Crawl-delay: 60 Disallow: /gc/ Disallow: /calendar/ Disallow: /today/story-image-display # Disallow: /calendar/ical-output.tcl User-agent: gsa-crawler Disallow: /gc/ Disallow: /today/story-image-display Disallow: /calendar/index.tcl Disallow: /calendar/index Disallow: /calendar/academic-calendar.tcl Disallow: /calendar/academic-calendar Disallow: /calendar/athletic-calendar.tcl Disallow: /calendar/athletic-calendar