# www.4smotors.co.uk User-agent: * Disallow: /bbclone/ Disallow: /style/ Disallow: /fpdf2/ User-agent: msnbot Crawl-delay: 10 User-agent: Teoma Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 User-agent: aipbot Disallow: / User-agent: BecomeBot Disallow: / User-agent: psbot Disallow: / #This is an example robots.txt file User-agent: TurnitinBot Disallow: / #Will disallow all urls on your site