# Alexa User-agent: ia_archiver Allow: # Ask User-agent: Teoma Allow: # Google User-agent: googlebot Allow: # MSN User-agent: msnbot Allow: # Yahoo! User-agent: Slurp Allow: # Baidu User-agent: baiduspider Allow: # Become.com User-agent: BecomeBot Allow: # DailyOrbit.com User-agent: Orbiter Allow: # dir.com User-agent: Pompos Allow: # IBM User-agent: Crawler Allow: # Picsearch User-agent: psbot Allow: # SpiderMonkey User-agent: SpiderMonkey Allow: # Voila.fr User-agent: VoilaBot Allow: # WebArchive User-agent: BruinBot Allow: # Yandex User-agent: Yandex Allow: User-agent: Mediapartners-Google Allow: User-agent: Adsbot-Google Allow: User-agent: Googlebot-Image Allow: User-agent: Googlebot-Mobile Allow: User-agent: ia_archiver-web.archive.org Allow: # Others User-agent: * Allow: