User-agent: * Disallow: /admin/ Disallow: /core/ Disallow: /modules/ Disallow: /html/