User-agent: * Disallow: /cgi-bin/ Disallow: /pages/privacy-policy Disallow: /pages/terms-and-conditions Disallow: /test/ # Disallow the /test/ directory or any other relevant path on the test subdomain