# File- Robots.txt by B. Armstrong 01-Jan-00 / 12-Jul-07 # # Purpose- # # This file prevents meta crawlers and site indexers from accessing # files in the /bin, /htbin, and /display directories # # Notes- # # To customize this file, it must be first copied by hand to # [dymax.?dbp?.web]robots.txt. WDExpr will look for the ?dbp? copy first, # and if it is not there, default to the master copy. # # We prefer that improvements to this file be made to the master copy # [dymax.web]robots.txt if possible so that all sites receive the # enhancements. # # Modifcations- # # 12-Jul-07 BG / PM / BG # - Clarified comment header about customization and policy for # making enhancements to master copy. # User-agent: * Disallow: /bin/ Disallow: /htbin/ Disallow: /display/ Disallow: /cat/ Disallow: /*/cat/