annotate web/robots.txt @ 736:1de6cbdcc619

Add some sanity checks
author Dongsheng Song <dongsheng.song@gmail.com>
date Mon, 16 Mar 2009 17:13:50 +0800
parents ad304b606163
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
673
ad304b606163 Initial cut at web comment system import
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1 User-agent: *
ad304b606163 Initial cut at web comment system import
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
2 Disallow: /feeds/