projects
/
irspy-moved-to-github.git
/ commitdiff
commit
grep
author
committer
pickaxe
?
search:
re
summary
|
shortlog
|
log
|
commit
| commitdiff |
tree
raw
|
patch
| inline |
side by side
(parent:
19a5102
)
block bot rogerbot for all URLs
author
Wolfram Schneider
<wosch@indexdata.dk>
Fri, 27 Jan 2012 11:58:48 +0000
(11:58 +0000)
committer
Wolfram Schneider
<wosch@indexdata.dk>
Fri, 27 Jan 2012 11:58:48 +0000
(11:58 +0000)
web/htdocs/robots.txt
patch
|
blob
|
history
diff --git
a/web/htdocs/robots.txt
b/web/htdocs/robots.txt
index
fed2b3a
..
a0fdc6c
100644
(file)
--- a/
web/htdocs/robots.txt
+++ b/
web/htdocs/robots.txt
@@
-10,3
+10,6
@@
Disallow: /raw.html
Disallow: /stats.html
Disallow: /upload.html
+User-agent: rogerbot
+Disallow: /
+