Commit c9b665244da3eda234b6d58ea89014eec2ef9684
Inform robots to ignore raw URLs in tags
Raw data isn't useful for search engines as it duplicates non-raw
things, previous change addressed only data in branches.
Author: xaizek
Author date (UTC): 2019-07-20 16:11
Committer name: xaizek
Committer date (UTC): 2019-07-20 16:11
Parent(s): a8b0d79c365ed6f9eb4445ed0be0f7918978616c
Signing key: 99DC5E4DB05F6BE2
Tree: 78218fee4c7b192eff00e991e32a99a94844b39a
File root/robots.txt changed (mode: 100644) (index 03d5baa..ef4ece5) |
... |
... |
Disallow: /nonexisting |
4 |
4 |
Disallow: /op/logout |
Disallow: /op/logout |
5 |
5 |
Disallow: /*?* |
Disallow: /*?* |
6 |
6 |
Disallow: /user/*/*/source/tree/branch/*/raw/ |
Disallow: /user/*/*/source/tree/branch/*/raw/ |
|
7 |
|
Disallow: /user/*/*/source/tree/tag/*/raw/ |
7 |
8 |
Disallow: /user/*/*/history |
Disallow: /user/*/*/history |