Commit 2c31f7cbd7b7567cc42dd7762e32d00cb01c2d97
Inform robots to ignore raw URLs in tags
Raw data isn't useful for search engines as it duplicates non-raw
things, previous change addressed only data in branches.
Author: xaizek
Author date (UTC): 2019-07-20 16:11
Committer name: xaizek
Committer date (UTC): 2020-07-30 21:15
Parent(s): a64b6f76163236e1cb2731bbea445f7817e3fd2e
Signing key: 99DC5E4DB05F6BE2
Tree: ee63d5e174c301c92186cd5426e44d27e76be22e
File root/robots.txt changed (mode: 100644) (index 03d5baa..ef4ece5) |
... |
... |
Disallow: /nonexisting |
4 |
4 |
Disallow: /op/logout |
Disallow: /op/logout |
5 |
5 |
Disallow: /*?* |
Disallow: /*?* |
6 |
6 |
Disallow: /user/*/*/source/tree/branch/*/raw/ |
Disallow: /user/*/*/source/tree/branch/*/raw/ |
|
7 |
|
Disallow: /user/*/*/source/tree/tag/*/raw/ |
7 |
8 |
Disallow: /user/*/*/history |
Disallow: /user/*/*/history |