Page MenuHomePhabricator

Add a robots.txt file to disallow /diffusion/
ClosedPublic

Authored by epriestley on Mar 14 2014, 2:23 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, May 26, 5:52 PM
Unknown Object (File)
Mon, May 26, 11:09 AM
Unknown Object (File)
Mon, May 26, 8:05 AM
Unknown Object (File)
Sun, May 25, 10:44 AM
Unknown Object (File)
Mon, May 5, 1:09 PM
Unknown Object (File)
Mon, May 5, 12:16 PM
Unknown Object (File)
Apr 27 2025, 3:12 AM
Unknown Object (File)
Apr 24 2025, 4:55 PM
Subscribers

Details

Summary

Fixes T4610. Open to suggestions, etc., if there's anything I'm missing.

Also:

  • Moves these "system" endpoints into a real application.
  • Makes isUnlisted() work a little more consistently.
Test Plan

Accessed /robots.txt, /status/ and /debug/.

Diff Detail

Repository
rP Phabricator
Branch
robotstxt
Lint
Lint Passed
Unit
Tests Passed

Event Timeline

epriestley retitled this revision from to Add a robots.txt file to disallow /diffusion/.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added reviewers: btrahan, chad.
btrahan edited edge metadata.

Makes sense to me.

I think diffusion being in SEO is really for cases where diviner should be in SEO. (i.e. code documentation, which should be getting pulled into diviner automagically.) We'll get there.

This revision is now accepted and ready to land.Mar 14 2014, 5:43 PM
epriestley updated this revision to Diff 20255.

Closed by commit rP838f78128544 (authored by @epriestley).