Differential D8782
Implement smart waits for rarely updated repositories Authored by epriestley on Apr 16 2014, 1:37 AM. Tags None Referenced Files
Subscribers Tokens
Details
Ref T4605. When figuring out how long to wait to update a repository, factor in when it was last pushed. For rarely updated repositories, wait longer between updates. (A slightly funky thing about this is that empty repos update every 15 seconds, but that seems OK for the moment.) Ran bin/phd debug pulllocal and saw sensible calculations and output: ... <VERB> PhabricatorRepositoryPullLocalDaemon Last commit to repository "rPOEMS" was 1,239,608 seconds ago; considering a wait of 6,198 seconds before update. >>> [79] <query> SELECT * FROM `repository` r ORDER BY r.id DESC <<< [79] <query> 514 us >>> [80] <query> SELECT * FROM `repository_statusmessage` WHERE statusType = 'needs-update' <<< [80] <query> 406 us <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIH" is not due for an update for 8,754 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rDUCK" is not due for an update for 14 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rMTESTX" is not due for an update for 21,598 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rQWER" is not due for an update for 14 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rBT" is not due for an update for 13 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rSVNX" is not due for an update for 21,598 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIG" is not due for an update for 13 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rHGTEST" is not due for an update for 21,598 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rBTX" is not due for an update for 14 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rGX" is not due for an update for 13 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rMTX" is currently updating. <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPOEMS" is not due for an update for 6,198 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPHU" is currently updating. <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rSVN" is not due for an update for 21,598 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPHY" is currently updating. <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rGTEST" is not due for an update for 21,598 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIS" is not due for an update for 6,894 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rARCLINT" is not due for an update for 21,599 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rLPHX" is not due for an update for 1,979 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rARC" is not due for an update for 1,824 second(s). <VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIHG" is not due for an update for 21,599 second(s). ...
Diff Detail
Event TimelineComment Actions I'm very down 'cuz of the arc land hook. :) Otherwise, I'd expect the "smart" part of the smart wait to cause confusion. I think the "not updated all that much" *implies* that folks won't want to look at the repository that soon after a land, but its a weak implication to me. Also, I don't know if "batching" (i.e. processing N commits instead of 1) ends up being actually less system-resource intensive, but if it does, I'd expect the best algorithm to be waiting a bit on frequently updated repositories to get that batch performance impact, waiting less for moderately updated repositories, and then waiting a long time again for infrequently updated repositories to exploit the "not updated all that much" implication. Comment Actions
Yeah, agreed. I'm hoping the "arc land" hook will fix a lot of this (we also get the hint for hosted repositories in all cases, so those are totally reliable). We can go add config or something if it's not good enough, but ideally this is close enough and it "just works" from users' points of view.
I think the break-even point on this is way above one commit per second, and we've never seen a repository which updates that fast. |