When custom fields are asked to apply their search constraints to a query in applyCustomFieldsToQuery (possibly elsewhere, I haven't dug), applyApplicationSearchConstraintToQuery is called before a viewer is set.
A minimal failing test case (drop in src/extensions and try to search for something in Maniphest): {F133066}
This can be fixed with something like $field->setViewer($this->requireViewer()); in that foreach loop, but I don't really understand the implications of doing that well enough to send you a diff.
My particular use case here is to allow a custom search field to use the viewer's timezone to parse a time.