Page MenuHomePhabricator

"Text" type custom field in Maniphest not displayed in Maniphest Advanced Search form
Closed, ResolvedPublic

Description

Steps to reproduce:

  1. RTFM at https://secure.phabricator.com/book/phabricator/article/custom_fields/
  2. Go to /config/edit/maniphest.custom-field-definitions/
  3. As Database value, enter:
{
  "fxxx": {
    "type": "text",
    "default": "none",
    "edit": false,
    "name": "Fxxx",
    "search": true,
    "view": true
  }
}
  1. Click Save Config Entry button
  2. Paranoidly double-check on /config/edit/maniphest.fields/ that only your custom field "Fxxx" is enabled.
  3. Restart webserver (systemctl restart httpd.service on Fedora 25)
  4. Go to /maniphest/query/advanced

Expected outcome:
Custom field Fxxx is shown in Maniphest Advanced Search form (between Spaces and Created After)

Actual outcome:
Custom field is not shown.

Other comments:

Versions:
phabricator: 0817eb14a94becd8478033742e339aef9ed6520d (Tue, Sep 20)
arcanist: 9e82ef979e8148c43b9b8439025d505b1219e213 (Thu, Aug 25)
phutil: fa6519f4cea1c059bd381c20971cac413cc4760e (Thu, Sep 15)

Event Timeline

I believed this was semi-intentional, although the downstream report suggests it worked at one point, so maybe I just made a mistake.

From digging, it looks like I disabled this support in D14766 with no explanation. I don't specifically recall why and it might have been a plain error or some debugging/testing code making it through, although I think I may not have wanted to expose these fields to Conduit search given the actual search behavior.

Overall, we support three general classes of text search. Although this isn't a totally accurate description of them, it's mostly how they work:

  • Search for exact matches: searching for "several wooly mammoths" finds "several wooly mammoths" only, not "several wooly mammoths knitting daintily" (technically: this search can still be case-insensitive).
  • Substring search: searching for "several wooly mammoths" finds "several wooly mammoths" and "several wooly mammoths knitting daintily", but not "several (daintily knitting) wooly mammoths", because the query string does not appear as an exact substring in the result (technically: some of our substring searches can still find this result).
  • Fulltext search: searching for "several wooly mammoths" finds all of "several wooly mammoths", "several wooly mammoths knitting daintily", and "several (daintily knitting) wooly mammoths". In some cases (particularly after T6740) this search can also use the query "wool mammoth knitter" to find "several (daintily knitting) wooly mammoths".

Exact match search is simple, but the least powerful. We rarely use this type of search because users often expect a more powerful kind of query and are confused when "mammoths" does not match "wooly mammoths". The disabled implementation uses exact match.

Substring search is good in some cases, and we use it for things like Almanac service names (and, once I get around to it, filenames in Files) -- generally, fields which act more like a single big word than a collection of smaller human-readable words. We seem to have a good implementation for it after T9979 based on building ngram indexes, but this requires some additional setup and work. The CustomField storage/index do not currently support ngram indexes, and adding support may be involved.

Fulltext search is yet more powerful, but also more complicated. The CustomField storage/index do not currently support fulltext indexes, and adding support is very likely involved.

I think the custom "text" field probably should use a more powerful search than "exact match". In the downstream case (RT ticket numbers), exact match is fine, but if the field was called something like "Keywords", "Labels", "Required Parts", "Related Vendors", "Zebra Stripe Colors", "Preferred Flavors", etc., a baker could not find a wedding cake request with "Preferred Flavors: jerky, cardamom" by searching for "cardamom". This seems contrary to reasonable expectations that users might have.

Fixing this by supporting substring or fulltext indexing on these fields is ultimately the pathway forward, but either is quite complex.

A quicker fix for the downstream issue might be to introduce a new type of builtin custom field that works exactly like "text" but has "exact match" search semantics. The existing "link" type already works somewhat like this, but has some additional behavior which isn't desirable in this case.

I don't have a great name for this offhand, but maybe "token" or "word"? If anyone has other ideas, let me know. If we implemented this, you'd just change "type": "text" to "type": "token" in the config and search should work properly again.

The UI should also warn you when you set "search": true for a field type which does not support search.

We use a text field for a 'reference number' which refers to tasks imported from bugzilla reference=bz123, or RT with reference=rt456

It definitely isn't clear, currently, that text fields are not supported and a plain exact-match would be better than nothing in this case.

There's a bunch of other Herald stuff we already do for this field so I'm just undoing the change from D14766, at least until someone complains that this field type isn't using the search logic they expect.

epriestley triaged this task as Normal priority.
epriestley added a project: Custom Fields.

This should now just work without any changes at HEAD of master. Let us know if you're still seeing issues after updating.