Goal: reduce frequency of bot responses that weakly match user intent. Reduce frequency of out-of-context bot responses.
Product Idea: Would it be possible to add a feature allowing organizations using Moveworks to configure (perhaps within Moveworks Setup?) a threshold of acceptability for enterprise search?
- For example, a minimum matching score rating for knowledge matches to allow them to be surfaced. If there are no knowledge matches that meet the minimum matching score, then the bot would let the user know and offer to file a ticket or initiate live chat / agent handoff.
I read about Moveworks Search “Ranking” at the link below, and understand Moveworks has specific ranking methodology to determine the priority order of which content to display.
However, this product idea addresses the scenario where Moveworks has nothing in its repository that meets the threshold and lets the user know, instead of citing a non-relevant or tangential topic. Instead, we would prefer the bot to reply with a statement along the lines of “I couldn't find a precise match for that. Would you like me to file a ticket for you?”
We also envision this could workout when a plug-in triggers: It would be nice to have a threshold so that any enterprise search that comes up after a plug-in trigger, will have a very high likelihood of matching the plug-in content. So for example, when entering time, it would be best to only have articles that pertain to entering Time and not one for Expense (which it would most likely do in organizations where the two topics are so intertwined, and vocabulary across similar topic areas overlaps significantly.)
I realize that if this feature existed, and an organization were to set the threshold too high, users could complain that the bot not knowing things. However if set too low, knowledge articles served would remain noisy, which can sometimes be the case currently.
Here are a few Observations from Recent Chat Experience, testing interacting with the bot and asking it unique / obscure questions:
- The bot will occasionally ask the user to clarify if the question is unclear. It will say things like “It sounds like you may mean one of two things…”, without serving non-relevant articles.
- However, I did also find that the bot will tell the user when it cannot find a source that answers a highly-specific question, but it still will reply with links to articles that do not address the user’s direct intent.
- (happy to try to provide further examples if helpful)