Skip to main content

    986 Ideas

    Triage Improvement1. New

    Hi Community,At our company, we were planning to use triage - however, the model’s precision hasn’t been ideal, partly due to data quality issues.We’ve been exploring ServiceNow Predictive Intelligence to retrieve the top 10 results using either classification or similarity models on our incident data.Our proposed improvement is to introduce an LLM (Large Language Model) to refine the prediction. The idea is for the LLM to evaluate the top 10 results from the ML mode - along with the description of each categor -  to determine the most accurate classification.We believe this could significantly increase the precision of both classification and similarity models, and in some cases, even replace the need for manual agent routing.Assuming triage currently uses OpenAI embeddings for indexing, implementation would be straightforward: Continue indexing all existing incident data, category descriptions, and service information. Use a RAG  approach to retrieve the top 10 results based on the user query. Have the LLM select the best match. Alternatively, using the classification model, we could: Retrieve the top 10 classification results based on the user query. Let the LLM determine the final category. Apply a confidence threshold (e.g., 70%). If the model’s precision is 60%, only high-confidence results go to the LLM, since the correct category is usually within the top few predictions. If results fall below the threshold, the ticket could be routed to an agent for review.The same logic applies when using similarity model - retrieve the most similar incidents, then have the LLM choose the best one.Additionally, since most of our agents follow knowledge articles linked to categories, we could extend this approach: Retrieve the top 5 relevant knowledge articles based on user issue. Have the LLM analyze them and select the one with the best category match. This could effectively automate L1 routing. Just some thoughts I’m planning to implement 🙂It might be worth considering this approach in genera - especially when ML alone isn’t sufficient due to data quality limitations. 

    Conflict Engine for Managing Plugin Overlap in NLP Systems5. Not Enough Info to Make Decision

    Overview:In systems that utilize multiple plugins to fulfill user requests—especially when those plugins serve related but distinct purposes—it can be difficult for the NLP engine to automatically determine which plugin is most appropriate. This often results in multiple plugins being triggered simultaneously, leading to redundant or confusing outputs.To address this, I propose the implementation of a Conflict Engine: a configurable component that detects potential plugin conflicts and prompts the user to select the most relevant plugin for their intent.Key Features: Conflict Detection Logic The Conflict Engine would activate only when multiple plugins that are eligible to respond to a request, but their purposes are distinct enough to warrant user input. For example, one plugin may retrieve live event data while another provides historical comparisons from SQL. User Prompting Mechanism When a conflict is detected, and the offending plugin’s are enabled as part of a Conflict Engine the system would present a prompt listing the relevant plugins along with short, curated descriptions. The user can then select the plugin that best matches their intent. Custom Plugin Descriptions Descriptions shown in the prompt would be defined within the Conflict Engine settings, rather than pulled from the plugin metadata. This ensures clarity and avoids NLP-specific helper text that may confuse end users. Selective Plugin Participation Developers can opt a plugins int the Conflict Engine via a checkbox and select a configured Conflict Engine from a dropdown setting. This ensures that only relevant plugins are considered during conflict resolution and avoids interference with plugins that serve as dependencies or support functions. Considerations:The Conflict Engine should not interfere with plugin chaining or dependency resolution. It should only activate when multiple plugins could independently fulfill a user’s request. Plugin grouping and tagging could further enhance the system’s ability to detect and resolve conflicts. A fallback mechanism such as a timeout should activate in case the user does not respond to the prompt (e.g., default to non Conflict Engine behavior). Conflict Engines could be created/Managed under Data Type, or similar.Benefits:Improves user experience by reducing ambiguity and redundant outputs. Enhances system transparency and user control. Provides a scalable solution for managing plugin complexity as systems grow.