How to use custom question answering using a decision tree approach?

Question

How to use custom question answering using a decision tree approach?

Biplab 0

We are building a speech enabled application and using custom question/answering to fetch the questions. This currently works in a sequential manner and I am able to fetch all questions from the QnA knowledge base.

However, the use case I am working on requires to ask relevant questions with respect to how the user is answering the previous one - something like a decision tree.

What relevant Azure offering should be used here? Will Azure AI Bot service help or I need to use CLU (Conversational Language Understanding) or is it going be Orchestration Workflow? The main problem is how to use a decision tree concept with question answering for a guided conversation.

Pavankumar Purilla 7,090 Reputation points Microsoft External Staff Moderator

2025-04-28T05:36:17.45+00:00

Hi Biplab,
Did you get any chance to check the response. Thank you!
Pavankumar Purilla 7,090 Reputation points Microsoft External Staff Moderator

2025-04-29T07:53:58.65+00:00

Hi Biplab,
Just following up to see if you had a chance to review the above response. Thank you!
Biplab 0 Reputation points

2025-05-09T16:49:14.1233333+00:00

Hi Pavan,

I was able to use custom Q&A along with Orchestration workflow. However, its getting difficult to build the decision tree directly into custom Q&A.

Do you think CLU can help with the decision tree? I saw you suggested this can be done using simple conditional logic or more advanced data structures like state machines or JSON-based flows. Is there an example that you can provide? I certainly don't want the decision tree branching logic to be defined with If/Else. This is where I am struggling currently.

I was able to use "Follow-up Prompts" in custom Q&A to define a branching structure but its becoming cumbersome.

Let me know.

Thanks,

Biplab
Pavankumar Purilla 7,090 Reputation points Microsoft External Staff Moderator

2025-05-09T22:01:26.4+00:00
Hi Biplab,
Thanks for the update — it’s great to hear you’ve been able to integrate Custom Q&A with Orchestration Workflow!

You’re correct that building a complex decision tree directly inside Custom Q&A, especially using follow-up prompts, can quickly become difficult to manage. While CLU can help, it’s important to clarify that CLU’s role is not to store or define the decision tree — instead, CLU helps understand the user’s intent and extract relevant entities from their responses, which you can then use to drive the branching logic in your application.

To avoid using many if/else conditions, I recommend adopting a data-driven approach where you define the decision tree in a structured format like a JSON file, YAML file, or database table. Your bot or application can then dynamically read this structure to determine:

What the next question is,

Which branch to follow, and

What condition or intent (detected by CLU) to evaluate for the next step.

Here’s a simple example of a JSON-based decision tree:

{ "question": "Do you have a medical condition?", "branches": { "yes": { "question": "Are you on medication?", "branches": { "yes": { "question": "What type of medication are you taking?" }, "no": { "question": "Have you consulted a doctor recently?" } } }, "no": { "question": "Do you exercise regularly?" } } }

In this setup:

CLU helps recognize the user’s intent or extract keywords like “yes,” “no,” or specific conditions.

Your bot or application uses the current position in the decision tree (defined in JSON) and the CLU result to determine the next question.

This eliminates hardcoded if/else logic and makes the system more flexible and easier to maintain.

To manage the conversation flow, you can also use Azure Bot Service with Bot Framework Composer, which supports adaptive dialogs and makes it easier to integrate CLU and read from the decision tree dynamically.
I hope this information helps.

1 answer

Your answer

Pavankumar Purilla 7,090 Reputation points Microsoft External Staff Moderator

2025-04-28T05:36:17.45+00:00

Hi Biplab,
Did you get any chance to check the response. Thank you!
Pavankumar Purilla 7,090 Reputation points Microsoft External Staff Moderator

2025-04-29T07:53:58.65+00:00

Hi Biplab,
Just following up to see if you had a chance to review the above response. Thank you!
Biplab 0 Reputation points

2025-05-09T16:49:14.1233333+00:00

Hi Pavan,

I was able to use custom Q&A along with Orchestration workflow. However, its getting difficult to build the decision tree directly into custom Q&A.

Do you think CLU can help with the decision tree? I saw you suggested this can be done using simple conditional logic or more advanced data structures like state machines or JSON-based flows. Is there an example that you can provide? I certainly don't want the decision tree branching logic to be defined with If/Else. This is where I am struggling currently.

I was able to use "Follow-up Prompts" in custom Q&A to define a branching structure but its becoming cumbersome.

Let me know.

Thanks,

Biplab

Answer 1

Hi Biplab,

To implement a decision tree approach for guided conversations using Custom Question Answering in a speech-enabled application, you can leverage a combination of Azure services. Here's how each relevant service contributes to the solution:

Custom Question Answering allows you to build a knowledge base from your own data such as FAQs, manuals, or structured documents. This service is used to fetch questions and answers relevant to your domain.

Conversational Language Understanding (CLU) plays a key role in understanding the user's intent and extracting meaningful entities from their responses. This contextual understanding helps determine the appropriate next question in a guided flow.

Orchestration Workflow lets you integrate different language components like Custom Question Answering and CLU into a single conversational experience. It intelligently routes user queries to the correct component based on intent and context.

To build your decision-tree-based guided conversation, follow these steps:

Step 1: Set Up Custom Question Answering – Create a QnA project in Azure Language Studio with your predefined questions and answers.

Step 2: Define Decision Tree Logic – In your application, implement logic that determines the next question based on user responses. This can be done using simple conditional logic or more advanced data structures like state machines or JSON-based flows.

Step 3: Use CLU for Understanding – Apply CLU to interpret user inputs more deeply, enabling your system to extract entities or intent that influence the decision tree's branching.

Step 4: Orchestrate with Workflow – Set up an Orchestration Project to seamlessly route between QnA and CLU, ensuring the right service is triggered at the right time.

By combining these services, you can design a conversational experience that dynamically adapts based on user input—much like a decision tree—while maintaining flexibility, contextual understanding, and voice interactivity using Azure Speech Services.

I hope this information helps. Thank you!

Share via

How to use custom question answering using a decision tree approach?

1 answer

Your answer