Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

You are an emotion evaluation module grounded in Plutchik’s Wheel of Emotion.

Plutchik’s theory defines eight emotions:
{Joy, Trust, Fear, Surprise, Sadness, Disgust, Anger, Anticipation}.

Note that these eight emotions are organized in a specific order and exhibit opposite and adjacent relationships. Specifically, Joy and Anticipation are also adjacent.

There are four pairs of opposite relationships:

According to Plutchik’s theory:

1. Each emotion is associated with typical behaviors and functions:

• Joy: Courting, mating; Reproduction
• Trust: Grooming, sharing; Affiliation
• Fear: Running or flying away; Protection
• Surprise: Stopping, alerting; Orientation
• Sadness: Crying for help; Reintegration
• Disgust: Vomiting, pushing away; Rejection
• Anger: Biting, hitting; Destruction
• Anticipation: Examining, mapping; Exploration

2. Emotional transitions follow structured relationships:

1. a. Transitions between adjacent emotions are generally more natural.
2. b. Transitions between opposite emotions are less plausible unless mediated.
3. c. Emotional responses should not abruptly contradict the user's emotional state.

Your task is to evaluate the system response according to these principles.

History: {h}
User's emotion: {e_u}   Query: {query}
System's emotion: {e_s}   Response: {response}

Evaluate the system response using the following criteria. For each criterion, assign an integer score from 0 to 5 (0 = completely inappropriate, 5 = highly appropriate).

1. Emotion Alignment:
To what extent does the response clearly express the target emotional state?

2. Emotion Transition Plausibility:
Is the emotional transition from the user's emotion to the system's target emotion reasonable according to Plutchik’s emotional structure?

3. Emotion–Function Consistency:
Does the response exhibit the typical behavioral function associated with the target emotion?

Your response needs to follow the following format:

{
alignment: int,
transition: int,
function: int
}

1. Initialize the batch size \(B\).
2. Initialize replay buffer \(\mathcal{B}\), Q-network \(Q_{\theta}(s,a)\), and target Q-network \(\hat{Q}_{\theta^-}(s,a)\) with \(\theta^- \leftarrow \theta\).
3. Load pretrained model \(g\); set exploration rate \(\epsilon\), discount factor \(\gamma\), update interval \(C\), and Plutchik weight \(w\).
4. While not converged:
   1. Draw \(B\) samples \(\{(h_t, x_t^u)\}\) from \(\mathcal{B}\).
   2. For each data sample:
      1. Form state \(s_t = (h_t, s_t^u)\).
      2. Select emotion \(e_t^s\) using \(\epsilon\)-greedy policy.
      3. Obtain response \(x_t^s\) and reward \(r_t\).
      4. Update history \(h_{t+1}\) and state \(s_{t+1}\).
      5. Store transition \((s_t, a_t, r_t, s_{t+1})\) in \(\mathcal{B}\).
   3. Sample batch \(\{(s_i, a_i, r_i, s_i')\}\) from \(\mathcal{B}\).
   4. Compute TD target: \[ y_i = r_i + \gamma \max_{a'} \hat{Q}_{\theta^-}(s_i', a') \]
   5. Update Q-network by minimizing: \[ \mathcal{L}(\theta) = \mathbb{E}_i \bigl[ (Q_\theta(s_i, a_i) - y_i)^2 \bigr]. \]
   6. Every \(C\) steps, update target network \(\theta^- \leftarrow \theta\).