Add streamed notify playback (#2191)

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>

docs/notify.md

@@ -0,0 +1,94 @@
+# Asynchronous Voice Notifications
+
+The `notify` message lets the cloud play a one-way voice notification while a device is idle. It does not open a conversation audio channel and never enables microphone uplink. Starting a conversation still requires an explicit wake action from the user.
+
+## Message format
+
+The cloud sends the following JSON through the device's current protocol control connection. For an
+MQTT device, this is the existing MQTT control topic:
+
+```json
+{
+  "type": "notify",
+  "audio_url": "https://cdn.example.com/audio/task-complete.ogg",
+  "subtitles": [
+    {
+      "start_ms": 0,
+      "text": "Your task is complete."
+    },
+    {
+      "start_ms": 1800,
+      "text": "The result has been saved."
+    }
+  ]
+}
+```
+
+`audio_url` is required. Both `http://` and `https://` URLs are accepted. HTTP is useful for local-network development, while production deployments can enforce HTTPS when generating the URL.
+
+`subtitles` is optional. Each entry contains the media start time in milliseconds and the text to display. The device sorts entries by `start_ms` and updates the display only when playback crosses a new subtitle entry.
+
+The message has no acknowledgement, notification ID, state, kind, or expiry field. Delivery is best effort and only applies to online devices.
+
+## Audio response
+
+`audio_url` must return a successful 2xx response containing a single-stream, mono Ogg Opus file. Responses with either `Content-Length` or chunked transfer encoding are supported. Redirects are not followed by the device.
+
+The file is read incrementally. The device does not allocate memory based on the complete response length and does not download the complete file before playback. Notification streaming uses the existing bounded HTTP response queue, a 2 KB Ogg logical-packet buffer, the shared 20-packet Opus decode queue, and the existing two-frame PCM playback queue. The decode queue represents 1.2 seconds at the firmware's normal 60 ms packet duration. These buffers provide TCP backpressure while keeping the feature usable on devices without PSRAM.
+
+On devices that use the standalone LiteAudioEngine WakeNet, WakeNet resources are released while a notification is playing and recreated when the device returns to `Idle`. AFE-based devices keep their existing local wake behavior.
+
+Opus packet duration is read from the Opus TOC byte instead of being supplied in the MQTT message. Integer packet durations from 5 ms through 120 ms that are supported by the firmware decoder are accepted. The current implementation rejects stereo streams, detected Ogg or Opus structural errors, incomplete streams, oversized logical packets, and 2.5 ms packets.
+
+## Device behavior
+
+The device accepts `notify` only while it is in `Idle`. It then performs the following actions:
+
+1. Enters the internal `Notifying` state and switches the board to performance mode.
+2. Disables normal voice processing and microphone uplink.
+3. Clears previous playback and queues the built-in popup sound.
+4. Starts one HTTP GET in a background task.
+5. Incrementally demultiplexes Ogg packets and sends them directly to the existing Opus decode queue.
+6. Displays subtitles according to the media position of Opus packets reaching the audio output task.
+7. Returns to `Idle` only after the HTTP stream has ended successfully and all queued audio has played.
+
+The popup is queued before the HTTP task starts, so remote audio cannot play before it. HTTP connection setup still overlaps the actual popup playback.
+
+If all playback queues drain after remote audio has started but before the HTTP stream has finished,
+the device logs `Notification playback underrun #<count> at <position> ms`. The popup-to-stream
+transition and normal end of playback are not reported as underruns.
+
+The device does not open the UDP audio channel, send `start-listening`, or automatically enter `Listening`. AFE-based devices may continue local wake-word detection during playback. Devices without playback echo cancellation retain the existing speaking-mode wake behavior; hardware wake controls can still cancel a notification.
+
+A wake action cancels the HTTP producer, clears queued notification audio, and continues through the normal wake flow. Network loss, HTTP errors, and invalid audio also cancel playback and return the device to `Idle`. A second notification received while the device is busy is ignored.
+
+## xz-mqtt forwarding
+
+`xz-mqtt` forwards the message with its existing Redis RPC method:
+
+```json
+{
+  "method": "forward",
+  "clientId": "device-client-id",
+  "params": {
+    "type": "notify",
+    "audio_url": "https://cdn.example.com/audio/task-complete.ogg",
+    "subtitles": [
+      {
+        "start_ms": 0,
+        "text": "Your task is complete."
+      }
+    ]
+  }
+}
+```
+
+The RPC result contains the boolean returned by the MQTT send operation:
+
+```json
+{
+  "success": true
+}
+```
+
+This result means that `xz-mqtt` wrote the publish message to the current online device connection. It does not confirm receipt or playback. `xz-mqtt` does not create a UDP session, start the chat bridge, proxy the Ogg file, or retain notifications for offline devices.

main/CMakeLists.txt

@@ -3,6 +3,7 @@ set(SOURCES "audio/audio_codec.cc"
             "audio/audio_debugger.cc"
             "audio/audio_service.cc"
             "audio/demuxer/ogg_demuxer.cc"
+            "notify/notify_player.cc"
             "audio/codecs/no_audio_codec.cc"
             "audio/codecs/box_audio_codec.cc"
             "audio/codecs/es8311_audio_codec.cc"

main/application.cc

@@ -16,10 +16,11 @@
 #include <arpa/inet.h>
 #include <cJSON.h>
 #include <cstring>
+#include <limits>
 
 #define TAG "Application"
 
-Application::Application() {
+Application::Application() : notify_player_(audio_service_) {
     event_group_ = xEventGroupCreate();
 
 #if CONFIG_USE_DEVICE_AEC && CONFIG_USE_SERVER_AEC
@@ -46,6 +47,7 @@ Application::Application() {
 }
 
 Application::~Application() {
+    notify_player_.Stop();
     if (clock_timer_handle_ != nullptr) {
         esp_timer_stop(clock_timer_handle_);
         esp_timer_delete(clock_timer_handle_);
@@ -83,6 +85,9 @@ void Application::Initialize() {
     callbacks.on_playback_drained = [this]() {
         xEventGroupSetBits(event_group_, MAIN_EVENT_PLAYBACK_DRAINED);
     };
+    callbacks.on_playback_progress = [this](uint32_t playback_id, uint32_t media_position_ms) {
+        notify_player_.OnPlaybackProgress(playback_id, media_position_ms);
+    };
     audio_service_.SetCallbacks(callbacks);
 
     // Add state change listeners
@@ -180,6 +185,9 @@ void Application::Run() {
         auto bits = xEventGroupWaitBits(event_group_, ALL_EVENTS, pdTRUE, pdFALSE, portMAX_DELAY);
 
         if (bits & MAIN_EVENT_ERROR) {
+            if (GetDeviceState() == kDeviceStateNotifying) {
+                StopNotification();
+            }
             SetDeviceState(kDeviceStateIdle);
             Alert(Lang::Strings::ERROR, last_error_message_.c_str(), "cancel",
                   Lang::Sounds::OGG_EXCLAMATION);
@@ -202,6 +210,9 @@ void Application::Run() {
         }
 
         if (bits & MAIN_EVENT_PLAYBACK_DRAINED) {
+            if (audio_service_.IsPlaybackIdle()) {
+                notify_player_.OnPlaybackDrained();
+            }
             // Deferred listening start (auto mode): the playback queue has
             // drained, so it is now safe to enable voice processing.
             if (pending_listening_start_ && GetDeviceState() == kDeviceStateListening &&
@@ -302,6 +313,9 @@ void Application::HandleNetworkConnectedEvent() {
 void Application::HandleNetworkDisconnectedEvent() {
     // Close current conversation when network disconnected
     auto state = GetDeviceState();
+    if (state == kDeviceStateNotifying) {
+        StopNotification();
+    }
     if (state == kDeviceStateConnecting || state == kDeviceStateListening ||
         state == kDeviceStateSpeaking) {
         ESP_LOGI(TAG, "Closing audio channel due to network disconnection");
@@ -558,7 +572,40 @@ void Application::InitializeProtocol() {
             ESP_LOGW(TAG, "Incoming JSON message has no type");
             return;
         }
-        if (strcmp(type->valuestring, "tts") == 0) {
+        if (strcmp(type->valuestring, "notify") == 0) {
+            auto audio_url = cJSON_GetObjectItem(root, "audio_url");
+            if (!cJSON_IsString(audio_url) || audio_url->valuestring[0] == '\0') {
+                ESP_LOGW(TAG, "Notify message requires audio_url");
+                return;
+            }
+
+            std::vector<NotifySubtitle> subtitles;
+            auto subtitles_json = cJSON_GetObjectItem(root, "subtitles");
+            if (subtitles_json != nullptr && !cJSON_IsArray(subtitles_json)) {
+                ESP_LOGW(TAG, "Notify subtitles must be an array");
+                return;
+            }
+            if (cJSON_IsArray(subtitles_json)) {
+                cJSON* item = nullptr;
+                cJSON_ArrayForEach (item, subtitles_json) {
+                    auto start_ms = cJSON_GetObjectItem(item, "start_ms");
+                    auto text = cJSON_GetObjectItem(item, "text");
+                    if (!cJSON_IsNumber(start_ms) || start_ms->valuedouble < 0 ||
+                        start_ms->valuedouble > std::numeric_limits<uint32_t>::max() ||
+                        !cJSON_IsString(text)) {
+                        ESP_LOGW(TAG, "Ignoring invalid notify subtitle");
+                        continue;
+                    }
+                    subtitles.push_back({.start_ms = static_cast<uint32_t>(start_ms->valuedouble),
+                                         .text = text->valuestring});
+                }
+            }
+
+            Schedule([this, url = std::string(audio_url->valuestring),
+                      subtitles = std::move(subtitles)]() mutable {
+                StartNotification(std::move(url), std::move(subtitles));
+            });
+        } else if (strcmp(type->valuestring, "tts") == 0) {
             auto state = cJSON_GetObjectItem(root, "state");
             if (!cJSON_IsString(state)) {
                 return;
@@ -717,6 +764,11 @@ void Application::StopListening() { xEventGroupSetBits(event_group_, MAIN_EVENT_
 void Application::HandleToggleChatEvent() {
     auto state = GetDeviceState();
 
+    if (state == kDeviceStateNotifying) {
+        StopNotification();
+        state = kDeviceStateIdle;
+    }
+
     if (state == kDeviceStateActivating) {
         SetDeviceState(kDeviceStateIdle);
         return;
@@ -776,6 +828,11 @@ void Application::ContinueOpenAudioChannel(ListeningMode mode) {
 void Application::HandleStartListeningEvent() {
     auto state = GetDeviceState();
 
+    if (state == kDeviceStateNotifying) {
+        StopNotification();
+        state = kDeviceStateIdle;
+    }
+
     if (state == kDeviceStateActivating) {
         SetDeviceState(kDeviceStateIdle);
         return;
@@ -807,7 +864,9 @@ void Application::HandleStartListeningEvent() {
 void Application::HandleStopListeningEvent() {
     auto state = GetDeviceState();
 
-    if (state == kDeviceStateAudioTesting) {
+    if (state == kDeviceStateNotifying) {
+        StopNotification();
+    } else if (state == kDeviceStateAudioTesting) {
         audio_service_.EnableAudioTesting(false);
         SetDeviceState(kDeviceStateWifiConfiguring);
         return;
@@ -830,6 +889,9 @@ void Application::HandleWakeWordDetectedEvent() {
 
     if (state == kDeviceStateIdle) {
         BeginWakeWordInvoke(wake_word);
+    } else if (state == kDeviceStateNotifying) {
+        StopNotification();
+        BeginWakeWordInvoke(wake_word);
     } else if (state == kDeviceStateSpeaking || state == kDeviceStateListening) {
         AbortSpeaking(kAbortReasonWakeWordDetected);
         // Clear send queue to avoid sending residues to server
@@ -969,6 +1031,11 @@ void Application::HandleStateChangedEvent() {
             }
             audio_service_.ResetDecoder();
             break;
+        case kDeviceStateNotifying:
+            display->SetStatus(Lang::Strings::SPEAKING);
+            audio_service_.EnableVoiceProcessing(false);
+            audio_service_.EnableWakeWordDetection(audio_service_.IsAfeWakeWord());
+            break;
         case kDeviceStateWifiConfiguring:
             audio_service_.EnableVoiceProcessing(false);
             audio_service_.EnableWakeWordDetection(false);
@@ -1009,6 +1076,72 @@ void Application::ConfigureWakeWordForListening() {
 #endif
 }
 
+void Application::StartNotification(std::string audio_url, std::vector<NotifySubtitle> subtitles) {
+    if (GetDeviceState() != kDeviceStateIdle || notify_player_.IsBusy()) {
+        ESP_LOGW(TAG, "Ignoring notify message while device is busy");
+        return;
+    }
+
+    auto& board = Board::GetInstance();
+    board.SetPowerSaveLevel(PowerSaveLevel::PERFORMANCE);
+    audio_service_.EnableVoiceProcessing(false);
+    audio_service_.EnableWakeWordDetection(audio_service_.IsAfeWakeWord());
+    audio_service_.ReleaseWakeWordResources();
+    while (audio_service_.PopPacketFromSendQueue()) {
+        // Discard microphone audio left over from a previous conversation.
+    }
+
+    if (!SetDeviceState(kDeviceStateNotifying)) {
+        board.SetPowerSaveLevel(PowerSaveLevel::LOW_POWER);
+        return;
+    }
+
+    audio_service_.ResetDecoder();
+    uint32_t playback_id = ++notification_playback_id_;
+    if (playback_id == 0) {
+        playback_id = ++notification_playback_id_;
+    }
+    audio_service_.PlaySound(Lang::Sounds::OGG_POPUP);
+
+    bool started = notify_player_.Start(
+        std::move(audio_url), std::move(subtitles), playback_id,
+        [this](uint32_t id, const std::string& text) {
+            Schedule([this, id, text]() {
+                if (GetDeviceState() == kDeviceStateNotifying && notification_playback_id_ == id) {
+                    Board::GetInstance().GetDisplay()->SetChatMessage("assistant", text.c_str());
+                }
+            });
+        },
+        [this](uint32_t id, bool success) {
+            Schedule([this, id, success]() { HandleNotificationFinished(id, success); });
+        });
+
+    if (!started) {
+        ESP_LOGE(TAG, "Failed to start notification playback");
+        StopNotification();
+    }
+}
+
+void Application::StopNotification() {
+    notify_player_.Stop();
+    audio_service_.ResetDecoder();
+    auto& board = Board::GetInstance();
+    board.GetDisplay()->SetChatMessage("assistant", "");
+    board.SetPowerSaveLevel(PowerSaveLevel::LOW_POWER);
+    if (GetDeviceState() == kDeviceStateNotifying) {
+        SetDeviceState(kDeviceStateIdle);
+    }
+}
+
+void Application::HandleNotificationFinished(uint32_t playback_id, bool success) {
+    if (GetDeviceState() != kDeviceStateNotifying || notification_playback_id_ != playback_id) {
+        return;
+    }
+    ESP_LOGI(TAG, "Notification playback %lu %s", static_cast<unsigned long>(playback_id),
+             success ? "completed" : "failed");
+    StopNotification();
+}
+
 void Application::Schedule(std::function<void()>&& callback) {
     {
         std::lock_guard<std::mutex> lock(mutex_);
@@ -1036,6 +1169,9 @@ ListeningMode Application::GetDefaultListeningMode() const {
 
 void Application::Reboot() {
     ESP_LOGI(TAG, "Rebooting...");
+    if (GetDeviceState() == kDeviceStateNotifying) {
+        StopNotification();
+    }
     // Disconnect the audio channel
     if (protocol_ && protocol_->IsAudioChannelOpened()) {
         protocol_->CloseAudioChannel();
@@ -1054,6 +1190,10 @@ bool Application::UpgradeFirmware(const std::string& url, const std::string& ver
     std::string upgrade_url = url;
     std::string version_info = version.empty() ? "(Manual upgrade)" : version;
 
+    if (GetDeviceState() == kDeviceStateNotifying) {
+        StopNotification();
+    }
+
     // Close audio channel if it's open
     if (protocol_ && protocol_->IsAudioChannelOpened()) {
         ESP_LOGI(TAG, "Closing audio channel before firmware upgrade");
@@ -1117,6 +1257,13 @@ void Application::WakeWordInvoke(const std::string& wake_word) {
                 BeginWakeWordInvoke(wake_word);
             }
         });
+    } else if (state == kDeviceStateNotifying) {
+        Schedule([this, wake_word]() {
+            if (GetDeviceState() == kDeviceStateNotifying) {
+                StopNotification();
+                BeginWakeWordInvoke(wake_word);
+            }
+        });
     } else if (state == kDeviceStateSpeaking) {
         Schedule([this]() { AbortSpeaking(kAbortReasonNone); });
     } else if (state == kDeviceStateListening) {
@@ -1192,6 +1339,9 @@ void Application::PlaySound(const std::string_view& sound) { audio_service_.Play
 
 void Application::ResetProtocol() {
     Schedule([this]() {
+        if (GetDeviceState() == kDeviceStateNotifying) {
+            StopNotification();
+        }
         // Close audio channel if opened
         if (protocol_ && protocol_->IsAudioChannelOpened()) {
             protocol_->CloseAudioChannel();

main/application.h

@@ -11,12 +11,15 @@
 #include <deque>
 #include <memory>
 #include <functional>
+#include <cstdint>
+#include <vector>
 
 #include "protocol.h"
 #include "ota.h"
 #include "audio_service.h"
 #include "device_state.h"
 #include "device_state_machine.h"
+#include "notify/notify_player.h"
 
 // Main event bits
 #define MAIN_EVENT_SCHEDULE             (1 << 0)
@@ -137,6 +140,8 @@ private:
     AecMode aec_mode_ = kAecOff;
     std::string last_error_message_;
     AudioService audio_service_;
+    NotifyPlayer notify_player_;
+    uint32_t notification_playback_id_ = 0;
     std::unique_ptr<Ota> ota_;
 
     std::function<void(const std::string&)> mcp_broadcast_callback_;
@@ -164,6 +169,9 @@ private:
     void ContinueWakeWordInvoke(const std::string& wake_word);
     void StartListeningAudio();
     void ConfigureWakeWordForListening();
+    void StartNotification(std::string audio_url, std::vector<NotifySubtitle> subtitles);
+    void StopNotification();
+    void HandleNotificationFinished(uint32_t playback_id, bool success);
 
     // Activation task (runs in background)
     void ActivationTask();

main/audio/audio_service.cc

@@ -333,6 +333,10 @@ void AudioService::AudioOutputTask() {
             codec_->EnableOutput(true);
         }
 
+        if (task->playback_id != 0 && callbacks_.on_playback_progress) {
+            callbacks_.on_playback_progress(task->playback_id, task->media_position_ms);
+        }
+
         codec_->OutputData(task->pcm);
 
         /* Update the last output time */
@@ -384,6 +388,8 @@ void AudioService::OpusCodecTask() {
             auto task = std::make_unique<AudioTask>();
             task->type = kAudioTaskTypeDecodeToPlaybackQueue;
             task->timestamp = packet->timestamp;
+            task->playback_id = packet->playback_id;
+            task->media_position_ms = packet->media_position_ms;
 
             SetDecodeSampleRate(packet->sample_rate, packet->frame_duration);
             bool decoded = false;
@@ -579,17 +585,19 @@ void AudioService::PushTaskToEncodeQueue(AudioTaskType type, std::vector<int16_t
 
 bool AudioService::PushPacketToDecodeQueue(std::unique_ptr<AudioStreamPacket> packet, bool wait) {
     std::unique_lock<std::mutex> lock(audio_queue_mutex_);
+    const uint32_t generation = playback_generation_;
     if (audio_decode_queue_.size() >= MAX_DECODE_PACKETS_IN_QUEUE) {
         if (wait) {
-            audio_queue_cv_.wait(lock, [this]() {
+            audio_queue_cv_.wait(lock, [this, generation]() {
                 return service_stopped_.load() ||
-                    audio_decode_queue_.size() < MAX_DECODE_PACKETS_IN_QUEUE;
+                       generation != playback_generation_ ||
+                       audio_decode_queue_.size() < MAX_DECODE_PACKETS_IN_QUEUE;
             });
         } else {
             return false;
         }
     }
-    if (service_stopped_.load()) {
+    if (service_stopped_.load() || generation != playback_generation_) {
         return false;
     }
     playback_drained_notified_ = false;
@@ -631,7 +639,18 @@ std::unique_ptr<AudioStreamPacket> AudioService::PopWakeWordPacket() {
 void AudioService::EnableWakeWordDetection(bool enable) {
     ESP_LOGD(TAG, "%s wake word detection", enable ? "Enabling" : "Disabling");
     if (enable) {
-        if (!InitializeAudioEngine() || !audio_engine_->HasWakeWord()) {
+        if (!InitializeAudioEngine()) {
+            xEventGroupClearBits(event_group_, AS_EVENT_WAKE_WORD_RUNNING);
+            return;
+        }
+#if !(CONFIG_IDF_TARGET_ESP32S3 || CONFIG_IDF_TARGET_ESP32P4 || CONFIG_IDF_TARGET_ESP32S31)
+        auto* lite_engine = static_cast<LiteAudioEngine*>(audio_engine_.get());
+        if (!lite_engine->RestoreWakeWordResources()) {
+            xEventGroupClearBits(event_group_, AS_EVENT_WAKE_WORD_RUNNING);
+            return;
+        }
+#endif
+        if (!audio_engine_->HasWakeWord()) {
             xEventGroupClearBits(event_group_, AS_EVENT_WAKE_WORD_RUNNING);
             return;
         }
@@ -651,6 +670,20 @@ void AudioService::EnableWakeWordDetection(bool enable) {
     }
 }
 
+void AudioService::ReleaseWakeWordResources() {
+#if !(CONFIG_IDF_TARGET_ESP32S3 || CONFIG_IDF_TARGET_ESP32P4 || CONFIG_IDF_TARGET_ESP32S31)
+    if (!audio_engine_initialized_) {
+        return;
+    }
+    if (xEventGroupGetBits(event_group_) &
+        (AS_EVENT_WAKE_WORD_RUNNING | AS_EVENT_AUDIO_PROCESSOR_RUNNING)) {
+        ESP_LOGW(TAG, "Cannot release WakeNet while the audio engine is active");
+        return;
+    }
+    static_cast<LiteAudioEngine*>(audio_engine_.get())->ReleaseWakeWordResources();
+#endif
+}
+
 void AudioService::EnableVoiceProcessing(bool enable) {
     ESP_LOGD(TAG, "%s voice processing", enable ? "Enabling" : "Disabling");
 
@@ -718,10 +751,10 @@ void AudioService::PlaySound(const std::string_view& ogg) {
     size_t size = ogg.size();
 
     auto demuxer = std::make_unique<OggDemuxer>();
-    demuxer->OnDemuxerFinished([this](const uint8_t* data, int sample_rate, size_t size){
+    demuxer->OnPacket([this](const uint8_t* data, int sample_rate, int frame_duration_ms, size_t size){
         auto packet = std::make_unique<AudioStreamPacket>();
         packet->sample_rate = sample_rate;
-        packet->frame_duration = 60;
+        packet->frame_duration = frame_duration_ms;
         packet->payload.resize(size);
         std::memcpy(packet->payload.data(), data, size);
         PushPacketToDecodeQueue(std::move(packet), true);

main/audio/audio_service.h

@@ -39,7 +39,7 @@
 #define OPUS_FRAME_DURATION_MS 60
 #define MAX_ENCODE_TASKS_IN_QUEUE 2
 #define MAX_PLAYBACK_TASKS_IN_QUEUE 2
-#define MAX_DECODE_PACKETS_IN_QUEUE (2400 / OPUS_FRAME_DURATION_MS)
+#define MAX_DECODE_PACKETS_IN_QUEUE (1200 / OPUS_FRAME_DURATION_MS)
 #define MAX_SEND_PACKETS_IN_QUEUE (2400 / OPUS_FRAME_DURATION_MS)
 #define AUDIO_TESTING_MAX_DURATION_MS 10000
 #define MAX_TIMESTAMPS_IN_QUEUE 3
@@ -82,6 +82,7 @@ struct AudioServiceCallbacks {
     std::function<void(void)> on_audio_testing_queue_full;
     // Fired when the decode/playback queues and their in-flight work are drained.
     std::function<void(void)> on_playback_drained;
+    std::function<void(uint32_t playback_id, uint32_t media_position_ms)> on_playback_progress;
 };
 
 
@@ -95,6 +96,8 @@ struct AudioTask {
     AudioTaskType type;
     std::vector<int16_t> pcm;
     uint32_t timestamp = 0;
+    uint32_t playback_id = 0;
+    uint32_t media_position_ms = 0;
 };
 
 struct DebugStatistics {
@@ -124,6 +127,7 @@ public:
     bool IsAfeWakeWord();
 
     void EnableWakeWordDetection(bool enable);
+    void ReleaseWakeWordResources();
     void EnableVoiceProcessing(bool enable);
     void EnableAudioTesting(bool enable);
     void EnableDeviceAec(bool enable);

main/audio/demuxer/ogg_demuxer.cc

@@ -1,48 +1,103 @@
 #include "ogg_demuxer.h"
+
+#include <algorithm>
+
 #include "esp_log.h"
 
 #define TAG "OggDemuxer"
 
-/// @brief 重置解封器
+/// @brief Reset the demuxer.
 void OggDemuxer::Reset()
 {
     opus_info_ = {
         .head_seen = false,
         .tags_seen = false,
+        .mono = false,
         .sample_rate = 48000
     };
 
+    has_error_ = false;
+    packet_count_ = 0;
+
     state_ = ParseState::FIND_PAGE;
     ctx_.packet_len = 0;
     ctx_.seg_count = 0;
     ctx_.seg_index = 0;
     ctx_.data_offset = 0;
-    ctx_.bytes_needed = 4;          // 需要4字节"OggS"
+    ctx_.bytes_needed = 4;          // Four bytes are needed for "OggS"
     ctx_.seg_remaining = 0;
     ctx_.body_size = 0;
     ctx_.body_offset = 0;
     ctx_.packet_continued = false;
     
-    // 清空缓冲区数据
+    // Clear buffered data.
     memset(ctx_.header, 0, sizeof(ctx_.header));
     memset(ctx_.seg_table, 0, sizeof(ctx_.seg_table));
     memset(ctx_.packet_buf, 0, sizeof(ctx_.packet_buf));
 }
 
-/// @brief 处理数据块
-/// @param data 输入数据
-/// @param size 输入数据大小
-/// @return 已处理的字节数
+int OggDemuxer::GetOpusPacketDurationMs(const uint8_t* data, size_t size) {
+    if (data == nullptr || size == 0) {
+        return -1;
+    }
+
+    const uint8_t toc = data[0];
+    const uint8_t config = toc >> 3;
+    int frame_duration_us = 0;
+    if (config < 12) {
+        static constexpr int kSilkDurationsUs[] = {10000, 20000, 40000, 60000};
+        frame_duration_us = kSilkDurationsUs[config & 0x03];
+    } else if (config < 16) {
+        frame_duration_us = (config & 0x01) ? 20000 : 10000;
+    } else {
+        static constexpr int kCeltDurationsUs[] = {2500, 5000, 10000, 20000};
+        frame_duration_us = kCeltDurationsUs[config & 0x03];
+    }
+
+    int frame_count = 0;
+    switch (toc & 0x03) {
+        case 0:
+            frame_count = 1;
+            break;
+        case 1:
+        case 2:
+            frame_count = 2;
+            break;
+        case 3:
+            if (size < 2) {
+                return -1;
+            }
+            frame_count = data[1] & 0x3f;
+            break;
+    }
+
+    const int packet_duration_us = frame_duration_us * frame_count;
+    if (frame_count == 0 || packet_duration_us > 120000 || packet_duration_us % 1000 != 0) {
+        return -1;
+    }
+    return packet_duration_us / 1000;
+}
+
+bool OggDemuxer::Finish() const {
+    return !has_error_ && opus_info_.head_seen && opus_info_.tags_seen && opus_info_.mono &&
+           packet_count_ > 0 && state_ == ParseState::FIND_PAGE && ctx_.bytes_needed == 4 &&
+           ctx_.packet_len == 0;
+}
+
+/// @brief Process an input block.
+/// @param data Input data.
+/// @param size Input size in bytes.
+/// @return Number of bytes processed.
 size_t OggDemuxer::Process(const uint8_t* data, size_t size)
 {
-    size_t processed = 0;  // 已处理的字节数
+    size_t processed = 0;  // Number of bytes processed
     
     while (processed < size) {
         switch (state_) {
           case ParseState::FIND_PAGE: {
-            // 寻找页头"OggS"
+            // Find the "OggS" page capture pattern.
             if (ctx_.bytes_needed < 4) {
-                // 处理不完整的"OggS"匹配（跨数据块）
+                // Continue a partial "OggS" match across input blocks.
                 size_t to_copy = std::min(size - processed, ctx_.bytes_needed);
                 memcpy(ctx_.header + (4 - ctx_.bytes_needed), data + processed, to_copy);
                 
@@ -50,27 +105,27 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 ctx_.bytes_needed -= to_copy;
                 
                 if (ctx_.bytes_needed == 0) {
-                    // 检查是否匹配"OggS"
+                    // Check whether the capture pattern matches "OggS".
                     if (memcmp(ctx_.header, "OggS", 4) == 0) {
                         state_ = ParseState::PARSE_HEADER;
                         ctx_.data_offset = 4;
-                        ctx_.bytes_needed = 27 - 4;  // 还需要23字节完成页头
+                        ctx_.bytes_needed = 27 - 4;  // 23 more bytes complete the header
                     } else {
-                        // 匹配失败，滑动1字节继续匹配
+                        // Shift by one byte and continue matching.
                         memmove(ctx_.header, ctx_.header + 1, 3);
                         ctx_.bytes_needed = 1;
                     }
                 } else {
-                    // 数据不足，等待更多数据
+                    // Wait for more data.
                     return processed;
                 }
             } else if (ctx_.bytes_needed == 4) {
-                // 在数据块中查找完整的"OggS"
+                // Search the input block for a complete "OggS" pattern.
                 bool found = false;
                 size_t i = 0;
                 size_t remaining = size - processed;
                 
-                // 搜索"OggS"
+                // Search for "OggS".
                 for (; i + 4 <= remaining; i++) {
                     if (memcmp(data + processed + i, "OggS", 4) == 0) {
                         found = true;
@@ -79,31 +134,31 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 }
                 
                 if (found) {
-                    // 找到"OggS"，跳过已搜索的字节
+                    // Skip bytes before the matched "OggS" pattern.
                     processed += i;
                     
-                    // 不记录找到的"OggS"，无必要
+                    // The matched "OggS" bytes do not need to be copied.
                     // memcpy(ctx_.header, data + processed, 4);
                     processed += 4;
                     
                     state_ = ParseState::PARSE_HEADER;
                     ctx_.data_offset = 4;
-                    ctx_.bytes_needed = 27 - 4;  // 还需要23字节
+                    ctx_.bytes_needed = 27 - 4;  // 23 more bytes are needed
                 } else {
-                    // 没有找到完整"OggS"，保存可能的部分匹配
+                    // Save a possible partial match when no complete pattern is found.
                     size_t partial_len = remaining - i;
                     if (partial_len > 0) {
                         memcpy(ctx_.header, data + processed + i, partial_len);
                         ctx_.bytes_needed = 4 - partial_len;
                         processed += i + partial_len;
                     } else {
-                        processed += i;  // 已搜索所有字节
+                        processed += i;  // All bytes have been searched
                     }
-                    return processed;  // 返回已处理的字节数
+                    return processed;
                 }
             } else {
                 ESP_LOGE(TAG, "OggDemuxer run in error state: bytes_needed=%zu", ctx_.bytes_needed);
-                Reset();
+                has_error_ = true;
                 return processed;
             }
             break;
@@ -113,16 +168,16 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
             size_t available = size - processed;
             
             if (available < ctx_.bytes_needed) {
-                // 数据不足，复制可用的部分
+                // Copy the available bytes and wait for more data.
                 memcpy(ctx_.header + ctx_.data_offset, 
                         data + processed, available);
                 
                 ctx_.data_offset += available;
                 ctx_.bytes_needed -= available;
                 processed += available;
-                return processed;  // 等待更多数据
+                return processed;
             } else {
-                // 有足够的数据完成页头
+                // Complete the page header.
                 size_t to_copy = ctx_.bytes_needed;
                 memcpy(ctx_.header + ctx_.data_offset, 
                         data + processed, to_copy);
@@ -131,13 +186,11 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 ctx_.data_offset += to_copy;
                 ctx_.bytes_needed = 0;
                 
-                // 验证页头
+                // Validate the page header.
                 if (ctx_.header[4] != 0) {
-                    ESP_LOGE(TAG, "无效的Ogg版本: %d", ctx_.header[4]);
-                    state_ = ParseState::FIND_PAGE;
-                    ctx_.bytes_needed = 4;
-                    ctx_.data_offset = 0;
-                    break;
+                    ESP_LOGE(TAG, "Invalid Ogg version: %d", ctx_.header[4]);
+                    has_error_ = true;
+                    return processed;
                 }
                 
                 ctx_.seg_count = ctx_.header[26];
@@ -146,15 +199,14 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                     ctx_.bytes_needed = ctx_.seg_count;
                     ctx_.data_offset = 0;
                 } else if (ctx_.seg_count == 0) {
-                    // 没有段，直接跳到下一个页面
+                    // Skip directly to the next page when there are no segments.
                     state_ = ParseState::FIND_PAGE;
                     ctx_.bytes_needed = 4;
                     ctx_.data_offset = 0;
                 } else {
-                    ESP_LOGE(TAG, "无效的段数: %u", ctx_.seg_count);
-                    state_ = ParseState::FIND_PAGE;
-                    ctx_.bytes_needed = 4;
-                    ctx_.data_offset = 0;
+                    ESP_LOGE(TAG, "Invalid Ogg segment count: %u", ctx_.seg_count);
+                    has_error_ = true;
+                    return processed;
                 }
             }
             break;
@@ -170,7 +222,7 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 ctx_.data_offset += available;
                 ctx_.bytes_needed -= available;
                 processed += available;
-                return processed;  // 等待更多数据
+                return processed;
             } else {
                 size_t to_copy = ctx_.bytes_needed;
                 memcpy(ctx_.seg_table + ctx_.data_offset, 
@@ -184,7 +236,7 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 ctx_.seg_index = 0;
                 ctx_.data_offset = 0;
                 
-                // 计算数据体总大小
+                // Calculate the total page body size.
                 ctx_.body_size = 0;
                 for (size_t i = 0; i < ctx_.seg_count; ++i) {
                     ctx_.body_size += ctx_.seg_table[i];
@@ -199,25 +251,22 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
             while (ctx_.seg_index < ctx_.seg_count && processed < size) {
                 uint8_t seg_len = ctx_.seg_table[ctx_.seg_index];
                 
-                // 检查段数据是否已经部分读取
+                // Continue a partially read segment.
                 if (ctx_.seg_remaining > 0) {
                     seg_len = ctx_.seg_remaining;
                 } else {
                     ctx_.seg_remaining = seg_len;
                 }
                 
-                // 检查缓冲区是否足够
+                // Check that the packet buffer has enough space.
                 if (ctx_.packet_len + seg_len > sizeof(ctx_.packet_buf)) {
-                    ESP_LOGE(TAG, "包缓冲区溢出: %zu + %u > %zu", ctx_.packet_len, seg_len, sizeof(ctx_.packet_buf));
-                    state_ = ParseState::FIND_PAGE;
-                    ctx_.packet_len = 0;
-                    ctx_.packet_continued = false;
-                    ctx_.seg_remaining = 0;
-                    ctx_.bytes_needed = 4;
+                    ESP_LOGE(TAG, "Ogg packet buffer overflow: %zu + %u > %zu",
+                             ctx_.packet_len, seg_len, sizeof(ctx_.packet_buf));
+                    has_error_ = true;
                     return processed;
                 }
                 
-                // 复制数据
+                // Copy segment data.
                 size_t to_copy = std::min(size - processed, (size_t)seg_len);
                 memcpy(ctx_.packet_buf + ctx_.packet_len, data + processed, to_copy);
                 
@@ -226,27 +275,51 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                 ctx_.body_offset += to_copy;
                 ctx_.seg_remaining -= to_copy;
                 
-                // 检查段是否完整
+                // Check whether the segment is complete.
                 if (ctx_.seg_remaining > 0) {
-                    // 段不完整，等待更多数据
+                    // Wait for the rest of the segment.
                     return processed;
                 }
                 
-                // 段完整
+                // The segment is complete.
                 bool seg_continued = (ctx_.seg_table[ctx_.seg_index] == 255);
                 
                 if (!seg_continued) {
-                    // 包结束
+                    // The packet ends at this segment.
                     if (ctx_.packet_len) {
                         if (!opus_info_.head_seen) {
                             if (ctx_.packet_len >=8 && memcmp(ctx_.packet_buf, "OpusHead", 8) == 0) {
                                 opus_info_.head_seen = true;
                                 if (ctx_.packet_len >= 19) {
-                                    opus_info_.sample_rate = ctx_.packet_buf[12] | 
-                                                            (ctx_.packet_buf[13] << 8) | 
-                                                            (ctx_.packet_buf[14] << 16) | 
-                                                            (ctx_.packet_buf[15] << 24);
+                                    opus_info_.mono = ctx_.packet_buf[9] == 1;
+                                    const uint32_t input_sample_rate =
+                                        static_cast<uint32_t>(ctx_.packet_buf[12]) |
+                                        (static_cast<uint32_t>(ctx_.packet_buf[13]) << 8) |
+                                        (static_cast<uint32_t>(ctx_.packet_buf[14]) << 16) |
+                                        (static_cast<uint32_t>(ctx_.packet_buf[15]) << 24);
+                                    switch (input_sample_rate) {
+                                        case 8000:
+                                        case 12000:
+                                        case 16000:
+                                        case 24000:
+                                        case 48000:
+                                            opus_info_.sample_rate = input_sample_rate;
+                                            break;
+                                        default:
+                                            // The OpusHead input rate is informational. Decode at a
+                                            // native Opus rate when it is not directly supported.
+                                            opus_info_.sample_rate = 48000;
+                                            break;
+                                    }
                                     ESP_LOGD(TAG, "OpusHead found, sample_rate=%d", opus_info_.sample_rate);
+                                    if (!opus_info_.mono) {
+                                        ESP_LOGE(TAG, "Only mono Ogg Opus streams are supported");
+                                        has_error_ = true;
+                                        return processed;
+                                    }
+                                } else {
+                                    has_error_ = true;
+                                    return processed;
                                 }
                                 ctx_.packet_len = 0;
                                 ctx_.packet_continued = false;
@@ -267,11 +340,20 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
                             }
                         }
                         if (opus_info_.head_seen && opus_info_.tags_seen) {
-                            if (on_demuxer_finished_) {
-                                on_demuxer_finished_(ctx_.packet_buf, opus_info_.sample_rate, ctx_.packet_len);
+                            const int frame_duration_ms =
+                                GetOpusPacketDurationMs(ctx_.packet_buf, ctx_.packet_len);
+                            if (frame_duration_ms <= 0) {
+                                ESP_LOGE(TAG, "Unsupported Opus packet duration");
+                                has_error_ = true;
+                                return processed;
+                            }
+                            ++packet_count_;
+                            if (on_packet_) {
+                                on_packet_(ctx_.packet_buf, opus_info_.sample_rate,
+                                           frame_duration_ms, ctx_.packet_len);
                             }
                         } else {
-                            ESP_LOGW(TAG, "当前Ogg容器未解析到OpusHead/OpusTags，丢弃");
+                            ESP_LOGW(TAG, "Dropping Ogg packet before OpusHead/OpusTags");
                         }
                     }
                     ctx_.packet_len = 0;
@@ -285,18 +367,18 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
             }
             
             if (ctx_.seg_index == ctx_.seg_count) {
-                // 检查是否所有数据体都已读取
+                // Check whether the complete page body was read.
                 if (ctx_.body_offset < ctx_.body_size) {
-                    ESP_LOGW(TAG, "数据体不完整: %zu/%zu", 
+                    ESP_LOGW(TAG, "Incomplete Ogg page body: %zu/%zu",
                             ctx_.body_offset, ctx_.body_size);
                 }
                 
-                // 如果包跨页，保持packet_len和packet_continued
+                // Preserve packet state when a packet continues on the next page.
                 if (!ctx_.packet_continued) {
                     ctx_.packet_len = 0;
                 }
                 
-                // 进入下一页面
+                // Continue with the next page.
                 state_ = ParseState::FIND_PAGE;
                 ctx_.bytes_needed = 4;
                 ctx_.data_offset = 0;
@@ -308,4 +390,3 @@ size_t OggDemuxer::Process(const uint8_t* data, size_t size)
     
     return processed;
 }
-

main/audio/demuxer/ogg_demuxer.h

@@ -4,6 +4,7 @@
 #include <functional>
 #include <cstdint>
 #include <cstring>
+#include <utility>
 #include <vector>
 
 class OggDemuxer {
@@ -18,24 +19,25 @@ private:
     struct Opus_t {
         bool    head_seen{false};
         bool    tags_seen{false};
+        bool    mono{false};
         int     sample_rate{48000};
     };
 
 
-    // 使用固定大小的缓冲区避免动态分配
+    // Use fixed-size buffers to avoid dynamic allocation.
     struct context_t {
-        bool packet_continued{false};   // 当前包是否跨多个段
-        uint8_t header[27];             // Ogg页头
-        uint8_t seg_table[255];         // 当前存储的段表
-        uint8_t packet_buf[8192];       // 8KB包缓冲区
-        size_t packet_len = 0;          // 缓冲区中累计的数据长度
-        size_t seg_count = 0;           // 当前页段数
-        size_t seg_index = 0;           // 当前处理的段索引
-        size_t data_offset = 0;         // 解析当前阶段已读取的字节数
-        size_t bytes_needed = 0;        // 解析当前字段还需要读取的字节数
-        size_t seg_remaining = 0;       // 当前段剩余需要读取的字节数
-        size_t body_size = 0;           // 数据体总大小
-        size_t body_offset = 0;         // 数据体已读取的字节数
+        bool packet_continued{false};   // Whether the current packet spans segments
+        uint8_t header[27];             // Ogg page header
+        uint8_t seg_table[255];         // Current segment table
+        uint8_t packet_buf[2048];       // 2 KB packet buffer
+        size_t packet_len = 0;          // Bytes accumulated in the packet buffer
+        size_t seg_count = 0;           // Segment count in the current page
+        size_t seg_index = 0;           // Current segment index
+        size_t data_offset = 0;         // Bytes read in the current parsing stage
+        size_t bytes_needed = 0;        // Bytes still needed for the current field
+        size_t seg_remaining = 0;       // Bytes remaining in the current segment
+        size_t body_size = 0;           // Total page body size
+        size_t body_offset = 0;         // Bytes read from the page body
     };
     
 public:
@@ -47,17 +49,23 @@ public:
     
     size_t Process(const uint8_t* data, size_t size);
 
-    /// @brief 设置解封装完毕后回调处理函数
-    /// @param on_demuxer_finished 
-    void OnDemuxerFinished(std::function<void(const uint8_t* data, int sample_rate, size_t len)> on_demuxer_finished) {
-        on_demuxer_finished_ = on_demuxer_finished;
+    bool Finish() const;
+    bool HasError() const { return has_error_; }
+
+    void OnPacket(std::function<void(const uint8_t* data, int sample_rate, int frame_duration_ms,
+                                     size_t len)> on_packet) {
+        on_packet_ = std::move(on_packet);
     }
 private:
 
     ParseState  state_ = ParseState::FIND_PAGE;
     context_t   ctx_;
     Opus_t      opus_info_;
-    std::function<void(const uint8_t*, int, size_t)> on_demuxer_finished_;
+    bool has_error_ = false;
+    size_t packet_count_ = 0;
+    std::function<void(const uint8_t*, int, int, size_t)> on_packet_;
+
+    static int GetOpusPacketDurationMs(const uint8_t* data, size_t size);
 };
 
-#endif
+#endif

main/audio/engines/lite_audio_engine.cc

@@ -12,25 +12,18 @@ LiteAudioEngine::~LiteAudioEngine() = default;
 
 bool LiteAudioEngine::Initialize(AudioCodec* codec, int frame_duration_ms, srmodel_list_t* models_list) {
     codec_ = codec;
+    models_list_ = models_list;
     frame_samples_ = frame_duration_ms * 16000 / 1000;
     output_buffer_.reserve(frame_samples_);
 
-    bool has_wakenet = models_list != nullptr &&
+    should_have_wake_word_ = models_list != nullptr &&
         esp_srmodel_filter(models_list, ESP_WN_PREFIX, nullptr) != nullptr;
 #if CONFIG_USE_ESP_WAKE_WORD
-    has_wakenet = has_wakenet || models_list == nullptr;
+    should_have_wake_word_ = should_have_wake_word_ || models_list == nullptr;
 #endif
-    if (has_wakenet) {
-        wake_word_ = std::make_unique<EspWakeWord>();
-        wake_word_->OnWakeWordDetected([this](const std::string& wake_word) {
-            wake_word_enabled_ = false;
-            if (wake_word_detected_callback_) {
-                wake_word_detected_callback_(wake_word);
-            }
-        });
-        if (!wake_word_->Initialize(codec_, models_list)) {
-            ESP_LOGE(TAG, "Failed to initialize standalone WakeNet");
-            wake_word_.reset();
+    {
+        std::lock_guard<std::mutex> lock(wake_word_mutex_);
+        if (!CreateWakeWordLocked()) {
             return false;
         }
     }
@@ -40,8 +33,11 @@ bool LiteAudioEngine::Initialize(AudioCodec* codec, int frame_duration_ms, srmod
 }
 
 void LiteAudioEngine::Feed(std::vector<int16_t>&& data) {
-    if (wake_word_enabled_ && wake_word_) {
-        wake_word_->Feed(data);
+    {
+        std::lock_guard<std::mutex> lock(wake_word_mutex_);
+        if (wake_word_enabled_ && wake_word_) {
+            wake_word_->Feed(data);
+        }
     }
     if (voice_processing_enabled_) {
         OutputRawAudio(data);
@@ -49,6 +45,7 @@ void LiteAudioEngine::Feed(std::vector<int16_t>&& data) {
 }
 
 void LiteAudioEngine::EnableWakeWordDetection(bool enable) {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
     if (!wake_word_) {
         wake_word_enabled_ = false;
         return;
@@ -77,6 +74,7 @@ void LiteAudioEngine::EnableDeviceAec(bool enable) {
 }
 
 bool LiteAudioEngine::HasWakeWord() const {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
     return wake_word_ != nullptr;
 }
 
@@ -89,6 +87,7 @@ bool LiteAudioEngine::IsVoiceProcessingEnabled() const {
 }
 
 size_t LiteAudioEngine::GetFeedSize() const {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
     if (wake_word_) {
         return wake_word_->GetFeedSize();
     }
@@ -108,12 +107,14 @@ void LiteAudioEngine::OnVadStateChange(std::function<void(bool speaking)> callba
 }
 
 void LiteAudioEngine::EncodeWakeWordData() {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
     if (wake_word_) {
         wake_word_->EncodeWakeWordData();
     }
 }
 
 bool LiteAudioEngine::GetWakeWordOpus(std::vector<uint8_t>& opus) {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
     return wake_word_ && wake_word_->GetWakeWordOpus(opus);
 }
 
@@ -121,6 +122,44 @@ const std::string& LiteAudioEngine::GetLastDetectedWakeWord() const {
     return wake_word_ ? wake_word_->GetLastDetectedWakeWord() : empty_wake_word_;
 }
 
+void LiteAudioEngine::ReleaseWakeWordResources() {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
+    wake_word_enabled_ = false;
+    if (wake_word_) {
+        wake_word_->Stop();
+        wake_word_.reset();
+        ESP_LOGI(TAG, "Released standalone WakeNet resources");
+    }
+}
+
+bool LiteAudioEngine::RestoreWakeWordResources() {
+    std::lock_guard<std::mutex> lock(wake_word_mutex_);
+    return CreateWakeWordLocked();
+}
+
+bool LiteAudioEngine::CreateWakeWordLocked() {
+    if (!should_have_wake_word_) {
+        return true;
+    }
+    if (wake_word_) {
+        return true;
+    }
+
+    auto wake_word = std::make_unique<EspWakeWord>();
+    wake_word->OnWakeWordDetected([this](const std::string& detected_wake_word) {
+        wake_word_enabled_ = false;
+        if (wake_word_detected_callback_) {
+            wake_word_detected_callback_(detected_wake_word);
+        }
+    });
+    if (!wake_word->Initialize(codec_, models_list_)) {
+        ESP_LOGE(TAG, "Failed to initialize standalone WakeNet");
+        return false;
+    }
+    wake_word_ = std::move(wake_word);
+    return true;
+}
+
 void LiteAudioEngine::OutputRawAudio(const std::vector<int16_t>& data) {
     if (!output_callback_ || codec_ == nullptr) {
         return;

main/audio/engines/lite_audio_engine.h

@@ -30,6 +30,11 @@ public:
     bool IsAfeWakeWord() const override { return false; }
     size_t GetFeedSize() const override;
 
+    // Release the standalone WakeNet allocation while it cannot be used, then
+    // recreate it before wake word detection is enabled again.
+    void ReleaseWakeWordResources();
+    bool RestoreWakeWordResources();
+
     void OnWakeWordDetected(std::function<void(const std::string& wake_word)> callback) override;
     void OnOutput(std::function<void(std::vector<int16_t>&& data)> callback) override;
     void OnVadStateChange(std::function<void(bool speaking)> callback) override;
@@ -41,17 +46,21 @@ public:
 private:
     AudioCodec* codec_ = nullptr;
     std::unique_ptr<EspWakeWord> wake_word_;
+    srmodel_list_t* models_list_ = nullptr;
+    bool should_have_wake_word_ = false;
     std::atomic<bool> wake_word_enabled_ = false;
     std::atomic<bool> voice_processing_enabled_ = false;
     int frame_samples_ = 0;
     std::vector<int16_t> output_buffer_;
     std::mutex output_mutex_;
+    mutable std::mutex wake_word_mutex_;
 
     std::function<void(const std::string&)> wake_word_detected_callback_;
     std::function<void(std::vector<int16_t>&&)> output_callback_;
     std::function<void(bool)> vad_state_change_callback_;
     std::string empty_wake_word_;
 
+    bool CreateWakeWordLocked();
     void OutputRawAudio(const std::vector<int16_t>& data);
 };
 

main/boards/common/wifi_board.cc

@@ -199,7 +199,8 @@ void WifiBoard::EnterWifiConfigMode() {
     auto& app = Application::GetInstance();
     auto state = app.GetDeviceState();
 
-    if (state == kDeviceStateSpeaking || state == kDeviceStateListening || state == kDeviceStateIdle) {
+    if (state == kDeviceStateSpeaking || state == kDeviceStateNotifying ||
+        state == kDeviceStateListening || state == kDeviceStateIdle) {
         // Reset protocol (close audio channel, reset protocol)
         Application::GetInstance().ResetProtocol();
 

main/boards/jiuchuan-s3/jiuchuan_dev_board.cc

@@ -231,7 +231,8 @@ private:
                 // 如果当前是聆听状态，切换到待命状态
                 ESP_LOGI(TAG, "从聆听状态切换到待命状态");
                 app.ToggleChatState(); // 切换到待命状态
-            } else if (current_state == kDeviceStateSpeaking) {
+            } else if (current_state == kDeviceStateSpeaking ||
+                       current_state == kDeviceStateNotifying) {
                 // 如果当前是说话状态，终止说话并切换到待命状态
                 ESP_LOGI(TAG, "从说话状态切换到待命状态");
                 app.ToggleChatState(); // 终止说话

main/boards/m5stack/atoms3r-echo-pyramid/atoms3r_echo_pyramid.cc

@@ -217,6 +217,7 @@ public:
                 ctrl_->SetStatusColor(0, 64, 0);   // green
                 break;
             case kDeviceStateSpeaking:
+            case kDeviceStateNotifying:
                 ctrl_->SetStatusColor(64, 0, 0);   // red
                 break;
             default:

main/device_state.h

@@ -9,6 +9,7 @@ enum DeviceState {
     kDeviceStateConnecting,
     kDeviceStateListening,
     kDeviceStateSpeaking,
+    kDeviceStateNotifying,
     kDeviceStateUpgrading,
     kDeviceStateActivating,
     kDeviceStateAudioTesting,

main/device_state_machine.cc

@@ -14,6 +14,7 @@ static const char* const STATE_STRINGS[] = {
     "connecting",
     "listening",
     "speaking",
+    "notifying",
     "upgrading",
     "activating",
     "audio_testing",
@@ -73,6 +74,7 @@ bool DeviceStateMachine::IsValidTransition(DeviceState from, DeviceState to) con
             return to == kDeviceStateConnecting ||
                    to == kDeviceStateListening ||
                    to == kDeviceStateSpeaking ||
+                   to == kDeviceStateNotifying ||
                    to == kDeviceStateActivating ||
                    to == kDeviceStateUpgrading ||
                    to == kDeviceStateWifiConfiguring;
@@ -92,6 +94,9 @@ bool DeviceStateMachine::IsValidTransition(DeviceState from, DeviceState to) con
             return to == kDeviceStateListening ||
                    to == kDeviceStateIdle;
 
+        case kDeviceStateNotifying:
+            return to == kDeviceStateIdle;
+
         case kDeviceStateFatalError:
             // Cannot transition out of fatal error
             return false;

main/led/circular_strip.cc

@@ -223,7 +223,8 @@ void CircularStrip::OnStateChanged() {
             SetAllColor(color);
             break;
         }
-        case kDeviceStateSpeaking: {
+        case kDeviceStateSpeaking:
+        case kDeviceStateNotifying: {
             StripColor color = { low_brightness_, default_brightness_, low_brightness_ };
             SetAllColor(color);
             break;

main/led/gpio_led.cc

@@ -236,6 +236,7 @@ void GpioLed::OnStateChanged() {
             StartFadeTask();
             break;
         case kDeviceStateSpeaking:
+        case kDeviceStateNotifying:
             SetBrightness(SPEAKING_BRIGHTNESS);
             TurnOn();
             break;

main/led/single_led.cc

@@ -149,6 +149,7 @@ void SingleLed::OnStateChanged() {
             TurnOn();
             break;
         case kDeviceStateSpeaking:
+        case kDeviceStateNotifying:
             SetColor(0, DEFAULT_BRIGHTNESS, 0);
             TurnOn();
             break;

main/notify/notify_player.cc

@@ -0,0 +1,284 @@
+#include "notify_player.h"
+
+#include <algorithm>
+#include <array>
+#include <cstring>
+#include <memory>
+
+#include <esp_log.h>
+
+#include "board.h"
+#include "http.h"
+#include "ogg_demuxer.h"
+
+namespace {
+constexpr int kHttpTimeoutMs = 5000;
+constexpr size_t kHttpReadBufferSize = 1024;
+constexpr uint32_t kNotifyTaskStackSize = 6144;
+constexpr UBaseType_t kNotifyTaskPriority = 2;
+const char* TAG = "NotifyPlayer";
+
+bool IsSupportedUrl(const std::string& url) {
+    return url.compare(0, 7, "http://") == 0 || url.compare(0, 8, "https://") == 0;
+}
+}  // namespace
+
+NotifyPlayer::NotifyPlayer(AudioService& audio_service) : audio_service_(audio_service) {}
+
+NotifyPlayer::~NotifyPlayer() { Stop(); }
+
+bool NotifyPlayer::Start(std::string audio_url, std::vector<NotifySubtitle> subtitles,
+                         uint32_t playback_id, SubtitleCallback subtitle_callback,
+                         FinishedCallback finished_callback) {
+    if (playback_id == 0 || !IsSupportedUrl(audio_url)) {
+        return false;
+    }
+
+    std::stable_sort(subtitles.begin(), subtitles.end(),
+                     [](const NotifySubtitle& left, const NotifySubtitle& right) {
+                         return left.start_ms < right.start_ms;
+                     });
+
+    {
+        std::lock_guard<std::mutex> lock(mutex_);
+        if (active_ || worker_running_) {
+            return false;
+        }
+        audio_url_ = std::move(audio_url);
+        subtitles_ = std::move(subtitles);
+        displayed_text_.clear();
+        subtitle_callback_ = std::move(subtitle_callback);
+        finished_callback_ = std::move(finished_callback);
+        playback_id_ = playback_id;
+        last_playback_position_ms_ = 0;
+        underrun_count_ = 0;
+        next_subtitle_index_ = 0;
+        active_ = true;
+        worker_running_ = true;
+        cancelled_ = false;
+        http_finished_ = false;
+        stream_started_ = false;
+        playback_drained_ = false;
+        completion_reported_ = false;
+    }
+
+    BaseType_t created = xTaskCreate(WorkerEntry, "notify_http", kNotifyTaskStackSize, this,
+                                     kNotifyTaskPriority, &task_handle_);
+    if (created != pdPASS) {
+        ESP_LOGE(TAG, "Failed to create notification HTTP task");
+        std::lock_guard<std::mutex> lock(mutex_);
+        active_ = false;
+        worker_running_ = false;
+        cancelled_ = true;
+        task_handle_ = nullptr;
+        return false;
+    }
+    return true;
+}
+
+void NotifyPlayer::Stop() {
+    std::lock_guard<std::mutex> lock(mutex_);
+    cancelled_ = true;
+    active_ = false;
+    subtitles_.clear();
+    subtitle_callback_ = nullptr;
+    finished_callback_ = nullptr;
+}
+
+bool NotifyPlayer::IsActive(uint32_t playback_id) const {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return active_ && (playback_id == 0 || playback_id == playback_id_);
+}
+
+bool NotifyPlayer::IsBusy() const {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return active_ || worker_running_;
+}
+
+bool NotifyPlayer::IsCancelled(uint32_t playback_id) const {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return cancelled_ || !active_ || playback_id != playback_id_;
+}
+
+void NotifyPlayer::OnPlaybackProgress(uint32_t playback_id, uint32_t media_position_ms) {
+    SubtitleCallback callback;
+    std::string text;
+    bool subtitle_changed = false;
+    {
+        std::lock_guard<std::mutex> lock(mutex_);
+        if (!active_ || playback_id != playback_id_) {
+            return;
+        }
+        last_playback_position_ms_ = media_position_ms;
+        while (next_subtitle_index_ < subtitles_.size() &&
+               subtitles_[next_subtitle_index_].start_ms <= media_position_ms) {
+            text = subtitles_[next_subtitle_index_].text;
+            ++next_subtitle_index_;
+            subtitle_changed = true;
+        }
+        if (!subtitle_changed || text == displayed_text_) {
+            return;
+        }
+        displayed_text_ = text;
+        callback = subtitle_callback_;
+    }
+    if (callback) {
+        callback(playback_id, text);
+    }
+}
+
+NotifyPlayer::FinishedCallback NotifyPlayer::CompleteLocked(uint32_t& playback_id) {
+    if (completion_reported_ || !active_) {
+        return nullptr;
+    }
+    completion_reported_ = true;
+    active_ = false;
+    playback_id = playback_id_;
+    return finished_callback_;
+}
+
+void NotifyPlayer::OnPlaybackDrained() {
+    FinishedCallback callback;
+    uint32_t playback_id = 0;
+    uint32_t underrun_count = 0;
+    uint32_t media_position_ms = 0;
+    {
+        std::lock_guard<std::mutex> lock(mutex_);
+        if (!active_) {
+            return;
+        }
+        playback_drained_ = true;
+        if (http_finished_) {
+            callback = CompleteLocked(playback_id);
+        } else if (stream_started_) {
+            underrun_count = ++underrun_count_;
+            media_position_ms = last_playback_position_ms_;
+        }
+    }
+    if (underrun_count != 0) {
+        ESP_LOGW(TAG, "Notification playback underrun #%lu at %lu ms",
+                 static_cast<unsigned long>(underrun_count),
+                 static_cast<unsigned long>(media_position_ms));
+    }
+    if (callback) {
+        callback(playback_id, true);
+    }
+}
+
+void NotifyPlayer::WorkerEntry(void* arg) {
+    auto* player = static_cast<NotifyPlayer*>(arg);
+    player->WorkerTask();
+    vTaskDelete(nullptr);
+}
+
+void NotifyPlayer::WorkerTask() {
+    std::string audio_url;
+    uint32_t playback_id = 0;
+    {
+        std::lock_guard<std::mutex> lock(mutex_);
+        audio_url = audio_url_;
+        playback_id = playback_id_;
+    }
+
+    bool success = false;
+    auto http = Board::GetInstance().GetNetwork()->CreateHttp(0);
+    if (http) {
+        http->SetTimeout(kHttpTimeoutMs);
+        http->SetHeader("Accept", "audio/ogg, application/ogg");
+        http->SetHeader("Accept-Encoding", "identity");
+        const bool opened = http->Open("GET", audio_url);
+        if (opened) {
+            const int status = http->GetStatusCode();
+            if (status >= 200 && status < 300 && !IsCancelled(playback_id)) {
+                auto demuxer = std::make_unique<OggDemuxer>();
+                uint32_t media_position_ms = 0;
+                bool packet_error = false;
+                demuxer->OnPacket(
+                    [this, playback_id, &media_position_ms, &packet_error](
+                        const uint8_t* data, int sample_rate, int frame_duration_ms, size_t size) {
+                        if (packet_error || IsCancelled(playback_id)) {
+                            packet_error = true;
+                            return;
+                        }
+
+                        auto packet = std::make_unique<AudioStreamPacket>();
+                        packet->sample_rate = sample_rate;
+                        packet->frame_duration = frame_duration_ms;
+                        packet->playback_id = playback_id;
+                        packet->media_position_ms = media_position_ms;
+                        packet->payload.assign(data, data + size);
+
+                        {
+                            std::lock_guard<std::mutex> lock(mutex_);
+                            if (cancelled_ || !active_ || playback_id != playback_id_) {
+                                packet_error = true;
+                                return;
+                            }
+                            playback_drained_ = false;
+                        }
+
+                        if (!audio_service_.PushPacketToDecodeQueue(std::move(packet), true)) {
+                            packet_error = true;
+                            return;
+                        }
+                        {
+                            std::lock_guard<std::mutex> lock(mutex_);
+                            stream_started_ = true;
+                        }
+                        media_position_ms += frame_duration_ms;
+                    });
+
+                std::array<char, kHttpReadBufferSize> buffer;
+                while (!packet_error && !IsCancelled(playback_id)) {
+                    int size = http->Read(buffer.data(), buffer.size());
+                    if (size < 0) {
+                        ESP_LOGE(TAG, "Notification HTTP read failed: %d", http->GetLastError());
+                        break;
+                    }
+                    if (size == 0) {
+                        success = demuxer->Finish();
+                        if (!success) {
+                            ESP_LOGE(
+                                TAG,
+                                "Notification Ogg stream ended before a complete audio stream");
+                        }
+                        break;
+                    }
+                    demuxer->Process(reinterpret_cast<const uint8_t*>(buffer.data()), size);
+                    if (demuxer->HasError()) {
+                        packet_error = true;
+                    }
+                }
+            } else {
+                ESP_LOGE(TAG, "Notification HTTP request returned status %d", status);
+            }
+        } else {
+            ESP_LOGE(TAG, "Failed to open notification HTTP request: %d", http->GetLastError());
+        }
+        http->Close();
+        http.reset();
+    }
+
+    FinishedCallback callback;
+    uint32_t completed_playback_id = 0;
+    bool report_success = false;
+    {
+        std::lock_guard<std::mutex> lock(mutex_);
+        worker_running_ = false;
+        task_handle_ = nullptr;
+        if (!cancelled_ && active_ && playback_id == playback_id_) {
+            if (!success) {
+                callback = CompleteLocked(completed_playback_id);
+            } else {
+                http_finished_ = true;
+                if (playback_drained_) {
+                    callback = CompleteLocked(completed_playback_id);
+                    report_success = true;
+                }
+            }
+        }
+    }
+    if (callback) {
+        callback(completed_playback_id, report_success);
+    }
+}

main/notify/notify_player.h

@@ -0,0 +1,63 @@
+#ifndef NOTIFY_PLAYER_H_
+#define NOTIFY_PLAYER_H_
+
+#include <freertos/FreeRTOS.h>
+#include <freertos/task.h>
+
+#include <cstdint>
+#include <functional>
+#include <mutex>
+#include <string>
+#include <vector>
+
+#include "audio_service.h"
+
+struct NotifySubtitle {
+    uint32_t start_ms = 0;
+    std::string text;
+};
+
+class NotifyPlayer {
+public:
+    using SubtitleCallback = std::function<void(uint32_t playback_id, const std::string& text)>;
+    using FinishedCallback = std::function<void(uint32_t playback_id, bool success)>;
+
+    explicit NotifyPlayer(AudioService& audio_service);
+    ~NotifyPlayer();
+
+    bool Start(std::string audio_url, std::vector<NotifySubtitle> subtitles, uint32_t playback_id,
+               SubtitleCallback subtitle_callback, FinishedCallback finished_callback);
+    void Stop();
+    void OnPlaybackProgress(uint32_t playback_id, uint32_t media_position_ms);
+    void OnPlaybackDrained();
+    bool IsActive(uint32_t playback_id = 0) const;
+    bool IsBusy() const;
+
+private:
+    AudioService& audio_service_;
+    mutable std::mutex mutex_;
+    std::string audio_url_;
+    std::vector<NotifySubtitle> subtitles_;
+    std::string displayed_text_;
+    SubtitleCallback subtitle_callback_;
+    FinishedCallback finished_callback_;
+    TaskHandle_t task_handle_ = nullptr;
+    uint32_t playback_id_ = 0;
+    uint32_t last_playback_position_ms_ = 0;
+    uint32_t underrun_count_ = 0;
+    size_t next_subtitle_index_ = 0;
+    bool active_ = false;
+    bool worker_running_ = false;
+    bool cancelled_ = false;
+    bool http_finished_ = false;
+    bool stream_started_ = false;
+    bool playback_drained_ = false;
+    bool completion_reported_ = false;
+
+    static void WorkerEntry(void* arg);
+    void WorkerTask();
+    bool IsCancelled(uint32_t playback_id) const;
+    FinishedCallback CompleteLocked(uint32_t& playback_id);
+};
+
+#endif  // NOTIFY_PLAYER_H_

main/protocols/protocol.h

@@ -11,6 +11,8 @@ struct AudioStreamPacket {
     int sample_rate = 0;
     int frame_duration = 0;
     uint32_t timestamp = 0;
+    uint32_t playback_id = 0;
+    uint32_t media_position_ms = 0;
     std::vector<uint8_t> payload;
 };