Skip to content

backend, Task exception was never retrieved at /utils/stt/streaming.py:60 #2175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
beastoin opened this issue Apr 9, 2025 · 2 comments
Open

backend, Task exception was never retrieved at /utils/stt/streaming.py:60 #2175

beastoin opened this issue Apr 9, 2025 · 2 comments

Comments

@beastoin
Copy link
Collaborator

beastoin commented Apr 9, 2025

issue


| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | websockets.exceptions.ConnectionClosedOK: sent 1000 (OK); then received 1000 (OK) |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | raise self.connection_closed_exc() |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | File "/opt/venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 948, in ensure_open |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | await self.ensure_open() |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | File "/opt/venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 635, in send |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | await transcript_socket_async_send(bytes(chunk)) |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | File "/app/utils/stt/streaming.py", line 70, in send_initial_file_path |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | Traceback (most recent call last): |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | future: <Task finished name='Task-6645' coro=<send_initial_file_path() done, defined at /app/utils/stt/streaming.py:60> exception=ConnectionClosedOK(Close(code=1000, reason=''), Close(code=<CloseCode.NORMAL_CLOSURE: 1000>, reason=''), False)> |
| 2025-04-09T01:29:30.562Z | "unknown" | "backend" | Task exception was never retrieved |

logs

Date Host Service Message
2025-04-09T01:29:30.746Z "unknown" "backend" INFO: 169.254.169.126:13318 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:29:30.746Z "unknown" "backend" get_conversations nmtP5s1jFmPZwGoRgQjBLoZw4G13 1 0 in_progress
2025-04-09T01:29:30.746Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" WebSocket disconnected nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" WebSocket disconnected nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" connect_to_trigger_pusher nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" send_initial_file_path
2025-04-09T01:29:30.746Z "unknown" "backend" speech_profile soniox duration 30.9875 nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:29:30.746Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:29:30.746Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:29:30.746Z "unknown" "backend" file_path _temp/nmtP5s1jFmPZwGoRgQjBLoZw4G13_speech_profile.wav
2025-04-09T01:29:30.746Z "unknown" "backend" speech_profile_duration 30.9875
2025-04-09T01:29:30.746Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:29:30.746Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $last_memory nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" finalize_processing_conversations len(processing): 0 nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" send_heartbeat nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:29:30.746Z "unknown" "backend" _listen nmtP5s1jFmPZwGoRgQjBLoZw4G13 en 16000 opus True None
2025-04-09T01:29:30.746Z "unknown" "backend" Could not process audio: error received 1000 (OK); then sent 1000 (OK) nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Soniox WebSocket closed in on_message.
2025-04-09T01:29:30.746Z "unknown" "backend" Error receiving from Soniox: Soniox error: 408 - Audio data decode timeout
2025-04-09T01:29:30.746Z "unknown" "backend" Soniox error: 408 - Audio data decode timeout
2025-04-09T01:29:30.746Z "unknown" "backend" Killing soniox_socket2 E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:29:30.746Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" connect_to_trigger_pusher nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" file_path None
2025-04-09T01:29:30.746Z "unknown" "backend" speech_profile_duration 0
2025-04-09T01:29:30.746Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 'mulaw', 'sample_rate': 8000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:29:30.746Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $last_memory nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" finalize_processing_conversations len(processing): 0 nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" send_heartbeat nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:29:30.746Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:29:30.746Z "unknown" "backend" _listen nmtP5s1jFmPZwGoRgQjBLoZw4G13 en 8000 pcm8 True None
2025-04-09T01:29:30.562Z "unknown" "backend" websockets.exceptions.ConnectionClosedOK: sent 1000 (OK); then received 1000 (OK)
2025-04-09T01:29:30.562Z "unknown" "backend" raise self.connection_closed_exc()
2025-04-09T01:29:30.562Z "unknown" "backend" File "/opt/venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 948, in ensure_open
2025-04-09T01:29:30.562Z "unknown" "backend" await self.ensure_open()
2025-04-09T01:29:30.562Z "unknown" "backend" File "/opt/venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 635, in send
2025-04-09T01:29:30.562Z "unknown" "backend" await transcript_socket_async_send(bytes(chunk))
2025-04-09T01:29:30.562Z "unknown" "backend" File "/app/utils/stt/streaming.py", line 70, in send_initial_file_path
2025-04-09T01:29:30.562Z "unknown" "backend" Traceback (most recent call last):
2025-04-09T01:29:30.562Z "unknown" "backend" future: <Task finished name='Task-6645' coro=<send_initial_file_path() done, defined at /app/utils/stt/streaming.py:60> exception=ConnectionClosedOK(Close(code=1000, reason=''), Close(code=<CloseCode.NORMAL_CLOSURE: 1000>, reason=''), False)>
2025-04-09T01:29:30.562Z "unknown" "backend" Task exception was never retrieved
2025-04-09T01:29:29.355Z "unknown" "backend" INFO: connection closed
2025-04-09T01:29:29.291Z "unknown" "backend" INFO: connection open
2025-04-09T01:29:29.124Z "unknown" "backend" INFO: ('169.254.169.126', 13314) - "WebSocket /v4/listen?language=en&sample_rate=16000&codec=opus&uid=nmtP5s1jFmPZwGoRgQjBLoZw4G13&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:29:28.685Z "unknown" "backend" INFO: connection closed
2025-04-09T01:29:06.925Z "unknown" "backend" INFO: connection open
2025-04-09T01:29:06.854Z "unknown" "backend" INFO: ('169.254.169.126', 36344) - "WebSocket /v4/listen?language=en&sample_rate=8000&codec=pcm8&uid=nmtP5s1jFmPZwGoRgQjBLoZw4G13&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:28:57.274Z "unknown" "backend" INFO: 169.254.169.126:33738 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:28:57.274Z "unknown" "backend" get_conversations ERaRpzlzykhVVBTW2xJOgFnDx5K2 1 0 in_progress
2025-04-09T01:28:56.437Z "unknown" "backend" INFO: 169.254.169.126:33724 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:28:50.501Z "unknown" "backend" INFO: 169.254.169.126:64494 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:28:50.501Z "unknown" "backend" send_initial_file_path 0.20164895057678223
2025-04-09T01:28:50.501Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Message: type $service_status E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" connect_to_trigger_pusher E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" send_initial_file_path
2025-04-09T01:28:50.501Z "unknown" "backend" speech_profile soniox duration 29.4896875 E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:28:50.501Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:28:50.501Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:28:50.501Z "unknown" "backend" file_path _temp/E72DsUrj72fombbiKA6hF2MNNbC2_speech_profile.wav
2025-04-09T01:28:50.501Z "unknown" "backend" speech_profile_duration 29.4896875
2025-04-09T01:28:50.501Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:28:50.501Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:28:50.501Z "unknown" "backend" Message: type $last_memory E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Message: type $service_status E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Message: type $service_status E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" finalize_processing_conversations len(processing): 0 E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Message: type $service_status E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" send_heartbeat E72DsUrj72fombbiKA6hF2MNNbC2
2025-04-09T01:28:50.501Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:28:50.501Z "unknown" "backend" _listen E72DsUrj72fombbiKA6hF2MNNbC2 en 16000 opus True None
2025-04-09T01:28:44.508Z "unknown" "backend" INFO: connection open
2025-04-09T01:28:44.349Z "unknown" "backend" INFO: ('169.254.169.126', 64480) - "WebSocket /v4/listen?language=en&sample_rate=16000&codec=opus&uid=E72DsUrj72fombbiKA6hF2MNNbC2&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:28:37.728Z "unknown" "backend" INFO: 169.254.169.126:22538 - "GET /v1/memories?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:28:37.728Z "unknown" "backend" get_memories wgPcnFbv04gzJo85A5hwXHwBFul2 1 0 in_progress
2025-04-09T01:28:30.412Z "unknown" "backend" INFO: 169.254.169.126:47248 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:28:30.412Z "unknown" "backend" _trigger_create_conversation_with_delay not conversation or not last session viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:28:23.967Z "unknown" "backend" INFO: 169.254.169.126:43492 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:28:23.967Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:28:06.545Z "unknown" "backend" INFO: 169.254.169.126:30084 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:28:06.545Z "unknown" "backend" get_conversations lrgeABnl6wSx5iY7LLEuscavqwz2 1 0 in_progress
2025-04-09T01:27:50.899Z "unknown" "backend" INFO: 169.254.169.126:52198 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:27:42.167Z "unknown" "backend" INFO: 169.254.169.126:50716 - "GET / HTTP/1.1" 404 Not Found
2025-04-09T01:27:41.663Z "unknown" "backend" INFO: 169.254.169.126:50712 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:27:41.663Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:27:34.541Z "unknown" "backend" INFO: 169.254.169.126:50708 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:27:34.541Z "unknown" "backend" get_conversations nmtP5s1jFmPZwGoRgQjBLoZw4G13 1 0 in_progress
2025-04-09T01:27:34.540Z "unknown" "backend" Error closing WebSocket: Cannot call "send" once a close message has been sent. lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:34.540Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:34.540Z "unknown" "backend" Message: type $service_status lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:34.540Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:34.540Z "unknown" "backend" connect_to_trigger_pusher lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:34.540Z "unknown" "backend" upsert_vector {'upserted_count': 1}
2025-04-09T01:27:34.540Z "unknown" "backend" Saving 2 facts for conversation 18618f29-213b-4b70-ab85-95ec9ec2c7b2
2025-04-09T01:27:34.540Z "unknown" "backend" _extract_facts: HOBBIES
2025-04-09T01:27:34.540Z "unknown" "backend" _extract_facts: CORE
2025-04-09T01:27:34.540Z "unknown" "backend" save_structured_vector creating vector
2025-04-09T01:27:32.278Z "unknown" "backend" INFO: connection closed
2025-04-09T01:27:22.285Z "unknown" "backend" INFO: 169.254.169.126:7172 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:27:22.277Z "unknown" "backend" INFO: 169.254.169.126:7158 - "GET /v3/speech-profile HTTP/1.1" 200 OK
2025-04-09T01:27:22.269Z "unknown" "backend" INFO: 169.254.169.126:38014 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:27:22.258Z "unknown" "backend" INFO: 169.254.169.126:7162 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:27:22.258Z "unknown" "backend" get_conversations lrgeABnl6wSx5iY7LLEuscavqwz2 1 0 in_progress
2025-04-09T01:27:22.258Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:27:22.258Z "unknown" "backend" WebSocket disconnected viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:27:22.258Z "unknown" "backend" Heartbeat error: Cannot call "send" once a close message has been sent. lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Initial processing error: lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Exception in process_audio_soniox:
2025-04-09T01:27:22.258Z "unknown" "backend" Message: type $last_memory lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Message: type $memory_created lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Message: type $service_status lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Message: type $service_status lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Error processing conversation: 400 Document 'projects/based-hardware/databases/(default)/documents/users/lrgeABnl6wSx5iY7LLEuscavqwz2/conversations/18618f29-213b-4b70-ab85-95ec9ec2c7b2' cannot be written because its size (1,070,452 bytes) exceeds the maximum allowed size of 1,048,576 bytes. lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" get_memories lrgeABnl6wSx5iY7LLEuscavqwz2 100 0
2025-04-09T01:27:22.258Z "unknown" "backend" delete_memories_for_conversation 18618f29-213b-4b70-ab85-95ec9ec2c7b2 2
2025-04-09T01:27:22.258Z "unknown" "backend" get_public_approved_plugins_data from cache
2025-04-09T01:27:22.258Z "unknown" "backend" get_google_maps_location 38.29669437854578 -85.72027120876126
2025-04-09T01:27:22.258Z "unknown" "backend" finalize_processing_conversations len(processing): 1 lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Message: type $service_status lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" send_heartbeat lrgeABnl6wSx5iY7LLEuscavqwz2
2025-04-09T01:27:22.258Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:27:22.257Z "unknown" "backend" _listen lrgeABnl6wSx5iY7LLEuscavqwz2 en 16000 opus True None
2025-04-09T01:27:22.247Z "unknown" "backend" INFO: connection closed
2025-04-09T01:26:55.900Z "unknown" "backend" INFO: connection open
2025-04-09T01:26:55.304Z "unknown" "backend" INFO: ('169.254.169.126', 20478) - "WebSocket /v4/listen?language=en&sample_rate=16000&codec=opus&uid=lrgeABnl6wSx5iY7LLEuscavqwz2&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:26:52.143Z "unknown" "backend" INFO: 169.254.169.126:15336 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:26:47.744Z "unknown" "backend" INFO: 169.254.169.126:15324 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:26:36.447Z "unknown" "backend" INFO: 169.254.169.126:58410 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:26:36.447Z "unknown" "backend" get_conversations lrgeABnl6wSx5iY7LLEuscavqwz2 1 0 in_progress
2025-04-09T01:26:36.420Z "unknown" "backend" INFO: 169.254.169.126:58404 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:26:36.420Z "unknown" "backend" get_conversations lrgeABnl6wSx5iY7LLEuscavqwz2 1 0 in_progress
2025-04-09T01:26:35.483Z "unknown" "backend" INFO: connection closed
2025-04-09T01:26:34.243Z "unknown" "backend" INFO: 169.254.169.126:58394 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:26:32.978Z "unknown" "backend" INFO: 169.254.169.126:53324 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:26:32.978Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:26:26.110Z "unknown" "backend" INFO: 169.254.169.126:53318 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:26:26.110Z "unknown" "backend" get_conversations jTbLZFVyJjduPPvf0KQDqqPYhyU2 1 0 in_progress
2025-04-09T01:26:26.050Z "unknown" "backend" INFO: 169.254.169.126:53304 - "GET /v1/conversations?include_discarded=false&limit=50&offset=0&statuses= HTTP/1.1" 200 OK
2025-04-09T01:26:26.050Z "unknown" "backend" Could not process audio: error received 1000 (OK); then sent 1000 (OK) nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:26.050Z "unknown" "backend" _get_in_progress_conversation new id='5734764c-82c0-4f42-91e3-87a16dae00df' created_at=datetime.datetime(2025, 4, 9, 1, 26, 23, 565721, tzinfo=datetime.timezone.utc) started_at=datetime.datetime(2025, 4, 9, 1, 26, 23, 565721, tzinfo=datetime.timezone.utc) finished_at=datetime.datetime(2025, 4, 9, 1, 26, 25, 245813, tzinfo=datetime.timezone.utc) source=<ConversationSource.omi: 'omi'> language='en' structured=Structured(title='', overview='', emoji='🧠', category=<CategoryEnum.other: 'other'>, action_items=[], events=[]) transcript_segments=[TranscriptSegment(id='8061638f-8d40-4cf3-b8f9-efe143fe8684', text='We love them. So,', speaker='SPEAKER_01', speaker_id=1, is_user=False, person_id=None, start=0.0, end=1.6800000000000002, translations=[])] geolocation=None photos=[] apps_results=[] plugins_results=[] external_data=None app_id=None discarded=False deleted=False visibility=<ConversationVisibility.private: 'private'> processing_memory_id=None processing_conversation_id=None status=<ConversationStatus.in_progress: 'in_progress'> viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" get_conversations viUv7GtdoHXbK1UBCDlPuTDuPgJ2 50 0
2025-04-09T01:26:26.050Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" send_initial_file_path 0.3442082405090332
2025-04-09T01:26:26.050Z "unknown" "backend" Message: type $service_status viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" connect_to_trigger_pusher viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" send_initial_file_path
2025-04-09T01:26:26.050Z "unknown" "backend" speech_profile soniox duration 42.745625 viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:26:26.050Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:26:26.050Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:26:26.050Z "unknown" "backend" file_path _temp/viUv7GtdoHXbK1UBCDlPuTDuPgJ2_speech_profile.wav
2025-04-09T01:26:26.050Z "unknown" "backend" speech_profile_duration 42.745625
2025-04-09T01:26:26.050Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 's16le', 'sample_rate': 16000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:26:26.050Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:26:26.050Z "unknown" "backend" Message: type $last_memory viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Message: type $service_status viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Message: type $service_status viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" finalize_processing_conversations len(processing): 0 viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Message: type $service_status viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" send_heartbeat viUv7GtdoHXbK1UBCDlPuTDuPgJ2
2025-04-09T01:26:26.050Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:26:26.050Z "unknown" "backend" _listen viUv7GtdoHXbK1UBCDlPuTDuPgJ2 en 16000 opus True None
2025-04-09T01:26:26.050Z "unknown" "backend" Soniox WebSocket closed in on_message.
2025-04-09T01:26:26.050Z "unknown" "backend" Error receiving from Soniox: Soniox error: 408 - Audio data decode timeout
2025-04-09T01:26:26.050Z "unknown" "backend" Soniox error: 408 - Audio data decode timeout
2025-04-09T01:26:20.403Z "unknown" "backend" INFO: connection open
2025-04-09T01:26:20.226Z "unknown" "backend" INFO: ('169.254.169.126', 46146) - "WebSocket /v4/listen?language=en&sample_rate=16000&codec=opus&uid=viUv7GtdoHXbK1UBCDlPuTDuPgJ2&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:26:15.324Z "unknown" "backend" INFO: 169.254.169.126:46120 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:26:15.324Z "unknown" "backend" Connected to Pusher transcripts trigger WebSocket. nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Connecting to Pusher transcripts trigger WebSocket... nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" connect_to_trigger_pusher nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" file_path None
2025-04-09T01:26:15.324Z "unknown" "backend" speech_profile_duration 0
2025-04-09T01:26:15.324Z "unknown" "backend" Sent initial request: {'api_key': '4d0cbc0805cfa21dde705ff102ff1fc636aca27af10e100b42d1eb955acb79eb', 'model': 'stt-rt-preview', 'audio_format': 'mulaw', 'sample_rate': 8000, 'num_channels': 1, 'enable_speaker_tags': True, 'language_hints': []}
2025-04-09T01:26:15.324Z "unknown" "backend" Connected to Soniox WebSocket.
2025-04-09T01:26:15.324Z "unknown" "backend" Message: type $last_memory nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" finalize_processing_conversations len(processing): 0 nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Message: type $service_status nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" send_heartbeat nmtP5s1jFmPZwGoRgQjBLoZw4G13
2025-04-09T01:26:15.324Z "unknown" "backend" Connecting to Soniox WebSocket...
2025-04-09T01:26:15.324Z "unknown" "backend" _listen nmtP5s1jFmPZwGoRgQjBLoZw4G13 en 8000 pcm8 True None
2025-04-09T01:26:05.289Z "unknown" "backend" INFO: connection open
2025-04-09T01:26:05.207Z "unknown" "backend" INFO: ('169.254.169.126', 56876) - "WebSocket /v4/listen?language=en&sample_rate=8000&codec=pcm8&uid=nmtP5s1jFmPZwGoRgQjBLoZw4G13&include_speech_profile=true&stt_service=soniox" [accepted]
2025-04-09T01:26:03.004Z "unknown" "backend" INFO: 169.254.169.126:38520 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:25:56.781Z "unknown" "backend" INFO: 169.254.169.126:38518 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:56.781Z "unknown" "backend" get_conversations lrgeABnl6wSx5iY7LLEuscavqwz2 1 0 in_progress
2025-04-09T01:25:55.193Z "unknown" "backend" INFO: 169.254.169.126:38512 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:55.193Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:25:18.814Z "unknown" "backend" INFO: 169.254.169.126:33104 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:25:13.811Z "unknown" "backend" INFO: 169.254.169.126:64772 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:13.811Z "unknown" "backend" get_conversations hnVeSnE7PLexsgq7DgF3Aa4CUdf2 1 0 in_progress
2025-04-09T01:25:13.445Z "unknown" "backend" INFO: 169.254.169.126:64770 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:13.445Z "unknown" "backend" get_conversations 1bNrBJ56mycPgTJzm4oQOnLlMaj2 1 0 in_progress
2025-04-09T01:25:07.159Z "unknown" "backend" INFO: 169.254.169.126:64764 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:07.159Z "unknown" "backend" get_conversations Qx9rgIwmGRQiCpB1hxX4ZxfHP4I2 1 0 in_progress
2025-04-09T01:25:04.662Z "unknown" "backend" INFO: 169.254.169.126:64754 - "GET /v1/conversations?include_discarded=true&limit=1&offset=0&statuses=in_progress HTTP/1.1" 200 OK
2025-04-09T01:25:04.662Z "unknown" "backend" get_conversations ORhE8c60f0TQP9hsJTiyYm8z6A32 1 0 in_progress
2025-04-09T01:24:53.964Z "unknown" "backend" INFO: 169.254.169.126:2336 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:24:53.781Z "unknown" "backend" INFO: 169.254.169.126:2326 - "GET /v2/firmware/latest?device_model=Omi%20DevKit%202&firmware_revision=2.0.8&hardware_revision=Seeed%20Xiao%20BLE%20Sense&manufacturer_name=Based%20Hardware HTTP/1.1" 200 OK
2025-04-09T01:24:52.012Z "unknown" "backend" INFO: 169.254.169.126:2322 - "GET /v3/speech-profile HTTP/1.1" 200 OK
2025-04-09T01:24:50.051Z "unknown" "backend" INFO: 169.254.169.126:2310 - "PATCH /v1/users/geolocation HTTP/1.1" 200 OK
2025-04-09T01:24:36.813Z "unknown" "backend" INFO: 169.254.169.126:16466 - "GET / HTTP/1.1" 404 Not Found

https://us5.datadoghq.com/logs?query=host%3Aunknown%20service%3Abackend%20container_id%3A00afae8043a75b770df5b4942b996d37065ceda9681bbe1c6e6fd9f5d744db378f95b3f8af902196e561eeff65b35bfbb6998f26b665b7387b88ab7e94d308793bf00fd0df8208&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=host%2Cservice&context_event=AZYYKq6AAAAvG4HV0e1NLgAF&event=&fromUser=true&messageDisplay=inline&refresh_mode=sliding&storage=hot&stream_sort=time%2Cdesc&to_event=AwAAAZYYKqd6m9Q-0wAAABhBWllZS3E2QUFBQXZHNEhWMGUxTkxnQVAAAAAkMDE5NjE4MmEtYjBiZi00MjNjLThlZDktNWI0OTA0OTdjZWRiAAANeg&viz=&from_ts=1744161870562&to_ts=1744162170747&live=false
@beastoin
Copy link
Collaborator Author

Image

@beastoin
Copy link
Collaborator Author

@nquang29 could you solve this 💪

@nquang29 nquang29 mentioned this issue Apr 10, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@beastoin