[speech-to-text] Connection immediately closed in Android build, works fine in editor #691
Description
Hi, I'm having issues with getting speech-to-text to work in an Android build. Everything works fine in the Unity Editor.
Unity version: 2021.3.6f1
Watson unity-sdk: v6.0.0
IBM unity-sdk-core: v1.2.4
Scripting backend: IL2CPP
Api compatibility level: .NET Framework (I tried switching to .NET Standard 2.1, but it results in a compilation error IBMSdkCore\Connection\RESTConnector.cs(668,24): error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create')
So everything works fine in the Editor, but when I run the Android build, the websocket connection is immediately closed. Similar to #542, but I'm using the Frankfurt endpoint (eu-de).
Edit: also tested on London (eu-gb) and Dallas (us-south), problem still the same.
Logs:
: RESTConnector: URL: https://iam.cloud.ibm.com/identity/token
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: RESTConnector: URL: https://iam.cloud.ibm.com/identity/token
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: RESTConnector: URL: https://api.eu-de.assistant.watson.cloud.ibm.com/v2/assistants/[redacted-assistant-id]/sessions?version=2021-11-27
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: [07-29-2022 08:55:14][WSConnector][WARNING] No case for URL for wss://. Replacing https:// with wss://.
: IBM.Cloud.SDK.Debug.DebugReactor:ProcessLog(LogRecord)
: IBM.Cloud.SDK.LogSystem:ProcessLog(LogRecord)
: IBM.Cloud.SDK.Log:Warning(String, String, Object[])
: IBM.Cloud.SDK.Connection.WSConnector:FixupURL(String)
: IBM.Cloud.SDK.Connection.WSConnector:CreateConnector(Authenticator, String, String, String)
: IBM.Watson.SpeechToText.V1.SpeechToTextService:CreateListenConnector()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:StartListening(OnRecognize, OnRecognizeSpeaker)
:
: SpeechToText.CreateListenConnector(): Created listen socket. Model: nl-NL_Telephony, parsedParams:
: IBM.Watson.SpeechToText.V1.SpeechToTextService:CreateListenConnector()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:StartListening(OnRecognize, OnRecognizeSpeaker)
:
: WSConnector.Send(): Sending TextMessage message: {"action":"start","content-type":"audio/l16;rate=16000;channels=1;","inactivity_timeout":-1,"interim_results":true,"max_alternatives":1,"profanity_filter":false,"smart_formatting":true,"speaker_labels":false,"timestamps":true,"word_confidence":true,"speech_detector_sensitivity":0,5,"background_audio_suppression":0,"processing_metrics":false}
: IBM.Cloud.SDK.Connection.WSConnector:Send(Message, Boolean)
: IBM.Watson.SpeechToText.V1.SpeechToTextService:SendStart()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListen(AudioData)
:
: SpeechToText.SendStart(): SendStart() with the following params: {"action":"start","content-type":"audio/l16;rate=16000;channels=1;","inactivity_timeout":-1,"interim_results":true,"max_alternatives":1,"profanity_filter":false,"smart_formatting":true,"speaker_labels":false,"timestamps":true,"word_confidence":true,"speech_detector_sensitivity":0,5,"background_audio_suppression":0,"processing_metrics":false}
: IBM.Watson.SpeechToText.V1.SpeechToTextService:SendStart()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListen(AudioData)
:
: WSConnector.OnWSClose(): Reason: An exception has occurred while receiving. - Code: 1006 - WasClean: False
: IBM.Cloud.SDK.Connection.WSConnector:OnWSClose(Object, CloseEventArgs)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocket:close(PayloadData, Boolean, Boolean, Boolean)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp
: SpeechToText.OnListenClosed(): OnListenClosed(), State = DISCONNECTED
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListenClosed(WSConnector)
: IBM.Cloud.SDK.Connection.<ProcessReceiveQueue>d__49:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: SpeechToText.KeepAlive(): KeepAlive exited.
: IBM.Watson.SpeechToText.V1.<KeepAlive>d__242:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
I have also enabled logging on the Websocket in WSConnector:
29-7-2022 10:55:15|Warn |WSConnector.SendMessages|The current logging level has been changed to Trace.
29-7-2022 10:55:15|Debug|WebSocket.sendHandshakeRequest|A request to the server:
GET /v1/recognize?model=nl-NL_Telephony HTTP/1.1
User-Agent: websocket-sharp/1.0
Host: api.eu-de.speech-to-text.watson.cloud.ibm.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: [redacted]
Sec-WebSocket-Version: 13
Authorization: Bearer [redacted]
29-7-2022 10:55:16|Debug|WebSocket.sendHandshakeRequest|A response to this request:
HTTP/1.1 101 Switching Protocols
sec-websocket-accept: [redacted]
Content-Type: application/octet-stream
Server: watson-gateway
strict-transport-security: max-age=31536000; includeSubDomains;
x-dp-watson-tran-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
x-request-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
x-global-transaction-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
X-EdgeConnect-MidMile-RTT: 0
X-EdgeConnect-Origin-MEX-Latency: 534
Date: Fri, 29 Jul 2022 08:55:15 GMT
Upgrade: websocket
Connection: Upgrade
29-7-2022 10:55:17|Fatal|TaskToApm.Begin|IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketException: The header part of a frame could not be read.
at IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketFrame.processHeader (System.Byte[] header) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketFrame+<>c__DisplayClass73_0.<readHeaderAsync>b__0 (System.Byte[] bytes) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at Mono.Net.Security.MobileAuthenticatedStream.StartOperation (Mono.Net.Security.MobileAuthenticatedStream+OperationType type, Mono.Net.Security.AsyncProtocolRequest asyncRequest, System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at Mono.Net.Security.MobileAuthenticatedStream.InnerRead (System.Boolean sync, System.Int32 requestedSize, System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.Tasks.ValueTask`1+ValueTaskSourceAsTask+<>c[TResult].<.cctor>b__4_0 (System.Object state) [0x00000] in <00000000000000000000000000000000>:0
at System.Net.Sockets.Socket+<>c.<.cctor>b__367_10 (System.IAsyncResult ares) [0x00000] in <00000000000000000000000000000000>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00000] in <00000000000000000000000000000000>:0
29-7-2022 10:55:17|Trace|TaskToApm.Begin|Begin closing the connection.
29-7-2022 10:55:17|Debug|WebSocket.close|Was clean?: False
sent: False
received: False
29-7-2022 10:55:17|Trace|TaskToApm.Begin|End closing the connection.
29-7-2022 10:55:17|Info |ExecutionContext.RunInternal|The connection has already been closed.