Can asynchronous I / O throw an elusive exception?

I have an application with multiple .NET threads that uses async I / O and AsyncCallbacks to handle I / O terminations. Rarely will an application work with an error, for example:

Arithmetic operation resulted in an overflow.
   at MyApp.ReadComplete(IAsyncResult ar) in c:\MyApp.cs:line 123
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.ContextAwareResult.CompleteCallback(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Net.ContextAwareResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
   at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

Line 123 in MyApp.cs is AsyncCallback's first executable line, and it is inside try / catch (Exception ex), but catch is not executed.

Is the .NET Framework for me about where the exception occurred? Did the exception really happen in an asynchronous world where I cannot catch it? Why can't I catch this exception?