Listener


Following on from Widcomm Thread restrictions on Win32 and Widcomm BluetoothListener stops getting connections, I first tried putting all calls to the Widcomm CRfCommPort API onto a new thread; that fulfilled that first part of the Widcomm statement: “An additional implication of this guideline is that derived functions must not call back into the stack with another SDK API.”

However it did not fix the problem where the WidcommBluetoothListener stopped accepting connections after a while.  So it seems like the second part of the statement really does mean we must use only one thread: “Any SDK API that must be called as a result of a callback from the stack must be executed from the application main thread, not the callback execution context.”

So now I’ve implemented a scheme where we create a thread and force all API calls to be done from that thread only.  (In class WidcommPortSingleThreader we have the commands (PortWriteCommand, OpenServerCommand, etc) which we add to a queue, and then wait for the single thread to take that off the queue and action it).

Some more testing is required, but this seems to have fixed this problem.  I was able to create one thousand connections one after the other without problem.  That’s much more that it could manage previously, so it looks good.

Hopefully this also fixes the problem where send operations were getting stuck on Win32  (Widcomm faults in OBEX operations) — one of the operations now done on the single thread is CRfCommPort.Write.  When I get a moment I’ll test that too.

Its been reported in the forums that BluetoothListener eventually stops accepting new connections.  I’ve reproduced this on Windows XP where new connections stop being accepted but it appears that the client device reports that it is still managing to form new connections.  I’ve still to go and test this on Windows Mobile.  If its Win32 only, then maybe its again related to Widcomm Thread restrictions on Win32  A workaround according to the reporter is to stop the BluetoothListener and create a new one.

I’ve added a new method to the ConsoleMenuTesting/DeviceMenuTesting test apps to test these scenarios.  Use “ListenAcceptMultiple” and “ConnectMultipleTimes” on the two sides.

The fault again seems to be that Widcomm stops reporting the respective event(s)…

UPDATE: Testing on WM (on my iPAQ) shows that this problem doesn’t occur there.  It just keeps on accepting connection.  So the problem is Win32 only!

Widcomm/Broadcom produces different SDKs for their different platforms, but the PDF documentation files in the the various SDKs looks very similar, so I printed the Windows CE/WM version and worked from that.  Recently however I was reading from the Win32 version and found this statement:

“An additional implication of this guideline is that derived functions must not call back into the stack with another SDK API.  Any SDK API that must be called as a result of a callback from the stack must be executed from the application main thread, not the callback execution context.”

So in a Widcomm-callback handler we shouldn’t call back into the stack. 😦  In the current code we do that — for instance when we have get pending data to send, and that stack sends us a ‘finished sending’ event, we call Write from there.  Maybe its this that causes the “gets stuck sending” problem

Anyway I’ve got some work in process to enable this, being tracked by bug 25410.

 I intend just to

  1. Mark threads with a flag noting whether they’re from a Widcomm callback or not.
  2. Any time I make a call back into the stack, check that flag, and if set force the call onto another thread, e.g. via a Delegate.BeginInvoke/EndInvoke pair.

Other much more complex mechanisms are available, e.g. one I tried earlier was to action all Widcomm-callbacks on a new thread, but that was very slow.

Note however that the documentation says “from the application main thread”.  As a library, we can’t know the application’s main thread.  We could though create a single thread and pass all calls to the Widcomm API via it, but is that really required??  It would make things much more complex…

The full BluetoothListener support as discussed at https://32feetnetdev.wordpress.com/2009/08/06/full-widcomm-bluetoothlistener/ is now committed. 🙂

The support for server-side in 2.4 (beta) was preliminary.  WidcommBluetoothListener only attempted to handle one pending connection at a time and rather than implementing its own initiating and event handling code it just piggy-backed on the consumer code’s Begin-/ AcceptBluetoothClient call.  (The latter aspect, as we discussed at http://inthehand.com/forums/p/2223/7752.aspx#7752, meaning of course that a connection couldn’t be accepted until BeginAcceptBluetoothClient had been called, and thus we couldn’t support the Pending method).

So for a full implementation we need to keep: 1. a queue of accepted connections, 2. a queue of waiting ports, and also 3. a queue of waiting acceptors (if the consumer has called Begin-/AcceptBluetoothClient one or more times before connection(s) have arrived).

If one looks at the UNIX Sockets internal e.g. via W. Richard Stevens’ book then one sees that there’s no “waiting sockets” queue.  The Widcomm API however requires a port is created and set into server-mode for a connection to be accepted.

I’v implemented this and my current version of the code is at https://32feetnetdev.wordpress.com/widcommbtlistener-2-0/.  However testing it threw up interesting behaviour so I have not committed it yet.  If anyone can have a play and confirm my solution then we can get it checked in.

The odd behaviour that I saw was that if I opened a connection to the server and closed it from the remote device, then the next connection would reuse the old port (Widcomm CRfCommPort) instead of using a port from the waiting queue.  This was surprising behaviour to me, and I had to add lots of diagnostic code to the stream and port classes (WidcommRfcommStream and WidcommRfcommPort) to confirm it.  (The odd behaviour became apparent promptly as an Assert on the old port instance reported that a CONNECT event was occurring when it was in PeerDidClose state, without the assert I may have been confused for much longer).  It is not discussed in the Widcomm documentation, neither explicitly not by implication as far as I can see.

Anyway, the solution to this should be simple.  Currently we don’t call the CRfCommPort.Close method until we Close/ Dispose the NetworkStream/ BluetoothClient (actually WidcommRfcommStream).   Thus the port is still active and apparently Widcomm will reuse it then.  So we simply have to Close the port as soon as we receive the event signalling that the peer close (i.e. CONNECT_ERR).

The diff is this:

--- d:\temp\temp\WidcommRfcommStream;C56017.cs	2009-08-06 21:54:50.250000000 +0100
+++ d:\working\InTheHand.Net.Personal\InTheHand.Net.Personal\Net.Bluetooth.Widcomm\WidcommRfcommStream.cs	2009-08-06 21:53:24.734375000 +0100
@@ -614,6 +614,7 @@
                         DebugId + ": " +
                         eventId + ": m_state wanted: " + "Connected" + ", but was: " + m_state_);
                     Console.WriteLine("HandlePortEvent: closed when open.");
+                    m_port.Close();
                     // No information whether this is a hard or clean close! :-(
                     m_state_ = State.PeerDidClose;
                     if (GetPendingReceiveDataLength() == 0) {

There’s another possibility, that is to return the disconnected port to a pool from where it can be used when we start the next waiter.  However we’re not creating enough ports for that to be necessary and it would be rather more complex and could make debugging more difficult so we’ll just go ahead and close the port.

This change affects all connection both client as well server.  We have to make sure it doesn’t break anything.  In particular we have to make sure that once a port is disconnected by the remote device (state PeerClose) that we don’t access the port.   All feedback  welcomed.