Connection Broker not working properly?

(Windows Sever 2012 R2 RDS eviroment)

My Connection Broker is on it's own box - not on the Session Host servers. I have 3 Session Host Servers ... SVRDS1, SVRDS2 and SVRDS3.

If a user is logged onto SVRDS2 via RdWeb and I shut down the server, the user can't log on (WebAccess) until the server is up and running again.

Why isn't the Connection Broker redirect to on of the other two RDSH servers?

I have this error in eventlog...

RD Connection Broker failed to process the connection request for user mydomain\myuser. 
Error: Element not found. 

Any idea?

Thanks.