All players leaving the game?

The last 3 skirmishes I've tried to get into have resulted in the other 3 players leaving the game. This is pretty unusual. It seems to be fixed by a restart, but I've just gotten the screen flickering bug to go away and now this. I don't want to have to restart every game. Anyone have any ideas whats going on?

My firewall is off. What gets me is that this happened just a few hours ago and continues to happen.