I had a weird habit about restaurants when my kids where this small. I avoided going with them, and if I did I usually took something(game, Nintendo, coloring book) to entertain them. They were more likely to be done before me and I hate the bathroom game. If you have to go, you have to go but don't play with me.
With that being said I had that habit because that was the time I used to bond with my husband. The time we spent to talk and have no worries about kids or anything. Just the two of us. Now that they are older we take them with us and we used the time to connect all four of us.
I know this may not be optional for you but I do recommend it if you can. In terms of the break have him go before the dinner arrive and then tell them you can go once he's done with his food. Schedules the breaks and explain that you would like to eat too. Call him on his game and be firm about your bathroom schedule. I do recommend to bring something so he can play while you eat. I think restaurants most be boring places for kids.