Since he is a villain, his "game ending" is bad one if he does not withdraw, and obviously he know a lot of game info. Including very crucial one, and states he has a contingency plan. Fast forward a little bit. His withdrawal is refused, and he does nothing. Then in chapter 20 or so, he decides to enact his contingency plan. Which is, act like a villainous noble.