Should I have been invited to the game?

Both. And as a guy we get annoyed with stuff like that. Do you follow this team hardcore? He prob thinks he knows more than you about the sport and doesnt want to hear you brag if your team wins