In his case, which may well extend to other non-fans, I can think of several possible reasons for Fan Avoidance Activities:

4 (admittedly simiar to 3 on Tavia's list) They just feel like Real Life is more important to them. My parents watch and read plenty of science fiction, but would rather visit relatives than go to a Con. They don't criticize my own interest in Cons and can talk SF with my friends (who think they are extremely cool), but they think the word Mundane is slightly offensive and aren't interested in joining Fandom.