Actually my AFC days might be helpful here. I once had a girlfriend who was 400 km away. But a few months later she moved nearer to me for the summer. This is where it applies to you:
She has a bunch of guy friends she hangs out with. Every day. They do things like play badminton or go out and eat, or whatever, and so whenever I try to get her to do something 'that night', her excuse is 'I have to go out with the guys'. And so this goes on for weeks on end, us arguing and ultimately after I told her off she ends the relationship.
But the point of this babble is that I wanted to spend time with her, and she didn't care enough to make time. So if I'm considered AFC for sticking with her, what does it say about you for doing the same?
eric live has the right idea -- if I was too busy to hang out with a girl, I would feel resentful to be made to hang out with said girl. But since I'm not around, should I expect her to just sit on her ass and wait for me? A big part of rapport is contact, and without contact she will drift. Flip the 'he' and 'she' and it applies to you -- if he won't make time, you can find somebody who will.