I, for one, hate the self-help books. I don't often speak against them, though, because they seem to help some people - but I don't be surprised if you feel that they aren't for you.
You sound like your on the verge of a breakthrough. You know that something doesn't feel right with the way you've been treating people and the way that you've been treated. I know it can be discouraging, because you feel that you're "just being yourself" and it isn't good enough for other people. The fact of the matter is that - if you're cool enough to be good friends with good people - you're cool enough to be more than friends with good women. Don't worry about what one girl said. That's her perception of you. I separate my women friends from my women interests for a very good reason. I have no trouble with girls, but I know that, if I asked, some of my female friends would tell me the same thing that girl told you. It doesn't matter, though, because I choose to allow them to view me that way. Girls have a way of seeing a man they're not interested in as an asexual being and then destroying his confidence with comments such as those. You just have to learn to expect them and shrug them off because no woman has the final say on whether you make a better friend or a lover. It's not like you were trying to seduce her anyway, so it doesn't matter if she has romantic intent for you.
I truly do think that confidence is your issue, just juding by the fact that you feel you've been ignored by women. The fact of the matter is that you will get the amount of attention from people that you feel you deserve just as long as you don't get on your hands and knees and beg for it, or try and bend over backwards to win people's approval. I used to be afraid of being arrogant, but really a little bit of arrogance is healthy for your personality. Balance that with compassion, friendliness, etc. and you're a well-rounded human being. There's nothing wrong with believing that you are a great person.
As for gaining confidence, there is no magic bullet. My life completely changed when I realized that I really didn't like any of the people that I was hanging out with. It's a long story, and it has to do with my (ex) girlfriend hooking up with a guy in my social circle. I'll spare you the boring details, but essentially I expected my friends to stick up for me and not to put up with that sh!t. They're loyalty was tentative at best, and completely towards the other guy at worst, and I just threw up my hands and said "Fvck it! They're not helping me the way they should be. They're not my friends." My new attitude made me expect them to prove themselves to me. I had written people off for the most part, and I was on the fence risking falling into the bitter and arrogant realm of doubting people and self-hating. Fortunately, there were a few good men in my circle afterall. They saw that I was at risk of becoming a loathful hermit, and they came to my aide when I needed it but really didn't deserve it. They brought me up and out of my clout. It was their selfless acts that made me realize that there is much more to life than worrying about what other people think of you.
Amazingly, by giving up on others and choosing to rely on myself, I found something incredible: I am everything I want in a person. I hold myself to my highest standards, and truly believe that I meet them. If ever I catch myself falling below those standards, I quickly correct my behavior and I will resort to punishing myself if I fail to do so. It sounds weird, but I see it as basic self-discipline, and it's a good practice.
The point to all this nonsense is that you can get the response out of other people that you want, but first you have to understand their motivations. You have to strip yourself of your expectations, and learn to trust yourself before trusting others. Dealing with other people, especially girls, is not a 'game'. Focusing on 'being a challenge!' and other dating 'techniques' will just leave you frustrated because it's unnatural and insincere. I find nothing to be less attractive than false behaviour that is clearly done trying to make me feel a certain way. You must be more concerned with how happy your actions make yourself than you are with how they make other people. In order to bring other people up, you have to focus on how doing so makes YOU feel - because you're emotions are the only barometer you have to judge the quality of your actions by. Once you begin living up to your own potential and truly being proud of yourself for it, others will notice that and appreciate that quality in you.
Don't write girls off. In fact, don't write anybody off. If your not happy with the way things are going, be aware that you can change them. You have everything you need already to be the person you've always wanted to be - all that you possibly lack is the motivation and the drive to do so. There is no magic bullet to confidence, this is true, but the key to it is realizing that you can live up to your own expectations no matter how high you set them, so set them high. Even having lofty goals will make you feel like you've discovered the keys to life that you were previously lacking. When you've truly found your focus, things become simple, and you'll wonder how you ever got by without it. I wish it was something I could package and mail to you, but it isn't that easy. Don't think for one seconed, however, that it is anything you don't already have. You've just got to learn to embrace it.
