@dustmuffin nailed it. Rollo has a great article about the 'buffers' we use to avoid rejection
here.
A LOT of the time, our physical preference for attraction is a buffer: we either go for girls who resemble girls we've had success with in the past (since we think we have a better chance with these girls to avoid rejection) or we go for girls who are exceedingly rare in our environment (to rationalize not having to face immediate rejection).
As for the BOTH problems, the more girls you get with, the more 'types' of girls--physically & personality-wise--you realize you have chemistry with. Your type now won't necessarily be your type in 10 years (in fact, if you're getting a taste of variety, I can almost assure you it won't be). For instance, there's been times in my life, I only liked girls with dark hair, until I had some awesome experiences with blondes. There's been times in my life I only liked girls with fat azzes, til I had some awesome experience with spinners. There's been times in my life I only liked short girls, til I had a tall girl or two rock my world. One of my most hard-headed rationalizations was that I only liked 'girls who were 10's'--which is a great way to avoid rejection. I could go out every weekend for a month and rationalize not approaching a single girl--'None of them met my standards.' But the truth is, I've had great times with girls who might not be total bombshells.
I'd say open your mind and approach--even if you don't think you're attracted to a girl; if you find after giving her a fair shake that you find yourself attracted (listening to your body and not your mind)--go for it. The past year or so, I've had awesome times with girls I wouldn't have given a second look at 4 or 5 years ago. Or you might find that even after experiencing all different types of women, you still have a preference for bigger girls, move. But it's like only having tasted chocolate icecream or maybe only chocolate and a tiny bit of vanilla and saying, "Well, I don't like any other flavors of icecream other than chocolate."