Yes, when I was weak and skinny I cared that I was no prince charming... putting on alot of muscle helped me out alot. I'm one ugly son of a *****, but I have a great body now (I feel so silly saying that). My advice would be to find something that you're good at and become great at it. Play guitar, work out, shoot pool... whatever you do, work with what you've got and work hard.
Besides, you don't have to be a 'pretty' boy to get the attention of an attractive woman. I find that being 'manly' rather than 'pretty' is of much more use.