Its not that their pulled forward LC haven;t you heard of cauliflower ear? It happens when the ear cartilage is smashed or hurt, so the cartilage fills up with fluid and becomes hard.
OP if you are that concerned about it you can have it surgically fixed, I'm sure you don;t have the career cauliflower of someone like dan henderson. I used headgear too but people don't use it for every single match or live practice, it only takes once to hurt the ear enough to make some cauliflower.
You can go under the knife, I think it is a plastic surgery procedure. I had gynecomastia when I was younger, since I was 13, not caused by steroids(all natural) or other drugs. I went under the knife to have it taken care of and it was hard, and it hurt, but in the end it was worth it not to have puffy nips. I felt like it was holding me back too because I had built some muscle and was still intimidated to take my shirt off. I recommend going for the procedure if it bothers you that much.
Wrestling comes with a price as other posters have indicated, I have had two knee surgeries and an upcoming shoulder surgery from injuries sustained while wrestling in high school, and made worse kickboxing and boxing in college. But you have not only the muscles and strength but also the ground know how to take out most people except for strikers with a good sprawl and good submission or JJ fighters. What you have suffered through and earned in wrestling will stay with you for the rest of your life in the form of inner strength and rugged belief in yourself. How many other sports do you not get to rely on your teammates? Not very many.