Yeah it's retarded, fapping is not the issue, porn is the issue. Internet porn literally causes identical changes in the brain to hardcore drug abuse (I can link to the studies). However, not-fapping is probably not good for you, the whole "semen retention" thing is broscience hype.
The fact that most "nofap" movements make no mention of porn tells me that most people don't understands what they're talking about.
It's like saying you're going to go on a caloric restriction diet or OMAD for a month, but still eat processed junk for every meal.
That said if you can give up porn for 3+ months and consistently decrease your overall usage over the years, you will see tangible benefits, but 99.99% of men cannot give up porn for any meaningful amount of time. The other issue is that many people just replace porn with other vices such as youtube, social media, or maladaptive internet use in general.