Once again, IT DEPENDS ON THE PERSON.
Some people get lethargic, sure. But I smoke every day and I work out every single day too. Infact, I enjoy working out more when I am stoned. I guess I feel like I can push myself further.
Also(time to play the Professor here), it depends on what kinda weed you're smoking. If you are smoking crap, you are more likely to be lethargic and here is why.
Weed contains things called Cannabinoids. They are responsible for the "high", however there is a certain kind of cannabinoid that does not have this effect. Read below.
The following is from Wikipedia:
Cannabinoids are a group of chemicals which activate the body's cannabinoid receptors. Before other types were discovered, the term referred to a unique group of secondary metabolites found in the cannabis plant, which are responsible for the plant's peculiar pharmacological effects. Currently, there are three general types of cannabinoids: herbal cannabinoids occur uniquely in the cannabis plant; endogenous cannabinoids are produced in the bodies of humans and other animals; and synthetic cannabinoids are similar compounds produced in a laboratory.
Cannabidiol, also known as CBD, is a non-psychoactive cannabinoid found in the hemp plant Cannabis sativa.
CBD alone is not psychoactive, but it appears to reduce the euphoric effect of THC (which is an isomer of cannabidiol) and add a sedative quality. It may decrease the rate of THC clearance from the body, perhaps by interfering with the metabolism of THC in the liver. Medically, it appears to relieve convulsion, inflammation, anxiety, and nausea. CBD has a greater affinity for the CB2 receptor than for the CB1 receptor. It is perceived to have more effect on the body.
Good weed will have lower amounts of CBD, which causes many of the more negative effects associated with marijuana use. I think it has something to do with the altitude the plant is grown at as far as potency, and the amount of Cannabidiols.