Improving self-confidence can be done in a number of ways.
One excellent way is to start following your dreams. Ever wanted to be a rockstar? Go buy a guitar and take some lessons. Want to be the next Tom Cruise? Sign up at some local theatre dealie. Gotta crawl before you can walk. If you want to be a surfer, buy a board and start practising. If you want to be a scientist, sign up for some biology classes at the local community college. You get the idea.
Another way is to exercise. Science has shown testosterone increases from regular exercise. Weights are relatively cheap and can be stored in your garage. Or join a gym. Obviously you have to start eating right as well. Information on this is readily available. Consult your doctor before beginning any new strenuous activity.
Something else you might want to investigate is self-hypnosis. Seeing as how confidence is all a state of mind, rather than a physical attribute, changing your mental states through hypnosis is an admittedly new-agey way to attain it. You can't try it thinking "this won't work" though, otherwise - surprise - it won't.
You can always check the DJ bible (OH NO WHY WOULD I DO THAT) for other suggestions. One way to overcome fear of people is to approach them in some non-threatening setting, like the local mall. Chat to them. Talk to girls who work in stores, they're paid to be nice.
========
Re: manager situation. If your manager is delegating all the lame jobs to only you and not your co-workers who are in the same position and get paid the same, you might want to quit and start anew at some other place. Other than that, it's probably a job requirement to follow your superiors orders.
Ignoring people is passive aggressive behaviour. Stay away from it. Be assertive instead. If some co-worker asks you to cover for you while they take a 30-minute smoke break, tell them "what's in it for me? Oh, nothing? You're out of luck, pal." You can even err on the side of jerkness if you like, just don't go overboard.