perfectly normal - your muscles are recovering and repairing themselves.
It's called the 'conditioning phase' after a few weeks of gym you won't feel no where near as sore.
Thats why when people usually start you tell them to do conditioning training where they lift half of the maximum weights they can push.
Now the key is the recovery process - the sooner your muscles recover the better. That's why you shouldn't work a muscle group more than once a week as it take about a week for muscles to recover.
Saying that to recover they also need fuel (food) - in particular protein which is needed to build muscle.
So what you need is:
1. A good Diet - specially high in protein - buy a protein powder with no fat or carbs that tastes nice. Have a double scoop - 4 times a day.
2. A good training regime -
http://www.bodybuilding.com/fun/wotw21.htm
3. Recovery supplements:
1. Glutamine
2. BCAA's
3. Creatine
THAT IS ALL - DONT LET THE SUPPLMENT STORE SELL YOU ANYTHING ELSE ALL YOU WILL GET IS EXPENSIVE URINE FROM ALL THE SHIT THEY SELL.