Well I'm not sure whether it's anything to worry about or not - I guess it's fairly common to remember back to about 6 or so?
I have some vague memories from around that age - 5 or 6 I guess. I remember going to kindergarten and sitting on the rug listening to the teachers reading stories. I remember telling my little sister I was going to run away at about 6 or 7 too - she said she wanted to come too but I told her she couldn't.
What always amazes me are these people that say they can remember so far back they were basically still in the womb.
People that say they remember their childhood vividly are basically exaggerating. If I add up all my memories before the age of 12 it'll add up to perhaps 1 hour of memories (just a wild guess, it could be a lot less) which is as good as nothing. It are nothing more than flashes which may or may not have happened.
Childhood memories are all memories of memories of remembering memories. The people who recall those years well are the people who spent a lot of time thinking about it in an unbroken chain. Photos and family members reinforcing the memories helps a lot too.
"Many people need desperately to receive this message: 'I feel and think much as you do, care about many of the things you care about, although most people do not care about them. You are not alone.'" ― Kurt Vonnegut