Hard to say from your post, but this sounds more like Generalized Anxiety Disorder to me. There's a brief comparison here
I have OCD, and ime there are particular things about it that set it apart. First of all, the fears are usually irrational.
For example, I have an irrational fear that I might lose control of my body and murder someone I care about. This fear is so intense that I can actually feel the impulses in my arms and hands to pick up a knife and stab them. A feeling like this, where you fear you have no control over your own actions, is terrifying beyond words to describe. My ritual response when this happens is to sit on my hands. Another example: I have an irrational fear that if I look at a balcony, someone in the apartment will decide to jump to their death. Just by looking at the balcony I have caused this to happen. So I avoid looking at balconies (my ritual) while I walk down the street.
I am perfectly aware that these fears are irrational, but when they arise (ie. I get the intrusive thoughts) they feel so convincingly possible to me that I find it extremely hard to avoid acting on my ritual compulsions. Other people have fears that they're homosexual and that they'll be compelled to have sex with people of the same sex even though they're not attracted to them. Others have fears that they're pedophiles. And (most closely related to the fear you've presented) some parents have fears that they'll harm their own children--eg. shake, strangle, etc.; ie. not 'harm' unintentionally through accident or neglect but intentionally (this is the kind of fear I would have). If the fear is not that you will harm the baby 'intentionally' (but contrary to your wishes) but that some kind of harm will come to the baby, that sounds more like Generalized Anxiety to me.
Another characteristic of OCD, for me, is that the intrusive thoughts typically come out of the blue as opposed to being the end product of a train of thought.
For example, I can be having a pleasant conversation with someone, with violence being the absolute furthest thing from my thoughts, when I will suddenly experience a very strong urge to stab them. This is not because I am angry at them or want to hurt them. It has nothing to do with how I feel about them at all. And my fear of stabbing them doesn't come from a long series of reflections about the possibility that I might someday become crazy and violent; the fear comes from the sudden intrusive thought itself, and it only persists for as long as the episode lasts. The urge comes 'out of the blue', out of nowhere, and imposes itself on me no matter what else I might be doing. But the urge is always in response to some environmental cue. I won't get an urge to stab someone if there is no one there to stab. It won't enter my thoughts at all. I won't fear that someone is going to throw themselves off the balcony if there is no balcony for me to look at. So the environmental cue triggers the intrusive thought, but these thoughts are entirely dependent on external cues in the environment, not on any process of reasoning.
By contrast, with Generalized Anxiety, the fear comes from rumination about the possible negative outcomes of various things going on in your life. You worry about all the bad things that could happen to the baby and can't stop thinking about those bad things happening. But the fears are tied to your thoughts in a way that is logical, a way that they aren't with OCD. You think about taking a trip on the plane and it's the thought of taking the trip that triggers the fear of dying in a crash. And then you keep going back to that possibility. But, by contrast, even though I experience strong urges to hurt people at times when other people are present, I don't ruminate about the possibility that I am actually going to hurt someone because I have no intention of hurting anyone; it's not part of any plan that I have, so there is no logical train of reasoning leading up to the fear. You have an intention to take a trip, so you think about the crash; I have no intention to hurt someone, so I don't think about the possibility of hurting someone ... until the intrusive thought arises. Now, if you were sitting around minding your own business, not thinking about anything at all having to do with your trip, and you had a sudden intrusive thought that you were going to die in that plane crash, that might be OCD. (And if you always had those thoughts about planes, and only about planes, that might be a specific phobia--fear of flying.)
Ofc, I'm describing "ideal" types. The symptoms sometimes overlap in various ways, or people sometimes suffer from both GAD and OCD. And if you have an environment that is always supplying you with external cues to trigger a thought, it might be hard to tell the difference. Health OCD and GAD about health issues are often extremely similar because you take your body with you everywhere. I am constantly being triggered by some minor problem I'm currently having with my body. Every time I notice the problem, I get irrational fears about the worse possible outcome and can't stop obsessing about it. I end up going online and researching it for hours and trying to find home remedies, etc. But I'm not actually sure whether this counts as OCD or GAD. I have the same problem with my fear of being homeless. (So maybe I have both OCD and GAD.)
If you have a baby in the house, it can be hard to avoid external cues about the baby because the baby demands constant attention. So this inability to avoid the cues triggering the thoughts might lead you to ruminate about what will seem like a real possibility that you will harm the baby. But if it were OCD it would more likely be a thought like, "I'm going to strangle the baby" or "if I don't change the diaper as soon as it's soiled the baby will get an infection and die". It will be about irrational and remote possibilities that will lead to irrational behaviors, like refusing to touch the baby to avoid strangling it, or checking the baby's diaper every ten minutes. By contrast, with GAD, your behaviors might be a little excessive, but not obviously irrational; eg. you might be especially careful about how you handle the baby to avoid unintentionally hurting it, or check for rashes every time you change the diaper. And you might experience a lot of anxiety about these possibilities and ruminate about them obsessively. But there won't be rituals involved. That being the third distinguishing characteristic of OCD--that there are always rituals involved (though they may only be mental, like always thinking a particular thing to 'dispel' an intrusive thought whenever you have that thought). At least, as far as I understand it.
The best way to find out is to talk to a professional. If you think OCD is the problem, I recommend talking to someone who deals specifically with OCD as opposed to, eg., your family doctor. You might also check out some videos by Chrissie Hodges
, an OCD advocate, and see if they resonate with you.