I understand that to you, the baby is not around much, but for the cat THE WORLD IS ENDING!! They're funny creatures. I'm no expert, but have a lot of experience fostering/rehoming cats (100+) as well as working with them. And I agree that it sounds behavioural. Sure, it could be physical but the level of diagnostics required to rule everything out would cost A LOT and be very time consuming & stressful for the cat. Feliway is great, but also v pricey. I think someone else has suggested special time with the cat, which is spot on. Does he have his own space? An area preferably with a door to close & window to look out of? Some cats prefer higher spaces (on top of wardrobes etc) while others prefer den-like spaces. Either way, give the cat a place that he knows is his, safe, and kid-free. To be honest, it sounds like you might want to think of re-homing him. As heart-breaking as that might be for you, it might be in his best interests. Shelters & rescues are always over stretched so you'd be better off asking around your friends & family (who have no littlies). Good luck