Homer spinning around with a bowling ball in his hand before throwing it into the air and exclaiming "I'm gonna make it after all!" is reminiscent of the opening of "The Mary Tyler Moore Show" where Mary throws her hat into the air.
This is the first episode to contain Maggie's name in the title.
When Homer finds out that Marge is pregnant with Bart and Lisa, he runs up the stairs, but on the wall you can see a picture of Lisa (or at least someone who looks like her).
Also when Homer finds out about Lisa he was upstairs in the bathroom with Bart when Bart was flushing the keys down the toilet, in Lisa's First Word.
Marge says that when Homer found she was pregnant with any of the kids, he would pull of his hair and scream and run off, but that doesn't happen in I Married Marge and Lisa's First Word. In I Married Marge, Dr. Hibbert says to Homer and Marge that she is pregnant, to which Homer exclaims "D'oh!". In Lisa's First Word, Marge tells Homer, to which Homer is happy, but Bart flushes his keys down the toilet.
The two scenes in which Homer discovers the birth of his first two kids, the background clearly shows 742 Evergreen Terrace, although they still live in the apartment at that point in Lisa's First Word.
Ruth Powers is among the people at Marge's baby shower despite the fact that she moved to Springfield in New Kid on the Block. She could have moved to another city after this episode and then moved back to Springfield.
When Homer and Marge come back from their night out, the door in their bedroom is on the wrong wall.
When Marge tells Homer her contractions are three minutes apart, a picture of Maggie can be seen hanging on the wall, despite not having been born yet.
Homer doesn't become the safety inspector of the power plant until Homer's Odyssey. And in that episode, Maggie was already born. Yet prior to Maggie's birth, Homer begs for his job back at the power plant before she is born.