Mikhail Gorbachev doesn't speak or understand English in real life, so he shouldn't have been able to understand what George H.W. Bush was saying or respond in English with a Russian accent.
Marge explains to Abe that she doesn't use or allow corporal punishment because she thinks it doesn't work, hence why she usually scolds Homer when she catches him choking Bart.
This is actually the second appearance for Barbara Bush, although in the earlier cameo in "Mr. Lisa Goes to Washington", she was voiced by Maggie Roswell (not Tress MacNeille) and showed off the Presidential bathroom.
This episode was inspired by the animosity towards the show by the Bushes from earlier in the series' run.
When Gerald Ford moves in at the end of the episode, the license plates for his two cars are MRDUH and LUV2SKI. The latter celebrates the former President's passion for skiing since he owned a house in Vail, Colorado in real life.
This episode sees the first appearance of Disco Stu.
In the DVD commentary for this episode, the writers state that they were not even vaguely aware that George Bush had a younger son who shared his name (at that time George W. Bush was Governor of Texas before becoming the 43rd President of the United States in 2000), and Homer's reference to one of the cardboard cut-out sons as "George Bush Jr." was just Homer being stupid (then again, Homer also mentioned Jeb Bush, who also exists and was governor of Florida around the time George W. Bush was President, so either the writers know that Jeb Bush is George's son or they just got lucky).
According to the DVD commentary, Wes Archer, the episodes' director, attempted to TP George H. W. Bush's house during his childhood.
When Bart is questioning George the first time they meet, Bart asks "How many times were YOU president, George?" This may be a reference to his loss to Bill Clinton when running for a second term. Also, the destruction of his memoirs after a lot of hard work may be meant as a comical reason to explain why, in real life, he does not plan to write them.
At the beginning of this episode, Bart and Homer interrupt Apu singing the Cheap Trick song "Dream Police" while washing his car.
The relationship between Bart and Mr. Bush is portrayed in a style reminiscent of Dennis the Menace and Mr. Wilson; Barbara Bush acts just like Mrs. Wilson, even using her catchphrase "Oh, George...".
Homer asks Bush to "apologize for the tax hike", a reference to Bush creating a 31% income tax as part of the 1990 Budget Reconciliation Act, despite his 1988 campaign promise of no new taxes.
When Homer and Bart shoot bottle rockets at the Bush house, it parodies a scene similar to the "Desert Storm" operation of the 1991 Gulf War, which occurred when Bush was in office. It is also surprisingly similar to an annual tradition in Vrodandos, Greece where churches fire bottle rockets at each other and 'score' by hitting the opposing church's bell.
Bush uses a trick he "learned in CIA". Bush was director of the CIA from January 30, 1976 to January 20, 1977. Bush also avoids using the phrase "the CIA," as do actual agents.
Bart references blowback from the "wig offensive", a reference to Bush's role as CIA chief.
Bush says that he'll ruin Homer "like a Japanese banquet", a reference to an incident that happened on January 8, 1992. During a state dinner, then-president Bush vomited on the lap of the Prime Minister of Japan, Kiichi Miyazawa.
Homer tricks Bush into coming to the door so he can glue a rainbow wig to his head by placing cardboard cutouts of Bush's sons in front of the door leading Bush to believe they're real. The sons represented are former President George W. Bush (whom the writers didn't know existed until years later when George W. Bush became President, as mentioned in the commentary) and Florida Governor Jeb Bush.
Bush is paid a visit by Mikhail Gorbachev. Homer calls him a "Commie friend" to Bush, referencing Gorbachev's role as the last leader of the Soviet Union before the fall of Communism, and the thaw in relations between the two countries during Bush's term in office. Ironically, Homer himself had befriended a "Commie friend" in the case of Adil Hoxha in The Crepes of Wrath.
During the shot in which the shredded memoir is falling, a torn piece of paper briefly falls past the screen, with the only non-shredded words reading "V.P. Quayle" and "embarrassment".
When Bart and Bush are looking through a photo album, Bart says that Bob Mosbacher is "a dumb name".
When Homer calls Bush a "wimp", this is a reference to the Wimp Factor, a criticism of Bush during the 1988 Election claiming that Bush looked "too weak" to be a president.
Homer and Ford simultaneously fall over the sidewalk incline when the show ends. This is a parody of Gerald Ford's perceived clumsiness and trick knee (most notably when he fell down the stairs of Air Force One several times) while in office, which was famously mocked on Saturday Night Live back in the 1970s.
Barbara Bush says George and Homer got off on the wrong foot, claiming their relationship is "just like the Noriega thing - now he and George are the best of friends." This reference to the former Panamanian leader Manuel Noriega is actually the opposite of the real situation: as CIA director, Bush had arranged for annual payments to General Noriega, but years later Bush launched Operation Just Cause to depose him.
His going into the sewer was a reference to the 1988 campaign and the dirty tactics involved as a result of advice from Bush's then-advisor Lee Atwater.
Grampa Simpson says he was "spanked by Grover Cleveland on two nonconsecutive occasions," a reference to Cleveland being the only U.S. President to serve two nonconsecutive terms.