At noon on a certain day, City A is $N$ degrees colder than City B. At 4:00, the temperature in City A has risen by $7$ degrees while the temperature in City B has fallen by $4$ degrees, at which time the temperatures in the two cities differ by $5$ degrees. What is the product of all possible values of $N$?