The Bright Minds Academy stationery shop sells erasers costing a whole number of cents. Some fifth graders each bought an eraser, paying a total of $2.09. Some of the $20$ fourth graders each bought an eraser, and they paid a total of $3.04. How many more fourth graders than fifth graders bought an eraser?