Two data sets of $23$ integers each are summarized in the histograms shown. For Data Set A, the first interval represents the frequency of integers greater than or equal to $20$, but less than $30$. The second interval represents the frequency of integers greater than or equal to $30$, but less than $40$, and so on. For Data Set B, the first interval represents the frequency of integers greater than or equal to $25$, but less than $35$. The second interval represents the frequency of integers greater than or equal to $35$, but less than $45$, and so on. What is the smallest possible difference between the mean of data set A and the mean of data set B?