The sums of three whole numbers taken in pairs are $15$, $20$, and $23$. What is the middle number?