The numbers $-5$, $-1$, $2$, $7$ and $11$ are rearranged according to these rules: The largest is not in the first two places, but is in one of the first four places. The smallest is not in the last two places, but is in one of the last four places. The median is not in the first or last place, but is in one of the middle three places. What is the average of the first and last numbers?