A basketball team played twelve games, scoring $1,2,3,4,5,6,7,8,9,10,11,$ and $12$ points. They lost by one point in exactly six games. In each of the other games, they scored twice as many points as their opponent. How many total points did their opponents score?