One morning each member of a family drank a $10$-ounce mixture of coffee with milk. The amounts of coffee and milk varied from cup to cup, but were never zero. One member drank a fifth of the total amount of milk and a seventh of the total amount of coffee. How many people are in the family?