A box contains 400 candies. $75\%$ of the candies are chocolate, and the rest are caramel. How many caramel candies must be removed from the box so that $80\%$ of the remaining candies are chocolate?