A horticulturist is tracking the growth of $8$ specific fruit trees. Some of these trees have $3$ new primary branches each, while the rest have no new primary branches. These new primary branches do not produce further branches within the tracking period. The horticulturist counts a total of $32$ trees and new primary branches. How many of these trees and new primary branches have no new primary branches?