Plant X and Plant Y were once the same height. Since then Plant X has grown $50\%$ while Plant Y has grown $\frac{1}{5}$ as many inches as Plant X. Plant X is now $45$ inches tall. How tall, in inches, is Plant Y now?