Are non-lytic viruses reproducing themselves in the host cell?
You are conflating production of infective viral particles with lysis of the host cell. While true for some well-characterized bacterial viruses, like lambda, or T4, there are other bacterial viruses that can produce infective particles without lysing their host cell (e.g., M13, and related viruses).

Likewise, in mammalian cells there are plenty of examples where either an RNA virus (like the RNA-tumour viruses, also called retroviruses) where the viral genome integrates into the host genome, but can also produce infective viral particles. These viruses have external coats that include both proteins and lipid bilayer membranes (the viruses pinch-off, or bud, from the host cells' surface).

Many DNA viruses also integrate a copy of their genome into the host cells' genome following infection (e.g., Epstein-Barr Virus (it can cause mononucleosis), and members of the Herpes Simplex virus family that infect neurons and can cause chicken pox, or genital herpes).  Once you are infected with these DNA viruses your body is never rid of them, rather your immune system and cellular defence mechanisms "fight" the infection such that the titre of infectious viruses drops to nil, but you still have a reservoir of circulating antibodies that can be detected--and used if you are exposed again.