How do the variable portions of antibody genes look in cells which don't produce antibodies?
Perhaps you want to ask how VDJ recombination is regulated in non lymphoid cells.. 

Well even in immature lymphoid cells VDJ recombination is regulated. And also Ig genes are suppressed in T-cell and TCR genes are suppressed in B-cells.. 
Also, the recombination of Ig is suppressed in T-cell and vice-versa. 

RAG-1,2(Recombination activating gene) downregulation partly does the job, but not fully. 

Earlier reports say that transcription in the locus is essential and epigenetic regulation of transcription in turn regulates recombination. 
But what it exactly means is that accessibility to the RSS (Recombination signal sequences) is limited in a repressed chromatin and recent reports say that nucleosomes are appropriately positioned over the RSS to control recombination.

Have a look at these: 

http://www.cell.com/abstract/S0092-8674%2802%2900675-X

http://www.sciencedirect.com/science/article/pii/S0952791506000057

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC204470/