Creating 3d model building notation using ASCII
As you’ve found, it’s hard to use a two-dimensional format, even from multiple viewpoints, to represent a 3D structure that’s not axis-aligned. I’m not sure I have a useful suggestion there—there’s a reason most 3D file formats support specifying translation/rotation/scaling of individual parts.

A different direction you might try would be using the existing work in the LDraw project; their file format is technically ASCII, and though it’s closer to a standard 3D format in that the fundamental parts (like studs) are built out of lists of vertices / triangles / etc., the higher-level pieces (bricks, plates, etc.) are built out of references to the fundamental ones, which sounds like it’s roughly along the lines of what you’re after. There’s an assortment of third-party design tools that provide GUIs on top of the LDraw parts library; of those, I’m partial to Bricksmith.