Import a .XYZ file as Displacement modifier? TL;DR  

I used a program called L3DT to create a mountain-y terrain, and I saved the height map to a .XYZ file. Is there any way I can import this .XYZ file as a displace modifier in Blender? I don't want to use a generic Black and white file for the heightmap because there can be some confusion between the map and Blender, but .XYZ files store raw coordinate data for each vertice that makes up the terrain. 

Process

First, I generate the .XYZ file in L3DT, which I have confirmed works and is not corrupt, as this is my .XYZ file in a Point-cloud viewer called LIDARview.



Then, in blender, I create a plane and subdivide it 100 times to add some geometry, then add a displace modifier:



How would I go from here? I can't use a .XYZ file as a texture..

Here's a link to the .XYZ file
It's an option. According to the document Using L3DT with Blender, you can export files in the OBJ format.

Played around with your file. Be carefully) a heavy construction. My computer freezes :

    import bpy
    import math
    import bmesh

    file_path = "D:\\terrain.xyz"

    def read_verts(file_path):
        verts = [] 
        with open(file_path) as myfile:
            for line in myfile:
                verts.append([float(x) / 500 for x in line.split()])
        return verts

    def create_faces(num_wid, num_heigh):
        faces = []
        j = 0
        for i in range(0, num_heigh * (num_wid - 1)):
            if j &lt; num_heigh - 1:
                face = (i, i + 1,  i + num_heigh + 1, i + num_heigh)
                faces.append(face)
                j = j + 1
            else:
                j = 0
        return faces



    verts = read_verts(file_path) 
    num_width = int(math.sqrt(len(verts)))
    faces = create_faces(num_width, num_width)


    # mesh
    mesh = bpy.data.meshes.new("terrain")
    mesh.from_pydata(verts,[],faces)
    mesh.update(calc_edges=True)

    # object
    object = bpy.data.objects.new("terrain",mesh)
    object.location = (-10, -10, 0)
    bpy.context.scene.objects.link(object)
    bpy.context.scene.objects.active = object
    object.select = True