title: Circular references not handled in Avro
body: <p>There is a tool called Avro-Tools which ships with Avro and can be used to convert between JSON, Avro-Schema (.avsc) and binary formats.
  But it does not work with circular references.</p>
  
  <p>We have two files:</p>
  
  <ol>
  <li><p><strong>circular.avsc</strong> (generated by Avro)</p></li>
  <li><p><strong>circular.json</strong> (generated by Jackson because it has circular reference and Avro doesn't like the same).</p></li>
  </ol>
  
  <p>circular.avsc</p>
  
  <pre><code>{
     "type":"record",
     "name":"Parent",
     "namespace":"bigdata.example.avro",
     "fields":[
        {
           "name":"name",
           "type":[
              "null",
              "string"
           ],
           "default":null
        },
        {
           "name":"child",
           "type":[
              "null",
              {
                 "type":"record",
                 "name":"Child",
                 "fields":[
                    {
                       "name":"name",
                       "type":[
                          "null",
                          "string"
                       ],
                       "default":null
                    },
                    {
                       "name":"parent",
                       "type":[
                          "null",
                          "Parent"
                       ],
                       "default":null
                    }
                 ]
              }
           ],
           "default":null
        }
     ]
  }
  </code></pre>
  
  <p>circular.json</p>
  
  <pre><code>{
     "@class":"bigdata.example.avro.Parent",
     "@circle_ref_id":1,
     "name":"parent",
     "child":{
        "@class":"bigdata.example.avro.DerivedChild",
        "@circle_ref_id":2,
        "name":"hello",
        "parent":1
     }
  }
  </code></pre>
  
  <p><strong>Command to run avro-tools on the above</strong></p>
  
  <p>java -jar avro-tools-1.7.6.jar fromjson --schema-file circular.avsc circular.json</p>
  
  <p><strong>Output</strong></p>
  
  <p>2014-06-09 14:29:17.759 java[55860:1607] Unable to load realm mapping info from SCDynamicStore
  Objavro.codenullavro.schema?
  {"type":"record","name":"Parent","namespace":"bigdata.example.avro","fields":[{"name":"name","type":["null","string"],"default":null},{"name":"child","type":["null",{"type":"record","name":"Child","fields":[{"name":"name","type":["null","string"],"default":null},{"name":"parent","type":["null","Parent"],"default":null}]}],"default":null}]}?'???K?jH!??Ė?Exception in thread "main" org.apache.avro.AvroTypeException: Expected start-union. Got VALUE_STRING
  at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)</p>
  
  <p>at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)</p>
  
  <p>at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)</p>
  
  <p>Some other JSON values tried with the same schema but that did not work</p>
  
  <p><strong>JSON 1</strong></p>
  
  <pre><code>{
     "name":"parent",
     "child":{
        "name":"hello",
        "parent":null
     }
  }
  </code></pre>
  
  <p><strong>JSON 2</strong></p>
  
  <pre><code>{
     "name":"parent",
     "child":{
        "name":"hello",
     }
  }
  </code></pre>
  
  <p><strong>JSON 3</strong></p>
  
  <pre><code> {
     "@class":"bigdata.example.avro.Parent",
     "@circle_ref_id":1,
     "name":"parent",
     "child":{
        "@class":"bigdata.example.avro.DerivedChild",
        "@circle_ref_id":2,
        "name":"hello",
        "parent":null
     }
  }
  </code></pre>
  
  <p>Removing some of the "optional" elements:</p>
  
  <p><strong>circular.avsc</strong></p>
  
  <pre><code>{
     "type":"record",
     "name":"Parent",
     "namespace":"bigdata.example.avro",
     "fields":[
        {
           "name":"name",
           "type":
              "string",
           "default":null
        },
        {
           "name":"child",
           "type":
              {
                 "type":"record",
                 "name":"Child",
                 "fields":[
                    {
                   "name":"name",
                   "type":
                      "string",
                   "default":null
                    },
                    {
                       "name":"parent",
                       "type":
                          "Parent",
                       "default":null
                    }
                 ]
              },
           "default":null
        }
     ]
  }
  </code></pre>
  
  <p><strong>circular.json</strong></p>
  
  <pre><code> {
     "@class":"bigdata.example.avro.Parent",
     "@circle_ref_id":1,
     "name":"parent",
     "child":{
        "@class":"bigdata.example.avro.DerivedChild",
        "@circle_ref_id":2,
        "name":"hello",
        "parent":1
     }
  }
  </code></pre>
  
  <p><strong>output</strong></p>
  
  <p>2014-06-09 15:30:53.716 java[56261:1607] Unable to load realm mapping info from SCDynamicStore
  Objavro.codenullavro.schema?{"type":"record","name":"Parent","namespace":"bigdata.example.avro","fields":[{"name":"name","type":"string","default":null},{"name":"child","type":{"type":"record","name":"Child","fields":[{"name":"name","type":"string","default":null},{"name":"parent","type":"Parent","default":null}]},"default":null}]}?x?N??O"?M?`AbException in thread "main" java.lang.StackOverflowError</p>
  
  <p>at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:212)</p>
  
  <p>at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)</p>
  
  <p>at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:216)</p>
  
  <p>at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)</p>
  
  <p>at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:216)</p>
  
  <p>at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)</p>
  
  <p>Does anyone know how I can make circular reference work with Avro?</p>