Why does calcined feldspar crumble?
You're replicating the natural weathering process of granite.
Granite is a rock comprising the minerals feldspar, quartz, and mica.
There is a continuous solid-solution ternary diagram here that illustrates the various feldspars as a function of calcium, potassium, and sodium content: the end members are K-spar (potassium variety), albite (sodium variety), and anorthite (calcium variety).
The weathering process is complex and involves so-called incongruent dissolution, where one component in the solid state reacts to form one or more nother solid state components. This is why you see granitic feldspar weathering faster than the quartz or mica components out in nature.
You are reacting various minerals (the exact composition of your feldspars and micas would be determined by XRD or a similar method) with oxygen, carbon dioxide, and water from the atmosphere as well as any waters of hydration that may be in your micas, which are sheet silicates from a general class of silicates known as phyllosilicates.
Back to your query about crumbling: you're dehydrating the micas (muscovite on the light-colored end, biotite on the dark-colored end) if they are hydrated at all, but mostly you're making various clays from the feldspars via the incongruent dissolution pathway. Those new minerals you're making are the crumbly ones, mostly clays and maybe some brittle mica scraps. But it's mostly clay.
Generally speaking, carbon dioxide and water form an aqueous solution of carbonic acid species which reacts in the presence of oxygen and a feldspar to form a clay and aqueous calcium and carbonate ions. There are many, many complex reactions.
Here is an excellent write-up about anorthite hydrolysis, specifically regarding the region around Egersund, Norway. In it, the author does some modeling using PHREEQC and gives specific examples of some of the weathering reactions. Here is one of the reactions that takes a mixed albanitic-anorthitic feldspar to kaolinite:
$$\ce{2NaCaAl3Si5O16 +
      8CO2 +
      9H2O =
      2Ca^2+ +
      2Na^+ +
      6HCO3- +
      4SiO2 +
      3~Al2Si2O5(OH)4 +
      2CO2}$$
High temperature speeds things up, and also will dictate what products you get. Same for water and carbon dioxide, of course.
In summary: you are making mostly crumbly clay from granite by speeding-up the chemical weathering process of (mostly) the feldspar component(s). The particular composition of the clay varies with the entire system composition.
This is a very complex sub-field of study in aqueous geochemistry. An excellent reference (which I used in my grad school days and beyond) is Stumm and Morgan.