text:
semantic / semantic.texi
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523
\input texinfo @c -*-texinfo-*-
@c $Id$
@setfilename semantic.info
@settitle Semantic Bovinator: Parser generation for Emacs
* semantic: (semantic). Semantic Parsing for Emacs
@end format
@end ifinfo
@sp 10
@center @titlefont{Semantic}
@vskip 0pt plus 1 fill
Copyright @copyright{} 1999, 2000, 2001, 2002 Eric M. Ludlam
@end titlepage
@node Top, Install, (dir), (dir)
@comment node-name, next, previous, up
Semantic is a program for Emacs which includes, at its core, a lexer,
and a compiler compiler (bovinator). Additional tools include a
bnf->semantic table converter, example tables, and a speedbar tool.
The core utility is the ``semantic bovinator'' which has similar
behaviors as yacc or bison. Since it is not designed to be as feature
rich as these tools, it uses the term ``bovine'' for cow, a lesser
cousin of the yak and bison.
To send bug reports, or participate in discussions about semantic,
use the mailing list cedet-semantic@@sourceforge.net via the URL:
* Install:: Installing semantic.
* Overview:: Introduce basic concepts.
* Semantic Components:: Enumerate all semantic modules.
* Lexing:: Setting up the lexer for your language.
* Bovinating:: Setting up the parser for your language.
* BNF conversion:: Using the BNF converter to make tables.
* Compiling:: Running the bovinator on a source file.
* Debugging:: Debugging bovine tables
* Programming:: How to program with a nonterminal stream.
* Current Context:: How to get the current code context.
* Tools:: User Tools which use semantic.
* Index::
@end menu
@node Install, Overview, Top, Top
@chapter Installation
To install semantic, untar the distribution into a subdirectory, such as
@file{/usr/share/emacs/site-lisp/semantic-#.#}. Next, add the following
lines into your individual @file{.emacs} file, or into
(setq semantic-load-turn-everything-on t)
(load-file "/path/to/semantic/semantic-load.el")
@end example
If you would like to turn individual tools on or off in your init
file, skip the first line.
@node Overview, Semantic Components, Install, Top
@chapter Overview
Semantic is a tool primarily for the Emacs-Lisp programmer.
However, it comes with ``applications'' that non-programmer might
find useful.
This chapter is mostly for the benefit of these non-programmers
as it gives brief descriptions of basic concepts such as
grammars, parsers, compiler-compilers, parse-tree, etc.
@cindex grammar
The grammar of a natural language defines rules by which valid phrases
and sentences can be composed using words, the fundamental units with
which all sentences are created.
@cindex context-free grammar
In a similar fashion, a ``context-free grammar'' defines the rules by which
programs can be composed using the fundamental units of the language,
i.e., numbers, symbols, punctuations, etc.
Context-free grammars are often specified in a well-known form called
@cindex Backus-Naur Form
@cindex BNF
Backus-Naur Form, BNF for short.
This is a systematic way of representing context-free grammars
such that programs can read files with grammars written in BNF
and generate code for ``parser'' of that language.
@cindex yacc
@cindex compiler-compiler
YACC (Yet Another Compiler Compiler) is one such program that has been
part of UNIX operating systems since the 1970's.
YACC is pronounced the same as ``yak'', the long-haired ox found in Asia.
The parser generated by YACC is usually a C program.
@cindex bison
@uref{http://www.gnu.org/software/bison/bison.html , Bison}
is also a ``compiler compiler'' that takes BNF grammars and produces
parsers in C language.
The difference between YACC and Bison is that Bison is
@cindex free software
@uref{http://www.gnu.org/philosophy/free-sw.html , free software}
and upward-compatible with YACC.
It also comes with an excellent manual.
Semantic is similar in spirit to YACC and Bison.
@cindex bovinator
Semantic, however, is referred to as a @dfn{bovinator} rather than
as a parser, because it is a lesser cousin of YACC and Bison.
It is lesser in that it does not perform a full parse
like YACC or Bison.
@cindex bovination
Instead, it @dfn{bovinates}.
``Bovination'' refers to partial parsing which
@cindex parse tree
creates @dfn{parse trees} of only the top most
expressions rather than parsing every nested expression.
This is sufficient for the purposes for which semantic was designed.
Semantic is meant to be used within Emacs for providing
editor-related features such as code browsers and translators rather
than for compiling which requires far more complex and complete parsers.
Semantic is not designed to be able to create full parse trees.
@cindex parser
One key benefit of semantic is that it creates parse trees
@cindex bovine tree
(perhaps the term @dfn{bovine tree} may be more accurate)
with the same structure regardless of the type of language involved.
Higher level applications written to work with bovine trees
will then work with any language for which the grammar is available.
For example, a code browser written today that supports C, C++, and
Java may work without any change on other languages that do not even
exist yet.
All one has to do is to write the BNF specification for the new language.
The rest of the work is done by semantic.
For certain languages, it is hard if not impossible to specify the syntax
of the language in BNF form, e.g.,
@uref{http://www.texinfo.org ,texinfo}
and other document oriented languages.
Semantic provides a parser for texinfo nevertheless.
Instead of BNF grammar, texinfo files are ``parsed'' using
Semantic comes with grammars for these languages:
@itemize @bullet
@item C
@item Emacs-Lisp
@item java
@item makefile
@item scheme
@end itemize
Several tools employing semantic that provide user observable features
are listed in @ref{Tools} section.
@node Semantic Components, Lexing, Overview, Top
@chapter Semantic Components
This chapter gives an overview of major components
of semantic and how they interact with each other to perform its job.
The first step of parsing is to break up the input file into
its fundamental components.
This step is called lexing.
The output of the lexer is a list of tokens that make up the file.
syntax table, keywords list, and options
input file ----> Lexer ----> token stream
@end example
The next step is the parsing shown below.
bovine table
token stream ---> Parser ----> parse tree
@end example
The end result, the parse tree, is created based on the ``bovine table'',
which is the internal representation of the BNF language grammar used by
Semantic database provides caching of the parse trees by saving them
into files named @file{semantic.cache} automatically then loading them
when appropriate instead of re-parsing. The reason for this is to save the
time it takes to parse a file which could take several seconds or more
for large files.
Finally, semantic provides an API for the Emacs-Lisp programmer to access the
information in the parse tree.
@node Lexing, Bovinating, Semantic Components, Top
@chapter Preparing your language for Lexing
In order to reduce a source file into a token list, it must first be
converted into a token stream. Tokens are syntactic elements such as
whitespace, symbols, strings, lists, and punctuation.
The lexer uses the major-mode's syntax table for conversion.
@xref{Syntax Tables,,,elisp}.
As long as that is set up correctly (along with the important
@code{comment-start} and @code{comment-start-skip} variable) the lexer
should already work for your language.
The primary entry point of the lexer is the @dfn{semantic-flex} function
shown below.
Normally, you do not need to call this function.
It is usually called by @emph{semantic-bovinate-toplevel} for you.
@defun semantic-flex start end &optional depth length
Using the syntax table, do something roughly equivalent to flex.
Semantically check between @var{START} and @var{END}. Optional
argument @var{DEPTH} indicates at what level to scan over entire
lists. The return value is a token stream. Each element is a list
of the form (symbol start-expression . end-expresssion). @var{END} does
not mark the end of the text scanned, only the end of the beginning of
text scanned. Thus, if a string extends past @var{END}, the end of
the return token will be larger than @var{END}. To truly restrict
scanning, use `narrow-to-region'. The last argument, @var{LENGTH}
specifies that @dfn{semantic-flex} should only return
@var{LENGTH} tokens.
@end defun
* Lexer Overview::
* Lexer Output::
* Lexer Options::
* Keywords::
* Keyword Properties::
@end menu
@node Lexer Overview, Lexer Output, Lexing, Lexing
@section Lexer Overview
Semantic lexer breaks up the content of an Emacs buffer into a list of
tokens. This process is based mostly on regular expressions which in
turn depend on the syntax table of the buffer's major mode being setup
@xref{Major Modes,,,emacs}.
@xref{Syntax Tables,,,elisp}.
Specifically, the following regular expressions which rely on syntax
tables are used:
@table @code
@item @code{\\s-}
whitespace characters
@item @code{\\sw}
word constituent
@item @code{\\s_}
symbol constituent
@item @code{\\s.}
punctuation character
@item @code{\\s<}
comment starter
@item @code{\\s>}
comment ender
@item @code{\\s\\}
escape character
@item @code{\\s)}
close parenthesis character
@item @code{\\s$}
paired delimiter
@item @code{\\s\"}
string quote
@item @code{\\s\'}
expression prefix
@end table
In addition, Emacs' built-in features such as
are employed.
@node Lexer Output, Lexer Options, Lexer Overview, Lexing
@section Lexer Output
The lexer, @ref{semantic-flex}, scans the content of a buffer and
returns a token list.
Let's illustrate this using this simple example.
00: /*
01: * Simple program to demonstrate semantic.
02: */
04: #include <stdio.h>
06: int i_1;
08: int
09: main(int argc, char** argv)
10: @{
11: printf("Hello world.\n");
12: @}
@end example
Evaluating @code{(semantic-flex (point-min) (point-max))}
within the buffer with the code above returns the following token list.
The input line and string that produced each token is shown after
each semi-colon.
((punctuation 52 . 53) ; 04: #
(INCLUDE 53 . 60) ; 04: include
(punctuation 61 . 62) ; 04: <
(symbol 62 . 67) ; 04: stdio
(punctuation 67 . 68) ; 04: .
(symbol 68 . 69) ; 04: h
(punctuation 69 . 70) ; 04: >
(INT 72 . 75) ; 06: int
(symbol 76 . 79) ; 06: i_1
(punctuation 79 . 80) ; 06: ;
(INT 82 . 85) ; 08: int
(symbol 86 . 90) ; 08: main
(semantic-list 90 . 113) ; 08: (int argc, char** argv)
(semantic-list 114 . 147) ; 09-12: body of main function
@end example
As shown above, the token list is a list of ``tokens''.
Each token in turn is a list of the form
@end example
where TOKEN-TYPE is a symbol, and the other two are integers indicating
the buffer position that delimit the token such that
(buffer-substring BEGINNING-POSITION ENDING-POSITION)
@end lisp
would return the string form of the token.
Note that one line (line 4 above) can produce seven tokens while
the whole body of the function produces a single token.
This is because the @var{depth} parameter of @code{semantic-flex} was
not specified.
Let's see the output when @var{depth} is set to 1.
Evaluate @code{(semantic-flex (point-min) (point-max) 1)} in the same buffer.
Note the third argument of @code{1}.
((punctuation 52 . 53) ; 04: #
(INCLUDE 53 . 60) ; 04: include
(punctuation 61 . 62) ; 04: <
(symbol 62 . 67) ; 04: stdio
(punctuation 67 . 68) ; 04: .
(symbol 68 . 69) ; 04: h
(punctuation 69 . 70) ; 04: >
(INT 72 . 75) ; 06: int
(symbol 76 . 79) ; 06: i_1
(punctuation 79 . 80) ; 06: ;
(INT 82 . 85) ; 08: int
(symbol 86 . 90) ; 08: main
(open-paren 90 . 91) ; 08: (
(INT 91 . 94) ; 08: int
(symbol 95 . 99) ; 08: argc
(punctuation 99 . 100) ; 08: ,
(CHAR 101 . 105) ; 08: char
(punctuation 105 . 106) ; 08: *
(punctuation 106 . 107) ; 08: *
(symbol 108 . 112) ; 08: argv
(close-paren 112 . 113) ; 08: )
(open-paren 114 . 115) ; 10: @{
(symbol 120 . 126) ; 11: printf
(semantic-list 126 . 144) ; 11: ("Hello world.\n")
(punctuation 144 . 145) ; 11: ;
(close-paren 146 . 147) ; 12: @}
@end example
The @var{depth} parameter ``peeled away'' one more level of ``list''
delimited by matching parenthesis or braces.
The depth parameter can be specified to be any number.
However, the parser needs to be able to handle the extra tokens.
This is an interesting benefit of the lexer having the full
resources of Emacs at its disposal.
Skipping over matched parenthesis is achieved by simply calling
the built-in functions @code{forward-list} and @code{forward-sexp}.
All common token symbols are enumerated below. Additional token
symbols aside from these can be generated by the lexer if user option
@var{semantic-flex-extensions} is set. It is up to the user to add
matching extensions to the parser to deal with the lexer
extensions. An example use of @var{semantic-flex-extensions} is in
@file{semantic-make.el} where @var{semantic-flex-extensions} is set to
the value of @var{semantic-flex-make-extensions} which may generate
@code{shell-command} tokens.
@anchor{Default syntactic tokens}
@subsection Default syntactic tokens if the lexer is not extended.
@table @code
@item bol
Empty string matching a beginning of line.
This token is produced only if the user set
@var{semantic-flex-enable-bol} to non-@code{nil}.
@item charquote
String sequences that match @code{\\s\\+}.
@item close-paren
Characters that match @code{\\s)}.
These are typically @code{)}, @code{@}}, @code{]}, etc.
@item comment
A comment chunk.
These token types are not produced by default.
They are produced only if the user set
@var{semantic-ignore-comments} to @code{nil}.
@item newline
Characters matching @code{\\s-*\\(\n\\|\\s>\\)}.
This token is produced only if the user set
@var{semantic-flex-enable-newlines} to
@item open-paren
Characters that match @code{\\s(}.
These are typically @code{(}, @code{@{}, @code{[}, etc.
Note that these are not usually generated unless the @var{depth}
argument to @ref{semantic-flex} is greater than 0.
@item punctuation
Characters matching @code{\\(\\s.\\|\\s$\\|\\s'\\)}.
@item semantic-list
String delimited by matching parenthesis, braces, etc. that
the lexer skipped over, because the @var{depth} parameter
to @ref{semantic-flex} was not high enough.
@item string
Quoted strings, i.e., string sequences that start and end with
characters matching @code{\\s\"}.
The lexer relies on @code{forward-sexp} to find the
matching end.
@item symbol
String sequences that match @code{\\(\\sw\\|\\s_\\)+}.
@item whitespace
Characters that match `\\s-+' regexp.
This token is produced only if the user set
@var{semantic-flex-enable-whitespace} to non-@code{nil}. If
@var{semantic-ignore-comments} is non-@code{nil} too comments are
considered as whitespaces.
@end table
@node Lexer Options, Keywords, Lexer Output, Lexing
@section Lexer Options
Although most lexer functions are called for you by other semantic
functions, there are ways for you to extend or customize the lexer.
Three variables shown below serve this purpose.
@defvar semantic-flex-unterminated-syntax-end-function
Function called when unterminated syntax is encountered.
This should be set to one function. That function should take three
parameters. The @var{SYNTAX}, or type of syntax which is unterminated.
@var{SYNTAX-START} where the broken syntax begins.
@var{FLEX-END} is where the lexical analysis was asked to end.
This function can be used for languages that can intelligently fix up
broken syntax, or the exit lexical analysis via @dfn{throw} or @dfn{signal}
when finding unterminated syntax.
@end defvar
@defvar semantic-flex-extensions
Buffer local extensions to the lexical analyzer.
This should contain an alist with a key of a regex and a data element of
a function. The function should both move point, and return a lexical
token of the form:
@end example
@code{nil} is a valid return value.
@var{TYPE} can be any type of symbol, as long as it doesn't occur as a
nonterminal in the language definition.
@end defvar
@defvar semantic-flex-syntax-modifications
Changes the syntax table for a given buffer.
These changes are active only while the buffer is being flexed.
This is a list where each element has the form
@end example
CHAR is the char passed to `modify-syntax-entry',
and CLASS is the string also passed to `modify-syntax-entry' to define
what syntax class CHAR has.
(setq semantic-flex-syntax-modifications '((?. "_"))
@end example
This makes the period @asis{.} a symbol constituent. This may be necessary if
filenames are prevalent, such as in Makefiles.
@end defvar
@defvar semantic-flex-enable-newlines
When flexing, report @code{'newlines} as syntactic elements.
Useful for languages where the newline is a special case terminator.
Only set this on a per mode basis, not globally.
@end defvar
@defvar semantic-flex-enable-whitespace
When flexing, report @code{'whitespace} as syntactic elements.
Useful for languages where the syntax is whitespace dependent.
Only set this on a per mode basis, not globally.
@end defvar
@defvar semantic-flex-enable-bol
When flexing, report beginning of lines as syntactic elements.
Useful for languages like python which are indentation sensitive.
Only set this on a per mode basis, not globally.
@end defvar
@defvar semantic-number-expression
Regular expression for matching a number.
If this value is @code{nil}, no number extraction is done during lex.
This expression tries to match C and Java like numbers.
| <HEX_LITERAL>[lL]?
| <OCTAL_LITERAL>[lL]?
| [.][0-9]+<EXPONENT>?[fFdD]?
| [0-9]+<EXPONENT>[fFdD]?
| [0-9]+<EXPONENT>?[fFdD]
@end example
@end defvar
@node Keywords, Keyword Properties, Lexer Options, Lexing
@section Keywords
Another important piece of the lexer is the keyword table (see
@ref{Settings}). You language will want to set up a keyword table for
fast conversion of symbol strings to language terminals.
The keywords table can also be used to store additional information
about those keywords. The following programming functions can be useful
when examining text in a language buffer.
@defun semantic-flex-keyword-p text
Return non-@code{nil} if @var{TEXT} is a keyword in the keyword table.
@end defun
@defun semantic-flex-keyword-put text property value
For keyword @var{TEXT}, set @var{PROPERTY} to @var{VALUE}.
@end defun
@defun semantic-token-put-no-side-effect token key value
For @var{TOKEN}, put the property @var{KEY} on it with @var{VALUE} without side effects.
If @var{VALUE} is @code{nil}, then remove the property from @var{TOKEN}.
All cons cells in the property list are replicated so that there
are no side effects if @var{TOKEN} is in shared lists.
@end defun
@defun semantic-flex-keyword-get text property
For keyword @var{TEXT}, get the value of @var{PROPERTY}.
@end defun
@defun semantic-flex-map-keywords fun &optional property
Call function @var{FUN} on every semantic keyword.
If optional @var{PROPERTY} is non-nil, call @var{FUN} only on every
keyword which has a @var{PROPERTY} value.
@var{FUN} receives a semantic keyword as argument.
@end defun
@defun semantic-flex-keywords &optional property
Return a list of semantic keywords.
If optional @var{PROPERTY} is non-nil, return only keywords which have
@var{PROPERTY} set.
@end defun
Keyword properties can be set up in a BNF file for ease of maintenance.
While examining the text in a language buffer, this can provide an easy
and quick way of storing details about text in the buffer.
@node Keyword Properties, , Keywords, Lexing
@section Standard Keyword Properties
Add known properties here when they are known.
@node Bovinating, BNF conversion, Lexing, Top
@chapter Preparing a bovine table for your language
When converting a source file into a nonterminal token stream
(parse-tree) it is important to specify rules to accomplish this. The
rules are stored in the buffer local variable
While it is certainly possible to write this table yourself, it is most
likely you will want to use the BNF converter (see @xref{BNF conversion}.)
This is an easier method for specifying your rules. You will still need
to specify a variable in your language for the table, however. A good
rule of thumb is to call it @code{language-toplevel-bovine-table} if it
part of the language, or @code{semantic-toplevel-language-bovine-table}
if you donate it to the semantic package.
When initializing a major-mode for your language, you will set the
variable @code{semantic-toplevel-bovine-table} to the contents of your
language table. @code{semantic-toplevel-bovine-table} is always buffer
Since it is important to know the format of the table when debugging ,
you should still attempt to understand the basics of the table.
Please see the documentation for the variable
@code{semantic-toplevel-bovine-table} for details on its format.
* add more doc here *
@node BNF conversion, Compiling, Bovinating, Top
@chapter Using the BNF converter to make bovine tables
The BNF converter takes a file in "Bovine Normal Form" which is similar
to "Backus-Naur Form". If you have ever used yacc or bison, you will
find it similar. The BNF form used by semantic, however, does not
include token precedence rules, and several other features needed to make
real parser generators.
It is important to have an Emacs Lisp file with a variable ready to take
the output of your table (see @xref{Bovinating}.) Also, make sure that the
file @file{semantic-bnf.el} is loaded. Give your language file the
extension @file{.bnf} and you are ready.
The comment character is @asis{#}.
When you want to test your file, use the keyboard shortcut @kbd{C-c C-c}
to parse the file, generate the variable, and load the new definition
in. It will then use the settings specified above to determine what to
do. Use the shortcut @kbd{C-c c} to do the same thing, but spend
extra time indenting the table nicely.
Make sure that you create the variable specified in the
@code{%parsetable} token before trying to convert the BNF file. A
simple definition like this is sufficient.
(defvar semantic-toplevel-lang-bovine-table
"Table for use with semantic for parsing LANG.")
@end example
If you use tokens (created with the @code{%token} specifier), also
make sure you have a keyword table available, like this:
(defvar semantic-lang-keyword-table
"Table for use with semantic for keywords.")
@end example
Specify the name of the keyword table with the @code{%keywordtable}
The BNF file has two sections. The first is the settings section, and
the second is the language definition, or list of semantic rules.
* Settings:: Setup for a language
* Rules:: Create rules to parse a language
* Optional Lambda Expression:: Actions to take when a rule is matched
* Examples:: Simple Samples
* Style Guide:: What the tokens mean, and how to use them.
@end menu
@node Settings, Rules, BNF conversion, BNF conversion
@section Settings
A setting is a keyword starting with a @asis{%}. (This syntax is taken
from yacc and bison. @xref{(bison)}.)
There are several settings that can be made in the settings section.
They are:
@deffn Setting %start <nonterminal>
Specify an alternative to @code{bovine-toplevel}. (See below)
@end deffn
@deffn Setting %scopestart <nonterminal>
Specify an alternative to @code{bovine-inner-scope}.
@end deffn
@deffn Setting %outputfile <filename>
Required. Specifies the file into which this files output is stored.
@end deffn
@deffn Setting %parsetable <lisp-variable-name>
Required. Specifies a lisp variable into which the output is stored.
@end deffn
@deffn Setting %setupfunction <lisp-function-name>
Required. Name of a function into which setup code is to be inserted.
@end deffn
@deffn Setting %keywordtable <lisp-variable-name>
Required if there are @code{%token} keywords.
Specifies a lisp variable into which the output of a keyword table is
stored. This obarray is used to turn symbols into keywords when applicable.
@end deffn
@deffn Setting %token <name> "<text>"
Optional. Specify a new token @var{NAME}. This is added to a lexical
keyword list using @var{TEXT}. The symbol is then converted into a new
lexical terminal. This requires that the @code{%keywordtable} specified
variable is available in the file specified by @code{%outputfile}.
@end deffn
@deffn Setting %token <name> type "<text>"
Optional. Specify a new token @var{NAME}. It is made from an existing
lexical token of type @var{TYPE}. @var{TEXT} is a string which will be
matched explicitly. @var{NAME} can be used in match rules as though they were
flex tokens, but are converted back to @var{TYPE} "text" internally.
@end deffn
@deffn Setting %put <NAME> symbol <VALUE>
@deffnx Setting %put <NAME> ( symbol1 <VALUE1> symbol2 <VALUE2> ... )
@deffnx Setting %put ( <NAME1> <NAME2>...) symbol <VALUE>
Tokens created without a type are considered keywords, and placed in a
keyword table. Use @code{%put} to apply properties to that keyword.
(see @ref{Lexing}).
@end deffn
@deffn Setting %languagemode <lisp-function-name>
@deffnx Setting %languagemode ( <lisp-function-name1> <lisp-function-name2> ... )
Optional. Specifies the Emacs major mode associated with the language
being specified. When the language is converted, all buffers of this
mode will get the new table installed.
@end deffn
@deffn Setting %quotemode backquote
Optional. Specifies how symbol quoting is handled in the Optional
Lambda Expressions. (See below)
@end deffn
@deffn Setting {%( <lisp-expression> )%}
Specify setup code to be inserted into the @code{%setupfunction}.
It will be inserted between two specifier strings, or added to
the end of the function.
@end deffn
When working inside @code{%( ... )%} tokens, any lisp expression can be
entered which will be placed inside the setup function. In general, you
probably want to set variables that tell Semantic and related tools how
the language works.
Here are some variables that control how different programs will work
with your language.
@defvar semantic-flex-depth
Default flexing depth.
This specifies how many lists to create tokens in.
@end defvar
@defvar semantic-number-expression
Regular expression for matching a number.
If this value is @code{nil}, no number extraction is done during lex.
Symbols which match this expression are returned as @code{number}
tokens instead of @code{symbol} tokens.
The default value for this variable should work in most languages.
@end defvar
@defvar semantic-flex-extensions
Buffer local extensions to the lexical analyzer.
This should contain an alist with a key of a regex and a data element of
a function. The function should both move point, and return a lexical
token of the form:
@end example
@code{nil} is also a valid return.
@var{TYPE} can be any type of symbol, as long as it doesn't occur as a
nonterminal in the language definition.
@end defvar
@defvar semantic-flex-syntax-modifications
Updates to the syntax table for this buffer.
These changes are active only while this file is being flexed.
This is a list where each element is of the form:
@end example
Where @var{CHAR} is the char passed to @dfn{modify-syntax-entry},
and @var{CLASS} is the string also passed to @dfn{modify-syntax-entry} to define
what class of syntax @var{CHAR} is.
@end defvar
@defvar semantic-flex-enable-newlines
When flexing, report @code{'newlines} as syntactic elements.
Useful for languages where the newline is a special case terminator.
Only set this on a per mode basis, not globally.
@end defvar
@defvar semantic-ignore-comments
Default comment handling.
@code{t} means to strip comments when flexing. @code{Nil} means to keep comments
as part of the token stream.
@end defvar
@defvar semantic-symbol->name-assoc-list
Association between symbols returned, and a string.
The string is used to represent a group of objects of the given type.
It is sometimes useful for a language to use a different string
in place of the default, even though that language will still
return a symbol. For example, Java return's includes, but the
string can be replaced with @code{Imports}.
@end defvar
@defvar semantic-case-fold
Value for @code{case-fold-search} when parsing.
@end defvar
@defvar semantic-expand-nonterminal
Function to call for each nonterminal production.
Return a list of non-terminals derived from the first argument, or @code{nil}
if it does not need to be expanded.
Languages with compound definitions should use this function to expand
from one compound symbol into several. For example, in @var{C} the
int a, b;
@end example
is easily parsed into one token, but represents multiple variables. A
functions should be written which takes this compound token and turns
it into two tokens, one for @var{A}, and the other for @var{B}.
Within the language definition (the @file{.bnf} sources), it is often
useful to set the NAME slot of a token with a list of items that
distinguish each element in the compound definition.
This list can then be detected by the function set in
@code{semantic-expand-nonterminal} to create multiple tokens.
This function has one additional duty of managing the overlays created
by semantic. It is possible to use the single overlay in the compound
token for all your tokens, but this can pose problems identifying
all tokens covering a given definition.
Please see @file{semantic-java.el} for an example of managing overlays
when expanding a token into multiple definitions.
@end defvar
@defvar semantic-override-table
Buffer local semantic function overrides alist.
These overrides provide a hook for a `major-mode' to override specific
behaviors with respect to generated semantic toplevel nonterminals and
things that these non-terminals are useful for.
Each element must be of the form: (SYM . FUN)
where SYM is the symbol to override, and FUN is the function to
override it with.
Available override symbols:
@multitable {@code{abbreviate-nonterminal}} {(token & nosnarf)} {Find token in buffer}
@item SYMBOL @tab PARAMETERS @tab DESCRIPTION
@item find-dependency @tab (token) @tab Find the dependency file
@item find-nonterminal @tab (token & parent) @tab Find token in buffer.
@item find-documentation @tab (token & nosnarf) @tab Find doc comments.
@item abbreviate-nonterminal @tab (token & parent) @tab Return summary string.
@item summarize-nonterminal @tab (token & parent) @tab Return summary string.
@item prototype-nonterminal @tab (token) @tab Return a prototype string.
@item concise-prototype-nonterminal' @tab (tok & parent color) @tab Return a concise prototype string.
@item uml-abbreviate-nonterminal' @tab (tok & parent color) @tab Return a UML standard abbreviation string.
@item uml-prototype-nonterminal' @tab (tok & parent color) @tab Return a UML like prototype string.
@item uml-concise-prototype-nonterminal' @tab (tok & parent color) @tab Return a UML like concise prototype string.
@item prototype-file @tab (buffer) @tab Return a file in which prototypes are placed
@item nonterminal-children @tab (token) @tab Return first rate children. These are children which may contain overlays.
@item nonterminal-external-member-parent @tab (token) @tab Parent of TOKEN
@item nonterminal-external-member-p @tab (parent token) @tab Non nil if TOKEN has PARENT, but is not in PARENT.
@item nonterminal-external-member-children @tab (token & usedb) @tab Get all external children of TOKEN.
@item nonterminal-protection @tab (token & parent) @tab Return protection as a symbol.
@item nonterminal-abstract @tab (token & parent) @tab Return if TOKEN is abstract.
@item nonterminal-leaf @tab (token & parent) @tab Return if TOKEN is leaf.
@item nonterminal-static @tab (token & parent) @tab Return if TOKEN is static.
@item beginning-of-context @tab (& point) @tab Move to the beginning of the
current context.
@item end-of-context @tab (& point) @tab Move to the end of the
current context.
@item up-context @tab (& point) @tab Move up one context level.
@item get-local-variables @tab (& point) @tab Get local variables.
@item get-all-local-variables@tab (& point) @tab Get all local variables.
@item get-local-arguments @tab (& point) @tab Get arguments to this function.
@item end-of-command @tab @tab Move to the end of the current
@item beginning-of-command @tab @tab Move to the beginning of the
current command
@item ctxt-current-symbol @tab (& point) @tab List of related symbols.
@item ctxt-current-assignment@tab (& point) @tab Variable being assigned to.
@item ctxt-current-function @tab (& point) @tab Function being called at point.
@item ctxt-current-argument @tab (& point) @tab The index to the argument of
the current function the cursor
is in.
@end multitable
Parameters mean:
@table @code
@item &
Following parameters are optional
@item buffer
The buffer in which a token was found.
@item token
The nonterminal token we are doing stuff with
@item parent
If a TOKEN is stripped (of positional information) then this will be the
parent token which should have positional information in it.
@end table
@end defvar
@defvar semantic-type-relation-separator-character
Character strings used to separation a parent/child relationship.
This list of strings are used for displaying or finding separators
in variable field dereferencing. The first character will be used for
display. In @var{C}, a type field is separated like this: ``type.field''
thus, the character is a ``.''. In @var{C}, and additional value of ``->''
would be in the list, so that ``type->field'' could be found.
@end defvar
@defvar semantic-dependency-include-path
Defines the include path used when searching for files.
This should be a list of directories to search which is specific to
the file being included.
This variable can also be set to a single function. If it is a
function, it will be called with one arguments, the file to find as a
string, and it should return the full path to that file, or nil.
@end defvar
This configures Imenu to use semantic parsing.
@defvar imenu-create-index-function
The function to use for creating a buffer index.
It should be a function that takes no arguments and returns an index
of the current buffer as an alist.
Simple elements in the alist look like @samp{(INDEX-NAME . INDEX-POSITION)}.
Special elements look like @samp{(INDEX-NAME INDEX-POSITION FUNCTION ARGUMENTS...)}.
A nested sub-alist element looks like (INDEX-NAME SUB-ALIST).
The function @code{imenu--subalist-p} tests an element and returns t
if it is a sub-alist.
This function is called within a @code{save-excursion}.
The variable is buffer-local.
@end defvar
These are specific to the document tool.
@table @code
@item document-comment-start
Comment start string.
@item document-comment-line-prefix
Comment prefix string. Used at the beginning of each line.
@item document-comment-end
Comment end string.
@end table
@node Rules, Optional Lambda Expression, Settings, BNF conversion
@section Rules
Writing the rules should be very similar to bison for basic syntax.
Each rule is of the form
RESULT : MATCH1 (optional-lambda-expression)
| MATCH2 (optional-lambda-expression)
@end example
@var{RESULT} is a non-terminal, or a token synthesized in your grammar.
@var{MATCH} is a list of elements that are to be matched if @var{RESULT}
is to be made. The optional lambda expression is a list containing
simplified rules for concocting the parse tree.
In bison, each time an element of a @var{MATCH} is found, it is
"shifted" onto the parser stack. (The stack of matched elements.) When
all of @var{MATCH1}'s elements have been matched, it is "reduced" to
@var{RESULT}. @xref{(bison)Algorithm}.
The first @var{RESULT} written into your language specification should
be @code{bovine-toplevel}, or the symbol specified with @code{%start}.
When starting a parse for a file, this is the default token iterated
over. You can use any token you want in place of @code{bovine-toplevel}
if you specify what that nonterminal will be with a @code{%start} token
in the settings section.
@var{MATCH} is made up of symbols and strings. A symbol such as
@code{foo} means that a syntactic token of type @code{foo} must be
matched. A string in the mix means that the previous symbol must have
the additional constraint of exactly matching it. Thus, the
symbol "moose"
@end example
means that a symbol must first be encountered, and then it must
@code{string-match "moose"}. Be especially careful to remember that the
string is a regular expression. The code:
punctuation "."
@end example
will match any punctuation.
For the above example in bison, a LEX rule would be used to create a new
token @var{MOOSE}. In this case, the @var{MOOSE} token would appear.
For the bovinator, this task was mixed into the language definition to
simplify implementation, though Bison's technique is more efficient.
To make a symbol match explicitly for keywords, for example, you can use
the @code{%token} command in the settings section to create new symbols.
%token MOOSE "moose"
find_a_moose: MOOSE
@end example
will match ``moose'' explicitly, unlike the previous example where moose
need only appear in the symbol. This is because ``moose'' will be
converted to @var{MOOSE} in the lexical analysis stage. Thus the symbol
@var{MOOSE} won't be available any other way.
If we specify our token in this way:
%token MOOSE symbol "moose"
find_a_moose: MOOSE
@end example
then @code{MOOSE} will match the string "moose" explicitly, but it won't
do so at the lexical level, allowing use of the text "moose" in other
forms of regular expressions.
Non symbol tokens are also allowed. For example:
%token PERIOD punctuation "."
filename : symbol PERIOD symbol
@end example
will explicitly match one period when used in the above rule.
@xref{Default syntactic tokens}.
@node Optional Lambda Expression, Examples, Rules, BNF conversion
@section Optional Lambda Expressions
The OLE (Optional Lambda Expression) is converted into a bovine lambda
(see @xref{Bovinating}.) This lambda has special short-cuts to simplify
reading the Emacs BNF definition. An OLE like this:
( $1 )
@end example
results in a lambda return which consists entirely of the string
or object found by matching the first (zeroth) element of match.
An OLE like this:
( ,(foo $1) )
@end example
executes `foo' on the first argument, and then splices its return
into the return list whereas:
( (foo $1) )
@end example
executes foo, and that is placed in the return list.
Here are other things that can appear inline:
@table @code
@item $1
the first object matched.
@item ,$1
the first object spliced into the list (assuming it is a list from a
@item '$1
the first object matched, placed in a list. i.e. ( $1 )
@item foo
the symbol foo (exactly as displayed)
@item (foo)
a function call to foo which is stuck into the return list.
@item ,(foo)
a function call to foo which is spliced into the return list.
@item '(foo)
a function call to foo which is stuck into the return list in a list.
@item (EXPAND $1 nonterminal depth)
a list starting with EXPAND performs a recursive parse on the token
passed to it (represented by $1 above.) The semantic list is a common
token to expand, as there are often interesting things in the list.
The @var{nonterminal} is a symbol in your table which the bovinator will
start with when parsing. @var{nonterminal}'s definition is the same as
any other nonterminal. @var{depth} should be at least 1 when
descending into a semantic list.
@item (EXPANDFULL $1 nonterminal depth)
is like EXPAND, except that the parser will iterate over
@var{nonterminal} until there are no more matches. (The same way the
parser iterates over @code{bovine-toplevel}. This lets you have
much simpler rules in this specific case, and also lets you have
positional information in the returned tokens, and error skipping.
@item (ASSOC symbol1 value1 symbol2 value2 ... )
This is used for creating an association list. Each @var{SYMBOL} is
included in the list if the associated @var{VALUE} is non-nil. While
the items are all listed explicitly, the created structure is an
association list of the form:
( ( symbol1 . value1) (symbol2 . value2) ... )
@end example
@end table
If the symbol @code{%quotemode backquote} is specified, then use
@code{,@@} to splice a list in, and @code{,} to evaluate the expression.
This lets you send @code{$1} as a symbol into a list instead of having
it expanded inline.
@node Examples, Style Guide , Optional Lambda Expression, BNF conversion
@section Examples
The rule:
SYMBOL : symbol
@end example
is equivalent to
SYMBOL : symbol
( $1 )
@end example
which, if it matched the string "A", would return
@end example
If this rule were used like this:
ASSIGN: SYMBOL punctuation "=" SYMBOL
( $1 $3 )
@end example
it would match "A=B", and return
@end example
The letters A and B come back in lists because SYMBOL is a nonterminal,
not an actual lexical element.
to get a better result with nonterminals, use @asis{,} to splice lists
in like this;
ASSIGN: SYMBOL punctuation "=" SYMBOL
( ,$1 ,$3 )
@end example
which would return
@end example
@node Style Guide , , Examples, BNF conversion
@section Semantic Token Style Guide
In order for a generalized program using Semantic to work with
multiple languages, it is important to have a consistent meaning for
the contents of the tokens returned. The variable
@code{semantic-toplevel-bovine-table} is documented with the complete
list of a tokens that a functional or OO language may use. While any
given language is free to create their own tokens, such a language
definition would not produce a stream of tokens usable by a
generalized tool.
@section Minimum Requirements
In general, all tokens returned from a parser should be generated with
the following form:
("NAME" type-symbol ... "DOCSTRING" PROPERTIES OVERLAY)
@end example
@var{NAME} and @var{type-symbol} are the only syntactic elements of a
nonterminal which are guaranteed to exist. This means that a parser
which uses @code{nil} for either of these two slots, or some value
which is not type consistent is wrong.
@var{NAME} is also guaranteed to be a string. This string represents
the name of the nonterminal, usually a named definition which the
language will use elsewhere as a reference to the syntactic element
@var{type-symbol} is a symbol representing the type of the
nonterminal. Valid @var{type-symbol}s can be anything, as long is it
is an Emacs Lisp symbol.
@var{DOCSTRING} is a required slot in the nonterminal, but can be
nil. Some languages have the documentation saved as a comment
nearby. In these cases, DOCSTRING is nil, and the function
@var{PROPERTIES} is a slot generated by the semantic parser harness,
and need not be provided by a language author. Programmatically access
nonterminal properties with @code{semantic-token-put} and
@code{semantic-token-get} to access properties.
@var{OVERLAY} represents positional information for this token. It is
automatically generated by the semantic parser harness, and need not
be provided by the language author, unless they provide a nonterminal
expansion function via @code{semantic-expand-nonterminal}.
The @var{OVERLAY} property is accessed via several functions returning
the beginning, end, and buffer of a token. Use these functions unless
the overlay is really needed (see @ref{Token Queries}). Depending on the
overlay in a program can be dangerous because sometimes the overlay is
replaced with an integer pair
@end example
when the buffer the token belongs to is not in memory. This happens
when a using has activated the Semantic Database @ref{semanticdb}.
@section Nonterminals for Functional Languages.
If a parser produces tokens for a functional language, then the
following token formats are available.
@table @asis
@item Variable
@itemx @code{("NAME" variable "TYPE" DEFAULT-VALUE EXTRA-SPEC}
@itemx @code{ "DOCSTRING" PROPERTIES OVERLAY)}
@var{TYPE} is a string representing the type of this variable.
@var{TYPE} can be @code{nil} for untyped languages. Languages which
support variable declarations without a type (Such as C) should supply
a string representing the default type for that language.
@var{DEFAULT-VALUE} can be a string, or something pre-parsed and
language specific. Hopefully this slot will be better defined in
future versions of Semantic.
@var{EXTRA-SPEC} are extra specifiers. See below.
@item Function
@itemx @code{ ("NAME" function "TYPE" ( ARG-LIST ) EXTRA-SPEC}
@itemx @code{ "DOCSTRING" PROPERTIES OVERLAY)}
@var{TYPE} is a string representing the return type of this function
or method. @var{type} can be @code{nil} for untyped languages, or for
procedures in languages which support functions with no return data.
See above for more.
@var{ARG-LIST} is a list of arguments passed to this function.
Each element in the arg list can be one of the following:
@table @asis
@item Semantic Token
A full semantic token with positional information.
@item A partial semantic token
Partial tokens may contain the @var{NAME} slot, @var{token-symbol},
and possibly a @var{TYPE}.
@item String
A string representing the name of the argument. Common in untyped
@end table
@item Type Declaration
@itemx @code{ ("NAME" type "TYPE" ( PART-LIST ) ( PARENTS ) EXTRA-SPEC}
@itemx @code{ "DOCSTRING" PROPERTIES OVERLAY)}
@var{TYPE} a string representing the of the type, such as (in C)
``struct'', ``union'', ``enum'', ``typedef'', or ``class''.
The @var{TYPE} for a type token should not be nil, as even untyped
languages with structures have type types.
@var{PART-LIST} is the list of individual entries inside compound
types. Structures, for example, can contain several fields which can
be represented as variables. Valid entries in a @var{PART-LIST} are:
@table @asis
@item Semantic Token
A full semantic token with positional information.
@item A partial semantic token
Partial tokens may contain the @var{NAME} slot, @var{token-symbol},
and possibly a @var{TYPE}.
@item String
A string representing the name of the slot or field. Common in untyped
@end table
@var{PARENTS} represents a list of parents of this type. Parents are used
in two situations.
@table @asis
@item Inheritance
For types which inherit from other types of the same type-type (Such
as classes).
@item Aliases
For types which are aliases of other types, the parent type is the
type being aliased. The Types' type is the command specifying that it
is an alias (Such as ``typedef'' in C or C++).
@end table
The structure of the @var{PARENTS} list is of this form:
@end example
@var{EXPLICIT-PARENTS} can be a single string (Just one parent) or a
list of parents (in a multiple inheritance situation. It can also be
@var{INTERFACE-PARENTS} is a list of strings representing the names of
all INTERFACES, or abstract classes inherited from. It can also be
This slot can be interesting because the form:
( nil "string")
@end example
is a valid parent where there is no explicit parent, and only an
@item Include files
@itemx @code{("FILE" include SYSTEM "DOCSTRING" PROPERTIES OVERLAY)}
A statement which gets additional definitions from outside the current
file, such as an @code{#include} statement in C.
In this case, instead of @var{NAME}, a @var{FILE} is specified.
@var{FILE} can be a subset of the actual file to be loaded.
@var{SYSTEM} is true if this include is part of a set of system
includes. This field isn't currently being used and may be
@item Package & Provide statements
@itemx @code{("NAME" package DETAIL "DOCSTRING" PROPERTIES OVERLAY)}
A statement which declares a given file is part of a package, such as
the Java @code{package} statement, or a @code{provide} in Emacs Lisp.
@var{DETAIL} might be an associated file name, or some other language
specific bit of information.
@end table
@section Extra Specifiers
Some default token types have a slot EXTRA-SPEC, for extra specifiers.
These specifiers provide additional details not commonly used, or not
available in all languages. This list is an alist, and if a given key
is nil, it is not in the list, saving space. Some valid extra
specifiers are:
@table @code
@item (parent . "text")
Name of a parent type/class. This is not the same as a parent for a
type. In C++ and CLOS allow the creation of a function outside the
body of that class. Such functions will set the @var{parent}
specifier to a plain text string which is the name of that parent.
@item (dereference . INT)
Number of levels of dereference.
In C, the number of array dimensions.
@item (pointer . INT)
Number of levels of pointers.
In C, the number of @code{*} characters.
@item (typemodifiers . ( "text" ... ))
Keyword modifiers for a type. In C, such words would include
@code{register'} and @code{volatile'}
@item (suffix . "text")
Suffix information for a variable. Not currently used.
@item (const . t)
This exists if the variable or function return value is constant.
@item (throws . ( "text" ... ))
For functions or methods in languages that support typed signal
throwing, this is a list of exceptions that can be thrown.
@item (destructor . t)
This exists for functions which are destructor methods in a class
definition. In C++, a destructor's name excludes the ~ character. When
producing the name of the function, the ~ is added back in.
@item (constructor . t)
This exists for functions which are constructors in a class
definition. In C++ this is t when the name of this function is
the same as the name of the parent class.
@item (user-visible . t)
For functions in interpreted languages such as Emacs Lisp, this
signals that a function or variable is user visible. In Emacs Lisp,
this means a function is @dfn{interactive}.
@item (prototype . t)
For functions or variables that are not declared locally, a prototype
is something that will define that function or variable for use.
In C, the term represents prototypes generally used in header files.
In Emacs Lisp, the @code{autoload} statement creates prototypes.
@end table
@node Compiling, Debugging, BNF conversion, Top
@chapter Compiling a language file with the bovinator
From a program you can use the function @code{semantic-bovinate-toplevel}.
This function takes one optional parameter specifying if the cache
should be refreshed. By default, the cached results of the last parse
are always used. Specifying that the cache should be checked will cause
it to be flushed if it is out of date.
Another function you can use is @code{semantic-bovinate-nonterminal}.
This command takes a token stream returned by the function
@code{semantic-flex} followed by a DEPTH (as above). This takes an
additional optional argument of NONTERMINAL which is the nonterminal in
your table it is to start parsing with.
@deffn Command bovinate &optional clear
Bovinate the current buffer. Show output in a temp buffer.
Optional argument @var{CLEAR} will clear the cache before bovinating.
@end deffn
@deffn Command semantic-clear-toplevel-cache
Clear the toplevel bovine cache for the current buffer.
Clearing the cache will force a complete reparse next time a token
stream is requested.
@end deffn
@defun semantic-bovinate-toplevel &optional checkcache
Bovinate the entire current buffer.
If the optional argument @var{CHECKCACHE} is non-@code{nil}, then flush the cache iff
there has been a size change.
@end defun
@node Debugging, Programming, Compiling, Top
@comment node-name, next, previous, up
@chapter Debugging
Writing language files using BNF is significantly easier than writing
then using regular expressions in a functional manner. Debugging
them, however, can still prove challenging.
There are two ways to debug a language definition if it is not
behaving as expected. One way is to debug against the source @file{.bnf}
file. The second is to debug against the lisp table created from the
@file{.bnf} source, or perhaps written by hand.
If your language definition was written in BNF notation, debugging is
quite easy. The command @code{bovinate-debug} will start you off.
@deffn Command bovinate-debug
Bovinate the current buffer and run in debug mode.
@end deffn
If you prefer debugging against the Lisp table, find the table in a
buffer, place the cursor in it, and use the command
@code{semantic-bovinate-debug-set-table} in it.
@deffn Command semantic-bovinate-debug-set-table &optional clear
Set the table for the next debug to be here.
Optional argument @var{CLEAR} to unset the debug table.
@end deffn
After the table is set, the @code{bovinate-debug} command can be run
at any time for the given language.
While debugging, two windows are visible. One window shows the file
being parsed, and the syntactic token being tested is highlighted. The
second window shows the table being used (either in the BNF source, or
the Lisp table) with the current rule highlighted. The cursor will
sit on the specific match rule being tested against.
In the minibuffer, a brief summary of the current situation is
listed. The first element is the syntactic token which is a list of
the form:
@end example
The rest of the display is a list of all strings collected for the
currently tested rule. Each time a new rule is entered, the list is
restarted. Upon returning from a rule into a previous match list, the
previous match list is restored, with the production of the dependent
rule in the list.
Use @kbd{C-g} to stop debugging. There are no commands for any
fancier types of debugging.
@node Programming, Current Context, Debugging, Top
@comment node-name, next, previous, up
@chapter Programming
Once a source file has been parsed, the following APIs can be used to
write programs that use the token stream most effectively.
* Token Queries:: Getting info about a parsed token (nonterminal).
* Nonterminal Streams:: Working with streams of nonterminals.
* Nonterminals at point:: Finding nonterminals at point.
* Nonterminal Sorting:: Reorganizing streams.
* Nonterminal Completion:: Completing read functions.
* Override Methods:: Language dependent functions covering
conversion to text strings, language dependent
queries and local context information
* Parser Hooks:: How to know when tags change.
* Example Programs:: Simple programming examples.
@end menu
@node Token Queries, Nonterminal Streams, Programming, Programming
@comment node-name, next, previous, up
@section Token Queries
When writing programs that use the bovinator, the following functions
are needed to find get details out of a nonterminal.
@defun semantic-equivalent-tokens-p token1 token2
Compare @var{TOKEN1} and @var{TOKEN2} and return non-@code{nil} if they are equivalent.
Use @dfn{eq} to test of two tokens are the same. Use this function if tokens
are being copied and regrouped to test for if two tokens represent the same
thing, but may be constructed of different cons cells.
@end defun
@defun semantic-token-token token
Retrieve from @var{TOKEN} the token identifier.
i.e., the symbol @code{'variable}, @code{'function}, @code{'type}, or other.
@end defun
@defun semantic-token-name token
Retrieve the name of @var{TOKEN}.
@end defun
@defun semantic-token-docstring token &optional buffer
Retrieve the documentation of @var{TOKEN}.
Optional argument @var{BUFFER} indicates where to get the text from.
If not provided, then only the @var{POSITION} can be provided.
@end defun
@defun semantic-token-overlay token
Retrieve the @var{OVERLAY} part of @var{TOKEN}.
The returned item may be an overlay or an unloaded buffer representation.
@end defun
@defun semantic-token-extent token
Retrieve the extent (@var{START} @var{END}) of @var{TOKEN}.
@end defun
@defun semantic-token-start token
Retrieve the start location of @var{TOKEN}.
@end defun
@defun semantic-token-end token
Retrieve the end location of @var{TOKEN}.
@end defun
@defun semantic-token-type token
Retrieve the type of @var{TOKEN}.
@end defun
@defun semantic-token-put token property value
On @var{token}, set @var{property} to @var{value}.
@end defun
@defun semantic-token-get token property
For @var{token} get the value of @var{property}.
@end defun
@defun semantic-token-extra-spec token spec
Retrieve a specifier for the variable @var{TOKEN}.
@var{SPC} is the symbol whose modifier value to get.
This function can get specifiers from any type of @var{TOKEN}.
Do not use this function if you know what type of token you are dereferencing.
Instead, use the function specific to that token type. It will be faster.
@end defun
@defun semantic-token-type-parts token
Retrieve the parts of the type @var{TOKEN}.
@end defun
@defun semantic-token-type-parent token
Retrieve the parent of the type @var{TOKEN}.
The return value is a list. @var{A} value of @code{nil} means no parents.
The @dfn{car} of the list is either the parent class, or a list
of parent classes. The @dfn{cdr} of the list is the list of
interfaces, or abstract classes which are parents of @var{TOKEN}.
@end defun
@defun semantic-token-type-parent-superclass token
Retrieve the parent super classes of type type @var{TOKEN}.
@end defun
@defun semantic-token-type-parent-implement token
Retrieve the parent interfaces of type type @var{TOKEN}.
@end defun
@defun semantic-token-type-modifiers token
Retrieve the type modifiers for the type @var{TOKEN}.
@end defun
@defun semantic-token-type-extra-specs token
Retrieve the extra specifiers for the type @var{TOKEN}.
@end defun
@defun semantic-token-type-extra-spec token spec
Retrieve a extra specifier for the type @var{TOKEN}.
@var{SPEC} is the symbol whose modifier value to get.
@end defun
@defun semantic-token-function-args token
Retrieve the arguments of the function @var{TOKEN}.
@end defun
@defun semantic-token-function-modifiers token
Retrieve the type modifiers of the function @var{TOKEN}.
@end defun
@defun semantic-token-function-destructor token
Non-@code{nil} if @var{TOKEN} is a destructor function.
@end defun
@defun semantic-token-function-extra-specs token
Retrieve the extra specifiers of the function @var{TOKEN}.
@end defun
@defun semantic-token-function-extra-spec token spec
Retrieve a specifier for the function @var{TOKEN}.
@var{SPEC} is a symbol whose specifier value to get.
@end defun
@defun semantic-token-function-throws token
Retrieve the throws signal of the function @var{TOKEN}.
This is an optional field, and returns @code{nil} if it doesn't exist.
@end defun
@defun semantic-token-function-parent token
The parent of the function @var{TOKEN}.
A function has a parent if it is a method of a class, and if the
function does not appear in body of its parent class.
@end defun
@defun semantic-token-variable-const token
Retrieve the status of constantness from the variable @var{TOKEN}.
@end defun
@defun semantic-token-variable-default token
Retrieve the default value of the variable @var{TOKEN}.
@end defun
@defun semantic-token-variable-modifiers token
Retrieve type modifiers for the variable @var{TOKEN}.
@end defun
@defun semantic-token-variable-extra-specs token
Retrieve extra specifiers for the variable @var{TOKEN}.
@end defun
@defun semantic-token-variable-extra-spec token spec
Retrieve a specifier value for the variable @var{TOKEN}.
@var{SPEC} is the symbol whose specifier value to get.
@end defun
@defun semantic-token-include-system token
Retrieve the flag indicating if the include @var{TOKEN} is a system include.
@end defun
For override methods that query a token, see @xref{Token Details}.
@node Nonterminal Streams, Nonterminals at point, Token Queries, Programming
@section Nonterminal streams
These functions take some key, and returns information found inside
the nonterminal stream. Some will return one token (the first
matching item found.) Others will return a list of all items matching
a given criterion. All these functions work regardless of a buffer
being in memory or not.
@defun semantic-find-nonterminal-by-name name streamorbuffer &optional search-parts search-include
Find a nonterminal @var{NAME} within @var{STREAMORBUFFER}. @var{NAME} is a string.
If @var{SEARCH-PARTS} is non-@code{nil}, search children of tokens.
If @var{SEARCH-INCLUDE} is non-@code{nil}, search include files.
@end defun
@defun semantic-find-nonterminal-by-property property value streamorbuffer &optional search-parts search-includes
Find all nonterminals with @var{PROPERTY} equal to @var{VALUE} in @var{STREAMORBUFFER}.
Properties can be added with @dfn{semantic-token-put}.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDES} are passed to
@end defun
@defun semantic-find-nonterminal-by-extra-spec spec streamorbuffer &optional search-parts search-includes
Find all nonterminals with a given @var{SPEC} in @var{STREAMORBUFFER}.
@var{SPEC} is a symbol key into the modifiers association list.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDES} are passed to
@end defun
@defun semantic-find-nonterminal-by-extra-spec-value spec value streamorbuffer &optional search-parts search-includes
Find all nonterminals with a given @var{SPEC} equal to @var{VALUE} in @var{STREAMORBUFFER}.
@var{SPEC} is a symbol key into the modifiers association list.
@var{VALUE} is the value that @var{SPEC} should match.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDES} are passed to
@end defun
@defun semantic-find-nonterminal-by-position position streamorbuffer &optional nomedian
Find a nonterminal covering @var{POSITION} within @var{STREAMORBUFFER}.
@var{POSITION} is a number, or marker. If @var{NOMEDIAN} is non-@code{nil}, don't do
the median calculation, and return nil.
@end defun
@defun semantic-find-innermost-nonterminal-by-position position streamorbuffer &optional nomedian
Find a list of nonterminals covering @var{POSITION} within @var{STREAMORBUFFER}.
@var{POSITION} is a number, or marker. If @var{NOMEDIAN} is non-@code{nil}, don't do
the median calculation, and return nil.
This function will find the topmost item, and recurse until no more
details are available of findable.
@end defun
@defun semantic-find-nonterminal-by-token token streamorbuffer &optional search-parts search-includes
Find all nonterminals with a token @var{TOKEN} within @var{STREAMORBUFFER}.
@var{TOKEN} is a symbol representing the type of the tokens to find.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDE} are passed to
@end defun
@defun semantic-find-nonterminal-standard streamorbuffer &optional search-parts search-includes
Find all nonterminals in @var{STREAMORBUFFER} which define simple token types.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDE} are passed to
@end defun
@defun semantic-find-nonterminal-by-type type streamorbuffer &optional search-parts search-includes
Find all nonterminals with type @var{TYPE} within @var{STREAMORBUFFER}.
@var{TYPE} is a string which is the name of the type of the token returned.
Optional argument @var{SEARCH-PARTS} and @var{SEARCH-INCLUDES} are passed to
@end defun
@defun semantic-find-nonterminal-by-function function streamorbuffer &optional search-parts search-includes
Find all nonterminals in which @var{FUNCTION} match within @var{STREAMORBUFFER}.
@var{FUNCTION} must return non-@code{nil} if an element of @var{STREAM} will be included
in the new list.
If optional argument @var{SEARCH-PARTS} is non-@code{nil}, all sub-parts of tokens
are searched. The over-loadable function @dfn{semantic-nonterminal-children} is
used for the searching child lists. If @var{SEARCH-PARTS} is the symbol
@code{'positiononly}, then only children that have positional information are
If @var{SEARCH-INCLUDES} is non-@code{nil}, then all include files are also
searched for matches.
@end defun
@defun semantic-find-nonterminal-by-function-first-match function streamorbuffer &optional search-parts search-includes
Find the first nonterminal which @var{FUNCTION} match within @var{STREAMORBUFFER}.
@var{FUNCTION} must return non-@code{nil} if an element of @var{STREAM} will be included
in the new list.
If optional argument @var{SEARCH-PARTS}, all sub-parts of tokens are searched.
The over-loadable function @dfn{semantic-nonterminal-children} is used for
If @var{SEARCH-INCLUDES} is non-@code{nil}, then all include files are also
searched for matches.
@end defun
@defun semantic-recursive-find-nonterminal-by-name name buffer
Recursively find the first occurrence of @var{NAME}.
Start search with @var{BUFFER}. Recurse through all dependencies till found.
The return item is of the form (@var{BUFFER} @var{TOKEN}) where @var{BUFFER} is the buffer
in which @var{TOKEN} (the token found to match @var{NAME}) was found.
@end defun
@node Nonterminals at point, Nonterminal Sorting, Nonterminal Streams, Programming
@comment node-name, next, previous, up
@section Nonterminals at point
When you just want to get at a nonterminal the cursor is on, there is
a more efficient mechanism than using
@code{semantic-find-nonterminal-by-position}. This mechanism
directly queries the overlays the parsing step leaves in the buffer.
This provides for very rapid retrieval of what function or variable
the cursor is currently in.
These functions query the current buffer's overlay system for tokens.
@defun semantic-find-nonterminal-by-overlay &optional positionormarker buffer
Find all nonterminals covering @var{POSITIONORMARKER} by using overlays.
If @var{POSITIONORMARKER} is @code{nil}, use the current point.
Optional @var{BUFFER} is used if @var{POSITIONORMARKER} is a number, otherwise the current
buffer is used. This finds all tokens covering the specified position
by checking for all overlays covering the current spot. They are then sorted
from largest to smallest via the start location.
@end defun
@defun semantic-find-nonterminal-by-overlay-in-region start end &optional buffer
Find all nonterminals which exist in whole or in part between @var{START} and @var{END}.
Uses overlays to determine position.
Optional @var{BUFFER} argument specifies the buffer to use.
@end defun
@defun semantic-current-nonterminal
Return the current nonterminal in the current buffer.
If there are more than one in the same location, return the
smallest token.
@end defun
@defun semantic-current-nonterminal-parent
Return the current nonterminals parent in the current buffer.
@var{A} token's parent would be a containing structure, such as a type
containing a field. Return @code{nil} if there is no parent.
@end defun
@node Nonterminal Sorting, Nonterminal Completion, Nonterminals at point, Programming
@section Nonterminal sorting
Sometimes it is important to reorganize a token stream into a form
that is better for display to a user. It is important to not use
functions with side effects when doing this, and that could effect
the token cache.
There are some existing utility functions which will reorganize the
token list for you.
@defun semantic-bucketize tokens &optional parent filter
Sort @var{TOKENS} into a group of buckets based on token type.
Unknown types are placed in a Misc bucket.
Type bucket names are defined by either `semantic-symbol->name-assoc-list'.
If @var{PARENT} is specified, then @var{TOKENS} belong to this @var{PARENT} in some way.
This will use `semantic-symbol->name-assoc-list-for-type-parts' to
generate bucket names.
Optional argument @var{FILTER} is a filter function to be applied to each bucket.
The filter function will take one argument, which is a list of tokens, and
may re-organize the list with side-effects.
@end defun
@defvar semantic-bucketize-token-token
Function used to get a symbol describing the class of a token.
This function must take one argument of a semantic token.
It should return a symbol found in `semantic-symbol->name-assoc-list'
which @dfn{semantic-bucketize} uses to bin up tokens.
To create new bins for an application augment
`semantic-symbol->name-assoc-list', and
`semantic-symbol->name-assoc-list-for-type-parts' in addition
to setting this variable (locally in your function).
@end defvar
@defun semantic-adopt-external-members tokens
Rebuild @var{TOKENS} so that externally defined members are regrouped.
Some languages such as @var{C}++ and @var{CLOS} permit the declaration of member
functions outside the definition of the class. It is easier to study
the structure of a program when such methods are grouped together
more logically.
This function uses @dfn{semantic-nonterminal-external-member-p} to
determine when a potential child is an externally defined member.
Note: Applications which use this function must account for token
types which do not have a position, but have children which *do*
have positions.
Applications should use @code{semantic-mark-external-member-function}
to modify all tokens which are found as externally defined to some
type. For example, changing the token type for generating extra
buckets with the bucket function.
@end defun
@defvar semantic-orphaned-member-metaparent-type
In @dfn{semantic-adopt-external-members}, the type of @code{'type} for metaparents.
@var{A} metaparent is a made-up type semantic token used to hold the child list
of orphaned members of a named type.
@end defvar
@defvar semantic-mark-external-member-function
Function called when an externally defined orphan is found.
Be default, the token is always marked with the @code{adopted} property.
This function should be locally bound by a program that needs
to add additional behaviors into the token list.
This function is called with one argument which is a shallow copy
of the token to be modified. This function should return the
token (or a copy of it) which is then integrated into the
revised token list.
@end defvar
@node Nonterminal Completion, Override Methods, Nonterminal Sorting, Programming
@section Nonterminal completion
These functions provide ways reading the names of items in a buffer with
@defun semantic-read-symbol prompt &optional default stream filter
Read a symbol name from the user for the current buffer.
@var{PROMPT} is the prompt to use.
Optional arguments:
@var{DEFAULT} is the default choice. If no default is given, one is read
from under point.
@var{STREAM} is the list of tokens to complete from.
@var{FILTER} is provides a filter on the types of things to complete.
@var{FILTER} must be a function to call on each element. (See !!!
@end defun
@defun semantic-read-variable prompt &optional default stream
Read a variable name from the user for the current buffer.
@var{PROMPT} is the prompt to use.
Optional arguments:
@var{DEFAULT} is the default choice. If no default is given, one is read
from under point.
@var{STREAM} is the list of tokens to complete from.
@end defun
@defun semantic-read-function prompt &optional default stream
Read a function name from the user for the current buffer.
@var{PROMPT} is the prompt to use.
Optional arguments:
@var{DEFAULT} is the default choice. If no default is given, one is read
from under point.
@var{STREAM} is the list of tokens to complete from.
@end defun
@defun semantic-read-type prompt &optional default stream
Read a type name from the user for the current buffer.
@var{PROMPT} is the prompt to use.
Optional arguments:
@var{DEFAULT} is the default choice. If no default is given, one is read
from under point.
@var{STREAM} is the list of tokens to complete from.
@end defun
@node Override Methods, Parser Hooks, Nonterminal Completion, Programming
@section Override Methods
These functions are called `override methods' because they provide
generic behaviors, which a given language can override. For example,
finding a dependency file in Emacs lisp can be done with the
`locate-library' command (which overrides the default behavior.) In C,
a dependency can be found by searching a generic search path which can
be passed in via a variable.
* Token->Text:: Converting Tokens into text strings
* Token Details:: Arbitrary token detail fetching
* Local Context:: Deriving information about a language specific local context.
* Making New Methods:: How to add your own methods for a tool
@end menu
@node Token->Text, Token Details, Override Methods, Override Methods
@subsection Token->Text
Any given token consists of Meta information which is best viewed in
some textual form. This could be as simple as the token's name, or as
a prototype to be added to header file in C. Not only are there
several default converters from a Token into text, but there is also
some convenient variables that can be used with them. Use these
variables to allow options on output forms when displaying tokens in
your programs.
@defvar semantic-token->text-functions
List of functions which convert a token to text.
Each function must take the parameters @var{TOKEN} &optional @var{PARENT} @var{COLOR}.
@var{TOKEN} is the token to convert.
@var{PARENT} is a parent token or name which refers to the structure
or class which contains @var{TOKEN}. @var{PARENT} is @var{NOT} a class which a @var{TOKEN}
would claim as a parent.
@var{COLOR} indicates that the generated text should be colored using
@end defvar
@defvar semantic-token->text-custom-list
@var{A} List used by customizable variables to choose a token to text function.
Use this variable in the @code{:type} field of a customizable variable.
@end defvar
Every token to text conversion function must take the same parameters,
which are @var{TOKEN}, the token to be converted, @var{PARENT}, the
containing parent (like a structure which contains a variable), and
@var{COLOR}, which is a flag specifying that color should be applied
to the returned string.
When creating, or using these strings, particularly with color, use
@dfn{concat} to build up larger strings instead of @dfn{format}. This
will preserve text properties.
@defun semantic-name-nonterminal token &optional parent color
Return the name string describing @var{TOKEN}.
The name is the shortest possible representation.
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@defun semantic-summarize-nonterminal token &optional parent color
Summarize @var{TOKEN} in a reasonable way.
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@defun semantic-prototype-nonterminal token &optional parent color
Return a prototype for @var{TOKEN}.
This function should be overloaded, though it need not be used.
This is because it can be used to create code by language independent
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@defun semantic-prototype-file buffer
Return a file in which prototypes belonging to @var{BUFFER} should be placed.
Default behavior (if not overridden) looks for a token specifying the
prototype file, or the existence of an @var{EDE} variable indicating which
file prototypes belong in.
@end defun
@defun semantic-abbreviate-nonterminal token &optional parent color
Return an abbreviated string describing @var{TOKEN}.
The abbreviation is to be short, with possible symbols indicating
the type of token, or other information.
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@defun semantic-concise-prototype-nonterminal token &optional parent color
Return a concise prototype for @var{TOKEN}.
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@defun semantic-uml-abbreviate-nonterminal token &optional parent color
Return a @var{UML} style abbreviation for @var{TOKEN}.
Optional argument @var{PARENT} is the parent type if @var{TOKEN} is a detail.
Optional argument @var{COLOR} means highlight the prototype with font-lock colors.
@end defun
@node Token Details, Local Context, Token->Text, Override Methods
@subsection Token Details
These functions help derive information about tokens that may not
be obvious for non-traditional languages with their own token types.
@defun semantic-nonterminal-children token &optional positionalonly
Return the list of top level children belonging to @var{TOKEN}.
Children are any sub-tokens which may contain overlays.
The default behavior (if not overridden with @code{nonterminal-children}
is to return type parts for a type, and arguments for a function.
If optional argument @var{POSITIONALONLY} is non-@code{nil}, then only return valid
children if they contain positions. Some languages may choose to create
lists of children without position/overlay information.
If this function is overridden, use @dfn{semantic-nonterminal-children-default}
to also include the default behavior, and merely extend your own.
Note for language authors:
If a mode defines a language that has tokens in it with overlays that
should not be considered children, you should still return them with
this function. If you do not, then token re-parsing, and database
saving will fail.
@end defun
@defun semantic-nonterminal-external-member-parent token
Return a parent for @var{TOKEN} when @var{TOKEN} is an external member.
@var{TOKEN} is an external member if it is defined at a toplevel and
has some sort of label defining a parent. The parent return will
be a string.
The default behavior, if not overridden with
@code{nonterminal-external-member-parent} is get the @code{'parent} extra
specifier of @var{TOKEN}.
If this function is overridden, use
@dfn{semantic-nonterminal-external-member-parent-default} to also
include the default behavior, and merely extend your own.
@end defun
@defun semantic-nonterminal-external-member-p parent token
Return non-@code{nil} if @var{PARENT} is the parent of @var{TOKEN}.
@var{TOKEN} is an external member of @var{PARENT} when it is somehow tagged
as having @var{PARENT} as it's parent.
The default behavior, if not overridden with
@code{nonterminal-external-member-p} is to match @code{'parent} extra specifier in
the name of @var{TOKEN}.
If this function is overridden, use
@code{semantic-nonterminal-external-member-children-p-default} to also
include the default behavior, and merely extend your own.
@end defun
@defun semantic-nonterminal-external-member-children token &optional usedb
Return the list of children which are not *in* @var{TOKEN}.
If optional argument @var{USEDB} is non-@code{nil}, then also search files in
the Semantic Database. If @var{USEDB} is a list of databases, search those
Children in this case are functions or types which are members of
@var{TOKEN}, such as the parts of a type, but which are not defined inside
the class. @var{C}++ and @var{CLOS} both permit methods of a class to be defined
outside the bounds of the class' definition.
The default behavior, if not overridden with
@code{nonterminal-external-member-children} is to search using
@dfn{semantic-nonterminal-external-member-p} in all top level definitions
with a parent of @var{TOKEN}.
If this function is overridden, use
@dfn{semantic-nonterminal-external-member-children-default} to also
include the default behavior, and merely extend your own.
@end defun
@defun semantic-nonterminal-protection token &optional parent
Return protection information about @var{TOKEN} with optional @var{PARENT}.
This function returns on of the following symbols:
@code{nil} - No special protection. Language dependent.
@code{'public} - Anyone can access this @var{TOKEN}.
@code{'private} - Only methods in the local scope can access @var{TOKEN}.
@code{'friend} - Like private, except some outer scopes are allowed
access to token.
Some languages may choose to provide additional return symbols specific
to themselves. Use of this function should allow for this.
The default behavior (if not overridden with @code{nonterminal-protection}
is to return a symbol based on type modifiers.
@end defun
@defun semantic-nonterminal-abstract token &optional parent
Return non @code{nil} if @var{TOKEN} is abstract.
Optional @var{PARENT} is the parent token of @var{TOKEN}.
In @var{UML}, abstract methods and classes have special meaning and behavior
in how methods are overridden. In @var{UML}, abstract methods are italicized.
The default behavior (if not overridden with @code{nonterminal-abstract}
is to return true if @code{abstract} is in the type modifiers.
@end defun
@defun semantic-nonterminal-leaf token &optional parent
Return non @code{nil} if @var{TOKEN} is leaf.
Optional @var{PARENT} is the parent token of @var{TOKEN}.
In @var{UML}, leaf methods and classes have special meaning and behavior.
The default behavior (if not overridden with @code{nonterminal-leaf}
is to return true if @code{leaf} is in the type modifiers.
@end defun
@defun semantic-nonterminal-static token &optional parent
Return non @code{nil} if @var{TOKEN} is static.
Optional @var{PARENT} is the parent token of @var{TOKEN}.
In @var{UML}, static methods and attributes mean that they are allocated
in the parent class, and are not instance specific.
@var{UML} notation specifies that @var{STATIC} entries are underlined.
The default behavior (if not overridden with @code{nonterminal-static}
is to return true if @code{static} is in the type modifiers.
@end defun
@defun semantic-find-dependency token
Find the filename represented from @var{TOKEN}.
@var{TOKEN} may be a stripped element, in which case @var{PARENT} specifies a
parent token that has positional information.
Depends on @code{semantic-dependency-include-path} for searching. Always searches
`.' first, then searches additional paths.
@defvar semantic-dependency-include-path
Defines the include path used when searching for files.
This should be a list of directories to search which is specific to
the file being included.
This variable can also be set to a single function. If it is a
function, it will be called with one arguments, the file to find as a
string, and it should return the full path to that file, or nil.
@end defvar
@end defun
@defun semantic-find-nonterminal token &optional parent
Find the location of @var{TOKEN}.
@var{TOKEN} may be a stripped element, in which case @var{PARENT} specifies a
parent token that has position information.
Different behaviors are provided depending on the type of token.
For example, dependencies (includes) will seek out the file that is
depended on, and functions will move to the specified definition.
@end defun
@defun semantic-find-documentation token
Find documentation from @var{TOKEN} and return it as a clean string.
@var{TOKEN} might have @var{DOCUMENTATION} set in it already. If not, there may be
some documentation in a comment preceding TOKEN's definition which we
can look for. When appropriate, this can be overridden by a language specific
@end defun
@node Local Context, Making New Methods, Token Details, Override Methods
@subsection Local Context
@defun semantic-up-context &optional point
Move point up one context from @var{POINT}.
Return non-@code{nil} if there are no more context levels.
Overloaded functions using @code{up-context} take no parameters.
@end defun
@defun semantic-beginning-of-context &optional point
Move @var{POINT} to the beginning of the current context.
Return non-@code{nil} if there is no upper context.
The default behavior uses @dfn{semantic-up-context}. It can
be overridden with @code{beginning-of-context}.
@end defun
@defun semantic-end-of-context &optional point
Move @var{POINT} to the end of the current context.
Return non-@code{nil} if there is no upper context.
Be default, this uses @dfn{semantic-up-context}, and assumes parenthetical
block delimiters. This can be overridden with @code{end-of-context}.
@end defun
@defun semantic-get-local-variables &optional point
Get the local variables based on POINT's context.
Local variables are returned in Semantic token format.
Be default, this calculates the current bounds using context blocks
navigation, then uses the parser with @code{bovine-inner-scope} to
parse tokens at the beginning of the context.
This can be overridden with @code{get-local-variables}.
@end defun
@defun semantic-get-local-arguments &optional point
Get arguments (variables) from the current context at @var{POINT}.
Parameters are available if the point is in a function or method.
This function returns a list of tokens. If the local token returns
just a list of strings, then this function will convert them to tokens.
Part of this behavior can be overridden with @code{get-local-arguments}.
@end defun
@defun semantic-get-all-local-variables &optional point
Get all local variables for this context, and parent contexts.
Local variables are returned in Semantic token format.
Be default, this gets local variables, and local arguments.
This can be overridden with @code{get-all-local-variables}.
Optional argument @var{POINT} is the location to start getting the variables from.
@end defun
These next set of functions handle local context parsing. This means
looking at the code (locally) and navigating, and fetching information
such as a the type of the parameter the cursor may be typing in.
@defun semantic-end-of-command
Move to the end of the current command.
Be default, uses @code{semantic-command-separation-character}.
Override with @code{end-of-command}.
@end defun
@defun semantic-beginning-of-command
Move to the beginning of the current command.
Be default, users @code{semantic-command-separation-character}.
Override with @code{beginning-of-command}.
@end defun
@defun semantic-ctxt-current-symbol &optional point
Return the current symbol the cursor is on at @var{POINT} in a list.
This will include a list of type/field names when applicable.
This can be overridden using @code{ctxt-current-symbol}.
@end defun
@defun semantic-ctxt-current-assignment &optional point
Return the current assignment near the cursor at @var{POINT}.
Return a list as per @dfn{semantic-ctxt-current-symbol}.
Return @code{nil} if there is nothing relevant.
Override with @code{ctxt-current-assignment}.
@end defun
@defun semantic-ctxt-current-function &optional point
Return the current function the cursor is in at @var{POINT}.
The function returned is the one accepting the arguments that
the cursor is currently in.
This can be overridden with @code{ctxt-current-function}.
@end defun
@defun semantic-ctxt-current-argument &optional point
Return the current symbol the cursor is on at @var{POINT}.
Override with @code{ctxt-current-argument}.
@end defun
@defun semantic-ctxt-scoped-types &optional point
Return a list of type names currently in scope at @var{POINT}.
Override with @code{ctxt-scoped-types}.
@end defun
For details on using these functions to get more detailed information
about the current context: @xref{Context Analysis}.
@node Making New Methods, , Local Context, Override Methods
@subsection Making New Methods
@node Parser Hooks, Example Programs, Override Methods, Programming
@section Parser Hooks
If you write a program that uses the stream of tokens in a persistent
display or database, it is necessary to know when tokens change so
that your displays can be updated. This is especially important as
tokens can be replaced, changed, or deleted, and the associated
overlays will then throw errors when you try to use them. Complete
integration with token changes can be achieved via several very
important hooks.
One interesting way to interact with the parser is to let it know that
changes you are going to make will not require re-parsing.
@defvar semantic-edits-are-safe
When non-@code{nil}, modifications do not require a reparse.
This prevents tokens from being marked dirty, and it
prevents top level edits from causing a cache check.
Use this when writing programs that could cause a full
reparse, but will not change the tag structure, such
as adding or updating top-level comments.
@end defvar
Next, it is sometimes useful to know what the current parsing state
is. These function can let you know what level of re-parsing may be
needed. Careful choices on when to reparse can make your program much
@defun semantic-bovine-toplevel-full-reparse-needed-p &optional checkcache
Return non-@code{nil} if the current buffer needs a full reparse.
Optional argument @var{CHECKCACHE} indicates if the cache check should be made.
@end defun
@defun semantic-bovine-toplevel-partial-reparse-needed-p &optional checkcache
Return non-@code{nil} if the current buffer needs a partial reparse.
This only returns non-@code{nil} if @dfn{semantic-bovine-toplevel-full-reparse-needed-p}
returns nil.
Optional argument @var{CHECKCACHE} indicates if the cache check should be made
when checking @dfn{semantic-bovine-toplevel-full-reparse-needed-p}.
@end defun
If you need very close interaction with the user's editing, then these
two hooks can be used to find out when a given tag is being changed.
These hooks could even be used to cut down on re-parsing if used correctly.
For all hooks, make sure you are careful to add it as a local hook if
you only want to effect a single buffer. Setting it globally can
cause unwanted effects if your program is concerned with a single buffer.
@defvar semantic-dirty-token-hooks
Hooks run after when a token is marked as dirty (edited by the user).
The functions must take @var{TOKEN}, @var{START}, and @var{END} as a parameters.
This hook will only be called once when a token is first made dirty,
subsequent edits will not cause this to run a second time unless that
token is first cleaned. Any token marked as dirty will
also be called with @code{semantic-clean-token-hooks}, unless a full
reparse is done instead.
@end defvar
@defvar semantic-clean-token-hooks
Hooks run after a token is marked as clean (re-parsed after user edits.)
The functions must take a @var{TOKEN} as a parameter.
Any token sent to this hook will have first been called with
@code{semantic-dirty-token-hooks}. This hook is not called for tokens
marked dirty if the buffer is completely re-parsed. In that case, use
@end defvar
@defvar semantic-change-hooks
Hooks run when semantic detects a change in a buffer.
Each hook function must take three arguments, identical to the
common hook @code{after-change-function}.
@end defvar
Lastly, if you just want to know when a buffer changes, use this hook.
@defvar semantic-after-toplevel-bovinate-hook
Hooks run after a toplevel token parse.
It is not run if the toplevel parse command is called, and buffer does
not need to be fully re-parsed.
This function is also called when the toplevel cache is flushed, and
the cache is emptied.
For language specific hooks, make sure you define this as a local hook.
This hook should not be used any more.
Use @code{semantic-after-toplevel-cache-change-hook} instead.
@end defvar
@defvar semantic-after-toplevel-cache-change-hook
Hooks run after the buffer token list has changed.
This list will change when a buffer is re-parsed, or when the token
list in a buffer is cleared. It is *NOT* called if the current token
list partially re-parsed.
Hook functions must take one argument, which is the new list of
tokens associated with this buffer.
For language specific hooks, make sure you define this as a local hook.
@end defvar
@defvar semantic-after-partial-cache-change-hook
Hooks run after the buffer token list has been updated.
This list will change when the current token list has been partially
Hook functions must take one argument, which is the list of tokens
updated among the ones associated with this buffer.
For language specific hooks, make sure you define this as a local hook.
@end defvar
@defvar semantic-before-toplevel-cache-flush-hook
Hooks run before the toplevel nonterminal cache is flushed.
For language specific hooks, make sure you define this as a local hook.
This hook is called before a corresponding
@code{semantic-after-toplevel-cache-change-hook} which is also called
during a flush when the cache is given a new value of nil.
@end defvar
@node Example Programs, , Parser Hooks, Programming
@section Programming Examples
Here are some simple examples that use different aspects of the
semantic library APIs. For fully functional example programs with
lots of comments, see the file @file{semantic-examples.el}.
@subsection Interactively querying for a token name
If you need a command that asks the user for a token name, you can
get full range completion using the query functions
@ref{Nonterminal Completion}.
(interactive (list (semantic-read-symbol "Symbol: ")))
@end example
@subsection Finding a symbol in a buffer
If you have the name of a function or variable, and need to find its
location in a buffer, you need a search function. There is a wide
range of searches you can perform @ref{Nonterminal Streams}.
t ;; look inside structures and classes for these symbols
nil) ;; do not look inside header files.
@end example
@subsection Finding a symbol in a project
If you have the name of a function or variable, and need to find its
location somewhere in a project, you need to use the Semantic Database
@ref{semanticdb}. There are many search functions similar to the
ones found in @ref{Nonterminal Streams}.
The Semantic Database is interesting in that the return structure is
@subsection Locating a token in a buffer
If you have a nonterminal token, or a list of them, you may want to
find their position in a buffer.
nil ;; Defaults to the current project's database list.
t ;; Search inside types
nil ;; Do not search include files
nil ;; Only search files in the same mode (all C files)
t ;; When a token is found, make sure it is loaded in a buffer.
@end example
Of interesting note above, semanticdb can find symbols in files that
are not loaded into an Emacs buffer. These tokens do not have an
associated overlay, and the function @dfn{semantic-token-buffer} will
The last parameter's tells the search function to
@dfn{find-file-noselect} any file in which a matching token was
found. This will allow you to merge all the tokens into a completion
list, or other flat list needed by most functions that use
association lists.
If you do not ask semanticdb to load those files, you will need to
explicitly request the database object (found in the @code{car} of
each sublist) get the file loaded. It is useful to not auto find all
files if you don't need to jump to that token.
@subsection Converting a token into a human readable string.
A nonterminal token is a rather unpleasant Lisp structure when trying
to decipher what is going on. As such, there is a wide range of
functions available that can convert a token into a human readable,
and colorful string @ref{Token->Text}.
If you program interfaces with lots of users, you will probably want
to have your program define a configurable variable that will let
users change the visible portion of your program.
(defcustom my-summary-function 'semantic-uml-prototype-nonterminal
"*Function to use when showing info about my token."
:group 'my-program
:type semantic-token->text-custom-list)
@end example
Note the special type provided by semantic.
Next, you can call this function to create a string.
(funcall my-summary-function token
t ; use color
@end example
In this case, @var{token-parent} is an optional argument. In many
cases, parent is not used by the outputting function. The parent
may be a struct or class that contains @var{token}, or nil for
top-level definitions. In particular, C++ needs the parent to
correctly calculate the protection of each method.
@node Current Context, Tools, Programming, Top
@comment node-name, next, previous, up
@chapter Deriving the Current Context
This chapter deals with how to derive the current context, and also
how a language maintainer can get the current context API to work
with their language.
By default, the behavior will function in C like languages. This
means languages with parenthetical blocks, and type dereferencing
which uses a similar form.
* Blocks::
* Local Variables:: Getting lists of local variables.
* Derived Context:: What goes at a given location?
* Context Analysis:: Analysis information about the local context.
@end menu
@node Blocks, Local Variables, Current Context, Current Context
@section Blocks and Navigation
Source code is typically built up of control structures, and blocks of
context, or lexical scope. Semantic terms these lexical scopes as a
``context''. The following functions can be used to navigate contexts.
Some of them are override functions. Language authors can override
a subset of them to make them work for their language.
@defun semantic-up-context &optional point
Move point up one context from @var{POINT}.
Return non-@code{nil} if there are no more context levels.
Overloaded functions using @code{up-context} take no parameters.
@end defun
@defun semantic-beginning-of-context &optional point
Move @var{POINT} to the beginning of the current context.
Return non-@code{nil} if there is no upper context.
The default behavior uses @code{semantic-up-context}. It can
be overridden with @code{beginning-of-context}.
@end defun
@defun semantic-end-of-context &optional point
Move @var{POINT} to the end of the current context.
Return non-@code{nil} if there is no upper context.
Be default, this uses @code{semantic-up-context}, and assumes parenthetical
block delimiters. This can be overridden with @code{end-of-context}.
@end defun
These next set of functions can be used to navigate across commands.
@defun semantic-end-of-command
Move to the end of the current command.
Be default, uses @code{semantic-command-separation-character}.
Override with @code{end-of-command}.
@end defun
@defun semantic-beginning-of-command
Move to the beginning of the current command.
Be default, users @code{semantic-command-separation-character}.
Override with @code{beginning-of-command}.
@end defun
@node Local Variables, Derived Context, Blocks, Current Context
@section Deriving local variables
Within a given context, or block of code, local variables are often
defined. These functions can be used to retrieve lists of locally
scoped variables.
@defun semantic-get-local-variables &optional point
Get the local variables based on POINT's context.
Local variables are returned in Semantic token format.
By default, this calculates the current bounds using context blocks
navigation, then uses the parser with @code{bovine-inner-scope} to
parse tokens at the beginning of the context.
This can be overridden with @code{get-local-variables}.
@end defun
@defun semantic-get-local-arguments &optional point
Get arguments (variables) from the current context at @var{POINT}.
Parameters are available if the point is in a function or method.
This function returns a list of tokens. If the local token returns
just a list of strings, then this function will convert them to tokens.
Part of this behavior can be overridden with @code{get-local-arguments}.
@end defun
@defun semantic-get-all-local-variables &optional point
Get all local variables for this context, and parent contexts.
Local variables are returned in Semantic token format.
Be default, this gets local variables, and local arguments.
This can be overridden with @code{get-all-local-variables}.
Optional argument @var{POINT} is the location to start getting the variables from.
@end defun
@node Derived Context, Context Analysis, Local Variables, Current Context
@section Deriving the Current Context
While a context has already been used to describe blocks of code,
other context include more local details, such as the symbol the
cursor is on, or the fact we are assigning into some other variable.
These context deriving functions can be overridden to provide language
specific behavior. By default, it assumes a C like language.
@defun semantic-ctxt-current-symbol &optional point
Return the current symbol the cursor is on at @var{POINT} in a list.
This will include a list of type/field names when applicable.
This can be overridden using @code{ctxt-current-symbol}.
@end defun
@defun semantic-ctxt-current-assignment &optional point
Return the current assignment near the cursor at @var{POINT}.
Return a list as per @code{semantic-ctxt-current-symbol}.
Return @code{nil} if there is nothing relevant.
Override with @code{ctxt-current-assignment}.
@end defun
@defun semantic-ctxt-current-function &optional point
Return the current symbol the cursor is on at @var{POINT}.
The function returned is the one accepting the arguments that
the cursor is currently in.
This can be overridden with `ctxt.current-function'.
@end defun
@defun semantic-ctxt-current-argument &optional point
Return the current symbol the cursor is on at @var{POINT}.
Override with @code{ctxt-current-argument}.
@end defun
@node Context Analysis, , Derived Context, Current Context
@section Analysis of the current context
The context parsing API is used in a context analysis library. This
library provides high level routines for scanning through token
databases to create lists of token associates. At it's core is a set
of EIEIO classes defining a context. The context contains information
about what was parsed at a given position, such as the strings there,
and they type of assignment. The analysis library then searches the
databases to determine the types and names available.
Two high level functions which can be run interactively are:
*NOTE TO SELF: Add more here*
@deffn Command semantic-analyze-current-context position
Analyze the current context at @var{POSITION}.
If called interactively, display interesting information about @var{POSITION}
in a separate buffer.
Returns an object based on symbol @code{semantic-analyze-context}.
@end deffn
@deffn Command semantic-analyze-possible-completions point
Return a list of semantic tokens which are possible completions.
Analysis is done at @var{POINT}.
@end deffn
@node Tools, Index, Current Context, Top
@comment node-name, next, previous, up
@chapter Tools
Several tools come with Semantic which would not be possible without it.
In general, these tools will work with any language supported by
* speedbar:: How to use Semantic speedbar support
* imenu:: Special support for Imenu.
* semanticdb:: Cache your parsed buffers between sessions.
* senator:: The Semantic Navigator
* analyzer:: Analyze local context
* class browser:: Class hierarchy browser
* document:: Document generation functions
* charts:: Charting code statistics
* minor modes:: Useful minor modes
@end menu
@node speedbar, imenu, Tools, Tools
@comment node-name, next, previous, up
@section Speedbar
Speedbar supports the display of tags through the Semantic parser. To
use this utility, add a line like this to your @file{.emacs} file:
(add-hook 'speedbar-load-hook (lambda () (require 'semantic-sb)))
@end example
or you can simply add:
(require 'semantic-sb)
@end example
Once installed, speedbar will use semantic to find tokens, and will
display them appropriately. Tags from semantic will have additional
details which can be seen, such as return type, or arguments to
If you use @file{semantic-load.el}, you do not need to add the above
lines in your .emacs file.
Two additional speedbar modes are described in @ref{Speedbar Analysis}, and
@ref{class browser}.
@node imenu, semanticdb, speedbar, Tools
@comment node-name, next, previous, up
@section Imenu support
There is special support for creating Imenu entries using semantic.
This is a highly customizable tool which can create specialized menu
systems for navigating your source file.
By default, each language that wants special imenu support will set
itself up for it. To setup imenu for your buffers, use this command
in your @file{.emacs} file:
(add-hook 'semantic-init-hooks (lambda ()
(imenu-add-to-menubar "TOKENS")))
@end example
Also supported is @dfn{which-func-mode}. This usually uses imenu
tags to show the current function. The semantic support for this
function uses overlays, which is much faster.
If you use @file{semantic-load.el}, you do not need to add the above
lines in your .emacs file.
You can customize imenu with the following options:
@deffn Option semantic-imenu-summary-function
Function to use when creating items in Imenu.
Some useful functions are:
@end deffn
@deffn Option semantic-imenu-bucketize-file
Non-@code{nil} if tokens in a file are to be grouped into buckets.
@end deffn
@deffn Option semantic-imenu-buckets-to-submenu
Non-@code{nil} if buckets of tokens are to be turned into submenus.
This option is ignored if @code{semantic-imenu-bucketize-file} is nil.
@end deffn
@deffn Option semantic-imenu-expand-type-parts
Non-@code{nil} if types should have submenus with parts in it.
@end deffn
@deffn Option semantic-imenu-bucketize-type-parts
Non-@code{nil} if elements of a type should be placed grouped into buckets.
@code{Nil} means to keep them in the same order.
Overridden to @code{nil} if @code{semantic-imenu-bucketize-file} is nil.
@end deffn
@deffn Option semantic-imenu-sort-bucket-function
Function to use when sorting tags in the buckets of functions.
@end deffn
@deffn Option semantic-imenu-index-directory
Non @code{nil} to index the entire directory for tags.
Doesn't actually parse the entire directory, but displays tags for all files
currently listed in the current Semantic database.
This variable has no meaning if semanticdb is not active.
@end deffn
@deffn Option semantic-imenu-auto-rebuild-directory-indexes
If non-@code{nil} automatically rebuild directory index imenus.
That is when a directory index imenu is updated, automatically rebuild
other buffer local ones based on the same semanticdb.
@end deffn
When adding support to a language, this variable may need to be set:
@defvar semantic-imenu-expandable-token
Tokens of this token type will be given submenu with children.
By default, a @code{type} has interesting children. In Texinfo, however,
a @code{section} has interesting children.
@end defvar
@node semanticdb, senator, imenu, Tools
@comment node-name, next, previous, up
@section Semantic Database
Semanticdb is a utility which tracks your parsed files, and saves the
parsed information to files. When you reload your source files,
semanticdb automatically associates the file with the cached copy,
saving time by not re-parsing your buffer.
Semanticdb also provides an API for programs to use. These functions
will return token information without loading the source file into
memory by checking the disk cache.
To use semanticdb, add the following to your @file{.emacs} file:
(require 'semanticdb)
(global-semanticdb-minor-mode 1)
@end example
If you have a tool which optionally uses the semantic database, it may
be important to track if the database mode is turned on or off.
@deffn Option semanticdb-mode-hooks
Hooks run whenever @dfn{global-semanticdb-minor-mode} is run.
Use @dfn{semanticdb-minor-mode-p} to determine if the mode has been turned
on or off.
@end deffn
@deffn Option semanticdb-persistent-path
List of valid paths that semanticdb will cache tokens to.
When @dfn{global-semanticdb-minor-mode} is active, token lists will
be saved to disk when Emacs exits. Not all directories will have
tokens that should be saved.
The value should be a list of valid paths. @var{A} path can be a string,
indicating a directory in which to save a variable. An element in the
list can also be a symbol. Valid symbols are @code{never}, which will
disable any saving anywhere, @code{always}, which enables saving
everywhere, or @code{project}, which enables saving in any directory that
passes a list of predicates in @code{semantic-project-predicates}.
@end deffn
@deffn Option semanticdb-project-roots
List of directories, where each directory is the root of some project.
All subdirectories of a root project are considered a part of one project.
Values in this string can be overridden by project management programs
via the @code{semanticdb-project-root-functions} variable.
@end deffn
The important difference between these two is that you may put just
``~'' in @code{semanticdb-persistent-path}, but you may put individual
project directories into @code{semanticdb-project-roots} so that
different database lists don't get cross referenced incorrectly.
@subsection Searching
You can search for tokens in the database using the following
functions. It is important to note that database search functions do
not return a plain list of tokens. This is because some tokens may
not be loaded in a buffer, which means that the found token would not
have an overlay, and no way to determine where it came from.
As such, all search functions return a list of the form:
@end example
@defun semanticdb-find-nonterminal-by-function function &optional databases search-parts search-includes diff-mode find-file-match
Find all occurrences of nonterminals which match @var{FUNCTION}.
Search in all @var{DATABASES}. If @var{DATABASES} is @code{nil}, search a range of
associated databases.
When @var{SEARCH-PARTS} is non-@code{nil} the search will include children of tokens.
When @var{SEARCH-INCLUDES} is non-@code{nil}, the search will include dependency files.
When @var{DIFF-MODE} is non-@code{nil}, search databases which are of a different mode.
@var{A} Mode is the @code{major-mode} that file was in when it was last parsed.
When @var{FIND-@var{FILE}-MATCH} is non-@code{nil}, the make sure any found token's file is
in an Emacs buffer.
@end defun
@defun semanticdb-find-nonterminal-by-name name &optional databases search-parts search-includes diff-mode find-file-match
Find all occurrences of nonterminals with name @var{NAME} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES}, @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
@end defun
@defun semanticdb-find-nonterminal-by-name-regexp regex &optional databases search-parts search-includes diff-mode find-file-match
Find all occurrences of nonterminals with name matching @var{REGEX} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES} @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
@end defun
@defun semanticdb-find-nonterminal-by-type type &optional databases search-parts search-includes diff-mode find-file-match
Find all nonterminals with a type of @var{TYPE} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES} @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
@end defun
@defun semanticdb-find-nonterminal-by-property property value &optional databases search-parts search-includes diff-mode find-file-match
Find all nonterminals with a @var{PROPERTY} equal to @var{VALUE} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES} @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
Return a list ((@var{DB-TABLE} . @var{TOKEN-LIST}) ...).
@end defun
@defun semanticdb-find-nonterminal-by-extra-spec spec &optional databases search-parts search-includes diff-mode find-file-match
Find all nonterminals with a @var{SPEC} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES} @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
Return a list ((@var{DB-TABLE} . @var{TOKEN-LIST}) ...).
@end defun
@defun semanticdb-find-nonterminal-by-extra-spec-value spec value &optional databases search-parts search-includes diff-mode find-file-match
Find all nonterminals with a @var{SPEC} equal to @var{VALUE} in databases.
See @dfn{semanticdb-find-nonterminal-by-function} for details on @var{DATABASES},
@var{SEARCH-PARTS}, @var{SEARCH-INCLUDES} @var{DIFF-MODE}, and @var{FIND-@var{FILE}-MATCH}.
Return a list ((@var{DB-TABLE} . @var{TOKEN-LIST}) ...).
@end defun
@defun semanticdb-file-stream file
Return a list of tokens belonging to @var{FILE}.
If file has database tokens available in the database, return them.
If file does not have tokens available, then load the file, and create them.
@end defun
@node senator, analyzer, semanticdb, Tools
@comment node-name, next, previous, up
@section Semantic Navigator
Senator stands for SEmantic NAvigaTOR and was written by David Ponce.
This library defines commands and a minor mode to navigate between
semantic language tokens in the current buffer.
@subsection Commands
The following user level commands are provided by Senator.
@subsubsection Navigation
@deffn Command senator-next-token
Move to the next token in the current buffer.
@end deffn
@deffn Command senator-previous-token
Move to the previous token in the current buffer.
@end deffn
@deffn Command senator-jump sym
Jump to the semantic symbol @var{SYM}.
If called interactively and a prefix argument is supplied jump in the
local type's context (see function @dfn{senator-current-type-context}).
@end deffn
@subsubsection Searching
Searching using senator mode restricts the search only to the
definition text, such as the name of the functions or variables in a
given buffer.
@deffn Command senator-isearch-toggle-semantic-mode
Toggles semantic search in isearch mode. When semantic search is
enabled, isearch is restricted to token names.
@end deffn
@deffn Command senator-search-forward string
@deffnx Command senator-search-backward string
Search forward and backward for a token matching @var{string}.
@end deffn
@deffn Command re-search-forward regex
@deffnx Command re-search-backward regex
Search forward and backward for a token matching the regular expression
@end deffn
@deffn Command word-search-forward word
@deffnx word-search-backward word
Search forward and backward for a token whose name matches @var{word}.
@end deffn
@subsubsection Completion
Completion in senator scans all known definitions in the local file,
and uses that information to provide the completion.
@deffn Command senator-complete-symbol
Complete the current symbol under point.
@end deffn
@deffn Command senator-completion-menu-keyboard-popup
Popup a completion menu for the symbol at point.
@end deffn
@subsubsection Copy/Paste
Token Copy/Paste is a high level form of the typical copy yank used by
Emacs. Copying a token saves the meta-information related to the
function or item the cursor is currently in. When that information is
yanked into a new buffer, the form of the text created is based on the
current status of the programming buffer.
For example, pasting a function into a different file results in a
function call template to be inserted. In a Texinfo file, a @@deffn
is created with documentation for that function or command.
@deffn Command senator-copy-token
Take the current token, and place it in the token ring.
@end deffn
@deffn Command senator-kill-token
Take the current token, place it in the token ring, and kill it.
Killing the token removes the text for that token, and places it into
the kill ring. Retrieve that text with @code{yank}.
@end deffn
@deffn Command senator-yank-token
Yank a token from the token ring.
The form the token takes is different depending on where it is being
yanked to.
@end deffn
@deffn Command senator-copy-token-to-register register &optional kill-flag
Copy the current token into @var{REGISTER}.
Optional argument @var{KILL-FLAG} will delete the text of the token to the
kill ring.
@end deffn
For programmers, to provide specialized pasting, created an override
function for @code{insert-foreign-token} (see @xref{Settings}.)
@subsubsection Minor Mode
@deffn Command senator-minor-mode
Toggle the SEmantic NAvigaTOR key bindings in the current buffer.
The following default key bindings are provided when semantic minor
mode is enabled:
@table @key
@item C-c , n
@item C-c , p
@item C-c , i
@item C-c , j
@item C-c , TAB
@item C-c , SPC
@item C-c , C-y
@item C-c , C-w
@item C-c , M-w
@item C-c ,
@end table
@end deffn
@subsection Customization
To enable the Senator keymap in all modes that support semantic parsing,
use this:
(add-hook 'semantic-init-hooks 'senator-minor-mode)
@end example
To customize navigation around different types of tokens, use the
following variables:
@deffn Option senator-step-at-token-ids
List of token identifiers where to step.
Token identifier is symbol @code{'variable}, @code{'function}, @code{'type}, or other. If
@code{nil} navigation steps at any token found. This is a buffer local
variable. It can be set in a mode hook to get a specific language
@end deffn
@deffn Option senator-step-at-start-end-token-ids
List of token identifiers where to step at start and end.
Token identifier is symbol @code{'variable}, @code{'function}, @code{'type}, or other. If
@code{nil} navigation only step at beginning of tokens. If @code{t} step at start
and end of any token where it is allowed to step. Also, stepping at
start and end of a token prevent stepping inside its children. This
is a buffer local variable. It can be set in a mode hook to get a
specific language navigation.
@end deffn
To have a mode specific customization, do something like this in a hook:
(add-hook 'mode-hook
(lambda ()
(setq senator-step-at-token-ids '(function variable))
(setq senator-step-at-start-end-token-ids '(function))
@end example
This will cause navigation and search commands to stop only between
functions and variables, and to step at start and end of functions
@subsection Contact information for Senator
Any comments, suggestions, bug reports or upgrade requests are
welcome. Please send them to David Ponce at david@@dponce.com
@node analyzer, class browser, senator, Tools
@comment node-name, next, previous, up
@section Semantic Analyzer
The semantic analyzer is a library tool that performs context
analysis and can derive useful information.
@deffn Command semantic-analyze-current-context position
Analyze the current context at @var{POSITION}.
If called interactively, display interesting information about @var{POSITION}
in a separate buffer.
Returns an object based on symbol @dfn{semantic-analyze-context}.
@end deffn
While this can be used as a command, it is mostly useful that way
while debugging the analyzer, or tools using the return value. Use
the Emacs command @dfn{describe-class} to learn more about using
Another command that uses the analyzer context derives a completion
@deffn Command semantic-analyze-possible-completions context
Return a list of semantic tokens which are possible completions.
@var{CONTEXT} is either a position (such as point), or a pre-calculated
context. Passing in a context is useful if the caller also needs
to access parts of the analysis.
Completions run through the following filters:
@table @asis
@item Elements currently in scope
@item Constants currently in scope
@item Elements match the @code{:prefix} in the @var{CONTEXT}.
@item Type of the completion matches the type of the context.
@end table
Context type matching can identify the following:
@table @asis
@item No specific type
@item Assignment into a variable of some type.
@item Argument to a function with type constraints.
@end table
When called interactively, this function displays the list of
possible completions. This is useful for debugging.
@end deffn
* Smart Completion :: Commands for smart completion.
* Speedbar Analysis :: Display an analysis in speedbar.
@end menu
@node Smart Completion, Speedbar Analysis, analyzer, analyzer
@comment node-name, next, previous, up
@subsection Smart Completion
The file @file{semantic-ia.el} contains two commands for performing
smart completion using the analysis library. Analysis to calculate
these completions are done through the analyzer and completion
mechanism. These functions just provide commands that can be bound
to key bindings.
@deffn Command semantic-ia-complete-symbol point
Complete the current symbol at @var{POINT}.
Completion options are calculated with
@end deffn
@deffn Command semantic-ia-complete-symbol-menu point
Complete the current symbol via a menu based at @var{POINT}.
Completion options are calculated with
@end deffn
@node Speedbar Analysis, , Smart Completion, analyzer
@comment node-name, next, previous, up
@subsection Speedbar Analysis
The Analyzer output can be used through a speedbar interface.
This interface lists details about the analysis, such as the current
function, local arguments and variables, details on the prefix (the
symbol the cursor is on), and a list of all possible completions.
Completions are specified in
Each entry can be jumped to by clicking on the name. For strongly
typed languages, this means you will jump to the definition of the
variable, slot, or type definition.
In addition each entry has an <i> button. Clicking on this will
display a summary of everything that is known about the variable or
type displayed on that line.
If you click on the name of a variable in the ``Completions'' menu,
then the text that was recently analyzed will be replaced with the
name of the token that was clicked on in speedbar.
@deffn Command semantic-speedbar-analysis
Start Speedbar in semantic analysis mode.
The analyzer displays information about the current context, plus a smart
list of possible completions.
@end deffn
You can also enter speedbar analyzer mode by selecting ``Analyze''
from the ``Display'' menu item on speedbar's menu.
@node class browser, document, analyzer, Tools
@comment node-name, next, previous, up
@section Class Browser
The semantic class browser is a library that can covert a project
group of files into an EIEIO based structure that contains links
between structures so that the inheritance links between semantic
tokens can be easily navigated.
The core to this library is one function in @file{semantic-cb.el}.
@defun semantic-cb-new-class-browser
Create an object representing this project's organization.
The object returned is of type @dfn{semantic-cb-project}, which contains
the slot `:types', a list of all top-level types. Each element is a
class of type @dfn{semantic-cb-token}, or @dfn{semantic-cb-type}.
@end defun
Use the Emacs function `describe-class' to learn more about these
You can access the class inheritance structure through a speedbar
interface. You can choose the ``Class Browser'' option from
Speedbar's ``Display'' menu item, or use the following command:
@deffn Command semantic-cb-speedbar-mode
Bring speedbar up, and put it into Class Browser mode.
This will use the Class Browser logic applied to the current Semantic
project database to build the available relations. The structure of
the class hierarchy can then be navigated using traditional speedbar
@end deffn
@node document, charts, class browser, Tools
@comment node-name, next, previous, up
@section Document generation
The document program uses semantic token streams to aid in the
creation of texinfo documentation.
For example, the following is a code fragment from @file{document.el}
that comes with semantic:
(defun document (&optional resetfile)
"Document the function or variable the cursor is in.
Optional argument RESETFILE is provided w/ universal argument.
When non-nil, query for a new documentation file."
@end lisp
While visiting @file{document.el}, put the cursor somewhere within the
function shown above. Then type @kbd{M-x document}.
After asking for the texinfo file name, which in this case is
@file{semantic.texi}, this will update the texinfo
documentation of the @code{document} function in that file.
The result is that the following texinfo text will be either created
or updated in @file{semantic.texi} file:
@@deffn Command document &optional resetfile
Document the function or variable the cursor is in.
Optional argument @@var@{RESETFILE@} is provided w/ universal argument.
When non-@@code@{nil@}, query for a new documentation file.
@@end deffn
@end example
Note that the function name, arguments and documentation string
is put in the right place.
Within the doc-string, the function arguments are marked with
the @@var command and the @code{nil} code fragment is marked with
@@code command.
This example provides just a glimpse of what is possible with the
syntactic information provided by semantic.
The main entry point for the
documentation generator are the following commands:
@deffn Command document &optional resetfile
Document the function or variable the cursor is in.
Optional argument @var{RESETFILE} is provided w/ universal argument.
When non-@code{nil}, query for a new documentation file.
@end deffn
@deffn Command document-inline
Document the current function with an inline comment.
@end deffn
@deffn Command document-insert-defun-comment nonterm buffer
Insert mode-comment documentation about @var{NONTERM} from @var{BUFFER}.
@end deffn
@deffn Command document-insert-new-file-header header
Insert a new header file into this buffer. Add reference to @var{HEADER}.
Used by @code{prototype} if this file doesn't have an introductory comment.
@end deffn
In addition to these base documentation commands, the texinfo semantic
parser includes a two convenience functions when working directly with
texinfo files.
@deffn Command semantic-texi-update-doc &optional token
Update the documentation for @var{TOKEN}.
If the current buffer is a texinfo file, then find the source doc, and
update it. If the current buffer is a source file, then get the
documentation for this item, find the existing doc in the associated
manual, and update that.
@end deffn
@deffn Command semantic-texi-goto-source &optional token
Jump to the source for the definition in the texinfo file @var{TOKEN}.
If @var{TOKEN} is @code{nil}, it is derived from the deffn under @var{POINT}.
@end deffn
@node charts, minor modes, document, Tools
@comment node-name, next, previous, up
@section Charting Commands
Some commands to draw charts of statistics generated from parsing:
@deffn Command semantic-chart-nonterminals-by-token &optional buffer-or-stream
Create a bar chart representing the number of nonterminals for a token.
Each bar represents how many toplevel nonterminal in @var{BUFFER-OR-STREAM}
exist with a given token type. See `semantic-symbol->name-assoc-list'
for tokens which will be charted.
@end deffn
@deffn Command semantic-chart-database-size &optional buffer-or-stream
Create a bar chart representing the size of each file in semanticdb.
Each bar represents how many toplevel nonterminals in @var{BUFFER-OR-STREAM}
exist in each database entry.
@end deffn
@deffn Command semantic-chart-nonterminal-complexity-token &optional symbol buffer-or-stream
Create a bar chart representing the complexity of some tokens.
Complexity is calculated for tokens with a token of @var{SYMBOL}. Each bar
represents the complexity of some nonterminal in @var{BUFFER-OR-STREAM}.
Only the most complex items are charted.
@end deffn
@node minor modes, , charts, Tools
@comment node-name, next, previous, up
@section Minor Modes
@deffn Command semantic-show-dirty-mode &optional arg
Minor mode for highlighting dirty tokens.
With prefix argument @var{ARG}, turn on if positive, otherwise off. The
minor mode can be turned on only if semantic feature is available and
the current buffer was set up for parsing. Return non-@code{nil} if the
minor mode is enabled.
@end deffn
@deffn Command global-semantic-show-dirty-mode &optional arg
Toggle global use of @code{semantic-show-dirty-mode}.
If @var{ARG} is positive, enable, if it is negative, disable.
If @var{ARG} is @code{nil}, then toggle.
@end deffn
@deffn Option semantic-dirty-token-face
Face used to show dirty tokens in @code{semantic-show-dirty-token-mode}.
@end deffn
@deffn Command semantic-show-unmatched-syntax-mode &optional arg
Minor mode to highlight unmatched-syntax tokens.
With prefix argument @var{ARG}, turn on if positive, otherwise off. The
minor mode can be turned on only if semantic feature is available and
the current buffer was set up for parsing. Return non-@code{nil} if the
minor mode is enabled.
@end deffn
@deffn Command global-semantic-show-unmatched-syntax-mode &optional arg
Toggle global use of @dfn{semantic-show-unmatched-syntax-mode}.
If @var{ARG} is positive, enable, if it is negative, disable.
If @var{ARG} is @code{nil}, then toggle.
@end deffn
@deffn Option semantic-unmatched-syntax-face
Face used to show unmatched-syntax in.
The face is used in @dfn{semantic-show-unmatched-syntax-mode}.
@end deffn
@deffn Command global-semantic-auto-parse-mode &optional arg
Toggle global use of @dfn{semantic-auto-parse-mode}.
If @var{ARG} is positive, enable, if it is negative, disable.
If @var{ARG} is @code{nil}, then toggle.
@end deffn
@deffn Command semantic-auto-parse-mode &optional arg
Minor mode to auto parse buffer following changes.
With prefix argument @var{ARG}, turn on if positive, otherwise off. The
minor mode can be turned on only if semantic feature is available and
the current buffer was set up for parsing. Return non-@code{nil} if the
minor mode is enabled.
@end deffn
@deffn Option semantic-auto-parse-no-working-message
Non-@code{nil} disable display of working message during parse.
@end deffn
@deffn Option semantic-auto-parse-idle-time
Time in seconds of idle time before auto-reparse.
This time should be short enough to ensure that auto-parse will be
run as soon as Emacs is idle.
@end deffn
@deffn Option semantic-auto-parse-max-buffer-size
Maximum size in bytes of buffers automatically re-parsed.
If this value is less than or equal to @var{0} buffers are automatically
re-parsed regardless of their size.
@end deffn
@node Index, , Tools, Top
@comment node-name, next, previous, up
@chapter Index
@unnumbered Function Index
@printindex fn
@c LocalWords: alist API APIs arg argc args argv asis assoc autoload
@c LocalWords: backquote bnf bovinate bovinates
@c LocalWords: bovinating bovination bovinator bucketize
@c LocalWords: cb cdr charquote checkcache cindex CLOS
@c LocalWords: concat concocting const constantness ctxt Decl defcustom
@c LocalWords: deffn deffnx defun defvar destructor's dfn diff dir
@c LocalWords: doc docstring EDE EIEIO elisp emacsman emph enum
@c LocalWords: eq Exp EXPANDFULL expresssion fn foo func funcall
@c LocalWords: ia ids iff ifinfo imenu imenus init int isearch itemx java kbd
@c LocalWords: keymap keywordtable lang languagemode lexer lexing Ludlam
@c LocalWords: menubar metaparent metaparents min minibuffer Misc mode's
@c LocalWords: multitable NAvigaTOR noindent nomedian nonterm noselect
@c LocalWords: nosnarf obarray OLE OO outputfile paren parsetable POINT's
@c LocalWords: popup positionalonly positiononly positionormarker pre
@c LocalWords: printf printindex Programmatically pt punctuations quotemode
@c LocalWords: ref regex regexp Regexps reparse resetfile samp sb
@c LocalWords: scopestart SEmantic semanticdb setfilename setq
@c LocalWords: settitle setupfunction sexp sp SPC speedbar speedbar's
@c LocalWords: streamorbuffer struct subalist submenu submenus
@c LocalWords: subsubsection sw sym texi texinfo titlefont titlepage
@c LocalWords: tok TOKEN's toplevel typemodifiers uml unset untar
@c LocalWords: uref usedb var vskip xref yak