text: added package target to Makefile.in some: doc-if 750: doc-ahead 751: doc-then 752: doc-begin 753: doc-until 754: doc-again 755: doc-cs-pick 756: doc: doc-else 767: doc-while 768: doc-repeat 769: 770: Counted loop words constitute a separate group of words: 771: 772: doc-?do 773: doc-do 774: doc-for 775: doc-loop 776: doc-s+loop 777: doc-+loop 778: doc-next 779: doc-leave 780: doc-?leave 781: doc-unloop 782: doc definition (@code{LOOP} etc. compile an @code{UNLOOP} on the 788: fall-through path). Also, you have to ensure that all @code{LEAVE}s are 789: resolved (by using one of the loop-ending words or @code{UNDO}). 790: 791: Another group of control structure words are 792: 793: doc-case 794: doc-endcase 795: doc-of 796: doc-endof 797: 798: @i{case-sys} and @i{of-sys} cannot be processed using @code{cs-pick} and 799: @code{cs-roll}. 800: 801: @subsubsection Programming Style 802: 803: In order to ensure readability we recommend that you do not create 804: arbitrary control structures directly, but define new control structure 805: words for the control structure you want and use these words in your 806: program. 807: 808: E.g., instead of writing 809: 810: @example 811: begin 812: ... 813: if [ 1 cs-roll ] 814: ... 815: again then 816: @end example 817: 818: we recommend defining control structure words, e.g., 819: 820: @example 821: : while ( dest -- orig dest ) 822: POSTPONE if 823: 1 cs-roll ; immediate 824: 825: : repeat ( orig dest -- ) 826: POSTPONE again 827: POSTPONE then ; immediate 828: @end example 829: 830: and then using these to create the control structure: 831: 832: @example 833: begin 834: ... 835: while 836: ... 837: repeat 838: @end example 839: 840: That's much easier to read, isn't it? Of course, @code{BEGIN} and 841: @code{WHILE} are predefined, so in this example it would not be 842: necessary to define them. 843: 844: @subsection Calls and returns 845: 846: A definition can be called simply be writing the name of the 847: definition. When the end of the definition is reached, it returns. An earlier return can be forced using 848: 849: doc-exit 850: 851: Don't forget to clean up the return stack and @code{UNLOOP} any 852: outstanding @code{?DO}...@code{LOOP}s before @code{EXIT}ing. The 853: primitive compiled by @code{EXIT} is 854: 855: doc-;s 856: 857: @subsection Exception Handling 858: 859: doc-catch 860: doc-throw 861: 862: @node Locals 863: @section Locals 864: 865: Local variables can make Forth programming more enjoyable and Forth 866: programs easier to read. Unfortunately, the locals of ANS Forth are 867: laden with restrictions. Therefore, we provide not only the ANS Forth 868: locals wordset, but also our own, more powerful locals wordset (we 869: implemented the ANS Forth locals wordset through our locals wordset). 870: 871: @menu 872: @end menu 873: 874: @subsection gforth locals 875: 876: Locals can be defined with 877: 878: @example 879: @{ local1 local2 ... -- comment @} 880: @end example 881: or 882: @example 883: @{ local1 local2 ... @} 884: @end example 885: 886: E.g., 887: @example 888: : max @{ n1 n2 -- n3 @} 889: n1 n2 > if 890: n1 891: else 892: n2 893: endif ; 894: @end example 895: 896: The similarity of locals definitions with stack comments is intended. A 897: locals definition often replaces the stack comment of a word. The order 898: of the locals corresponds to the order in a stack comment and everything 899: after the @code{--} is really a comment. 900: 901: This similarity has one disadvantage: It is too easy to confuse locals 902: declarations with stack comments, causing bugs and making them hard to 903: find. However, this problem can be avoided by appropriate coding 904: conventions: Do not use both notations in the same program. If you do, 905: they should be distinguished using additional means, e.g. by position. 906: 907: The name of the local may be preceded by a type specifier, e.g., 908: @code{F:} for a floating point value: 909: 910: @example 911: : CX* @{ F: Ar F: Ai F: Br F: Bi -- Cr Ci @} 912: \ complex multiplication 913: Ar Br f* Ai Bi f* f- 914: Ar Bi f* Ai Br f* f+ ; 915: @end example 916: 917: GNU Forth currently supports cells (@code{W:}, @code{W^}), doubles 918: (@code{D:}, @code{D^}), floats (@code{F:}, @code{F^}) and characters 919: (@code{C:}, @code{C^}) in two flavours: a value-flavoured local (defined 920: with @code{W:}, @code{D:} etc.) produces its value and can be changed 921: with @code{TO}. A variable-flavoured local (defined with @code{W^} etc.) 922: produces its address (which becomes invalid when the variable's scope is 923: left). E.g., the standard word @code{emit} can be defined in therms of 924: @code{type} like this: 925: 926: @example 927: : emit @{ C^ char* -- @} 928: char* 1 type ; 929: @end example 930: 931: A local without type specifier is a @code{W:} local. Both flavours of 932: locals are initialized with values from the data or FP stack. 933: 934: Currently there is no way to define locals with user-defined data 935: structures, but we are working on it. 936: 937: GNU Forth allows defining locals everywhere in a colon definition. This poses the following questions: 938: 939: @subsubsection Where are locals visible by name? 940: 941: Basically, the answer is that locals are visible where you would expect 942: it in block-structured languages, and sometimes a little longer. If you 943: want to restrict the scope of a local, enclose its definition in 944: @code{SCOPE}...@code{ENDSCOPE}. 945: 946: doc-scope 947: doc-endscope 948: 949: These words behave like control structure words, so you can use them 950: with @code{CS-PICK} and @code{CS-ROLL} to restrict the scope in 951: arbitrary ways. 952: 953: If you want a more exact answer to the visibility question, here's the 954: basic principle: A local is visible in all places that can only be 955: reached through the definition of the local@footnote{In compiler 956: construction terminology, all places dominated by the definition of the 957: local.}. In other words, it is not visible in places that can be reached 958: without going through the definition of the local. E.g., locals defined 959: in @code{IF}...@code{ENDIF} are visible until the @code{ENDIF}, locals 960: defined in @code{BEGIN}...@code{UNTIL} are visible after the 961: @code{UNTIL} (until, e.g., a subsequent @code{ENDSCOPE}). 962: 963: The reasoning behind this solution is: We want to have the locals 964: visible as long as it is meaningful. The user can always make the 965: visibility shorter by using explicit scoping. In a place that can 966: only be reached through the definition of a local, the meaning of a 967: local name is clear. In other places it is not: How is the local 968: initialized at the control flow path that does not contain the 969: definition? Which local is meant, if the same name is defined twice in 970: two independent control flow paths? 971: 972: This should be enough detail for nearly all users, so you can skip the 973: rest of this section. If you relly must know all the gory details and 974: options, read on. 975: 976: In order to implement this rule, the compiler has to know which places 977: are unreachable. It knows this automatically after @code{AHEAD}, 978: @code{AGAIN}, @code{EXIT} and @code{LEAVE}; in other cases (e.g., after 979: most @code{THROW}s), you can use the word @code{UNREACHABLE} to tell the 980: compiler that the control flow never reaches that place. If 981: @code{UNREACHABLE} is not used where it could, the only consequence is 982: that the visibility of some locals is more limited than the rule above 983: says. If @code{UNREACHABLE} is used where it should not (i.e., if you 984: lie to the compiler), buggy code will be produced. 985: 986: Another problem with this rule is that at @code{BEGIN}, the compiler 987: does not know which locals will be visible on the incoming 988: back-edge. All problems discussed in the following are due to this 989: ignorance of the compiler (we discuss the problems using @code{BEGIN} 990: loops as examples; the discussion also applies to @code{?DO} and other 991: loops). Perhaps the most insidious example is: 992: @example 993: AHEAD 994: BEGIN 995: x 996: [ 1 CS-ROLL ] THEN 997: { x } 998: ... 999: UNTIL 1000: @end example 1001: 1002: This should be legal according to the visibility rule. The use of 1003: @code{x} can only be reached through the definition; but that appears 1004: textually below the use. 1005: 1006: From this example it is clear that the visibility rules cannot be fully 1007: implemented without major headaches. Our implementation treats common 1008: cases as advertised and the exceptions are treated in a safe way: The 1009: compiler makes a reasonable guess about the locals visible after a 1010: @code{BEGIN}; if it is too pessimistic, the 1011: user will get a spurious error about the local not being defined; if the 1012: compiler is too optimistic, it will notice this later and issue a 1013: warning. In the case above the compiler would complain about @code{x} 1014: being undefined at its use. You can see from the obscure examples in 1015: this section that it takes quite unusual control structures to get the 1016: compiler into trouble, and even then it will often do fine. 1017: 1018: If the @code{BEGIN} is reachable from above, the most optimistic guess 1019: is that all locals visible before the @code{BEGIN} will also be 1020: visible after the @code{BEGIN}. This guess is valid for all loops that 1021: are entered only through the @code{BEGIN}, in particular, for normal 1022: @code{BEGIN}...@code{WHILE}...@code{REPEAT} and 1023: @code{BEGIN}...@code{UNTIL} loops and it is implemented in our 1024: compiler. When the branch to the @code{BEGIN} is finally generated by 1025: @code{AGAIN} or @code{UNTIL}, the compiler checks the guess and 1026: warns the user if it was too optimisitic: 1027: @example 1028: IF 1029: { x } 1030: BEGIN 1031: \ x ? 1032: [ 1 cs-roll ] THEN 1033: ... 1034: UNTIL 1035: @end example 1036: 1037: Here, @code{x} lives only until the @code{BEGIN}, but the compiler 1038: optimistically assumes that it lives until the @code{THEN}. It notices 1039: this difference when it compiles the @code{UNTIL} and issues a 1040: warning. The user can avoid the warning, and make sure that @code{x} 1041: is not used in the wrong area by using explicit scoping: 1042: @example 1043: IF 1044: SCOPE 1045: { x } 1046: ENDSCOPE 1047: BEGIN 1048: [ 1 cs-roll ] THEN 1049: ... 1050: UNTIL 1051: @end example 1052: 1053: Since the guess is optimistic, there will be no spurious error messages 1054: about undefined locals. 1055: 1056: If the @code{BEGIN} is not reachable from above (e.g., after 1057: @code{AHEAD} or @code{EXIT}), the compiler cannot even make an 1058: optimistic guess, as the locals visible after the @code{BEGIN} may be 1059: defined later. Therefore, the compiler assumes that no locals are 1060: visible after the @code{BEGIN}. However, the useer can use 1061: @code{ASSUME-LIVE} to make the compiler assume that the same locals are 1062: visible at the BEGIN as at the point where the item was created. 1063: 1064: doc-assume-live 1065: 1066: E.g., 1067: @example 1068: { x } 1069: AHEAD 1070: ASSUME-LIVE 1071: BEGIN 1072: x 1073: [ 1 CS-ROLL ] THEN 1074: ... 1075: UNTIL 1076: @end example 1077: 1078: Other cases where the locals are defined before the @code{BEGIN} can be 1079: handled by inserting an appropriate @code{CS-ROLL} before the 1080: @code{ASSUME-LIVE} (and changing the control-flow stack manipulation 1081: behind the @code{ASSUME-LIVE}). 1082: 1083: Cases where locals are defined after the @code{BEGIN} (but should be 1084: visible immediately after the @code{BEGIN}) can only be handled by 1085: rearranging the loop. E.g., the ``most insidious'' example above can be 1086: arranged into: 1087: @example 1088: BEGIN 1089: { x } 1090: ... 0= 1091: WHILE 1092: x 1093: REPEAT 1094: @end example 1095: 1096: @subsubsection How long do locals live? 1097: 1098: The right answer for the lifetime question would be: A local lives at 1099: least as long as it can be accessed. For a value-flavoured local this 1100: means: until the end of its visibility. However, a variable-flavoured 1101: local could be accessed through its address far beyond its visibility 1102: scope. Ultimately, this would mean that such locals would have to be 1103: garbage collected. Since this entails un-Forth-like implementation 1104: complexities, I adopted the same cowardly solution as some other 1105: languages (e.g., C): The local lives only as long as it is visible; 1106: afterwards its address is invalid (and programs that access it 1107: afterwards are erroneous). 1108: 1109: @subsubsection Programming Style 1110: 1111: The freedom to define locals anywhere has the potential to change 1112: programming styles dramatically. In particular, the need to use the 1113: return stack for intermediate storage vanishes. Moreover, all stack 1114: manipulations (except @code{PICK}s and @code{ROLL}s with run-time 1115: determined arguments) can be eliminated: If the stack items are in the 1116: wrong order, just write a locals definition for all of them; then 1117: write the items in the order you want. 1118: 1119: This seems a little far-fetched and eliminating stack manipulations is 1120: unlikely to become a conscious programming objective. Still, the 1121: number of stack manipulations will be reduced dramatically if local 1122: variables are used liberally (e.g., compare @code{max} in \sect{misc} 1123: with a traditional implementation of @code{max}). 1124: 1125: This shows one potential benefit of locals: making Forth programs more 1126: readable. Of course, this benefit will only be realized if the 1127: programmers continue to honour the principle of factoring instead of 1128: using the added latitude to make the words longer. 1129: 1130: Using @code{TO} can and should be avoided. Without @code{TO}, 1131: every value-flavoured local has only a single assignment and many 1132: advantages of functional languages apply to Forth. I.e., programs are 1133: easier to analyse, to optimize and to read: It is clear from the 1134: definition what the local stands for, it does not turn into something 1135: different later. 1136: 1137: E.g., a definition using @code{TO} might look like this: 1138: @example 1139: : strcmp @{ addr1 u1 addr2 u2 -- n @} 1140: u1 u2 min 0 1141: ?do 1142: addr1 c@ addr2 c@ - ?dup 1143: if 1144: unloop exit 1145: then 1146: addr1 char+ TO addr1 1147: addr2 char+ TO addr2 1148: loop 1149: u1 u2 - ; 1150: @end example 1151: Here, @code{TO} is used to update @code{addr1} and @code{addr2} at 1152: every loop iteration. @code{strcmp} is a typical example of the 1153: readability problems of using @code{TO}. When you start reading 1154: @code{strcmp}, you think that @code{addr1} refers to the start of the 1155: string. Only near the end of the loop you realize that it is something 1156: else. 1157: 1158: This can be avoided by defining two locals at the start of the loop that 1159: are initialized with the right value for the current iteration. 1160: @example 1161: : strcmp @{ addr1 u1 addr2 u2 -- n @} 1162: addr1 addr2 1163: u1 u2 min 0 1164: ?do @{ s1 s2 @} 1165: s1 c@ s2 c@ - ?dup 1166: if 1167: unloop exit 1168: then 1169: s1 char+ s2 char+ 1170: loop 1171: 2drop 1172: u1 u2 - ; 1173: @end example 1174: Here it is clear from the start that @code{s1} has a different value 1175: in every loop iteration. 1176: 1177: @subsubsection Implementation 1178: 1179: GNU Forth uses an extra locals stack. The most compelling reason for 1180: this is that the return stack is not float-aligned; using an extra stack 1181: also eliminates the problems and restrictions of using the return stack 1182: as locals stack. Like the other stacks, the locals stack grows toward 1183: lower addresses. A few primitives allow an efficient implementation: 1184: 1185: doc-@local# 1186: doc-f@local# 1187: doc-laddr# 1188: doc-lp+!# 1189: doc-lp! 1190: doc->l 1191: doc-f>l 1192: 1193: In addition to these primitives, some specializations of these 1194: primitives for commonly occurring inline arguments are provided for 1195: efficiency reasons, e.g., @code{@@local0} as specialization of 1196: @code{@@local#} for the inline argument 0. The following compiling words 1197: compile the right specialized version, or the general version, as 1198: appropriate: 1199: 1200: doc-compile-@@local 1201: doc-compile-f@@local 1202: doc-compile-lp+! 1203: 1204: Combinations of conditional branches and @code{lp+!#} like 1205: @code{?branch-lp+!#} (the locals pointer is only changed if the branch 1206: is taken) are provided for efficiency and correctness in loops. 1207: 1208: A special area in the dictionary space is reserved for keeping the 1209: local variable names. @code{@{} switches the dictionary pointer to this 1210: area and @code{@}} switches it back and generates the locals 1211: initializing code. @code{W:} etc.@ are normal defining words. This 1212: special area is cleared at the start of every colon definition. 1213: 1214: A special feature of GNU Forths dictionary is used to implement the 1215: definition of locals without type specifiers: every wordlist (aka 1216: vocabulary) has its own methods for searching 1217: etc. (@xref{dictionary}). For the present purpose we defined a wordlist 1218: with a special search method: When it is searched for a word, it 1219: actually creates that word using @code{W:}. @code{@{} changes the search 1220: order to first search the wordlist containing @code{@}}, @code{W:} etc., 1221: and then the wordlist for defining locals without type specifiers. 1222: 1223: The lifetime rules support a stack discipline within a colon 1224: definition: The lifetime of a local is either nested with other locals 1225: lifetimes or it does not overlap them. 1226: 1227: At @code{BEGIN}, @code{IF}, and @code{AHEAD} no code for locals stack 1228: pointer manipulation is generated. Between control structure words 1229: locals definitions can push locals onto the locals stack. @code{AGAIN} 1230: is the simplest of the other three control flow words. It has to 1231: restore the locals stack depth of the corresponding @code{BEGIN} 1232: before branching. The code looks like this: 1233: @format 1234: @code{lp+!#} current-locals-size @minus{} dest-locals-size 1235: @code{branch} <begin> 1236: @end format 1237: 1238: @code{UNTIL} is a little more complicated: If it branches back, it 1239: must adjust the stack just like @code{AGAIN}. But if it falls through, 1240: the locals stack must not be changed. The compiler generates the 1241: following code: 1242: @format 1243: @code{?branch-lp+!#} <begin> current-locals-size @minus{} dest-locals-size 1244: @end format 1245: The locals stack pointer is only adjusted if the branch is taken. 1246: 1247: @code{THEN} can produce somewhat inefficient code: 1248: @format 1249: @code{lp+!#} current-locals-size @minus{} orig-locals-size 1250: <orig target>: 1251: @code{lp+!#} orig-locals-size @minus{} new-locals-size 1252: @end format 1253: The second @code{lp+!#} adjusts the locals stack pointer from the 1254: level at the {\em orig} point to the level after the @code{THEN}. The 1255: first @code{lp+!#} adjusts the locals stack pointer from the current 1256: level to the level at the orig point, so the complete effect is an 1257: adjustment from the current level to the right level after the 1258: @code{THEN}. 1259: 1260: In a conventional Forth implementation a dest control-flow stack entry 1261: is just the target address and an orig entry is just the address to be 1262: patched. Our locals implementation adds a wordlist to every orig or dest 1263: item. It is the list of locals visible (or assumed visible) at the point 1264: described by the entry. Our implementation also adds a tag to identify 1265: the kind of entry, in particular to differentiate between live and dead 1266: (reachable and unreachable) orig entries. 1267: 1268: A few unusual operations have to be performed on locals wordlists: 1269: 1270: doc-common-list 1271: doc-sub-list? 1272: doc-list-size 1273: 1274: Several features of our locals wordlist implementation make these 1275: operations easy to implement: The locals wordlists are organised as 1276: linked lists; the tails of these lists are shared, if the lists 1277: contain some of the same locals; and the address of a name is greater 1278: than the address of the names behind it in the list. 1279: 1280: Another important implementation detail is the variable 1281: @code{dead-code}. It is used by @code{BEGIN} and @code{THEN} to 1282: determine if they can be reached directly or only through the branch 1283: that they resolve. @code{dead-code} is set by @code{UNREACHABLE}, 1284: @code{AHEAD}, @code{EXIT} etc., and cleared at the start of a colon 1285: definition, by @code{BEGIN} and usually by @code{THEN}. 1286: 1287: Counted loops are similar to other loops in most respects, but 1288: @code{LEAVE} requires special attention: It performs basically the same 1289: service as @code{AHEAD}, but it does not create a control-flow stack 1290: entry. Therefore the information has to be stored elsewhere; 1291: traditionally, the information was stored in the target fields of the 1292: branches created by the @code{LEAVE}s, by organizing these fields into a 1293: linked list. Unfortunately, this clever trick does not provide enough 1294: space for storing our extended control flow information. Therefore, we 1295: introduce another stack, the leave stack. It contains the control-flow 1296: stack entries for all unresolved @code{LEAVE}s. 1297: 1298: Local names are kept until the end of the colon definition, even if 1299: they are no longer visible in any control-flow path. In a few cases 1300: this may lead to increased space needs for the locals name area, but 1301: usually less than reclaiming this space would cost in code size. 1302: 1303: 1304: @subsection ANS Forth locals 1305: 1306: The ANS Forth locals wordset does not define a syntax for locals, but 1307: words that make it possible to define various syntaxes. One of the 1308: possible syntaxes is a subset of the syntax we used in the gforth locals 1309: wordset, i.e.: 1310: 1311: @example 1312: @{ local1 local2 ... -- comment @} 1313: @end example 1314: or 1315: @example 1316: @{ local1 local2 ... @} 1317: @end example 1318: 1319: The order of the locals corresponds to the order in a stack comment. The 1320: restrictions are: 1321: 1322: @itemize @bullet 1323: @item 1324: Locals can only be cell-sized values (no type specifers are allowed). 1325: @item 1326: Locals can be defined only outside control structures. 1327: @item 1328: Locals can interfere with explicit usage of the return stack. For the 1329: exact (and long) rules, see the standard. If you don't use return stack 1330: accessing words in a definition using locals, you will we all right. The 1331: purpose of this rule is to make locals implementation on the return 1332: stack easier. 1333: @item 1334: The whole definition must be in one line. 1335: @end itemize 1336: 1337: Locals defined in this way behave like @code{VALUE}s 1338: (@xref{values}). I.e., they are initialized from the stack. Using their 1339: name produces their value. Their value can be changed using @code{TO}. 1340: 1341: Since this syntax is supported by gforth directly, you need not do 1342: anything to use it. If you want to port a program using this syntax to 1343: another ANS Forth system, use @file{anslocal.fs} to implement the syntax 1344: on the other system. 1345: 1346: Note that a syntax shown in the standard, section A.13 looks 1347: similar, but is quite different in having the order of locals 1348: reversed. Beware! 1349: 1350: The ANS Forth locals wordset itself consists of the following word 1351: 1352: doc-(local) 1353: 1354: The ANS Forth locals extension wordset defines a syntax, but it is so 1355: awful that we strongly recommend not to use it. We have implemented this 1356: syntax to make porting to gforth easy, but do not document it here. The 1357: problem with this syntax is that the locals are defined in an order 1358: reversed with respect to the standard stack comment notation, making 1359: programs harder to read, and easier to misread and miswrite. The only 1360: merit of this syntax is that it is easy to implement using the ANS Forth 1361: locals wordset. 1362: 1363: @node Internals 1364: @chapter Internals 1365: 1366: Reading this section is not necessary for programming with gforth. It 1367: should be helpful for finding your way in the gforth sources. 1368: 1369: @section Portability 1370: 1371: One of the main goals of the effort is availability across a wide range 1372: of personal machines. fig-Forth, and, to a lesser extent, F83, achieved 1373: this goal by manually coding the engine in assembly language for several 1374: then-popular processors. This approach is very labor-intensive and the 1375: results are short-lived due to progress in computer architecture. 1376: 1377: Others have avoided this problem by coding in C, e.g., Mitch Bradley 1378: (cforth), Mikael Patel (TILE) and Dirk Zoller (pfe). This approach is 1379: particularly popular for UNIX-based Forths due to the large variety of 1380: architectures of UNIX machines. Unfortunately an implementation in C 1381: does not mix well with the goals of efficiency and with using 1382: traditional techniques: Indirect or direct threading cannot be expressed 1383: in C, and switch threading, the fastest technique available in C, is 1384: significantly slower. Another problem with C is that it's very 1385: cumbersome to express double integer arithmetic. 1386: 1387: Fortunately, there is a portable language that does not have these 1388: limitations: GNU C, the version of C processed by the GNU C compiler 1389: (@pxref{C Extensions, , Extensions to the C Language Family, gcc.info, 1390: GNU C Manual}). Its labels as values feature (@pxref{Labels as Values, , 1391: Labels as Values, gcc.info, GNU C Manual}) makes direct and indirect 1392: threading possible, its @code{long long} type (@pxref{Long Long, , 1393: Double-Word Integers, gcc.info, GNU C Manual}) corresponds to Forths 1394: double numbers. GNU C is available for free on all important (and many 1395: unimportant) UNIX machines, VMS, 80386s running MS-DOS, the Amiga, and 1396: the Atari ST, so a Forth written in GNU C can run on all these 1397: machines@footnote{Due to Apple's look-and-feel lawsuit it is not 1398: available on the Mac (@pxref{Boycott, , Protect Your Freedom--Fight 1399: ``Look And Feel'', gcc.info, GNU C Manual}).}. 1400: 1401: Writing in a portable language has the reputation of producing code that 1402: is slower than assembly. For our Forth engine we repeatedly looked at 1403: the code produced by the compiler and eliminated most compiler-induced 1404: inefficiencies by appropriate changes in the source-code. 1405: 1406: However, register allocation cannot be portably influenced by the 1407: programmer, leading to some inefficiencies on register-starved 1408: machines. We use explicit register declarations (@pxref{Explicit Reg 1409: Vars, , Variables in Specified Registers, gcc.info, GNU C Manual}) to 1410: improve the speed on some machines. They are turned on by using the 1411: @code{gcc} switch @code{-DFORCE_REG}. Unfortunately, this feature not 1412: only depends on the machine, but also on the compiler version: On some 1413: machines some compiler versions produce incorrect code when certain 1414: explicit register declarations are used. So by default 1415: @code{-DFORCE_REG} is not used. 1416: 1417: @section Threading 1418: 1419: GNU C's labels as values extension (available since @code{gcc-2.0}, 1420: @pxref{Labels as Values, , Labels as Values, gcc.info, GNU C Manual}) 1421: makes it possible to take the address of @var{label} by writing 1422: @code{&&@var{label}}. This address can then be used in a statement like 1423: @code{goto *@var{address}}. I.e., @code{goto *&&x} is the same as 1424: @code{goto x}. 1425: 1426: With this feature an indirect threaded NEXT looks like: 1427: @example 1428: cfa = *ip++; 1429: ca = *cfa; 1430: goto *ca; 1431: @end example 1432: For those unfamiliar with the names: @code{ip} is the Forth instruction 1433: pointer; the @code{cfa} (code-field address) corresponds to ANS Forths 1434: execution token and points to the code field of the next word to be 1435: executed; The @code{ca} (code address) fetched from there points to some 1436: executable code, e.g., a primitive or the colon definition handler 1437: @code{docol}. 1438: 1439: Direct threading is even simpler: 1440: @example 1441: ca = *ip++; 1442: goto *ca; 1443: @end example 1444: 1445: Of course we have packaged the whole thing neatly in macros called 1446: @code{NEXT} and @code{NEXT1} (the part of NEXT after fetching the cfa). 1447: 1448: @subsection Scheduling 1449: 1450: There is a little complication: Pipelined and superscalar processors, 1451: i.e., RISC and some modern CISC machines can process independent 1452: instructions while waiting for the results of an instruction. The 1453: compiler usually reorders (schedules) the instructions in a way that 1454: achieves good usage of these delay slots. However, on our first tries 1455: the compiler did not do well on scheduling primitives. E.g., for 1456: @code{+} implemented as 1457: @example 1458: n=sp[0]+sp[1]; 1459: sp++; 1460: sp[0]=n; 1461: NEXT; 1462: @end example 1463: the NEXT comes strictly after the other code, i.e., there is nearly no 1464: scheduling. After a little thought the problem becomes clear: The 1465: compiler cannot know that sp and ip point to different addresses (and 1466: the version of @code{gcc} we used would not know it even if it could), 1467: so it could not move the load of the cfa above the store to the 1468: TOS. Indeed the pointers could be the same, if code on or very near the 1469: top of stack were executed. In the interest of speed we chose to forbid 1470: this probably unused ``feature'' and helped the compiler in scheduling: 1471: NEXT is divided into the loading part (@code{NEXT_P1}) and the goto part 1472: (@code{NEXT_P2}). @code{+} now looks like: 1473: @example 1474: n=sp[0]+sp[1]; 1475: sp++; 1476: NEXT_P1; 1477: sp[0]=n; 1478: NEXT_P2; 1479: @end example 1480: This can be scheduled optimally by the compiler (see \sect{TOS}). 1481: 1482: This division can be turned off with the switch @code{-DCISC_NEXT}. This 1483: switch is on by default on machines that do not profit from scheduling 1484: (e.g., the 80386), in order to preserve registers. 1485: 1486: @subsection Direct or Indirect Threaded? 1487: 1488: Both! After packaging the nasty details in macro definitions we 1489: realized that we could switch between direct and indirect threading by 1490: simply setting a compilation flag (@code{-DDIRECT_THREADED}) and 1491: defining a few machine-specific macros for the direct-threading case. 1492: On the Forth level we also offer access words that hide the 1493: differences between the threading methods (@pxref{Threading Words}). 1494: 1495: Indirect threading is implemented completely 1496: machine-independently. Direct threading needs routines for creating 1497: jumps to the executable code (e.g. to docol or dodoes). These routines 1498: are inherently machine-dependent, but they do not amount to many source 1499: lines. I.e., even porting direct threading to a new machine is a small 1500: effort. 1501: 1502: @subsection DOES> 1503: One of the most complex parts of a Forth engine is @code{dodoes}, i.e., 1504: the chunk of code executed by every word defined by a 1505: @code{CREATE}...@code{DOES>} pair. The main problem here is: How to find 1506: the Forth code to be executed, i.e. the code after the @code{DOES>} (the 1507: DOES-code)? There are two solutions: 1508: 1509: In fig-Forth the code field points directly to the dodoes and the 1510: DOES-code address is stored in the cell after the code address 1511: (i.e. at cfa cell+). It may seem that this solution is illegal in the 1512: Forth-79 and all later standards, because in fig-Forth this address 1513: lies in the body (which is illegal in these standards). However, by 1514: making the code field larger for all words this solution becomes legal 1515: again. We use this approach for the indirect threaded version. Leaving 1516: a cell unused in most words is a bit wasteful, but on the machines we 1517: are targetting this is hardly a problem. The other reason for having a 1518: code field size of two cells is to avoid having different image files 1519: for direct and indirect threaded systems (@pxref{image-format}). 1520: 1521: The other approach is that the code field points or jumps to the cell 1522: after @code{DOES}. In this variant there is a jump to @code{dodoes} at 1523: this address. @code{dodoes} can then get the DOES-code address by 1524: computing the code address, i.e., the address of the jump to dodoes, 1525: and add the length of that jump field. A variant of this is to have a 1526: call to @code{dodoes} after the @code{DOES>}; then the return address 1527: (which can be found in the return register on RISCs) is the DOES-code 1528: address. Since the two cells available in the code field are usually 1529: used up by the jump to the code address in direct threading, we use 1530: this approach for direct threading. We did not want to add another 1531: cell to the code field. 1532: 1533: @section Primitives 1534: 1535: @subsection Automatic Generation 1536: 1537: Since the primitives are implemented in a portable language, there is no 1538: longer any need to minimize the number of primitives. On the contrary, 1539: having many primitives is an advantage: speed. In order to reduce the 1540: number of errors in primitives and to make programming them easier, we 1541: provide a tool, the primitive generator (@file{prims2x.fs}), that 1542: automatically generates most (and sometimes all) of the C code for a 1543: primitive from the stack effect notation. The source for a primitive 1544: has the following form: 1545: 1546: @format 1547: @var{Forth-name} @var{stack-effect} @var{category} [@var{pronounc.}] 1548: [@code{""}@var{glossary entry}@code{""}] 1549: @var{C code} 1550: [@code{:} 1551: @var{Forth code}] 1552: @end format 1553: 1554: The items in brackets are optional. The category and glossary fields 1555: are there for generating the documentation, the Forth code is there 1556: for manual implementations on machines without GNU C. E.g., the source 1557: for the primitive @code{+} is: 1558: @example 1559: + n1 n2 -- n core plus 1560: n = n1+n2; 1561: @end example 1562: 1563: This looks like a specification, but in fact @code{n = n1+n2} is C 1564: code. Our primitive generation tool extracts a lot of information from 1565: the stack effect notations@footnote{We use a one-stack notation, even 1566: though we have separate data and floating-point stacks; The separate 1567: notation can be generated easily from the unified notation.}: The number 1568: of items popped from and pushed on the stack, their type, and by what 1569: name they are referred to in the C code. It then generates a C code 1570: prelude and postlude for each primitive. The final C code for @code{+} 1571: looks like this: 1572: 1573: @example 1574: I_plus: /* + ( n1 n2 -- n ) */ /* label, stack effect */ 1575: /* */ /* documentation */ 1576: { 1577: DEF_CA /* definition of variable ca (indirect threading) */ 1578: Cell n1; /* definitions of variables */ 1579: Cell n2; 1580: Cell n; 1581: n1 = (Cell) sp[1]; /* input */ 1582: n2 = (Cell) TOS; 1583: sp += 1; /* stack adjustment */ 1584: NAME("+") /* debugging output (with -DDEBUG) */ 1585: { 1586: n = n1+n2; /* C code taken from the source */ 1587: } 1588: NEXT_P1; /* NEXT part 1 */ 1589: TOS = (Cell)n; /* output */ 1590: NEXT_P2; /* NEXT part 2 */ 1591: } 1592: @end example 1593: 1594: This looks long and inefficient, but the GNU C compiler optimizes quite 1595: well and produces optimal code for @code{+} on, e.g., the R3000 and the 1596: HP RISC machines: Defining the @code{n}s does not produce any code, and 1597: using them as intermediate storage also adds no cost. 1598: 1599: There are also other optimizations, that are not illustrated by this 1600: example: Assignments between simple variables are usually for free (copy 1601: propagation). If one of the stack items is not used by the primitive 1602: (e.g. in @code{drop}), the compiler eliminates the load from the stack 1603: (dead code elimination). On the other hand, there are some things that 1604: the compiler does not do, therefore they are performed by 1605: @file{prims2x.fs}: The compiler does not optimize code away that stores 1606: a stack item to the place where it just came from (e.g., @code{over}). 1607: 1608: While programming a primitive is usually easy, there are a few cases 1609: where the programmer has to take the actions of the generator into 1610: account, most notably @code{?dup}, but also words that do not (always) 1611: fall through to NEXT. 1612: 1613: @subsection TOS Optimization 1614: 1615: An important optimization for stack machine emulators, e.g., Forth 1616: engines, is keeping one or more of the top stack items in 1617: registers. If a word has the stack effect {@var{in1}...@var{inx} @code{--} 1618: @var{out1}...@var{outy}}, keeping the top @var{n} items in registers 1619: @itemize 1620: @item 1621: is better than keeping @var{n-1} items, if @var{x>=n} and @var{y>=n}, 1622: due to fewer loads from and stores to the stack. 1623: @item is slower than keeping @var{n-1} items, if @var{x<>y} and @var{x<n} and 1624: @var{y<n}, due to additional moves between registers. 1625: @end itemize 1626: 1627: In particular, keeping one item in a register is never a disadvantage, 1628: if there are enough registers. Keeping two items in registers is a 1629: disadvantage for frequent words like @code{?branch}, constants, 1630: variables, literals and @code{i}. Therefore our generator only produces 1631: code that keeps zero or one items in registers. The generated C code 1632: covers both cases; the selection between these alternatives is made at 1633: C-compile time using the switch @code{-DUSE_TOS}. @code{TOS} in the C 1634: code for @code{+} is just a simple variable name in the one-item case, 1635: otherwise it is a macro that expands into @code{sp[0]}. Note that the 1636: GNU C compiler tries to keep simple variables like @code{TOS} in 1637: registers, and it usually succeeds, if there are enough registers. 1638: 1639: The primitive generator performs the TOS optimization for the 1640: floating-point stack, too (@code{-DUSE_FTOS}). For floating-point 1641: operations the benefit of this optimization is even larger: 1642: floating-point operations take quite long on most processors, but can be 1643: performed in parallel with other operations as long as their results are 1644: not used. If the FP-TOS is kept in a register, this works. If 1645: it is kept on the stack, i.e., in memory, the store into memory has to 1646: wait for the result of the floating-point operation, lengthening the 1647: execution time of the primitive considerably. 1648: 1649: The TOS optimization makes the automatic generation of primitives a 1650: bit more complicated. Just replacing all occurrences of @code{sp[0]} by 1651: @code{TOS} is not sufficient. There are some special cases to 1652: consider: 1653: @itemize 1654: @item In the case of @code{dup ( w -- w w )} the generator must not 1655: eliminate the store to the original location of the item on the stack, 1656: if the TOS optimization is turned on. 1657: @item Primitives with stack effects of the form {@code{--} 1658: @var{out1}...@var{outy}} must store the TOS to the stack at the start. 1659: Likewise, primitives with the stack effect {@var{in1}...@var{inx} @code{--}} 1660: must load the TOS from the stack at the end. But for the null stack 1661: effect @code{--} no stores or loads should be generated. 1662: @end itemize 1663: 1664: @subsection Produced code 1665: 1666: To see what assembly code is produced for the primitives on your machine 1667: with your compiler and your flag settings, type @code{make engine.s} and 1668: look at the resulting file @file{engine.c}. 1669: 1670: @section System Architecture 1671: 1672: Our Forth system consists not only of primitives, but also of 1673: definitions written in Forth. Since the Forth compiler itself belongs 1674: to those definitions, it is not possible to start the system with the 1675: primitives and the Forth source alone. Therefore we provide the Forth 1676: code as an image file in nearly executable form. At the start of the 1677: system a C routine loads the image file into memory, sets up the 1678: memory (stacks etc.) according to information in the image file, and 1679: starts executing Forth code. 1680: 1681: The image file format is a compromise between the goals of making it 1682: easy to generate image files and making them portable. The easiest way 1683: to generate an image file is to just generate a memory dump. However, 1684: this kind of image file cannot be used on a different machine, or on 1685: the next version of the engine on the same machine, it even might not 1686: work with the same engine compiled by a different version of the C 1687: compiler. We would like to have as few versions of the image file as 1688: possible, because we do not want to distribute many versions of the 1689: same image file, and to make it easy for the users to use their image 1690: files on many machines. We currently need to create a different image 1691: file for machines with different cell sizes and different byte order 1692: (little- or big-endian)@footnote{We consider adding information to the 1693: image file that enables the loader to change the byte order.}. 1694: 1695: Forth code that is going to end up in a portable image file has to 1696: comply to some restrictions: addresses have to be stored in memory 1697: with special words (@code{A!}, @code{A,}, etc.) in order to make the 1698: code relocatable. Cells, floats, etc., have to be stored at the 1699: natural alignment boundaries@footnote{E.g., store floats (8 bytes) at 1700: an address dividable by~8. This happens automatically in our system 1701: when you use the ANSI alignment words.}, in order to avoid alignment 1702: faults on machines with stricter alignment. The image file is produced 1703: by a metacompiler (@file{cross.fs}). 1704: 1705: So, unlike the image file of Mitch Bradleys @code{cforth}, our image 1706: file is not directly executable, but has to undergo some manipulations 1707: during loading. Address relocation is performed at image load-time, not 1708: at run-time. The loader also has to replace tokens standing for 1709: primitive calls with the appropriate code-field addresses (or code 1710: addresses in the case of direct threading). 1711: 1712: @contents 1713: @bye 1714: