   C/C++ and Assembler
  I'm wanting to learn about whats happening at a level below C/C++. I was wondering if anyone knew any good books that teach Assembler, but also relate it to C or C++. As in explain things how C's/C++'s stack, function, class, exception, and memory management work at the lower level. This might seem like a specific request, but i thought I'd give it a shot.As a side note:1) Is there one pretty common/standard assembler language? Do compilers each use different ones? How does this kind of thing work?2) From my knowledge, the steps for C/C++ are   - Preprocess   - Compile to ASM   - Link   At which point we have a chunck of Assembler code that can be run by the CPU   What happens for a language like C# or Java ? Im thinking   - Preprocess   - Compile to CIL   - JIT it to ASM ??   At which point we have a chunck of Assembler code that can be run by the CPU   I guess there must be a lot more going on than this   And then what happens for scripting languages like Python?3) Is a C/C++ exe ASM instructions? or lower level than that? Could I just open up an exe in some application and view these instructions?   
  1) Each processor has an instruction set that it understands.  The instruction set usually has both an assembly-language representation and a binary representation (sometimes these representations match nearly 1:1).2) You're pretty much correct for C/C++ and C#/Java.  I don't know what Python does.3) A 'native C/C++' EXE has code section(s) which contain the binary representation of instructions for whatever processor it was compiled for.  A 'disassembler' can read the binary instructions and convert them back to a readable assembly-language form.Some IDEs (like Visual Studio, for example) have a 'Disassembly' window that you can use when you're debugging which will let you see the instructions (both in binary and assembly language format) that are actually being executed.  You usually have to pause the program at a breakpoint before you can view the disassembly, though.One way to learn how low-level stuff works is to make test apps and then run them in Visual Studio and examine the disassembly.  This works better than trying to open the EXE in a stand-alone disassembler because Visual Studio can usually show you the C/C++ source code mixed into the assembly.MSVC(x86) stores its call stack like that listed in the Wikipedia article: Call stack (Structure).  On the x86/x64, the locations in memory *decrease* as more information is pushed onto the stack.Exceptions are implementation specific.  Two examples are 'Structured Exception Handling' and 'Vectored Exception Handling'.Memory management is implementation specific.  C/C++ 'new/delete/malloc/free' use a heap.  In Windows, the heap is built on top of a Virtual Memory manager.  C# uses whatever memory manager the CLR it's executed on uses (The compact framework and micro frameworks have different memory managers than the standard framework, for example).[Edited by - Nypyren on April 4, 2010 2:20:37 PM];Quote:Original post by hick18I'm wanting to learn about whats happening at a level below C/C++. I was wondering if anyone knew any good books that teach Assembler, but also relate it to C or C++. As in explain things how C's/C++'s stack, function, class, exception, and memory management work at the lower level. This might seem like a specific request, but i thought I'd give it a shot.Check out the Wikipedia page on that topic. It has a lot of info and links to even more info on what you've asked about.Quote:Original post by hick181) Is there one pretty common/standard assembler language? Do compilers each use different ones? How does this kind of thing work?Each processor model has its own instruction set. The instruction sets of processor families (i.e. x86, PowerPC, MIPS, ARM,...) are often very similar to each other, but there are often significant differences between those families and between generations in those families. This is caused by the difference in features.Human read/written assembly is expressed by mnemonics. Each compiler/assemlber may use a different set of mnemonics, but in general they are similar. Those mnemonics are then assembled to machine code.Machine code is (usually) a 1:1 representation of assembler code, and can be converted back and forth with little loss of information (Assembly / Disassembly). However named symbols like variable names and jump labels (and comments of course) are not useful for the processor and therefor not present in machine code. That's what make disassembled code more difficult to read than the original assembly.Also, many assembler support macros (pretty much like C++ macros), which are resolved during assembling the machine code and won't be present in the disassembled code.Quote:Original post by hick182) From my knowledge, the steps for C/C++ are   - Preprocess   - Compile to ASM   - Link   At which point we have a chunck of Assembler code that can be run by the CPUIt's actually compiled to machine code (with the optional step of going through ASM code). I have listed the differences above.Quote:Original post by hick18   What happens for a language like C# or Java ? Im thinking   - Preprocess   - Compile to CIL   - JIT it to ASM ??   At which point we have a chunck of Assembler code that can be run by the CPU   I guess there must be a lot more going on than thisPretty much, but again, replace ASM with machine code.Quote:Original post by hick18   And then what happens for scripting languages like Python?Those are either interpreted from their source code or compiled to a bytecode (kinda like machine code, but not runnable by a (real) maching) and then interpreted (that's what Python and all the other somewhat performant scripting languages do).You can think of interpretation of bytecode as a big switch statement that does something for each valid value of the operator part of the bytecode, using the operand values as input/output.Quote:Original post by hick183) Is a C/C++ exe ASM instructions? or lower level than that? Could I just open up an exe in some application and view these instructions?See above.   ;  If you are looking for a book this probably has the simplest and best explanation for someone new to assembly:Assembly Language Step-by-Step: Programming with LinuxBut in the end as everyone mentioned it doesn't matter if you use Java, C#,etc they are all reduced to 1's and 0's in a file and what happens depends on how your specific cpu interprets those.This is all explained in the above book.;  A slightly more expensive way to learn ASM is to order an Atmel AVR (for example, the STK500 in combination with the Atmega8/16/32). These little toys are very handy because one can use the C programming language, but it is also possible to use ASM. In the datasheet and manual a lot of things are both explained in ASM and C. Furthermore, the AVR instruction set is a lot smaller and simpler then Intel's X86 or AMD64. (Link to picture: https://wiki.tkk.fi/download/attachments/16221372/stk500.jpg).Ive learned a lot about ASM using this AVR, so it might come in handy for you to experiment with :)A good ASM resource can be found at: http://www.drpaulcarter.com/pcasm/   ;Quote:Original post by ForsithA slightly more expensive way to learn ASM is to order an Atmel AVR (for example, the STK500 in combination with the Atmega8/16/32). These little toys are very handy because one can use the C programming language, but it is also possible to use ASM. In the datasheet and manual a lot of things are both explained in ASM and C.As an alternative, the STM8S-Discovery gets you a development kit and demo board for $7 RRP. The documentation (and community) isn't as good as that for the AVR, but it's certainly cheaper and gets you a CISC microcontroller (as opposed to the RISC AVR or PIC).At least with microcontrollers you have a good reason to use assembly. [smile]   ;  Thanks for your replies. But Im really looking for a book to read on the subject. Something that I can play with in Visual Studio or some other IDE's. I noticed that most of the books on Amazon seem to be related to linux. I couldnt really find any on windows. Which I thought was surprising.   ;Quote:Original post by hick18Thanks for your replies. But Im really looking for a book to read on the subject. Something that I can play with in Visual Studio or some other IDE's. I noticed that most of the books on Amazon seem to be related to linux. I couldnt really find any on windows. Which I thought was surprising.Actually, the older edition of the book I mentioned, uses MASM so you should get that edition if you want to go that route.If you want a newer book that covers MASM you can take a look at the Kip Irvine ASM book. He has a good tutorial on setting up MASM with Visual Studio anyways.;  Assembly usage in Windows (really for any modern PC) is generally for advanced reasons only. Utilizing SIMD instruction sets, VMX work (virtualization) and the like are the usual areas where you'll see it employed. Other areas you might see it are interop layers between languages (for instance, C -> Ruby and visa versa). And of course operating system boot loaders are generally written in assembly, at least until you get to a point where you can use a higher level language like C. For pretty much anything else assembly language is mostly useless.Platforms where assembly language is much more prevalent is where it's at. That being embedded platforms (for the most part), where compilers and libraries are not yet advanced enough to outdo handwritten assembly. Those targets, such as the iPhone (for the SIMD instructions), DS, and other similar devices... along with many microchips (ARM variants for instance), and even things like the Propeller micro processor....   ;Quote:Original post by hick18I'm wanting to learn about whats happening at a level below C/C++. Always a good choice. Knowing how something works helps you use it better.There are a million ways to make computers do what you want. As you learn more, you will see familiar patterns all over the place (for example, the use of stacks, the basic asm instructions: load, store, arithmetic, compares, jumps, type coercion, and procedure calls). But there is no definitive way of doing things. Most people come from a background of C and C++ on an x86 platform. But there are a million architectures out there which are just as viable and do everything slightly differently. In the gcc pipeline, your process is pretty close to correct. When you compile a .c file, gcc (by default) turns it into an object file (a .o). Technically, this is an ELF file. It stores raw machine code, but also other information, such as debugging info (what lines correspond to which offsets in the machine code), the symbol table (the names of all the functions defined and their corresponding offsets), and other stuff (global data to be initialized, probably stack size information and shared library linking code).When you take a number of .o files and link them together, the output is an executable. Confusingly, it is *also* in ELF format. But it's an executable ELF file. Again, it contains machine code, but it also has a debugging table and a name table and global data stuff.The next trick is where the OS comes in. In unix, when you execute a file, the operating system has to figure out HOW to execute it. If the file looks like an ELF file, it creates a system process, copies all the machine code into the process, initializes the data, links the shared libraries, sets the IP to the address of your main method, and your program begins running. The point here is that there's more to your program than code :) Now, this is how "traditional" compiled languages work. Other languages might be totally different. In Java (and in C#... the two are very similar), when you compile code, you compile to a .class file. The class file format is similar to the ELF format. The major difference is that ELF format lets you use whatever machine code you want. In the class format, you can only have Java bytecode. Furthermore, all constants have to conform to the Java platform (the endianness, integer sizes, and floating point formats are all rigorously specified by the specs). This is what is meant when we say Java bytecode is "cross platform".The JVM bytecode, of course, is meaningless to the CPU. So what you have to do is interpret it. Writing an interpreter is conceptually really simple. It loads the bytecode into an array. You read the first instruction. Do whatever the instruction supposed to do. Then, move to the next instruction. The interpreter also provides a number of useful services, such as garbage collection. Garbage collection is really important for many languages. It's pretty big topic, too. But effectively, you just malloc/new without worrying about free/deleting anything. When you run out of space, you figure out what memory is no longer in use and kill it. Memory is considered "in use" when it's a global, a variable on the stack, or a variable which can be found by following a pointer on an "in use" object.Python implementations also use interpreted bytecode. But interpreters are slow. SLOOOOOOOW. You can see why by counting the number of actual machine code instructions executed. If I want to increment an integer in C, it takes 1 instruction to fetch the value, 1 to increment it, and 1 to store it. Sometimes, you can even optimize it by storing the integer in a register.... you are averaging no more than 2 instructions to increment to a variable.But what about in Java? Remember, we're counting machine instructions, not bytecode instructions. It takes the same 3 Java bytecode instructions in the worst case.... but each Java bytecode instruction might take 10 or more machine code instructions: read the next bytecode, look it up in a table to see what to do with it, if it has any arguments (an immediate value, a jump offset) then read that too, jump to a procedure to increment a number, if the number is boxed then unbox it, increment the number, if it needs to be boxed then box it, write the number back to memory, increment the program counter.The relatively new technique for solving this problem is called Just-In-Time compilation. It's still mostly black magic to me, but the philosophy behind it is that you can perform much better optimizations at runtime. Furthermore, code execution follows the 80/20 rule: 20% of the code accounts for 80% of execution time. So a JIT virtual machine is always keeping track of what functions/methods are being called. If a method is called repeatedly, it's said to be "hot". If a method gets hot enough, the VM compiles that functions to native machine code, optimizes it, then redirects future calls to the optimized version. The compiled method may not be 100% same as the old one -- it might only work for a certain subset of inputs. But it's FAST for that subset. It uses the fast compiled version whenever it can, and it falls back on interpretation elsewhere.Finally, I want to mention that the JVM (and most every virtual machine) supports native methods. That is, when a native method is called, the virtual machine hands the request to a shared library or some other C-compiled code. The result is a very fast, but platform dependent library. If you want a fun exercise, try writing your own mini VM.   