Design of Arithmetic Section
The diagrams below are from a textbook called Computer Organization and Design: The Hardware Software Interface: (ARM Edition) and this can be found in Appendix A. While it's not the best textbook on the market, there are some pretty good diagrams. These diagrams below will show you exactly what's inside an ALU.

An ALU is typically designed with these logical and mathematical operations: AND, OR, ADD, SUBTRACT, SLT (which is a comparison operation "Set if Less Than"), and NOR.


This is your typical 1-bit ALU. Very simple. The oval shapes with the 0 and 1 represent a multiplexer that controls whether or not bits A or B negative (by using the 2's compliement). The oval shape with the 0,1,2, and 3 is another MUX that controls which mathematical or logical operation is performed and lastly, that box with the "+" symbol indicates a full adder. There are other components like the "overflow detector" that does exactly what is sounds like. You also have the "set" flag from the SLT operation. This is just a simple 1 bit ALU. Now let's look at a 64-bit ALU.


I know what you might be thinking: "Why do the numbers on a63 and b63 look different from the other numbers?" Well, it's because I had to photoshop some typos that were found in the textbook (told you that it's not the most ideal textbook on the market). But I digress...

Now, as you can see. There are more operations! Now there's a zero detector with that giant OR gate the end with an inverter. If you're basically adding 0+0, the OR gate at the end with send out a zero then the inverter will make that signal a 1 to send out zero flag.

SO TO CONCLUDE WITH YOUR QUESTION: You're right to assume that ALU performs arithmetic operations and that's how computers can add, subtract, compare, etc. particular data that is given by the user. You can also create your own operations like multiplication or division but multiplication or division in binary is essentially recursive addition or subtraction. Hence, when you look at the Assembly language, the multiplication and division operations are pseudo instructions, meaning that it really uses addition and subtraction instructions to perform that instruction and thus the execution time typically takes a slightly longer time to execute.