How can a computer run an infinite loop?
In a contemporary processor there is, among many other things, a register (digital electronic component to hold some bits) called the Program Counter (PC). It holds the memory address to the current machine instruction.

During normal flow of the program, the PC will always be updated to address the next instruction. However, any loop will be implemented with so called branch or jump type instructions which will cause the PC to address some other instruction, rather than the next one.

One, if not the, simplest infinite loop, is one in which the PC is updated again to the same instruction.

An assembly (MIPS) code example (beq stands for Branch on Equal):

LABEL: beq $0, $0, LABEL # Comment: If 0 == 0,then goto LABEL.