How to probe into the internal signals and registers in FPGA without using JTAG?
One of the most useful ways to gain access to the internal register contents is to design it in!!

First you have to decide the most convenient mechanism with which to access the FPGA. You could use a SPI interface, UART, I2C ... what ever suits connection to your monitoring device.

I one instance I decided to design in monitoring through the use of LEDs connected to the FPGA. I had registers up to 16-bits wide in the FPGA so I connected 16 LEDs up to output pins in the FPGA design. I also equipped the design with a set of four jumpers that were inputs into the FPGA design. In the logical design of the FPGA the LED outputs were connected to an internal MUX that was 16-bits wide. The inputs to the MUX were connected to all of the various registers and counters that I wanted to be able to monitor. The selector controls for the mux channel came from the four jumper pin inputs. This allowed me to monitor up to 16 different internal sources of information (although I only needed to use 11 or 12 for that particular design).

I could easily watch the internal state machines and counters operate by viewing the LEDs. During the debug phase I needed to be able to monitor more closely than what you can view on an LED and so each of the 16 outputs also had a test point attached where I could connect my USB logic analyzer to capture the selected outputs in real time.

In your case most general purpose FPGA evaluation board provide for switches that connect to the FPGA as inputs and a set of LEDs driven by outputs of the FPGA. You can leverage these existing resources to implement exactly what I did (in my case I had made my own board). If your kit support only 8 LEDs and you have larger registers you can design the MUX to show the upper and lower halves of the register via two separate MUX selections from the switches.