When to use an RTOS?
You would use an operating system when you want or need to isolate the tasks performed by the system so that each can be implemented with no implementation knowledge about the other tasks. It's a sliding scale how much of an operating system you have.

So, if you have a piece of hardware that allows you to define an interrupt handler that reacts to a GPIO pin, and put another function into the main loop, your operating system is fairly minimal: set up the interrupt handler and then start the second task, no further arbitration is needed.

If the reaction to the GPIO pin causes something to happen inside the main loop, and for example, the loop should immediately handle that event, you need some coordination, which can be as simple as setting a flag and checking it in the main loop. At this point, your code has gained knowledge of the implementation of other code running on the same system: the interrupt handler knows that it can write to a variable that the main loop will read periodically, and the main loop knows that the variable can change due to external influence.

You can however generalize this code: the main loop, when idle, calls an idle() function, and the interrupt, when it wants to signal a state change, calls an action() function. Now, you have abstracted the different modules from each other, the beginning of an operating system.

Several improvements to the operating system later, you use a system-defined main loop that iterates through a list of tasks to be performed and keeps track of the time allotted to each. Adding another task to the system does not require modifying any of the existing tasks.

A RTOS is a special kind of operating system that also ensures timely response to external events, by further limiting what individual tasks may do; for example a typical requirement is that task interrupt handlers run while interrupts are unlocked, so a single handler failing to return in a timely manner can be preempted.

Whether an operating system, with or without realtime support, is required depends on your application and the constraints it is working under.

For example, if the hardware supports writing data to memory directly and notifies your application once some data is ready, you have time until the buffer is full to actually fetch the data.

If you miss that deadline and the data lost can be recovered, or if the condition is unlikely enough and not dangerous, you do not need any realtime capabilities. Examples for these are Ethernet on desktop computers (lost data is simply retransmitted after a timeout) or a keyboard (people type slow enough that you have several seconds to collect the data).

If the data is lost after missing the deadline, that is a "soft realtime" condition: recording a sensor value from a buffer would fit here. You cannot correct the error if you are missing values, but you usually have lots of time to collect the data because of the buffers.

"Hard realtime" are constraints where action needs to be taken within a certain timeframe, and failure to do so would have consequences; the most extreme example would be the emergency stop button on a machine, where you immediately need to reverse motor power to bring the machine to a stop as quickly as possible, and as soon as the motor is stopped, turn off the power.

You usually have a mixture of these in your system, and the hardest problem defines what kind of realtime support you need in your operating system. If you can offload "harder" problems, for example by handling the emergency stop button in an interrupt handler and everything else in the main program, or by building a hardwired stop mechanism that is not controlled by the microcontroller, you can often reduce the complexity of the programming significantly.

For example, I have a CNC mill that has a bunch of 74xx counters and gates that will inhibit motion more than 10 steps (for braking) across the endstops , when the emergency stop button has been pressed or if the machine is open, no matter what the microcontroller does, so programming bugs have less of a safety impact. The step generation code lives in a high-frequency timer IRQ with highest priority, so step generation will be cycle exact (hard realtime requirement to keep motion smooth), while G-Code interpretation works on data that is received either via the UART (soft realtime), USB (bulk, no realtime) or Ethernet (TCP with retransmits, no realtime).