What are the details about link files and startup code one needs to know to write an Operating System for a uC?
Here are some things that spring to mind right away.


You must know all of the details of what happens when the processor comes out of a power-on reset. There will be a number of control registers and these have default values. You need to know these, down cold.
You must know all of the CPU architecture details. All of the registers, special modes using them, etc. These things will give you clues about how to arrange the registers for calling and returning from functions, etc. If you are using C, or want to be compatible with C, you will have to read about the choices made by various C compiler vendors (not always the same choices.)
You must know all of the details regarding various functional units and library code that you will permit to be linked in. Particularly so, if you will support pre-emption. In some cases, you cannot allow pre-emption since there is no way to save the state of some operations that are in progress (the MSP430 multiplier is like this) despite the fact that an interval timer can interrupt that process. In some cases regarding libraries, they also cannot be interrupted since they have static state that would be corrupted by a separate thread accessing the same code. Etc.
Your startup code will have to be linked to the power-on reset address for the MCU. It's job is to do whatever initialization is promised. This might be tabulating memory areas, initializing some memory, and whatever else will be expected in some known state prior to the execution of the first thread/process. If C is involved, this means initializing all initialized static variables per their appropriate values and initializing all other static variables to their semantic equivalent of 0, whatever that may be.
You need to know all of the ways that exceptions can take place and how to differentiate, for example, between a watchdog timer event vs a power on reset event.
The linker input file merely lays out the areas where code and initialized data may be placed and how big they are. It also uses names so that the linking process can assign named code or named initialized data to the appropriate places as indicated in the linker input file. This isn't complicated. But the details do have to be correct. If you are using a C compiler, that C compiler will probably make all kinds of assumptions about the names of sections (segments) and you will have to either obey them or else write a tool that modifies the object code before linking.
You may need to write a special tool for linking, anyway. This can happen if you have to support multiple C compilers generating different named segments in their object files; or an assembler output that also uses differing names; etc. Sometimes, the routine prologues and epilogues don't make the same assumptions, even, and you may need to patch those things prior to linking. You may also need to write a patching tool, anyway, to patch or "fix up" addresses that are specified in the object files prior to linking. (I've had to do this with 6502/65816 processor code destined for ROM file production with weird memory-mappers that are different and also incompatible with each other.)


And that's just a very short list that flows from my fingers without thinking about it. I'm sure that if I spent another 5 minutes, I could double this list. (For example, I've not even addressed anything about what a debugger might want in terms of information and/or modified or inserted code to support its operations. Nor have I discussed the differences involved in Harvard vs von Neumann memory systems. Nor have I discussed the standard "program model" of organization [code; constants; init data; uninit data; heap; stack, etc.])

I'd recommend taking this in slow steps. Since you claim to know already about bare metal programming and know about operating systems, as well, let me simply recommend that you read the very first XINU book by Douglas Comer (it is circa 1984, has a red cover and there is no volume 2, etc.) Then see if you can cobble up a cooperative switching operating system. This means NO PRE-EMPTION. It means just THREADS -- not complete, separate processes -- but threads that share the same code space, constants, static data, and heap; with the only difference being that they have separate stacks. Support a switch() call to make this cooperative thread switching work. It must also support hardware driver events without accidentally overflowing some thread's stack in the process. You need to carefully design how you handle hardware events and their driver code. (I split this into low level hardware response code + high level thread-accessible code separated by buffers to decouple the two from each other so they can operate independently, while also supporting the hardware fully.)

If you want to go the next step, add inter-thread messaging. Use a simple word -- just one word -- for each thread and let any other thread write it. Overwrite it, in fact. Don't make this complicated. See if you can get single word messages going between processes okay.

Then add a sleep queue and provide a timer that can move threads from the sleep queue back onto the run queue.

Then add semaphore queues.

If you can get this far, you will have learned a great deal.

And by the way, it took me less than two working days (Monday, until early Tuesday afternoon) to get all of the above working -- from scratch and without a single line of code from prior projects, so just typing as fast as I could think -- including a complete mixture of assembly and C to handle hardware events and so on. I had run/sleep/semaphore queues, timers, and co-operative threads along with per-thread exception handling added ... in less than two days.

So this is NOT a particularly difficult task ahead.

Have at it.