   Coroutines under Windows, Linux and Mac
  Hello,I am now considering making my application cross-platform (for the above mentionned platforms) and am evaluating various sources of problems.One of them are coroutines. Currently my application is implemented for the Windows platform, and makes uses of fibres. I have been researching about a portable coroutine library, but apparently the implementations are difficult and not working flawlessly (why?). Maybe it is because my knowledge about other platforms is equal to nil, or because I don't know enough about how operating systems work, but I was wondering why this would be so difficult. In my understanding, all that is needed is an equivalent os-specific function to start/end a coroutine and a few other routines, like:ConvertThreadToFiberCreateFiberSwitchToFiberDeleteFiberGetCurrentThreadIdSo my question is: don't I simply need to implement the equivalent functions available on Linux and Mac? Then in the source code put preprocessor commands like "#ifdef LINUX" with their respective coroutine functions?Or am I missing an important point?Thanks for any advice or clarification!   
  IIRC, the equivalent of the Windows Fiber functions in POSIX is  setcontext family of functions.   ;Quote:Original post by floatingwoodsbut I was wondering why this would be so difficult.Due to the fact that each kernel is implement differently. C and C++ have no concept of operating system (scheduling, stack, process) and as such can only use each OSes syscalls. Other languages do not implement fiber-like functionality due to inherent and unsolvable problems with their design - threads do just fine.Quote:In my understanding, all that is needed is an equivalent os-specific function to start/end a coroutine and a few other routinesYes - but in most other kernels there is no viable userspace equivalent, at least not adequately standardized on.Apple offers GCD which serves the same purpose.Google's Go offer similar functionality, but Go is poorly supported on Windows.For POSIX platforms setjmp/longjmp can be used to simulate this but with plethora of pitfalls.Another way to achieve it is via manually generating cooperative multitasking via manipulation of stack pointers and choosing appropriate calling convetion - this is CPU-specific solution due to use of assembly generation (Go uses a trick to achieve this).Quote:don't I simply need to implement the equivalent functions available on Linux and Mac?No - their process/thread models are incompatible with Windows API.Quote:Or am I missing an important point?Why use fibers? Between the problem with exception propagation, lack of stack smashing protection and detection, the difficulty of managing ownership for cleanup and reliable scheduling that avoids freezes (Win 3.1 and below), a robust solution means reinventing the OS.As far as cost effectiveness goes, finding a good thread job dispatch mechanism (GCD, TBB, even boost::asio) or using manual dispatch via select() loop will be simple, portable and robust.Alternatively, use a language which supports coroutines as first class members.Quote:IIRC, the equivalent of the Windows Fiber functions in POSIX is  setcontext family of functions.If is, but POSIX, despite fancy sounding name, has all kinds of compatibility issues even among (supposedly) POSIX-certified platforms. For anything cross-platform, the number of workarounds will not be worth it.If working on true Unix, then relying on certain behavior is viable, but mentioning Mac may quickly mean ARM and as soon as difference architectures are mixed things get tricky fast.IMHO, low level portable development for sake of portable development isn't viable anymore, at least not without a solid business model that support it.   ;  Thank you very much SiCrane and Antheus for the lengthly explanation!Well, it seems I underestimated the disadvantages of using fibres in my application. Actually I decided using them because I need to synchronize threads among them (i.e. I need to control when (in terms of main thread location) an auxiliary thread gets started/resumed and when it switches to give control back to the main thread (switch controlled from within the auxiliary threads)). Would this also be feasible by using traditional threads? From my understanding yes, but it might require calling "Sleep" more often than desired (everytime a given thread should execute a bit more, but already automatically switched, then other threads should call "Sleep" until that thread get a bit more execution time), effectively reducing execution speed of my application... because there is no other thread control mechanism than "Sleep" IIRC   ;  Sleep() is never a good solution to anything.Manual switching is also never a good idea, unless the underlying task scheduler is known and deterministic or if it's a hard RTOS.yield() can sometimes be used for specific parts of code.In my experience, the above is indication of incorrect design abstractions.Even if problem does benefit from microthreads, such as one way of designing an AI, there are some inherent ordering and deadlock issues which are usually solved by message passing.Quote:because I need to synchronize threads among themWhich is why fork/join or map/reduce was invented. It's standard approach available in almost every parallel toolkit.Fibers (green threads) are typically used since they avoid complicated resource management. In VMs they are a cheap and popular way to solve concurrent garbage collection problem and external C binding issues with implicit static state. They solve a very different system-related problem. Windows Fibers also come with tons of gotchas which make them very annoying to use and frequently for no inherent performance benefit.Almost all uses for high-level application design prefer a much higher level implementation with explicit state storage, such as continuations or functors. Scripting languages often keep their own per-function state so they avoid OS issues completely.   ;Quote:Origional post by floatingwoodsFrom my understanding yes, but it might require calling "Sleep" more often than desired (everytime a given thread should execute a bit more, but already automatically switched, then other threads should call "Sleep" until that thread get a bit more execution time),You might just be looking for condition variables. You'd tell threads to 'wait' on a CV, and then some other process would 'signal' the CV when there is more work to do. So, the standard example of this would be a producer-consumer. The producer adds items, if it was the first item, it signals the consumer. If the queue is full it sleeps on a "isFull" CV. The consumer is waiting on a "isEmpty" CV, and when signaled will wake up and process data. If the queue was full, it signals the producer to start producing after it eats the first item. If the queue becomes empty again, it waits on the "isEmpty" CV again.   ;  Thanks again Antheus and thank you Kulseran!Yes, I knew that Sleep was somewhat evil, and until now I didn't realize there were other mechanisms allowing to control threads. Gotta look more into that ;)Cheers!   