   Multithreaded Resource Loader
  I've decided I want to make my file system/resource loader multi-threaded and also cross platform. I'm just wondering what sort of design ideas and tips everyone has on this topic.I'm also a bit confused as to how you would manage loading a resource in a work thread and then knowing when the resource is ready to use by the renderer. Would the resource have an IsValid() function or something and when trying to use the resource in the renderer you would check that?In terms of using it in DirectX I'm guessing you would load the raw data in a worker thread and then in the main thread call the D3DX functions to create the DX resource. Again, I'm unsure of how you determine when the raw data is loaded and ready for a D3DX function call to use it and also then what is valid for the renderer to use.Cheers!   
  I'd use one of two designs.1) Request queue + response queue:Main thread(s) push filenames into a request queue.Loader thread(s) pop filenames from request queue and load them (sleep if queue is empty).When done, loader thread(s) push pair<filename,data> into response queue.Main thread(s) poll the response queue each frame to see if any resources have arrived.2) Futures:Main thread allocates a 'future value' and keeps a pointer to it:struct FutureResource{  FutureResource() : data(NULL), loaded(false) {}  byte* data;  atomic<bool> loaded;};Main thread then pushes a pair<filename,FutureResource*> into a request queue.Worker thread(s) pop filename/future pairs from the queue and load the files into the future's data member variable, and then set the loaded flag to true.Main thread polls the loaded flag of any FutureResources that it's waiting for.Some articles that might help out:Writing Lock-Free Code: A Corrected QueueAvoid Exposing Concurrency: Hide It Inside Synchronous MethodsUse Threads Correctly = Isolation + Asynchronous MessagesSome other food for thought: * You can implement this by protecting the queues and the 'loaded' flags with mutexes (AKA "critical sections" on windows). * You can also do it without mutexes for better performance! But it's a lot more complex. * Without mutexes, it's only straightforward if you have one main thread and one loader thread -- a single-reader/single-writer queue can be fairly straightforward to implement without mutexes, but multi-reader/multi-writer is very hard. * If you set the loaded-flag without using a mutex, you will have to use the "Interlocked" family of functions to set the flag and make the flag volatile. This ensures that the flag is only set *after* the data has been written (the compiler and your CPU may try to optimise things and set the flag before the data reaches your RAM, which would be bad). * If you're new to all this, you may just want to stick to locking/unlocking mutexes whenever you access something shared by both threads...   ;Quote:Original post by ShaelI want [...] cross platform [...] DirectXWell, since you're apparently using DirectX, I would forget about the "cross platform" bit, which means you can use an IO Completion port for the request queue. That will make your life a lot happier. If you do insist on "cross platform", it gets a lot harder.Once you have used IOCPs, you will want to forget about everything else you ever used or anyone ever told you about, as they simply rock. They are ultra fast and very intuitive to use once you've dug through the approximately 10 pages of documentation, and they have none of the pitfalls that do-yourself-lockfree has (which is almost guaranteed to end in a desaster unless you're willing to spend 1-2 years on research and implementation).Plus, you can handle your asynchronous completion events (file reads) and load requests on the same IOCP. All in all, they provide an "it just works" solution. I wish Linux had asio and epoll working in a similarly good fashion.For checking readiness in the main thread, you can use a future construct like Hodgman proposed, or you can post an asynchronous procedure call onto the calling thread (need to make sure the loader thread(s) know the respective thread handle). APCs have the advantage of being fast and safe, and not requiring the main thread to "remember" what it requested. Instead, you fire off a request and forget about it. Whenever it's non-critical, you do a SleepEx(0,1); and either nothing happens, or you get poked that something has completed (or has failed) with a pointer to the structure you handed to the resource manager in the first place. I find that much more convenient. It has the additional bonus that you can do allocations and frees in one thread.Alternatively, instead of using SleepEx, you can become alertable "for free" by using any of the other functions that make a thread alertable, for example when you check window messages anyway.   ;Quote:Original post by samothI wish Linux had asio and epoll working in a similarly good fashion.I will have to first admit I have little experience of this but I'm still going to have to raise the fact that this sounds a bit like 'FUD'.First off; Boost ASIO is not Linux kernels responsibility its Boosts, and it lists the Linux kernel 2.4 and 2.6 as supported here.And I see no mention of epoll not working after a quick google.It also slaps my as a little surprising that the most prevalent webserver platform is not capable of Async IO.Now if you can point evidence to the contrary I will certainly accept.;Quote:Original post by HodgmanI'd use one of two designs.1) Request queue + response queue:Main thread(s) push filenames into a request queue.Loader thread(s) pop filenames from request queue and load them (sleep if queue is empty).When done, loader thread(s) push pair<filename,data> into response queue.Main thread(s) poll the response queue each frame to see if any resources have arrived.2) Futures:Main thread allocates a 'future value' and keeps a pointer to it:struct FutureResource{  FutureResource() : data(NULL), loaded(false) {}  byte* data;  atomic<bool> loaded;};Main thread then pushes a pair<filename,FutureResource*> into a request queue.Worker thread(s) pop filename/future pairs from the queue and load the files into the future's data member variable, and then set the loaded flag to true.Main thread polls the loaded flag of any FutureResources that it's waiting for.Some articles that might help out:Writing Lock-Free Code: A Corrected QueueAvoid Exposing Concurrency: Hide It Inside Synchronous MethodsUse Threads Correctly = Isolation + Asynchronous MessagesSome other food for thought: * You can implement this by protecting the queues and the 'loaded' flags with mutexes (AKA "critical sections" on windows). * You can also do it without mutexes for better performance! But it's a lot more complex. * Without mutexes, it's only straightforward if you have one main thread and one loader thread -- a single-reader/single-writer queue can be fairly straightforward to implement without mutexes, but multi-reader/multi-writer is very hard. * If you set the loaded-flag without using a mutex, you will have to use the "Interlocked" family of functions to set the flag and make the flag volatile. This ensures that the flag is only set *after* the data has been written (the compiler and your CPU may try to optimise things and set the flag before the data reaches your RAM, which would be bad). * If you're new to all this, you may just want to stick to locking/unlocking mutexes whenever you access something shared by both threads...Thanks for the thorough explanation. That makes good sense to me. The only bit I'm still unsure of is how it links in with the rest of the program knowing what is ready to use. My current thoughts are as followed:- When asking for a resource I would get two objects; a FutureResource and an IResource (Texture, Mesh, etc inherit this and are used by the user).- The FutureResource would work just like you have explained. Once the Main Thread polls the loaded flag and it is true, it would find the IResource associated with that FutureResource and pass it the data to create the resource and set a flag in the IResource to say it is ready to use.- The user then, before using any IResource checks IResource::IsValid() to determine whether the resource is ready to use by the renderer or whatever needs it.How does that sound?PS. What is the equivalent of atomic in normal C++?[Edited by - Shael on April 14, 2010 9:17:01 PM];Quote:Original post by ShaelThe only bit I'm still unsure of is how it links in with the rest of the program knowing what is ready to use. My current thoughts are as followed:...How does that sound?Oh ok, I was assuming that you simply wouldn't create the resource until the data for it had been loaded, but if you want to create an empty/zombie resource which later becomes usuable, then your idea should be ok.Quote:PS. What is the equivalent of atomic in normal C++?There isn't one in the current C++ standard (there will be in the next C++ standard, eventually).To make your own atomic variable, you should use a volatile long.However, you can't read/write that variable directly. On windows you have to use the 'Interlocked functions' and on GCC you have to use the __sync_* functions.If you're using mutexes/critical-sections though, then you don't have to worry about using atomic variables.   ;  Hm ok. Just trying to get my head around having the main thread continue doing what it is doing while the resource loader is doing its thing along side. Do you simply just not allow the main thread to move into a state where it needs the resource until it is loaded?For the case where say a game level is already loaded and you dynamically create a new entity which contains a mesh. Would you just not add the entity to the list until its resources are loaded? What I had thought was that the entity would be created and added but it's resource might not be ready yet. So in a render loop it'll call each entities draw method and each draw method would check if its resource is usable before using it.I latter method I don't think I like and would seem simpler to just not add the entity till all its resources are ready.Your thoughts?;Quote:First off; Boost ASIONot sure where boost comes in, I've been referring to kernel asynchronous io support.Nevertheless, Boost ASIO has similar limitations/problems, since it's basically only an abstraction of the underlying OS mechanism. It works just fine on all platforms if you use it as a message queue or for sockets -- in fact, it works so well that you would be stupid to ever use anything different. However, things look a lot different if you want asynchronous file IO as well. No problem under Windows, but try and get it to work under Linux. Then come again and say 'FUD'  :-)Quote:And I see no mention of epoll not working after a quick google.Of course you don't, because first the documentation is not clear about when it works and when it doesn't, and second, it does "work" for the most part. However, it is not immediately obvious what works and how it works, and it doesn't do what you need in this case.For example, epoll will wake up all waiting threads when an event arrives. This surely can be regarded as "works fine", but it is totally stupid. You would normally want no more than one thread to wake up. Seeing how epoll was meant to prevent the thundering herd, this is somewhat ironical.Also, at first glance the documentation for both IOCP and epoll says (roughly summarized in one sentence) "you can wait for events on file descriptors and stuff". Sounds good, just what you need.Turns out that epoll really only works with sockets and some special objects, but not normal files. More precisely, it does "work" with files, but it always signals readiness, whether the next call would block or not. Of course you have to search for quite a while to learn about that.Certainly, there is a (undocumented) feature in libaio which allows you to signal an event object on newer kernels. That will in fact provide a functional workaround if you use kernel asynchronous IO. Except for the cases where it fails (you can find out about this in another thread on this forum). Plus, you'll need to dig through the system headers to find those undocumented function calls. And, it still doesn't work with ordinary, buffered files.My point is, IOCP just works. No hidden "if" and "when", no shit. You throw something at it, then go to sleep, and you can be sure it will poke you when something has happened.Quote:It also slaps my as a little surprising that the most prevalent webserver platform is not capable of Async IO.Funnily, the most prevalent webserver still uses a thread-per-connection model (although there is a more modern "experimental" implementation around by now) and no asynchronous IO at all.It is stunning that this model has been working perfectly well for 20 years and still works just fine today for all but the busiest sites. That's probably because you don't really notice whether a website takes 0.001 seconds or 0.5 seconds to load, and even 2 seconds probably wouldn't make you feel like something is wrong with the server.   ;  *bump*Any thoughts on my actual question? heh   