   How to structure a game engine.
  I've been working my way step by step through designing my first 2D game engine. I've done similar projects in the past (actually more like attempts) and I always get hung up at the same point.    How do I separate the different parts of the engine?I know how to use classes, create DLLs, precompiled headers etc. as well as how to use design patterns such as the singleton, and I use them in my code where I think it's appropriate. However every time I think I've successfully separated my classes and code so that I have something like an Engine which runs a Graphics system, an Input Manager, a File I/O manager, etc. I can't take the next step and actually build anything upon this because I quickly realize all of these are so interconnected in order to function properly that for something like a sprite to be rendered it has to know about all of the inner aspects of the engine itself, which just doesn't seem right.I guess what I'm asking is if there is a way to do something similar to what HAL does on an Operating System that works between the engine and the game code, an EAL?I've searched all over the web for good articles on how to design an engine, but they all just tell me the different pieces that I need, not how they communicate. I've thought about events but I have little knowledge of how to implement them either...   
  It seems to me that you focus too much on classes while programming.It's never a good idea to rely on implementation, so no code of yours _ever_ should use a certain class directly.The trick is to stick to the rule "program to interfaces, not implementations".Interfaces define the outer receptance of a group of classes, a single class or a complex sub-system.The thing is, you don't know what's behind an interface, and you don't have to -you simply stick to using what the interface offers, period.Why this imperative for abstraction?It keeps your code clean. It keeps your design clean.And, if done properly, it reduces coupling between classes.Let me give an example...Imagine you have your scene graph update all the items in it.If you stick to just classes, you'd have to differentiate between huge amounts of classes to do so, but if every class requiring updates implemented an IUpdatable interface with a public update() method, you could treat all kinds of objects the same way, and could be presented with all kinds of objects without even knowing their implementation at all - they could be even outside of your own assembly unit, so e.g. plugins written by some users or whatever you could possibly think of.All of these classes would simply just follow one rule - implement a method called update(), specified by an interface IUpdatable.I hope you get the idea ;)Another thing you mentioned is design patterns - especially Singleton.Use them with caution - they tend to bloat your code when you don't fully understand them, as in the saying "When all you know is nails, everything looks like a hammer".Design patterns are great in many places - but, in many places they aren't, because you're not using the right pattern at the right place.And using the _wrong_ tool to do the job just complicates things.Finally you spoke about events.Events and event-driven programming are a complete paradigm of their own, which is the right tool for some applications, and for some it simply isn't.The problem is, events are ideally suited for asynchronous operations, as they are non-deterministic by nature.For use in a game engine, at _some_ places events are just fine and have their advantages, but at a greater scale, they make things worse.Events can be used to locally communicate changes in state between objects, but they do at a cost (event handler execution, instantiation of event objects, destruction of event objects, and possibly raising other events, too).This cost is it which has to be taken into account when using them.For example, updating an entire engine with events, where each and every object receives the same event, may seem thrilling - but, updating objects with method calls may be more efficient.This may seem a bit negative, but there is no super-remedy for everything in software design.All these things are a bit generalized, though, maybe it would help you more if you could give some insight into your engine as it is right now, so one could point out possible solutions.[Edited by - lucem on January 30, 2008 5:21:18 PM];  This is a common question and one that has a lot of answers.  Some answers are better than others, but for the most part there are shades of grey.  I can tell you my thoughts.  Others will have different opinions, equally valid.I use a hierarchy of C++ classes.  I'll have a top-level "GAME" class, which has (as private members) a GRAPHICS class, an INPUT class, a SOUND class, and so on.  I called these classes "systems" or "subsystems".  For example, I'd call my SOUND class a subsystem.  It exposes a public interface to do the job and contains a number of subclasses and objects as private data.I think it’s good practice to try to minimize subsystem dependencies. If one subsystem has to know a lot about other subsystems, it may mean that the way you’ve architected what data and functionality goes into each class isn’t optimal.I recommend that you architect your classes so that they can do their job without having to know about other subsystems.  Put any interconnection code in a higher level system that knows about both of the subsystems that need to work together.Let’s take a simple example of a keyboard input subsystem that takes care of the low level stuff associated with reading key-press events (and possibly re-mapping them based on key assignments), and a player subsystem that has a bunch of actions that can be performed. If implemented the way I'd recommend, the keyboard subsystem doesn’t need to know anything about a player, and a player class doesn’t need to know anything about the keyboard. Instead, there needs to be a higher level system that knows about both the keyboard and player subsystem. For this example let's just say that our GAME class is going to be the higher level system — but really it’s any appropriate higher level system (and doesn't have to be the top level system). Anyway, the game system ties the keyboard input to player actions via some method. Since this is a simple example, I’ll say the game system will, each time through its main/update/tick loop, ask the keyboard input subsystem if the “jump” key is pressed, and if so, it tells the player subsystem to jump (by, for example, calling a player::jump() function).I try to avoid singletons and globals as much as possible, because they introduce interconnectivity where you really don't want any. I think the advantage of avoiding singletons and globals is that you don’t need the player or keyboard to know about each other. This improves compile times, but much more than that, it makes your code very modular. If you want to add joystick support, you don’t have to change the keyboard or player classes at all. Likewise, if you want to add network support, you don’t have to touch your keyboard or player classes. If you need to re-write the keyboard class, you don’t have to touch your player class.  Also, if you are working on a different project later, you can often just pull out any subsystems you might need and use them in a completely different application without modification.  It's almost like each subsystem is a separate library.This example is pretty simple, and I might just be telling you things you already know. But it extends to more complicated cases without trouble and alleviates headaches down the line, even if it takes a bit more work (and a bit more thought) to code than using a convenient singleton or global.Again, just my 2 cents based on my experiences -- I'm not saying other ways aren't just as good or better -- this is just what I do.   ;  I'd say, don't build an engine, build a game instead.After all, that's what game engines are for anyway: to make game development easier by taking care of common functionality. But how can you tell what is common if you haven't made any games before? How can you tell which approach worked and which didn't if you've never gone through the process completely?I've tried creating some 2D engines before too, but at some point, I simply decided to build games. The interesting part is that, once you've built a game, you can usually reuse some of it's code. After a few games, you can end up with a vastly improved framework or engine that has proven to be usefull. It worked for me. :)Anyway, since you mentioned singletons, you should be aware that, in most cases, they're just globals in a shiny design pattern wrapper. Since they're globally accessible, it's easy for anything to access them. As a result, as venzon already mentioned, you'll likely end up with many dependencies. If some system needs to know about another system, pass it a reference. If you end up sending many references, it's a clear sign that your design is flawed and that you should spend some time thinking about what dependencies you really need.   ;  I do pretty much the same thing with my games. If its done correctly it works great, if not, well, it has a tendency to paint you into a corner as you've discovered.Now, as for that nasty use of singletons you've alluded to... Please learn to live without the Singleton pattern. As the other poster said, they tend to introduce a sort of indirect coupling between systems that use the same singleton. Furthermore, singletons are really just glorified psuedo-OO globals and they also have potential issues with their order of creation.Instead, locate systems in the smallest scope in which they can accomplish their duties and pass references to those who need access.Also, adopt some other practices and patterns such as the Single Responsibility Principle, Message Passing and/or the observer pattern, and the Visitor Pattern, among others.Also, I'm very much agreed with Captain P: build a game, not an engine. What that means is that you should come up with a specific game and develop an engine to support exactly the features it will require (and no more!) -- Some of this code you will be able to carry into your next game, but most will be re-written or thrown away. What you do keep will probably be made more general. Over the course of 3 or more games, your core code-base will grow into a framework of general and proven engine components.Now, it doesn't mean that you shouldn't plan ahead or write hackish code full of magic numbers, it simply means don't try to engineer your solution for all sorts of crazy hypothetical use-cases. What it does mean is that you should try to write an elegant solution for the exact requirements at hand.   ;Quote:Original post by Captain PI'd say, don't build an engine, build a game instead.After all, that's what game engines are for anyway: to make game development easier by taking care of common functionality. But how can you tell what is common if you haven't made any games before? How can you tell which approach worked and which didn't if you've never gone through the process completely?I've tried creating some 2D engines before too, but at some point, I simply decided to build games. The interesting part is that, once you've built a game, you can usually reuse some of it's code. After a few games, you can end up with a vastly improved framework or engine that has proven to be usefull. It worked for me.AMEN.  Make a game, and make it playable.  That'll teach you so much more than making an 'engine'.  There's always a balance between making your game work as quickly as possible, and making your code elegant.  Don't spin you wheels, bogging down it designing a proper 'engine'.   ;  Maybe it's a good idea to differentiate a bit on that...It's always a good idea to put some thought on what you need and how it's going to work before you start coding, and laying out an "engine" is just one way to do it.The thing is, for one-man-projects, the word "engine" should not have the same meaning as in large-scale development teams.An engine can well be some sort of abstraction from the libraries used, with some glue code attached.That's just fine, and it helps you keep your focus on the game.You should not, however, start thinking about how to organize your game data in an engine, or about game code in general, in terms of an engine.So, you can design some classes which do the painting of sprites for you, and add the capability of playing some sound.You also can design a class that takes input device data and translates it into a more generalized form.Something like that, with a bot of experience, is what you can call your "engine", and build on top of that - but, do your self a favour and keep things simple.No one ever completed a program once he started writing frameworks....   ;  Wow, a lot of valuable info there! Thanks for the quick responses!First off regarding the singletons, could one of you possibly give me a brief example of when to use a singleton vs. when not to? At the moment I'm implementing it for a few things where I don't see a reason for there to be more than one instance of ever (My File I/O, Input and the Renderer to name a few), is this wrong?Venzon your description involving one top-most system operating many subsystems with their own private subsystems is more or less the design approach I've taken so far, as well as in the past. My problems come when I want to start drawing objects and I see two possible options:A) I include the rendering code headers in the source for the main object class, and then in a Render() function or something similar for the object I want to draw (let's say of type "Sprite") I could all the drawing functions possibly from some interface as Lucem suggested.B) The more complicated idea that I like more is that I store all relevant rendering info in some sort of structure with the objects I want to draw, and then have the engine pickup this info and feed it to the interface when it's rendering time. (My main problem with this seems to be that it just seems like a sloppy solution, if it isn't please tell me so.)Also on the idea of using interfaces, my main problem with that idea in the past (and really the only reason I haven't tried them yet) is that I always figured I would just end up with a header file that had functions defined, that did nothing more than get the sole instance of the render, and call the same function from it (thus removing direct communication with the render itself). After reading your comments it looks like I might have the wrong idea of how such an interface should be structured...Finally, Captain P I like your idea of building multiple games until I've refined my code into something like an engine, my only problem with that is that I've written small games before, I could write pong or tetris in a day or less depending on how much to add, the problem is that doing so would mean deliberately ignoring certain things such as a manager for input versus just checking for 4 or 5 keys that only work in THAT game, or something similar for other aspects.EDIT: Some of this has been answered by you guys in the time it took me to write this originally, thanks!   ;  Singletons:Simple - use them never.When you run in a situation where you need them, you'll see for yourself.In all other circumstances, when something's crying for Singleton, ignore it.Singletons are more of an anti-pattern today, with good reason.After reading what you wrote about your experience with interfaces, it's obvoius to me that you haven't understood object oriented programming at all.There should be no direct interaction of a game object with the renderer.A game object is responsible for managing it's own data, such as it's world position (not screen position!), and the way it behaves.It's not responsible for drawing itself, it's not responsible for updating its screen coordinates, if visible, it shouldn't even know about the fact that it's a renderable object at all.The renderer is responsible for drawing everything, finding the right sprite to draw, and all stuff graphics.So, how to do that?Well, there comes in an architectural pattern that's called MVP.Model View Presenter is about a three-tier design where there is(in this case) game data (model), a renderer (presenter) and someonein the middle that intermediates between them (view).So, in your case, the view is the center of all attention, and it interacts with the model data, and is interacted with by the presenter.The presenter doesn't talk to any game object in the model tier.It gets all that there is to know using a defined interface that is implemented either by view alone (using delegation of certain request to model) or by view and model.So, each system is responsible for one task only, and using interfaces here provides you with the possibility of e.g. re-using the renderer in the next game, while the view and the model change.;  Ahh that makes more sense and gives me another direction to head in.I'd like to just say that I truly do understand how OOP SHOULD work if it's implemented correctly, I just tried to emphasize that I don't quite understand how to do this myself to the extent that something larger than a few-hundred lines of code would require. With my interface examples I was just trying to say that I only saw two possible ways of doing it at all, example A: the truly wrong way that was never really an option in my mind (now that I read that again I can see that I didn't really show that I wasn't considering it, a mistake on my part) and example B: which actually doesn't seem to be what you suggested I tried, but also isn't what you said is a horrible idea either (it wouldn't interact with the renderer directly, and that private structure of display information for the sprite class constitutes being responsible for managing its own data (texture to use, world-position etc.) if not, let me know.Also when I saw that the sprite class would know what texture to use, I'm referring to it having some sort of ID tag and the renderer would look up the texture it corresponds to.Oh one more question. With this "MVP" architecture, if I'm using DirectX for example in my renderer, what would each part of this be. Obviously model is self-explanatory, but in this case would I even need to write a "renderer" at all? or would "view" just be an interface between the model and the DirectX functions needed to display it?[Edited by - ChugginWindex on January 30, 2008 6:37:55 PM]