   BIG world
  Hi,Thanks to the memory, we see more and more huge world in games (GTA IV / Far Cry 2) that are loading the upcoming part in the background. Textures, geometry, objects, etc. I was amazed by the Far Cry 2 tech video that showed the life of just one of the so many trees in the landscape. These trees actually grow! How the heck does the system store/manage the life of all thousands/million trees? And how to save that while playing? Savegame == the current state of the ENTIRE world?My engine must support something similiar, although my world is indoor (a little bit like Halflife1). There is a little twist though. Games like GTA IV forget almost everything that you leave behind. People/cars/knocked over boxes, etc. Everything spawns at random, and once leaving the scene, the lovely ladies of the memory cleaning services come by.In my case I want to have "global entities". These are active objects that can be anywhere on the scene. For example, an enemy that is trying to search you. I only need to render a tiny portion of the world for the player himself, but that enemy does need some information about his environment as well. Pathfinding (walk from door A to B in scene X). No problem so far, but it would be nice if the enemy could interact with other "global entities" there as well. For example, knock over a table, or attack another enemy that is in the same room. This makes the whole story alot more difficult. I would need physics there, and thus a model of that chamber there. It's expensive, but it opens lots of gameplay options.First I was thinking about using a low-poly mesh of the entire world. This low-poly model is used for physics anywhere where needed, maybe to support some decals that are created outside the players current scene, etc. However, making an extra low-poly model costs time, and it's not really accurate for physics or decals. When I enter a scene where had been a fight, blood will float in the air and objects will stick in the ground because the world physic model != the detailed mesh model.Maybe there is a much better way to manage entities outside the loaded "detailed scene". But I don't want to fake too much. So I was thinking... why not load the entire detailed world into the memory? I'm not talking about textures/lights or tiny objects here, just the polygons. I did some dirty math, and I guess I would need 1 gig memory for the entire world structure, IF I also load tangents, normals, texcoords, ... Average scene polycount = 10.000Entire world ~= 200 scenes --> 2 million polygonsaverage Scene size (polies + portal structure + little overhead) = 5 MB.200 x 5 MB = 1 GIGThe good thing is that I don't have to load the detailed mesh anymore when traveling through the world. And global entities at any place in the world can make use of it. The bad thing is the memory of course. But since modern computers have 2 or more Gig RAM... Why not use it? I mean, why did Halflife2 load such small chunks of the world each time? I know the computers had less memory those days, but still.I'm worried that I forget a couple of things here. First, the physics engine (probable Newton) makes a copy of the world as well. I don't know how carefull Newton is with the memory, but it wouldn't surprise me if 200 MB extra is needed. Are there other important things that I forget or have to consider here? Anyone experience here? Greetings, and sorry for the long post :)Rick   
Quote:Original post by spekHow the heck does the system store/manage the life of all thousands/million trees? And how to save that while playing? Savegame == the current state of the ENTIRE world?My guess: it doesn't. Many things aren't important enough to be saved. Other things don't need to be saved in full detail (for example, you wouldn't store every particle position - you'd simply store the fact that the given particle emitter is already active). That, and you wouldn't save things that are miles away - unless they're important somehow. As for updating, only those objects within a certain radius of the player are important. Some games also employ a LOD-system for their logic: accurate updates for nearby objects, rough but cheap updates for others, no updates for distant objects.Quote:No problem so far, but it would be nice if the enemy could interact with other "global entities" there as well. For example, knock over a table, or attack another enemy that is in the same room. This makes the whole story alot more difficult. I would need physics there, and thus a model of that chamber there. It's expensive, but it opens lots of gameplay options.Does it? "If the player didn't see it, it didn't happen" is a (common?) saying among level-designers.Quote:First I was thinking about using a low-poly mesh of the entire world.That's common for collision-detection in general, yes. Most AAA games use tools to generate a simplified collision-mesh. Then again, most of those games also use editors developed in-house, so that's usually very tech-specific.Quote:However, making an extra low-poly model costs time, and it's not really accurate for physics or decals. When I enter a scene where had been a fight, blood will float in the air and objects will stick in the ground because the world physic model != the detailed mesh model.How about correcting those blood-stains when the player comes close to that particular room? If, of course, you absolutely need fights to actually play out without the player being there.Quote:Maybe there is a much better way to manage entities outside the loaded "detailed scene". But I don't want to fake too much.Games are one big fake. The catch is to make it a believable fake. Let's put it like this: if the player sees no difference between a faked fight and a real fight, why spend all the resources on actually simulating it?Quote:I mean, why did Halflife2 load such small chunks of the world each time? I know the computers had less memory those days, but still.Those small chunks already took quite some time to load on my system. Not that I run a fast one, but loading that stuff into memory is something to keep in mind.Quote:I'm worried that I forget a couple of things here. First, the physics engine (probable Newton) makes a copy of the world as well.Sounds troublesome to me. However, do you really need all of this? What kind of game are you working on? Why does the design demand far-away fights to be simulated?   ;  "If the player didn't see it, it didn't happen" Totally agree with that. But in my case the actions of each character in the world are very important. Besides of the player, a handfull of other characters are traveling around. They should be able to find their path, make damage, interact/fight with each other, change the state of "puzzles" (light switch on/off, lock a door, etc.).Faking their actions is 1 option. You might wonder how important it really is if a turd made by another character at the other side of the world, is in the exactly right position. But when faking it, it must be done good. I don't want pre-scripted events or positions. Because of the destructive behavuours of the other characters, the damage they leave behind is important. In fact, it could be used to trace another character, or vice-versa, they trace you. I don't want chambers to magically reset to their initial state, so I need to store alot of things.I think the low-poly mesh is good enough. Correcting things like you mentioned is not a bad idea. But IF its possible to allocate ~1 gig for a big world... then why not doing it? It makes certain things easier/more accurate, plus it helps loading the next scene faster in the background for the player. But since I never seen another game doing this, I have my doubts of course...>>Those small chunks already took quite some time to load on my system. Not >>that I run a fast one, but loading that stuff into memoryTrue, and that's why I have doubts. So far my ~10.000 polygon scenes are loaded in 1 second or less, alot faster. Maybe Halflife2 uses a bigger variaty of textures that need to be loaded/scaled, big lightmaps, other structures to manage rendering/physics/AI/... ?Maybe their loading time is just terrible. Some games need ages to load, other ones are pretty fast. Crysis for example loads relative fast for example, while this game does have quite big worlds, and lots of high-res textures...Thanks for the adviceRick   ;  Procedural generation of foliage and trees is another option very commonly used. For instance, one big random number could contain all the information for a level - including heightmap, trees, enemies, global politics, amount of crust on the public pies and so on...Another trick is to always store a set of very low detail textures (placeholders) and dynamically load the high-res versions of the textures when needed.Streaming data is a must if you want to have a loading-free environment.Also, storing/loading chunks of the terrain is fully possible. For instance, if another player is doing something in zone C, the player in zone A should not be bothered with that information UNTIL he enters zone C. Simply keep a database of objects in each zone and update these when needed.If your level increases in size, you can't possible hold everything in memory. I guess the reason for Half Life being so slow to load is that it didn't use threaded loading (at least not that I've noticed) and still needs to load physics, shadowmaps, textures, models etc for that new region.(There is however a parameter that allows you to adjust how much you should load at the start of the level, you could very well load the entire level in ram if you have enough)Pathfinding must use zones. I.e use a coarse layer to say that a path between zone A and C exist, and re-calculate the closest path to the next zone upon entry of a new zone./Robert   ;  Thanks for the advice. Lots of terrain data can just get loaded or even random generated at the fly. There is no need to remember each flower position for example. Then again, my game is not really outside for most of the time. I'm talking more about objects like chairs, tables, crates, stuff that can be picked up, and decals that have been generated somehow. Some of them are not important to store all the time. Lots of small detail objects/decals/lights are not needed by the AI, and thus can be unloaded once they become invisible. Other objects are actually important for the AI in my case. For example, a blood decal could be used by an enemy to trace you. So if I really want to store all these things, I have 2 problems: 1.- RAM, 2.- Saving/loading all changed entities.With problem 2 I mean that I must save the state of an object somehow. Not only in the RAM, but also on the hard-drive. If its really true that individual trees in Far Cry 2 can grow, then the 'lifetime' or something should be stored for each tree that uses this system. Not nessesarely into the RAM, we don't care about how big a tree is if we are miles away from it. But when we enter that zone again, the tree lifetime must be loaded from the hard-drive again. Maybe there are smarter tricks, but I assume the "savegames" in such a game would actually contain a copy of the entire world, or at least all the "changes" to the initial state of the world. Right?As for the first problem, its clear that the RAM is not endless. I was just wondering how much we can actually put into it. I'm trying to do "worst case" calculations, but it turns out that my world *could* fit (the geometry + basis entity properties, not the textures and tiny details). That would be nice, but I'm afraid I've made a mistake somewhere. That's why I was talking about Halflife2. The scene geometry can be compared with ours a little bit, but the Halflife2 scenes are relative pretty small...Hmmm, difficult. I think its smart to make a low-poly version of the world, because it allows a bigger overal world. But it also means more streaming, a less accurate world for characters operating outside the player "visible range", and the artist must construct a low-poly version. There are probably tools to do that automatically, but so far we don't have them, and automatic tools aren't always working either. Maybe I should just test with a random generated huge world with lots of entities and look where the memory limitations are.Greetings,Rick   ;  Well one example of a game that has a massive active play area would be the X games from Egosoft, since the entire X Universe is constantly running.Of course no home computer could actauly run the entire universe in full detail, so theres basicly two sets of code for large potions of the game. In-Sector(IS) code for the sector the players in, and Out Of Sector(OOS) code for the rest of the universe. This allows the game to keep track of the entire universe, with things happening (pirate/xenon/khaak attacks for example) throughout the universe, not just where the player is. Everything is also persistant so if the player destroys say a military capital ship, it will stayt dead rather than respawning the next time the player enters the sector.IS you have all the normal collisions and stuff, with combat playing out in real time.OOS is updated at a much slower rate, and most things are reduduced to simple maths calculations (Eg theres no collision detection etc, so combat is just done by maths based on the ships weapons, shields, hull, etc.).A down side to this is some diffrences between the outcome of events in sector.eg IS the PPC's on cap ships are not really a threat to fighters as the bullets are too slow, but OOS PPC's murder fighters since the equations don't seem to take the low hit rate into account.;  Yep, that is what I mean :)You also mentioned that the 'OOS' part is updated much slower. Luckily I don't have that much AI creatures. There won't be hundreds of guys doing pathfinding for example. So, that means that 99% of the entities is doing actually nothing, except parking their @sses on my precious memory slots :)However, IF something happens (2 characters don't like each other and start a riot, clumsy guy bumps over a table, or a character is just scanning surrounding objects looking for clues), it does need physics/collision detection though. If the physics weren't nessesary, I wouldn't need that entire world. Pathfinding can do without for example. Well, having the geometry already loaded is nice for faster loading the next scene when the player opens a new door, but then again, you can also decide to load the current scene plus the neighbours for that. No need to load everything.We're pretty keen on the little details that another character can cause, so even tiny details as a bumped over chair or a decal somewhere are important for our gameplay. So, when we need to create a decal or throw around objects, the geometry is needed there. We should at least have a lower-polygon model that includes walls/pillars. Although the physics outside the view don't have to be very accurate, I don't want find decals floating in the air or ragdolls with their upperbody sticking through a wall. Besides of that, we might want camera's that allow the player to view elsewhere in the world.I think my question is not very clear here, I talk too much :) Basically I already have some ideas/solutions, but I was wondering how "smart" it is to load the entire "detailed" world, instead of working with tricks or lower polygon versions. If the RAM is there, why not use it? On the other hand, I'm afraid I will run out of it much faster than I expect...Greetings,Rick   ;  As far as distant entities interacting with their environment... I'm not sure if "you can't see it then it didn't happen" is the right attitude.  I would be inclined to say "if you can't see it then it doesn't matter how it happened, all that matters is that it happened".  I had done some design on a world that could have potentially millions of entities at once doing "their own thing", so naturally this is one of the first problems to come up.For instance, take the situation of what happened when Guy 1 got into a fight with Guy 2 over trying to flip a switch on the other side of town.  Now it might be quite important that this happens, and it might be a dynamic outcome, one or the other might win depending on some random seed or whatever.  It's not important about the physics involved unless the player sees it.  All that matters is that it happened.Keep track of what the intent is, draw some random numbers up, splatter some blood decals around the area, kill one of the players and place his body close by, toggle the switch, and set a variable indicating the other guy hit the thing.  Nothing actually has to happen.  The game is just told that it did, and the proper consequences/reactions were set up. Cheers!-Scott   ;Quote:Original post by popsoftheyearAs far as distant entities interacting with their environment... I'm not sure if "you can't see it then it didn't happen" is the right attitude.  I would be inclined to say "if you can't see it then it doesn't matter how it happened, all that matters is that it happened".  I had done some design on a world that could have potentially millions of entities at once doing "their own thing", so naturally this is one of the first problems to come up.For instance, take the situation of what happened when Guy 1 got into a fight with Guy 2 over trying to flip a switch on the other side of town.  Now it might be quite important that this happens, and it might be a dynamic outcome, one or the other might win depending on some random seed or whatever.  It's not important about the physics involved unless the player sees it.  All that matters is that it happened.Keep track of what the intent is, draw some random numbers up, splatter some blood decals around the area, kill one of the players and place his body close by, toggle the switch, and set a variable indicating the other guy hit the thing.  Nothing actually has to happen.  The game is just told that it did, and the proper consequences/reactions were set up. Cheers!-ScottThats along the lines of what I was thinking.  If it is far enough away just simplify it to a random number generator outcome.  Then make the situation "look" believable to the outcome.   ;Quote:Original post by popsoftheyearAs far as distant entities interacting with their environment... I'm not sure if "you can't see it then it didn't happen" is the right attitude.  I would be inclined to say "if you can't see it then it doesn't matter how it happened, all that matters is that it happened".  I had done some design on a world that could have potentially millions of entities at once doing "their own thing", so naturally this is one of the first problems to come up.That's exactly what I meant, yes. As long as you don't see it, it didn't happen. As soon as you see the result, then something happened. How it happened doesn't really matter: you weren't there to check it anyway. As long as it's consistent with what you've seen so far it's fine.So I'm still wondering, are those details really important to the game. As in, does the player actually notice. Did you guys prototype this idea already? There are probably quite a few shortcuts you can take without hindering the overall game experience.As for using RAM if it's there anyway, not everyone has an equal amount of RAM. Higher system requirements translate to a smaller audience. That could mean lower possible sales. That, of course, is something to consider with the team. Either way, you don't want to waste resources just because you can - it's going to cost you something somewhere anyway.As for low-poly versions of game-worlds, that's fairly common, typically for collision-detection purposes. Generating them would be the way to go, with some way for the level-designers to specify non-solid things (a few wires should probably not block the player, for example).   