Handling Projective Aliasing In Shadow Mapping I'm fiddling with simple procedural 3D modeling to make teeny buildings for 3D printing. An Example:





To make the models more readable onscreen, I've implemented basic shadow mapping for a single directional light. Since my scene isn't changing, I just render the shadow map once using an axis aligned box of the scene to figure out what the bounds of the shadow map should be. The shadows and bounds look like I've got all the matrices right, but coming in a bit closer looks terrible:



From the reading I've done, I understand the peter panning and what I might do about it, but the ragged edges, which I believe is a form of projective aliasing, look so bad that I think something is wrong in my basic implementation.

I hacked the pixel shader to show the shadow texel boundaries:



I've got bilinear filtering on in the texture (without it, I get serious shadow acne). Sadly, my attempt at PCF failed too - it looks just as ragged:



Here's the single sample case with bilinear filtering disabled:



Does this look like "typical" projective aliasing? Could dynamic rendering of the shadow map, using the view frustum clipped to the scene, possibly with cascading, address the issue?

Edit: Adding a closeup of bilinear filtering, post shadow comparison, just to show what I get. Bad shadow acne shows up because of interior edges; I'm modeling with stacked virtual blocks and not doing a proper union operation. From my reading, implementing boolean operations on polyhedra is not trivial, but will also allow me to implement static shadow volumes as well as clean up for some 3D printing software.
Shadow mapping with acceptable quality is quite a journey. So you implemented first step - basic shadow map that encompass whole scene statically. This means that shadow map texel size in light view space to texel size of rendered scene in camera view space ratio is quite big, resulting in aliasing. To reduce this ratio closer to 1:1 there are techniques you mentioned like:


the easiest way is increasing size of your shadow map, to find the value with still optimal performance,
align shadow map with camera view frustum. Then shadow map encompass smaller part of the scene, so it reduces aliasing,
this can be improved to CSM, which does the same but you need to render scene for each of cascade with different center position of light view matrix based on center point of each slice of frustum.


Some of the basic techniques of smoothing the edge of shadow:


obviously PCF, but is it recommended to combine your PCF implementiation with built in hardware PCF (in Opengl - using sampler2DShadow). This would result in less loop iterations from your side and cheaper PCF,
blockiness of edges can be exchanged for noise using rotated Poisson Disk. Samples used to sample from shadow map are rotated randomly with each iteration of PCF loop.


You can also research the edge smoothing techniques like:


rendering shadow map to MSAA applied framebuffer,
further enhancing PCF by interpolation,
increasing number of samples with temporal blend.


PCF basic technique suffers from shadow acne which can be fixed by applying some offset in shadow test. Calculating the size of this offset is also area of research.

Other more advanced shadow map methods include:


Exponential Shadow Maps and Variance Shadow Maps - in most cases do not suffer from shadow acne and offer smooth edges without blockiness, but in cost of some light leaking situations,
Exponential Variance Shadow Maps - probably most advanced technique of 2D depth map shadow mapping, combines great smoothness of VSM and resolves light leaks, in a cost of memory usage doubling,
Percentage Closer Soft Shadows - basic technique of enhancing traditional shadow maps with varying penumbra size,
Multiview Shadow Maps - multiple shadow maps of some technique, arranged on area light, for computing realistic soft shadows with varying penumbra size,
Deep Shadow Maps - for enhancing volumetric scattering with shadows,
Real time ray traced shadows - probably the future.


You didn't fail with PCF, it looks like that:)

I wish there was a way to apply gaussian blur on shadow map in camera view space, but it doesn't work as expected.