I have recently been doing a study of Unreal Engine's, still experimental, Procedural Content Generation system. It's been an exciting experience using a procedural system like this for the first time and I'd like to share the current status of my project in this blog post. Eventually, I would like to bring this study farther along and create a full post for it, showcasing a higher level of quality and polish but for now a blog will do while since I have to shelve it temporarily.

Up until this point in my 3D journey, the farthest I had gone in procedural workflows had been within the Substance Designer texture graph and the Blender and Unreal material graphs. I always found working within systems like that to be satisfying, getting to lay out the logic in a customized way to achieve exactly what you need in a non-destructive workflow that's super easy to iterate on. When I found out Unreal Engine had their own system for in-level real time procedural generation, I jumped on the opportunity to learn it and started planning out what I wanted to create.

To begin, I started researching some of the basic systems and fundamental nodes PCGs had to offer. I found out right away that the documentation and amount of tutorials regarding the subject were much fewer than other systems I am used to learning, I imagine due to it still being in beta (as of 5.6). Although the amount of support online was fairly slim, the community never ceases to disappoint and I was still able to learn all the basics from videos and forums. Shout out to pwnisher on YouTube for his super well laid out introduction to PCGs tutorial.

After I had a grasp on the fundamentals, I began sketching some concepts for an asset kit and what kinds of interactions I wanted to happen between them. I imagined a snowy landscape in which I could draw zones of different types of forests, mountainous terrain, paths, or even rivers through, where the zones would understand how to interact and generate alongside one another. I wanted to create a system that would make it super easy to drop in a few spline blueprints and say "you're the spruce forest zone" and "you 're the mountains" and have them just generate what you need within the zones you choose. I continued sketching what I thought an environment like that could look like and what kinds of things I would need to fill it with.

Once I had roughly what I figured I'd need laid out, I made a handful of super rough block-out meshes in Maya with the general proportions of the objects that would eventually end up taking their place and threw them into engine so I could get started.

The first struggle I encountered was understanding attributes. Most of my coding knowledge is limited to Python scripting and HTML so I hadn't really used using data groupings like this to create logic before. Luckily it didn't take me too long to figure out what was going on and I figured out what most of them did and how they were passed through and altered in the inputs and outputs of the nodes.

I started off with extremely basic logic testing out how to use different sampler types, projections, filters, and modifiers. I even learned how to create and expose parameters as override values that were usable while constructing. Generating geometry using splines felt so cool to figure out even if it isn't that complex.

Pretty soon I had some decent foundational generation going on although it was still a little messy and obviously not looking too great.

After I had some better basic logic laid out, I switched gears and decided to move through the pipeline with my assets so I could better fine tune my graph to the specific meshes I would be using.

I started off in Zbrush and sculpted out my main meshes using the block-outs as a base for proportions. I also created a set of false cards for my foliage that would later be baked onto some planes and turned into usable card assets.

Next up was retopology and baking. For most of these assets I used some simple auto remeshing or decimation in ZBrush to get a low poly version to bake onto in Substance painter. The cards were done simply by creating base planes in Maya for the high poly to bake onto and would later be cut out and given proper geometry and shape. For the spruce branches specifically, I used vertex paint to bake an ID mask allowing me to more easily separate the needles from the branch parts of the card. The rest was a pretty straightforward baking and texturing process.

After all the texturing was done, I cut out the cards and assembled the meshes that required components into their full assets and put them in engine. For the foliage materials, I used some wind functionality I had learned in a previous foliage project just to get some basic sway on the branches and wind on the grass and leaves.

Next, I knew I needed some tile-able landscape textures so I got to work creating both snow and dirt PBR textures in Substance Designer and getting them both hooked up to a layered landscape material with some nice height blending for use with landscape painting.

Now that I had the snow textures, I wanted to create a fresh fallen powder layer effect for the rocks that I could apply to them without having to create new geometry. For this I created a material function that would take input from the rock material textures and mask out the zones that should have snow on them based on upward facing vertex normals. Then it would apply the snow as a world aligned texture so all snow lines up properly and to the same scale throughout the map, and finally displace the snow mask using world position offset based on the height map of the snow texture to create a thick layered effect that gave me control over how thick the layer is and the slope falloff for where snow should generate. This function would also ensure that no matter the placement of my meshes, snow would always sit on top.

At this point, all of my assets were pretty much ready to go and I started to plug them into my system to replace the old meshes. At this same time I was also removing my temporary "children" spline components that existed underneath the PCG asset in level so I could replace them with dedicated and preconfigured blueprint classes for each part of the generation.

This is where a huge learning opportunity happened for me. I had forgotten to account for the relative scaling of the spline components when building my initial graph. This was a problem because it meant that most of the samples my graph depended on had been designed around an arbirtary extent upscale I had made on one of those temporary spline components early on. Because of this massive oversight, all the filters, adjustments, distance filters, culling, etc. was all completely off and I would need to almost entirely start again with a system that would base all generation off of a consistently scaled landscape or world ray hit sample rather than the samples within each spline component. Here's a video showing samples taken directly from a spline interior, and another video showing samples taken from the landscape filtered by distance of the spline interior samples.

Scrapping my initial graph and starting again with a new one ended up being super great for the over all organization, performance, and fundamentals behind what I have now. I learned a lot in that first go around and while it was a bummer to have it all break when I was so close to getting some better looking results, it taught me a lot about what not to do and gave me an opportunity to better apply my knowledge to a brand new system.

My new system was much more organized and had a much healthier "layered" system that operated in just an overall cleaner way. While building up this new version, the named reroute node became my new best friend. When keeping track of and filtering so many sets of attributes the reroutes allowed for a much easier grouping of information. I have used reroutes before to organize material or blueprint graphs but this was the first time I have ever needed them on this large a scale.

Now I'll showcase my current generation logic and try to break down each step of the process as I go through it.

First, I gather all the relevant data necessary for creating samples from all the components in my scene. I have blueprint actors containing spline components for the mountainous terrain, the spruce forest, the birch forest, and the pathway which can separate/exclude the others. Additionally, I need the landscape data to project onto and later on, data from a world ray hit query.

The data from those assets can be read and sampled for their base points. In this step I also filter out all the sample space close enough to the path spline on the landscape. Doing this early on means I won't have to sample for and filter out the path for each component later.

The rocks will be the first set of points to be generated, as some of the other objects have the ability to be placed on top of them. The initial point set is created using noise and set up so that larger rocks will always bias more sloped areas. Extreme sloping = Extreme rocks!

The points are filtered down into two sizes that I called upper and lower tier rocks here. They are each given a range of points they can spawn on and are then filtered down again to create a more randomized and "bunched" look. The ratios between the two rock sizes can be configured here as well. In the box with the actual spawning, I give the poins a range of transformations it can take at random to randomly rotate and scale the rocks before they are spawned in, then I create a set of bound for each type of rock which will be useful for pruning out self-overlapping objects or determining the distance between objects. Then they are self pruned only against themselves so that one type of rock cannot overlap another of that same variety. This is also where the dependency for the world ray hit query happens. We want to generate the world ray hit after the rocks spawn in so that they can be sampled on.

Based on distance from the placements of the large rocks, medium rocks are then placed in around them. Similar transform and bounds modifications will happen every time we spawn in objects.

The large rock points and the rocks that do not query in the world ray hit are designated here for later interactions with the forests and ambient spawns.

This is where we will be sampling our forests. This is the first use of the world ray hit query as a sampler, as we want the trees to be able to spawn on designated rocks. I have a simple control here allowing to change the density of the forest types. I then make sure that the trees can't ever spawn too close to a rock that the ray hit query cant touch. This way no clipping will occur between trees and rocks. Next, I check for points that would be too steep for a tree to naturally spawn on and filter them out. Then we sample the border of the forest splines and shrink the trees down and limit their frequency of spawning based on how close they are to the edge. I did this to make the forest feel more "full" and healthy the farther into it you go. For later use with ambient objects, the area I am using to sample the trees has some "bald spots" filtered out of its samples based on noise and only in flat zones.

I apply some noise to the points and separate them to designate short and tall trees. For the purpose of my system, I want the two forest types to be able to overlap and generate together if needed without clipping through one another. To achieve this, I have to apply the proper transforms and bounds for each tree type, then I give each type of tree a designated density value, merge all the point data together and use a self prune on all trees at the same time, and then filter the points back out for individual spawning based on those density values I just assigned. This way, no matter how many forests are layered on top of each other, they will prune out properly so no clipping or density issues happen. Then the final points of all trees is saved in a reroute to be referenced later.

Now to start generating ambient objects in areas not designated by splines but within the bounds of the PCG zone, I filter out everything within the splines by distance so ambient objects won't overlap with rocks, trees, or paths. Then I determine how frequent ambient objects should be and filter out the noise four ways one for each class of ambient object I'll be spawning.

Before we can follow the ambient points to their finale, I first also designate where logs and stumps will be spawning within their relative forests. I do this based on distance away from the dense forest forcing the generation to lean towards those bald spots we created earlier.

Now we can grab all the points for all the logs, stumps, and berry bushes that need to be spawned and do another group pruning. This gets us some nice ambient and "bald spot" fallen tree generation.

Now I can finally generate in the rest of the smaller foliage. Some simple grass is spawned in around ambient objects and within the forests. Things like pine cones and dead branches are found near their respective trees.

That's the whole system as of now. It's a pretty simple system when it's boiled down, and like any good nature mimicking tech-art, it's essentially a bunch of noise on top of noise on top of noise that's dialed in just a little bit and put into the right order. This is what the whole system looks like:

There were also a couple of things I just couldn't get working within the time I've spent on this project so far. The first one had to do with the world position offset wind I use in my foliage material being applied to the PCGs instanced versions of the meshes. I know it has something to do with the relative pivots wanting to be at the origin of the PCG asset instead of each local object. As you can see, with my wind mask applied to the regular asset (right) vs an instanced one (left), something is definitely super off. I couldn't quite figure out a fix so far so if you're seeing this and think you know whats up please drop a message!

The second thing I just could not get to work was the experimental "Landscape Patch" plugin. I tried for so long to get this to work for so long and just could not figure out how to make it do what it is supposed to. I must me missing something here but I couldn't figure it out with the documentation I found. Once again if you know how to make this work please drop a message, I'd love to know what I'm doing wrong. This is the reason my poor dirt textures unfortunately never made it into the final generation.

I really enjoyed making this system and learning about the PCG process for the first time. Like I said before, I plan on polishing this up as some point in the future to bring it to a higher quality bar, taking some more time to polish and optimize my meshes potentially using nanite, making the textures a little more interesting, finding a higher quality way for texture bombing the landscape, and potentially some wind and snow VFX as well. There was also a set of icicles and an ice river I had planned that didn't end up making it in either that I would love to include.

Thanks so much to anyone who was willing to read this far!

Special thanks to my mentor on this study, Owen Miakilani for helping me plan this all out and guide me in the right directions.

Here's one last video of the kit in action.