In our team we use this pipeline to distribute new binaries to our designers and artists when code has been changed. The SCM gets polled every 5 minutes for possible code changes and builds new editor binaries when needed. Programmers exclude the binaries folder from their view mappings in Perforce because they do not want to deal with large binary files as they compile their own.
This tutorial will explain how to set up build automation for Unreal Engine 4 Editor Binaries using Perforce. The batch files used in this tutorial should work with any version control system although it might require some minor changes.
Prerequisites
Before we begin this tutorial, make sure you have the following set up and available.
- You know how to use Jenkins and it's interface, this is also slightly explained here.
- You have the Jenkins P4 plugin installed.
- A Linux or Windows server running Jenkins CI and (optional) Perforce Helix.
- Jenkins Perforce plugin (optional)
- A local workstation with Unreal Engine 4 installed that will work as our build executor.
- Make sure you have the correct Visual Studio setup (Setting Up Visual Studio for UE4).
- Make sure you have installed the Java Runtime Environment (JRE).
Setting up a new Freestyle project
From the home screen you can visit "New Item" to create a new project. In our case we will create a new "Freestyle project", give the project a appropriate name and press Ok.
Give the build a clear description. We also want to set up the amount of builds to keep. In my case, my server disk space is limited so I have set up Jenkins to keep a maximum of 2 artifacts per project, for a maximum of 7 days. You should pick the options that suite your project.
Parameterize the build
To provide yourself with easy access to some settings, we want to parameterize the build.
Tick the "This project is parameterized" option.
The following script allows us to easily change our build configuration.
Choice Parameter
==================================================================================
NAME: BUILD_CONFIGURATION
==================================================================================
Development
Shipping
Test
DebugGame
==================================================================================
Development = This configuration is equivalent to Release. Unreal Editor uses the Development configuration by default. Compiling your project using the Development configuration enables you to see code changes made to your project reflected in the editor.
Shipping = This is the configuration for optimal performance and shipping your game. This configuration strips out console commands, stats, and profiling tools.
Test = This configuration is the Shipping configuration, but with some console commands, stats, and profiling tools enabled.
DebugGame = This configuration builds the engine as optimized, but leaves the game code debuggable. This configuration is ideal for debugging only game modules.
We might also want to change our engine version per build.
Choice Parameter
==================================================================================
NAME: ENGINE_VERSION
==================================================================================
UE_4.20
==================================================================================
What Unreal Engine 4 version we want to build the project with.
And we might not want to use pak files for our build, so let's include an option. We will however have the option for PAK enabled by default.
String Parameter
==================================================================================
NAME: USE_PAK
==================================================================================
-pak
We want to easily be able to duplicate a Jenkins project for a new project. So I find it easy to have a choice parameter for my project name. So I don't have to change the project name in all the batch scripts.
Choice Parameter
==================================================================================
NAME: PROJECT_NAME
==================================================================================
ProjectName
==================================================================================
The name of the uproject file.
Add Perforce as Source Control (Optional)
If you do not want to use Perforce, you can skip this step. You will have to look up how to set up a different SCM with Jenkins.
Next up we need to supply which version control we are using. In my case this is Perforce. If you are not using Perforce you will have to set it up for Git or SVN. Note: You will need to add your Perforce Credentials, just click on "add" and follow the instructions provided by Jenkins.
I picked the manual workspace behaviour, here I can supply the stream to check and specify my workspace view mappings. Set up yours according to your project.
Important: Under polling build filters we add a new build filter, pick the "Exclude changes outside view mask" option. We only want to trigger binary builds when changes are made to our code. To prevent the build from starting from changes that are made by artists and designers we exclude every change that is made outside of our source folder. Add the following to the Perforce View Mask:
//depot_name/stream_name/path_to_project/Source
-//depot_name/stream_name/path_to_project/Binaries
-//depot_name/stream_name/path_to_project/Content
Setting up SCM polling
We probably also want Jenkins to poll our SCM once in a while to check for code changes. I have set up Jenkins to poll the SCM every 5 minutes because I want to get the latest binaries to our artists and designers as quickly as possible.
Building the Unreal Engine 4 project
Now we will have to set up the build commands that will be executed on our build executer.
Click on "Add Build Step" and choose "Execute Windows Batch Command".
First, we got to make sure we do not have any leftover files from the previous build.
# remove the temporary directory if it still exists
rd /s /q temp
# cleanup binaries, they will be rebuild during the process
rmdir /s /q "%WORKSPACE%\Game\Binaries"
# cleanup intermediates, they will be rebuild during the process
rmdir /s /q "%WORKSPACE%\Game\Intermediate"
Now we can use the Unreal Build Tool to start building the project.
"%ENGINE_ROOT%\%ENGINE_VERSION%\Engine\Binaries\DotNET\UnrealBuildTool.exe" -projectfiles -project="%WORKSPACE%\Game\%PROJECT_NAME%.uproject" -game -rocket -progress
Now that we have linked our project with the Unreal Build Tool, we want to rebuild our project code and make sure our binaries are up to date.
# Rebuild our binaries
"%ENGINE_ROOT%/%ENGINE_VERSION%/Engine/Binaries/DotNET/UnrealBuildTool.exe" %PROJECT_NAME% %BUILD_CONFIGURATION% Win64 -project="%WORKSPACE%/Game/%PROJECT_NAME%.uproject" -rocket -editorrecompile -progress -noubtmakefiles -NoHotReloadFromIDE -2017
Submitting the binaries to Perforce
Before we publish our assets to Perforce, there are 2 important steps that we have to take. First we need to reconcile changes in our project and plugins binaries folder and add them to the default change list. Next we output a list of all changed files and submit them to the depot. Your project is now set up and ready to go! Finally, you might want to add Slack Notifications.
# Reconcile our editor and plugin binaries, add them to the default changelist
p4 -c %P4_CLIENT% -u %P4_USER% reconcile -c default //disillusion/development/Game/Binaries/...
p4 -c %P4_CLIENT% -u %P4_USER% reconcile -c default //disillusion/development/Game/Plugins/*/Binaries/...
# Output a list of all changed files in the default changelist. Then submit the files to the depot
p4 -c %P4_CLIENT% -u %P4_USER% opened -c default
p4 -c %P4_CLIENT% -u %P4_USER% submit -d "Submitted by Jenkins. Build: %JOB_NAME%-%BUILD_NUMBER%"