Building VirtualBox for Windows

 Intro

It is a well-known fact to many users of the Windows version of VirtualBox (from now on, VB; not to be confused with Visual Basic) that starting with 4.3.14 the developers added the so-called «hardening» designed to prevent malicious injections into VB. Although the intentions were good, the implementation happened to cause numerous conflicts with totally legitimate products such as antiviruses, cryptographic modules and even some updates of the Windows itself, and when such a conflict occurs VB simply stops working. Users have to wait for at least a month till the new VB version is released with the proper exclusions added. Worst case is, the conflicting application or update has to be uninstalled, or VB itself has to be downgraded to the version 4.3.12 which was the latest one without hardening. Numerous requests to add a user-controlled exclusion list, or an option to disable hardening, are all left unanswered. The only reply from developers sounds like «if you don't want it build it from source code yourself». Well, looks like we'll have to.

Although the build instructions are described on the official project Wiki, they are incomplete and somewhat outdated, while the build procedure often fails with vague error messages. So when, in the end, I got it working I thought it was worth documenting in full details in a separate article. This instruction is being updated from time to time, and at the moment it is adapted to building VB version 6.1.18. However, if you need information on building earlier versions of VB or auxiliary libraries you can always get it from the history of changes.

 Table of Contents

» Problem Statement
» A Word of Warning
» Build Environment Preparation
» Applications Installation Quirks
» Final Touches
» Building VirtualBox
» Epilogue
» Amendments

 Problem Statement

At first, I wanted to keep it simple by just rebuilding the binary components and copying them on top of the version installed from the official distribution. However, that occurred to be far from simple because installation relies on some obscure system API and requires drivers installation and COM components registration. I wondered whether it was viable to learn how all this worked and to write a script for automating these tasks, but after some consideration I decided to go big and build a full distribution which would be as close as possible to the official one, only without hardening.

I have to say that this task proved to be really hard, and I failed to fulfill it to 100%. What I've stumbled upon were the Guest Additions which are provided for Windows (32- and 64-bit), OS/2, Linux, and some other *NIX systems. The comments in the corresponding Makefile mention that they are all built on different remote machines, and I was definitely not looking forward to managing such a build factory. So my final solution was to build from source codes everything, except the Additions, but take the official ISO image with them and just put into my distribution as is. I have not checked whether the Additions have the same hardening mechanism, but even if they do I've never heard any complains about it.

 A Word of Warning

• Security considerations

Hardening was not added on a whim, it was a solution for closing a vulnerability in VB. Unfortunately, Oracle refuses to provide the detailed description of the issue, even though it was fixed many years ago. Generally, it has something to do with the DLL injection mechanism in Windows which, in case of VB, can lead to unauthorized privilege elevation on the host computer, and there are working exploits for this vulnerability. So be warned if you decide to use hardening-less version anyway.

• Drivers signing

Starting with Vista, in 64-bit Windows versions users cannot load arbitrary drivers anymore. The drivers must be signed by a certificate with a cross-certificate chain that goes up to the Microsoft root CA (and in Windows 10 with Secure Boot enabled the drivers must be signed by Microsoft themselves). Before you set out on the journey of building VB you need to decide how to solve this problem. You can either spend some money and buy your own certificate, or try contacting some third-party companies which provide signing services for open source projects (if they agree to sign vulnerable drivers), or set your Windows into the test mode that allows loading drivers signed with a locally generated test certificate.

In the article I will mainly imply the latter scenario, but I'll mention how the procedure changes if you do have a «full-grown» commercial certificate.

• End of support for 32-bit host OS versions

Starting from 6.0, VirtualBox stopped supporting 32-bit host Windows versions (guest systems are OK), however it still could be used there. In version 6.1 the 32-bit version was completely removed from the installer package (apart from the API library). In my own build I decided to follow suit, switched to 64-bit-only package, and removed the parts of the article which are no longer relevant. If you still have to work with 32-bit host versions, you could try to build the 32-bit VirtualBox from source, using instructions from the older article revisions. But as time goes, it will become an increasingly more difficult task, and you will reach a point when you are unable to fill all the gaps. A reasonable approach would be using the older versions of VirtualBox, or switching to a 64-bit host OS.

 Build Environment Preparation

The official guide recommends to use Windows 8.1 or 10 as a build system. My build environment has been using Windows 7 SP1 x64 since the times that was the officially recommended OS; but it still works fine, so I keep using it. Please, keep in mind that the machine (real or virtual) you are going to use for building VB must have Internet connection.

The build environment requires quite a bunch of tools. If some program has a portable version I prefer to take that instead of using the installer.

Now, the following programs can only be obtained as installable distributions (at least, officially). For Visual Studio and SDK/WDK it is important to follow the installation order as specified below. After you've finished with them I highly recommend to run Windows Update and fetch the latest updates for all Microsoft products.

• Visual Studio 2010 Professional
  You need exactly the version 2010, and it should be Professional or higher. The version 2010 Express lacks ATL which is required for building COM API, used by all the frontends for communicating. I tried to switch to VS 2013 or 2015 Community Edition so that it wasn't a requirement to have a paid program (which is even no longer sold), but failed so far.
• Windows SDK v7.1
• Visual Studio 2010 SP1
• Visual C++ 2010 SP1 Compiler Update for SDK 7.1
• Windows Driver Development Kit (WDK) v7.1
• Windows SDK v8.1
• ActivePerl
• ActivePython 2.7
• Cygwin

The other programs are downloaded as portable archives or source codes:

• Qt 5.6.3 (source code)
• MinGW-w64 4.5.4 x86_64
• SDL v1.2.x (development library for Visual C++)
• cURL (source code)
• OpenSSL 1.1.1 (source code)
• gSOAP 2.8.x (version 2.8.41 or higher is recommended)
• libvpx 1.7.0 (source code; newer versions do not support VS 2010)
• libopus 1.3.1 (source code)
• MiKTeX Portable
• NASM
  I recommend using the 64-bit portable version.
• WiX
  I recommend using portable archive (named like wix311-binaries.zip).

You will also need the following archives:

• DocBook XML DTD 4.5
• DocBook XSL Stylesheets 1.69.1

Here is the summary of all the tools and libraries I'm using, including their exact versions (where possible) and installation paths I chose on my build system. Designation «{x32|x64}» is used for separate 32- and 64-bit version installation locations.

 Applications Installation Quirks

Here I provide some instructions on installation of some of the aforementioned tools where the procedure is not obvious or requires additional actions.

• Windows SDK v7.1

Installation of this SDK will probably fail because it includes the outdated versions of the compiler and runtime libraries. After you installed VS 2010, there are newer versions of those present in the system, so the older ones fail to install, and the main installer considers it a critical error. For working this around, you need to either untick the corresponding checkboxes, or before installation of the SDK, uninstall the packages named like «Microsoft Visual C++ 2010 <arch> Redistributable», «Microsoft Visual C++ 2010 <arch> Runtime», «Microsoft Visual C++ Compilers…». Let the SDK install the older versions of those, and then run Windows Update to get the latest available versions.

When you install the SDK, make sure you check the program samples (Windows Native Code Development -> Samples): they contain some scripts used by the VB build rules. Without them you won't be able to build the MSI packages.

• Windows SDK v8.1

You only need to install the component «Windows Software Development Kit».

• WDK v7.1

You only need to install the component «Build Environments».

• Qt 5.6.3

Starting with Qt 5.7.0, for building it you need Visual Studio 2012 or later, so we're stuck with 5.6.x.
Since there are no official builds for Visual Studio 2010 we'll have to build Qt from source code ourselves.

1. Unpack the Qt source code archive into C:\Programs\Qt\ and rename the directory qt-everywhere-opensource-src-5.6.3 into 5.6.3-src.
2. Beside it create a directory build-x64 where the build will take place.
3. Open the console and run the following commands to prepare the environment:

   md C:\Programs\Qt\build-x64
   cd /d C:\Programs\Qt\build-x64
   SET QTVER=5.6.3
   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
   COLOR 07
   SET QTDIR=C:\Programs\Qt\%QTVER%-x64
   SET PATH=%QTDIR%\bin;%PATH%
   SET QMAKESPEC=win32-msvc2010

   The color command is optional, it gets rid of the green colour which was set by the script SetEnv.Cmd.
4. Now we need to run the configure.bat script from 5.6.3-src. VB does not use most of what Qt includes, so we can significantly reduce the build time by disabling various components, but please note that some options are crucial for VB. Particularly, I've discovered that:
   • OpenGL ES 2 is not supported (when building VB, the compiler fails to find some header files).
   • FreeType support must be enabled (otherwise the plugin qoffscreen will not be built, and it is required for VB).
   This is the full command line I've ended up with:

   ..\5.6.3-src\configure.bat -prefix c:\Programs\Qt\5.6.3-x64 -mp -opensource -confirm-license -nomake tests -nomake examples -no-compile-examples -release -shared -pch -no-ltcg -accessibility -no-sql-sqlite -opengl desktop -no-openvg -no-nis -no-iconv -no-evdev -no-mtdev -no-inotify -no-eventfd -largefile -no-system-proxies -qt-zlib -qt-pcre -no-icu -qt-libpng -qt-libjpeg -qt-freetype -no-fontconfig -qt-harfbuzz -no-angle -incredibuild-xge -no-plugin-manifests -qmake -qreal double -rtti -strip -no-ssl -no-openssl -no-libproxy -no-dbus -no-audio-backend -no-wmf-backend -no-qml-debug -no-direct2d -directwrite -no-style-fusion -native-gestures -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtconnectivity -skip qtdeclarative -skip qtdoc -skip qtenginio -skip qtgraphicaleffects -skip qtlocation -skip qtmacextras -skip qtmultimedia -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns

5. The Qt installation path specified here (in the -prefix option) is hardcoded into the intermediate source code files by the configure script, so the binary files will also remember it. Any Qt-based application built using this library will search for the required plugins in that path first, and only if they could not be found, it will use its own current location. In most cases this is OK, but suppose on the computer where our VB is installed, there is another build of Qt located at c:\Programs\Qt\5.6.3-x64, but compiled with different options or by a different compiler? Then VB will try to load those (incompatible) plugins and crash.
   There are two possible ways to avoid this issue. First is adding to the installation an additional file named qt.conf with the following text:

   [Paths]
   Plugins=.

   The other is to fix that saved installation path before building Qt, so that it pointed to the application's directory. I decided on following the latter way, I didn't like the idea of having additional files to what the Oracle VB distribution had. You need to open the file C:\Programs\Qt\build-x64\qtbase\src\corelib\global\qconfig.cpp created by the configure script, find the following line:

   static const char qt_configure_prefix_path_str       [512 + 12] = "qt_prfxpath=c:/Programs/Qt/5.6.3-x64";

   and replace the path with a single dot, so that this line looked like this:

   static const char qt_configure_prefix_path_str       [512 + 12] = "qt_prfxpath=.";

   This change will only affect the VB's runtime behaviour. The installation step of Qt itself will use the original path, because it is now stored in the Makefiles which we do not modify.
6. Next, run the build using the command nmake
7. And install the built library using nmake install

After you've finished with installing, the directories build-x64, and 5.6.3-src can be deleted.

• MinGW

Simply unpack the archive into the chosen installation directory.

• Cygwin

During installation you need to choose the packages make and yasm.

• SDL

1. Unpack the SDL archive into C:\Programs\SDL\x64\.
2. Move all the contents of the subdirectory C:\Programs\SDL\x64\lib\x64\ to its parent directory (that is, into C:\Programs\SDL\x64\lib\), then delete the subdirectories C:\Programs\SDL\x64\lib\x86 and x64.

• NASM

Unpack the archive nasm-2.14.02-win64.zip into C:\Programs\, then rename the resultant directory nasm-2.14.02 as nasm.

• OpenSSL

1. This library still needs to be built for both 32- and 64-bit architectures. Unpack the OpenSSL archive into C:\Programs\OpenSSL\ twice, renaming the copies of the directory openssl-1.1.1i as openssl-1.1.1i-x32 and openssl-1.1.1i-x64.
2. Open the console, build and install the 32-bit version by running:

   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x86 /win7
   COLOR 07
   set PATH=%PATH%;C:\Programs\nasm
   cd /d C:\Programs\OpenSSL\openssl-1.1.1i-x32\
   perl Configure VC-WIN32 no-shared --prefix=C:\Programs\OpenSSL\x32 --openssldir=C:\Programs\OpenSSL\x32\ssl
   nmake
   nmake test
   nmake install

   If the configure step outputs a scary warning about a missing compiler, please, disregard this message, it's lying outrageously.
   If you don't want to use NASM just skip the command with PATH modification, and for the Configure script specify the additional parameter no-asm.
3. Now open a new console window and do the same for the 64-bit version:

   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
   COLOR 07
   set PATH=%PATH%;C:\Programs\nasm
   cd /d C:\Programs\OpenSSL\openssl-1.1.1i-x64\
   perl Configure VC-WIN64A no-shared --prefix=C:\Programs\OpenSSL\x64 --openssldir=C:\Programs\OpenSSL\x64\ssl
   nmake
   nmake test
   nmake install

   Disabling NASM is the same as in 32-bit.
4. Now you can delete the directories C:\Programs\OpenSSL\openssl-1.1.1i-x32 and openssl-1.1.1i-x64.

• cURL

1. Like with OpenSSL, we need the 32-bit version in addition to the 64-bit one. Unpack the cURL archive into C:\Programs\curl\ and rename the resultant subdirectory curl-7.74.0 into curl-7.74.0-x32.
2. Open the file C:\Programs\curl\curl-7.74.0-x32\winbuild\MakefileBuild.vc in a text editor, and look for the following block of code (around lines No.61–69):

   !IF "$(VC)"=="6"
   CC_NODEBUG  = $(CC) /O2 /DNDEBUG
   CC_DEBUG    = $(CC) /Od /Gm /Zi /D_DEBUG /GZ
   CFLAGS      = /I. /I../lib /I../include /nologo /W4 /wd4127 /GX /DWIN32 /YX /FD /c /DBUILDING_LIBCURL
   !ELSE
   CC_NODEBUG  = $(CC) /O2 /DNDEBUG
   CC_DEBUG    = $(CC) /Od /D_DEBUG /RTC1 /Z7 /LDd
   CFLAGS      = /I. /I ../lib /I../include /nologo /W4 /wd4127 /EHsc /DWIN32 /FD /c /DBUILDING_LIBCURL
   !ENDIF

   After these lines add the new directive:

   CFLAGS      = $(CFLAGS) /DCURL_DISABLE_LDAP

   If you don't, the build of VB will fail with link errors.
3. Now open the file C:\Programs\curl\curl-7.74.0-x32\winbuild\gen_resp_file.bat, and after the very first line (@echo OFF) insert this command:

   cd .

   It does not do anything, except resetting the ERRORLEVEL code. The script is so simple that it is possible for it to complete without running any command that would change the error code. So if that code happened to be non-zero before starting the script, it will keep its value on the script exit, and nmake would think it was the script that returned this error code, and terminate the build with failure. Adding that fake cd fixes the issue.
4. Now make a full copy of curl-7.74.0-x32 under the name curl-7.74.0-x64.
5. Open the console, build the 32-bit version and copy the files we need:

   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x86 /win7
   COLOR 07
   cd /d C:\Programs\curl\curl-7.74.0-x32\winbuild
   md C:\Programs\curl\x32
   nmake /f Makefile.vc mode=static WITH_SSL=static DEBUG=no MACHINE=x86 SSL_PATH=C:\Programs\OpenSSL\x32 ENABLE_SSPI=no ENABLE_WINSSL=no ENABLE_IDN=no
   copy ..\builds\libcurl-vc-x86-release-static-ssl-static-ipv6\lib\libcurl_a.lib ..\..\x32\libcurl.lib
   xcopy /E ..\builds\libcurl-vc-x86-release-static-ssl-static-ipv6\include\curl ..\..\x32\include\curl\

6. Open another console window and build the 64-bit version:

   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
   COLOR 07
   cd /d C:\Programs\curl\curl-7.74.0-x64\winbuild
   md C:\Programs\curl\x64
   nmake /f Makefile.vc mode=static WITH_SSL=static DEBUG=no MACHINE=x64 SSL_PATH=C:\Programs\OpenSSL\x64 ENABLE_SSPI=no ENABLE_WINSSL=no ENABLE_IDN=no
   copy ..\builds\libcurl-vc-x64-release-static-ssl-static-ipv6\lib\libcurl_a.lib ..\..\x64\libcurl.lib
   xcopy /E ..\builds\libcurl-vc-x64-release-static-ssl-static-ipv6\include\curl ..\..\x64\include\curl\
   copy ..\builds\libcurl-vc-x64-release-static-ssl-static-ipv6\bin\curl.exe ..\..\x64\curl.exe

   Notice that in addition to what we did for the 32-bit version, here we also copy the file curl.exe. We'll use it later, for downloading the guest additions image.
7. The directories C:\Programs\curl\curl-7.74.0-x32 and curl-7.74.0-x64 are no longer needed and can be removed.

• libvpx

1. Unpack the libvpx archive into C:\Programs\libvpx-build\.
2. Launch the Cygwin terminal, the build will be performed there. Our target platform is Visual Studio 2010; the build system supports it only partially: it will attempt to run the actual build using msbuild.exe but it does not work for some reason; I failed to find out why. Instead, we can simply run the Visual Studio itself and use it for building the project. Luckily it can be used from command line, so it is not necessary to launch the actual IDE (although you can if you want to, in which case you will have to modify the PATH variable and add the path C:\Programs\cygwin64\bin to it, or in any other way allow VS to find yasm.exe located there and required for the build). So, to perform the build of both architectures, run the following commands in the Cygwin terminal:

   mkdir -p /cygdrive/c/Programs/libvpx-build/build64
   cd /cygdrive/c/Programs/libvpx-build/build64
   ../libvpx-1.7.0/configure --target=x86_64-win64-vs10 --disable-install-bins --disable-examples --disable-tools --disable-docs --prefix=../../libvpx
   make
   "/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/Common7/IDE/devenv.com" vpx.sln /Project vpx.vcxproj /Rebuild "Release|x64"
   make install

3. Close Cygwin, we won't need it anymore. The directory C:\Programs\libvpx-build can be removed.

• libopus

1. Unpack the opus archive into C:\Programs\libopus-build\, go to the subdirectory opus-1.3.1\win32\VS2015.
2. This project is designed for the newer Visual Studio version, so we need to modify it to make it build in our version 2010. You can do it either using the IDE, or a plain text editor. I chose the latter one. Open the file opus.vcxproj and do the following:
   1. Find all the lines containing the text

      <PlatformToolset>v140</PlatformToolset>

      and replace v140 with v100. If you are using IDE, this is the configuration option «Platrofm Toolset» located on the page Configuration Properties -> General. Don't forget about the Configuration and Platform selectors at the top of the dialog.
   2. Next, look for this block of code:

        <ItemDefinitionGroup>
          <ClCompile>

      and add a new tag inside it:

            <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>

      In the IDE you can do it by opening the page Configuration Properties -> C/C++ -> General and setting «Debug Information Format» into «ProgramDatabase (/Zi)». Actually, you can set it to any other valid value, we don't need the debugging database at all, but with an invalid value the project will fail to build.
3. Now let's build the Release version for both platforms (using either the VS IDE, or the command line) and copy the library opus.lib and the subdirectory include\ into our destination path:

   cd /d C:\Programs\libopus-build\opus-1.3.1\win32\VS2015
   md C:\Programs\libopus\lib\x64
   xcopy /E C:\Programs\libopus-build\opus-1.3.1\include C:\Programs\libopus\include\
   
   "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\devenv.com" opus.sln /Project opus.vcxproj /Rebuild "Release|x64"
   copy x64\Release\opus.lib C:\Programs\libopus\lib\x64\

4. The directory C:\Programs\libopus-build can be removed.

• gSOAP

Open the archive, go into the subdirectory gsoap-2.8\gsoap and unpack its contents into C:\Programs\gSOAP\. OpenSSL 1.1.x requires gSOAP version 2.8.41 or higher. If for some reason you want to use an older version you will need to apply the patch created by Mattias Ellert. Applying a patch can be done either manually (the format is easy: open the mentioned files, delete the lines marked with minus, add lines marked with plus; the rest is context), or get the patch tool ported for Windows and use it.

• MiKTeX

1. Unpack the archive into C:\Programs\MiKTeX\.
2. Open the console and install additional modules:

   "C:\Programs\MiKTeX\texmfs\install\miktex\bin\mpm.exe" --verbose --install=koma-script --install=ucs --install=tabulary --install=url --install=fancybox --install=fancyvrb --install=bera --install=charter --install=mptopdf

• DocBook

For XML DTD you need to create a directory and unpack the archive contents there. For XSL Stylesheets, all the contents is already in a single directory, so you need to extract that and just rename it afterwards.

 Final Touches

The build environment is almost ready, just a few more steps are needed. Download the VirtualBox source code archive (if you haven't done it yet) and unpack it to where you'd like to work. I chose C:\Devel\ for that purpose, put there the unpacked source codes and renamed the subdirectory into VirtualBox-src.

• Adding certificates

If you don't have a paid certificate I recommend you to at least create a self-signed personal one: it is easier to make Windows load self-signed drivers than unsigned ones. To do so, open the console with elevated privileges, and run the following commands which will add two certificates (SHA-1 and SHA-256) and import them into your personal storage:

"C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
COLOR 07
makecert.exe -a sha1 -r -pe -ss my -n "CN=Roga and Kopyta Ltd" C:\Devel\testcert_1.cer
makecert.exe -a sha256 -r -pe -ss my -n "CN=Roga and Kopyta Ltd" C:\Devel\testcert_256.cer
certmgr.exe -add C:\Devel\testcert_1.cer -s -r localMachine root
certmgr.exe -add C:\Devel\testcert_256.cer -s -r localMachine root

You can, of course, choose your own name for the certificates instead of «Roga and Kopyta Ltd», and path for storing the files. Now we need to get the thumbprints of the generated certificates. Open the Certificates management console (by running certmgr.msc) and navigate to the Personal storage. You'll see the two new certificates named «Roga and Kopyta Ltd» there. Double-click the first one; in the dialog that appears, go to the Details tab. The «Signature algorithm» field contains the algorithm name: sha256RSA or sha1RSA. Scroll down for the «Thumbprint» field which contains the sequence of hexadecimal numbers. Copy this full value and write it down somewhere. Do the same for the second certificate; don't forget to mark which thumbprint was for SHA-1, and which for SHA-256.

• Building xmllint

One of the build steps will need the xmllint program. I did not put it into the build requirements list because its sources are bundled in the VirtualBox archive. But you will still have to build it yourself, because the VB build rules do not do it. I chose C:\Programs\xmllint as destination directory.

1. Copy the directory C:\Devel\VirtualBox-src\src\libs\libxml2-2.9.4 into C:\Programs\ to make sure its build does not affect VB in any way.
2. Open the console and run the following commands:

   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
   COLOR 07
   cd /d C:\Programs\libxml2-2.9.4\win32
   cscript.exe configure.js cruntime=/MT prefix=C:\Programs\xmllint iconv=no
   nmake /f Makefile.msvc
   nmake /f Makefile.msvc install

3. Remove the directory C:\Programs\libxml2-2.9.4.

• Modifications of VB files

Before starting the build we also need to bring some changes into the VB source code. The full set of them is presented here as a downloadable patch file which can be applied either manually, or using the patch tool (which you need to download separately):

» vbox_build.patch

If there were no problems applying the patch you can jump to the next step. However, if you had some issues and need more information on particular changes, or if you simply want to understand what is being changed and why, you are welcome to go on and read the details provided below. Please, note that the changes come here in a different order compared to the patch file. Paths to the files are specified relative to the VB source code directory C:\Devel\VirtualBox-src.

1. File configure.vbs:
   • The line of code:

     if Shell(DosSlashes(strPathVC & "/bin/cl.exe"), True) <> 0 then

     is replaced with:

     if Shell(DosSlashes(strPathVC & "/bin/cl.exe") & " /?", True) <> 0 then

     This line performs the compiler check but it forgets that cl.exe without any arguments returns an error, and thus is considered an invalid compiler. Adding the «/?» argument makes it output the help and exit with non-error code 0.
   • Now let's jump to the function CheckForCurlSub, and find the following block of code:

        if   LogFileExists(strPathCurl, "include/curl/curl.h") _
         And LogFindFile(strPathCurl, "libcurl.dll") <> "" _
         And LogFindFile(strPathCurl, "libcurl.lib") <> "" _

     It checks the validity of libcurl, but it requires dynamically linked version and fails if no DLL file can be found. We are using statically linked version, so the check for the DLL sadly has to go and leave us with:

        if   LogFileExists(strPathCurl, "include/curl/curl.h") _
         And LogFindFile(strPathCurl, "libcurl.lib") <> "" _

   • Our next function is CheckForPython, where the variable VBOX_BLD_PYTHON is generated:

           CfgPrint "VBOX_BLD_PYTHON       := " & strPathPython & "\python.exe"

     The backslash in front of python.exe should be replaced with the forward one: "/python.exe" (without this fix, some checks during the build procedure will fail; I haven't noticed any other ill-effects, but it's prettier without failure messages, and in any case better safe than sorry).
   • Windows version of the configure script does not support libvpx and libopus, so I've added them myself. Of course, the simplest way was just to hardcode paths to the libraries, but I preferred to implement it just like all the rest of the components, with passing the path in the command line, and proper validity checks. The main code for this consists of the two check functions:

     ''
     ' Checks for libvpx
     sub CheckForVpx(strOptVpx)
        dim strPathVpx, str
        strVpx = "libvpx"
        PrintHdr strVpx
     
        if strOptVpx = "" then
           MsgError "Invalid path specified!"
           exit sub
        end if
     
        if g_strTargetArch = "amd64" then
           strVsBuildArch = "x64"
        else
           strVsBuildArch = "Win32"
        end if
        strLibPathVpx = "lib/" & strVsBuildArch & "/vpxmd.lib"
     
        strPathVpx = ""
        if   LogFileExists(strOptVpx, "include/vpx/vpx_encoder.h") _
         And LogFileExists(strOptVpx, strLibPathVpx) _
           then
              strPathVpx = UnixSlashes(PathAbs(strOptVpx))
              CfgPrint "SDK_VBOX_VPX_INCS := " & strPathVpx & "/include"
              CfgPrint "SDK_VBOX_VPX_LIBS := " & strPathVpx & "/" & strLibPathVpx
           else
              MsgError "Can't locate " & strVpx & ". " _
                     & "Please consult the configure.log and the build requirements."
              exit sub
           end if
     
        PrintResult strVpx, strPathVpx
     end sub
     
     
     
     ''
     ' Checks for libopus
     sub CheckForOpus(strOptOpus)
        dim strPathOpus, str
        strOpus = "libopus"
        PrintHdr strOpus
     
        if strOptOpus = "" then
           MsgError "Invalid path specified!"
           exit sub
        end if
     
        if g_strTargetArch = "amd64" then
           strVsBuildArch = "x64"
        else
           strVsBuildArch = "Win32"
        end if
        strLibPathOpus = "lib/" & strVsBuildArch & "/opus.lib"
     
        strPathOpus = ""
        if   LogFileExists(strOptOpus, "include/opus.h") _
         And LogFileExists(strOptOpus, strLibPathOpus) _
           then
              strPathOpus = UnixSlashes(PathAbs(strOptOpus))
              CfgPrint "SDK_VBOX_OPUS_INCS := " & strPathOpus & "/include"
              CfgPrint "SDK_VBOX_OPUS_LIBS := " & strPathOpus & "/" & strLibPathOpus
           else
              MsgError "Can't locate " & strOpus & ". " _
                     & "Please consult the configure.log and the build requirements."
              exit sub
           end if
     
        PrintResult strOpus, strPathOpus
     end sub

     The function usage is used for printing the list of command line arguments; append our two new arguments there:

        Print "  --with-libvpx=PATH    "
        Print "  --with-libopus=PATH   "

     In the beginning of the function Main many variables are defined for storing paths to the components, we need to add two new ones there:

        strOptVpx = ""
        strOptOpus = ""

     A bit further down goes the select-case block that processes the arguments and fills in the variables; adding our contribution:

              case "--with-libvpx"
                 strOptVpx = strPath
              case "--with-libopus"
                 strOptOpus = strPath

     And, finally, almost at the very end of the file there is the chain of all those check function calls, this is where we add calling of our two new functions:

        CheckForVpx strOptVpx
        CheckForOpus strOptOpus

2. The next file is src\VBox\Runtime\Makefile.kmk. We need to find the definitions of variables VBoxRT_LIBS.win and VBoxRT-x86_LIBS.win, and add two new libraries to them, crypt32.lib and bcrypt.lib. So the following code:

   VBoxRT_LIBS.win                = \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB)/vccomsup.lib \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB)/wbemuuid.lib \
   	$(PATH_TOOL_$(VBOX_VCC_TOOL)_LIB)/delayimp.lib

   turns into:

   VBoxRT_LIBS.win                = \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB)/vccomsup.lib \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB)/wbemuuid.lib \
   	$(PATH_TOOL_$(VBOX_VCC_TOOL)_LIB)/delayimp.lib \
   	$(PATH_SDK_$(VBOX_WINPSDK)_LIB)/crypt32.lib \
   	$(PATH_SDK_$(VBOX_WINPSDK)_LIB)/bcrypt.lib

   (don't miss the trailing backslash after delayimp.lib!); and, respectively, this block:

   VBoxRT-x86_LIBS.win                = \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB.x86)/vccomsup.lib \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB.x86)/wbemuuid.lib \
   	$(PATH_TOOL_$(VBOX_VCC_TOOL_STEM)X86_LIB)/delayimp.lib

   becomes:

   VBoxRT-x86_LIBS.win                = \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB.x86)/vccomsup.lib \
   	$(PATH_SDK_$(VBOX_WINDDK)_LIB.x86)/wbemuuid.lib \
   	$(PATH_TOOL_$(VBOX_VCC_TOOL_STEM)X86_LIB)/delayimp.lib \
   	$(PATH_SDK_$(VBOX_WINPSDK)_LIB.x86)/crypt32.lib \
   	$(PATH_SDK_$(VBOX_WINPSDK)_LIB.x86)/bcrypt.lib

   This change is needed for successful linking of VBoxRT.dll. I'm not 100% sure why that is so, the Oracle version does not have the load-time dependency on crypt32.dll, it is loaded in run-time, so the LIB file should not be required. However without it the linker cannot find some functions and fails. I suspect it might be related to the OpenSSL build options, but I haven't checked this, adding the dependency is easier. And the second dependency, bcrypt.dll, is a requirement for the new OpenSSL 1.1.1.
3. If you have gSOAP of version 2.8.79 or higher you'll have to edit the file src\VBox\Runtime\r3\win\VBoxRT-openssl-1.1plus.def and add the following lines to the exports list:

       OpenSSL_version_num
       DH_generate_parameters_ex
       DH_new
       ASN1_STRING_get0_data

   This list defines which functions are exported by the library VBoxRT.dll (which contains OpenSSL inside). When the VBoxWebSrv.exe tool is being linked, depending on the gSOAP version, it may require additional OpenSSL functions. Since they are missing in the exports list, the linker adds OpenSSL itself and immediately complains about tons of conflicts between this external OpenSSL and its own copy already built into VBoxRT. Adding the aforementioned exports fixes the problem.
4. As I've mentioned in the beginning, I don't build the guest additions, but I do need to have their ISO image as part of the distribution. The build rules are designed for this scenario, but they expect that the ready ISO image will magically appear at the right place at the right time. I've added this magic into the file src\VBox\Makefile.kmk. Look for the following code:

     ifdef VBOX_WITH_ADDITIONS
      include $(PATH_SUB_CURRENT)/Additions/Makefile.kmk
     endif

   and just below it goes the rule for automatic download of the image file:

    ifndef VBOX_WITHOUT_ADDITIONS_ISO
     $(VBOX_PATH_ADDITIONS_ISO)/VBoxGuestAdditions.iso:
   	$(QUIET)$(MKDIR) -p $(@D)
   	$(VBOX_RETRY) $(TOOL_CURL_FETCH) http://download.virtualbox.org/virtualbox/$(VBOX_VERSION_STRING_RAW)/VBoxGuestAdditions_$(VBOX_VERSION_STRING_RAW).iso -o $@
    endif

   If you edit the files manually instead of applying the patch, please, note that the rule commands must begin with the tabulation character.
5. Building documentation is very fiddly. Before the version 6.0 it used to work fine without any changes. I don't know what was wrong with that, but it was significantly redesigned in such a way that it no longer works on my build system. I have no idea how Oracle themselves build the documentation (maybe on a *NIX system), but for me the build rules kept losing slashes in the paths, or adding an excess of them, and as a result they failed to find the target files because of mismatches in catalog files. I finally managed to combine some changes which allowed to build the documentation without errors. First of all, one of the target directories was missing, so some of the intermediate files could not be created. This issue is fixed in the file doc\manual\Makefile.kmk, inside the following code block:

   define def_vbox_refentry_to_user_sect1
   $$(VBOX_PATH_MANUAL_OUTBASE)/$(1)/user_$(2): $(3) \
   		$$(VBOX_PATH_MANUAL_SRC)/docbook-refentry-to-manual-sect1.xsl \
   		$$(VBOX_XML_CATALOG) $$(VBOX_XML_CATALOG_DOCBOOK) $$(VBOX_XML_CATALOG_MANUAL) \
   		$$(VBOX_XML_ENTITIES) $$(VBOX_VERSION_STAMP) | $$$$(dir $$$$@)
   	$$(call MSG_TOOL,xsltproc $$(notdir $$(filter %.xsl,$$^)),,$$(filter %.xml,$$^),$$@)
   	$$(QUIET)$$(RM) -f "$$@"
   	$$(QUIET)$$(call VBOX_XSLTPROC_WITH_CAT) --output $$@ $$(VBOX_PATH_MANUAL_SRC)/docbook-refentry-to-manual-sect1.xsl $$<
   endef

   Just after the $$(RM) I've added a command for creating the missing directory:

   	$$(QUIET)$$(MKDIR) -p "$$(@D)"

   
   As for the slashes, the fixes are located in the file doc\manual\Config.kmk. I couldn't find a «normal» solution for that, but there is a workaround good enough for practical purposes, which implies duplicating the path substitution rules for the incorrect paths. First, below the line:

     VBOX_FILE_URL_MAYBE_SLASH = $(if $(eq $(KBUILD_HOST),win),/,)

   I've created two new variables which basically contain the same paths as their origins, but with triple slash after the disk letter instead of a single slash:

    VBOX_PATH_MANUAL_SRC_SLASHED = $(subst :/,:///,$(VBOX_PATH_MANUAL_SRC))
    VBOX_PATH_MANUAL_OUTBASE_SLASHED = $(subst :/,:///,$(VBOX_PATH_MANUAL_OUTBASE))

   A little bit below that, there is a rule for creating the catalog file:

    $(VBOX_XML_CATALOG): $(VBOX_PATH_MANUAL_SRC)/Config.kmk | $$(dir $$@)
   	$(call MSG_L1,Creating catalog $@)
   	$(QUIET)$(APPEND) -tn "$@" \
   		'<?xml version="1.0"?>' \
   		'<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">' \
   		'<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">' \
   		'  <delegatePublic publicIdStartString="-//OASIS/ENTITIES DocBook XML"      catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegatePublic publicIdStartString="-//OASIS/DTD DocBook XML"           catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/" catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="http://docbook.org/"                catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateURI uriStartString="http://www.oasis-open.org/docbook/"         catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateURI uriStartString="http://docbook.org/"                        catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_SRC)"            catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_OUTBASE)"        catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_SRC)"                    catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_OUTBASE)"                catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_SRC)"     catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_OUTBASE)" catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'</catalog>'

   For each line which contains the variable VBOX_PATH_MANUAL_SRC or VBOX_PATH_MANUAL_OUTBASE (except for the lines with the file:// prefix), I've appended the copy of the whole line, but with the variable replaced with its triple-slash counterpart. The result looks like this:

    $(VBOX_XML_CATALOG): $(VBOX_PATH_MANUAL_SRC)/Config.kmk | $$(dir $$@)
   	$(call MSG_L1,Creating catalog $@)
   	$(QUIET)$(APPEND) -tn "$@" \
   		'<?xml version="1.0"?>' \
   		'<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">' \
   		'<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">' \
   		'  <delegatePublic publicIdStartString="-//OASIS/ENTITIES DocBook XML"      catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegatePublic publicIdStartString="-//OASIS/DTD DocBook XML"           catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/" catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="http://docbook.org/"                catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateURI uriStartString="http://www.oasis-open.org/docbook/"         catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateURI uriStartString="http://docbook.org/"                        catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_DOCBOOK)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_SRC)"            catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_SRC_SLASHED)"    catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_OUTBASE)"        catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateSystem systemIdStartString="$(VBOX_PATH_MANUAL_OUTBASE_SLASHED)" catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_SRC)"                    catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_SRC_SLASHED)"            catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_OUTBASE)"                catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="$(VBOX_PATH_MANUAL_OUTBASE_SLASHED)"        catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_SRC)"     catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'  <delegateURI uriStartString="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_OUTBASE)" catalog="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_XML_CATALOG_MANUAL)"/>' \
   		'</catalog>'

   Even further down, there is another rule for creating an auxiliary catalog file; the starting line is:

    $(VBOX_XML_CATALOG_MANUAL): $(VBOX_PATH_MANUAL_SRC)/Config.kmk | $$(dir $$@)

   Here I'm doing the same operation as above (note that one of the lines is in fact a part of a for-macro; you need to be careful with round brackets there). In addition, in the beginning of the generated file there are several lines defining the entries in the common/ subdirectory:

   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-accessibility-en.xml"            uri="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-accessibility-en.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-diversity.xml"					uri="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-diversity.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-support-en.xml"                  uri="file://$(VBOX_FILE_URL_MAYBE_SLASH)$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-support-en.xml"/>' \

   With these we have the opposite problem: triple slash after the file is replaced with a single slash. I worked this around by getting rid of the file protocol altogether and replacing those URIs with direct file system paths in the target address (the uri attribute). The result for these lines looks like this (including the aforementioned fix):

   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-accessibility-en.xml"            uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-accessibility-en.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC_SLASHED)/common/oracle-accessibility-en.xml"            uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-accessibility-en.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-diversity.xml"					uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-diversity.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC_SLASHED)/common/oracle-diversity.xml"					uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-diversity.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC)/common/oracle-support-en.xml"                  uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-support-en.xml"/>' \
   		'  <system systemId="$(VBOX_PATH_MANUAL_SRC_SLASHED)/common/oracle-support-en.xml"                  uri="$(VBOX_PATH_MANUAL_SRC)/en_US/oracle-support-en.xml"/>' \

6. When VB is built with signing, most of its binaries receive the integrity check flag (the linker option /IntegrityCheck) which forces Windows to check digital signatures and forbids launching applications which are signed incorrectly. If you have a valid paid certificate that's not a problem; however with a self-signed certificate VB will refuse to start, even if Windows is booted in the test mode. I've modified the file Config.kmk in such a way that this flag is only added when you have a full-grown certificate (the criterion of that is presence of a cross-certificate in LocalConfig.kmk; see below). The set of changes looks like this:
   • A new variable VBOX_INTEGRITY_CHECK is added, which contains the desired value of the option:

     if defined(VBOX_SIGNING_MODE) && defined(VBOX_CROSS_CERTIFICATE_FILE)
     	VBOX_INTEGRITY_CHECK := /IntegrityCheck
     else
     	VBOX_INTEGRITY_CHECK := /IntegrityCheck:NO
     endif

   • Below that there is the editbin call:

     	$(VBOX_VCC_EDITBIN) /LargeAddressAware /DynamicBase /NxCompat /Release /IntegrityCheck \
     		/Version:$(VBOX_VERSION_MAJOR)0$(VBOX_VERSION_MINOR).$(VBOX_VERSION_BUILD) \
     		"$@"

     Here I've replaced the unconditional /IntegrityCheck with the new variable $(VBOX_INTEGRITY_CHECK).
   • Next, look for the blocks of the following kinds:

     ifdef VBOX_SIGNING_MODE
      TEMPLATE_XXXXXX_LDFLAGS          += -IntegrityCheck
     endif

     or

     if defined(VBOX_SIGNING_MODE) && defined(VBOX_WITH_HARDENING)
      TEMPLATE_XXXXXX_LDFLAGS          += -IntegrityCheck
     endif

     where «XXXXXX» stands for various component names. There are 6 such blocks in total, 3 of each kind. Here I've modified the condition by adding a check for cross-certificate. The first line then turns into, respectively:

     if defined(VBOX_SIGNING_MODE) && defined(VBOX_CROSS_CERTIFICATE_FILE)

     or

     if defined(VBOX_SIGNING_MODE) && defined(VBOX_CROSS_CERTIFICATE_FILE) && defined(VBOX_WITH_HARDENING)

7. One more file modified by me does not take immediate part in building of VB, it's src\VBox\Installer\win\Scripts\UnpackBlessedDrivers.cmd. This auxiliary script can be used for unpacking a ZIP archive with drivers, which were signed by Microsoft for Windows 10. With my patch I added ability to specify path to the signtool program, and got rid of the unzip tool by replacing it with a small Perl script. The signing procedure is described below. If you don't plan to get the Microsoft signature you can simply ignore all the changes in this script.

• VB build configuration file

Now we need to create the file LocalConfig.kmk in the VB sources directory, and write there all the paths and build parameters. You can use the following text as a template:

VBOX_WITH_HARDENING :=
VBOX_PATH_WIX := C:\Programs\WiX
VBOX_GSOAP_INSTALLED := 1
VBOX_PATH_GSOAP := C:\Programs\gSOAP
VBOX_WITH_COMBINED_PACKAGE :=
VBOX_WITH_QT_PAYLOAD := 1
VBOX_WITH_QTGUI_V5 := 1
VBOX_SIGNING_MODE := release
VBOX_CERTIFICATE_SUBJECT_NAME := Roga and Kopyta Ltd
VBOX_CERTIFICATE_FINGERPRINT := XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
VBOX_CERTIFICATE_SHA2_SUBJECT_NAME := Roga and Kopyta Ltd
VBOX_CERTIFICATE_SHA2_FINGERPRINT := XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
VBOX_TSA_URL := http://timestamp.digicert.com
VBOX_TSA_SHA2_URL := http://timestamp.digicert.com
VBOX_TSA_URL_ARGS := /t "$(VBOX_TSA_URL)"
VBOX_TSA_SHA2_URL_ARGS := /tr "$(VBOX_TSA_SHA2_URL)" /td sha256
VBOX_CROSS_CERTIFICATE_FILE :=
VBOX_CROSS_CERTIFICATE_FILE_ARGS :=
VBOX_CROSS_CERTIFICATE_SHA2_FILE :=
VBOX_CROSS_CERTIFICATE_SHA2_FILE_ARGS :=
VBOX_PATH_SIGN_TOOLS := C:\Programs\DevKits\8.1\bin\x64
VBOX_PATH_SELFSIGN := C:\WinDDK\7600.16385.1\bin\selfsign
VBOX_PATH_WISUMINFO := "C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts\WiSumInf.vbs"
VBOX_PATH_WISUBSTG  := "C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts\WiSubStg.vbs"
VBOX_WITH_DOCS := 1
VBOX_WITH_DOCS_CHM := 1
VBOX_WITH_DOCS_PACKING := 1
VBOX_WITH_ADDITIONS :=
VBOX_WITH_ADDITIONS_PACKING := 1
VBOX_HAVE_XMLLINT := 1
VBOX_XMLLINT := C:\Programs\xmllint\bin\xmllint.exe
VBOX_PATH_DOCBOOK := C:/Programs/DocBook/xsl
VBOX_PATH_DOCBOOK_DTD := C:/Programs/DocBook/xml
VBOX_PATH_HTML_HELP_WORKSHOP := "C:\Program Files (x86)\HTML Help Workshop"
VBOX_PDFLATEX := C:\Programs\MiKTeX\texmfs\install\miktex\bin\pdflatex.exe
VBOX_PDFLATEX_CMD := $(VBOX_PDFLATEX) -halt-on-error -interaction batchmode
TOOL_CURL_FETCH := C:\Programs\curl\x64\curl.exe
PATH_TOOL_NASM := C:/Programs/nasm
VBOX_INSTALLER_LANGUAGES := en_US
VBOX_WITH_TESTCASES :=
VBOX_WITH_VALIDATIONKIT :=
VBOX_WITH_VBOX_IMG := 1
VBOX_WITH_RECORDING := 1
VBOX_WITH_AUDIO_RECORDING := 1
SDK_VBOX_VPX := 1
VBOX_WITH_LIBVPX := 1
SDK_VBOX_OPUS := 1
VBOX_WITH_LIBOPUS := 1
VBOX_BUILD_PUBLISHER := _OSE

You'll need to edit this template:

• The variables VBOX_CERTIFICATE_SUBJECT_NAME and VBOX_CERTIFICATE_SHA2_SUBJECT_NAME should contain the names of the SHA-1 and SHA-256 certificates, respectively.
• The variables VBOX_CERTIFICATE_FINGERPRINT and VBOX_CERTIFICATE_SHA2_FINGERPRINT should contain the thumbprints of those certificates; you've copied them earlier from the Certificates management console.
• If you have a paid certificate you should delete the lines defining the variables VBOX_CROSS_CERTIFICATE_FILE_ARGS and VBOX_CROSS_CERTIFICATE_SHA2_FILE_ARGS, then in the variables VBOX_CROSS_CERTIFICATE_FILE and VBOX_CROSS_CERTIFICATE_SHA2_FILE (without «_ARGS») put the full path to the cross-certificate (without it the drivers will not be accepted). You can download it from the web site of the company that issued the certificate, or from Microsoft.
• You can fine-tune the signing process using various additional variables and macros to redefine the certificate storage, timestamp server, or even construct a full command line for the signtool program. You can take a look into the file Config.kmk below the comment «Code Signing», there you'll find which variables are defined and how they are used.
• If you've installed some of the programs into paths different from mine, you need to fix those paths in the template. It's strongly recommended to keep the path style for each variable (forward/backward slashes), sometimes it's critical for successful build.
• Fox WiX you need to specify the path to its binaries. With the portable version, it is just the directory where you unpacked it; if you used the installer, the binaries will be located in the subdirectory bin. Please, note, that the path must not contain spaces! If it does you have to convert it into the 8.3 format (you can use dir /x for that). Unfortunately, enclosing the path in quotes does not work with this variable.
• The variable VBOX_BUILD_PUBLISHER specifies the branding suffix in the version number. By default it is «_OSE» (that is, the full product version is «6.1.18_OSE»). You can specify any other suffix here, or set it to empty to get rid of it altogether. But if you delete the variable completely, it will use the default «_OSE».

The rest of the variables are mostly used to declare which components are to be built. And, of course, the main declaration comes at the very top: turn the hardening off.

 Building VirtualBox

Finally, we can now start building VirtualBox itself. Since there are not many people around, who can get their hands on Windows 10 signing, I'll split the instruction into two items. First, the main, «simple» sequence of commands; and then an explanation on which additional steps you need for obtaining the Microsoft signature.

1. Open the console and run the following commands:

   cd /d C:\Devel\VirtualBox-src
   "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64 /win7
   COLOR 07
   set BUILD_TARGET_ARCH=amd64
   cscript configure.vbs --with-DDK=C:\WinDDK\7600.16385.1 --with-MinGW-w64=C:\Programs\mingw64 --with-libSDL=C:\Programs\SDL\x64 --with-openssl=C:\Programs\OpenSSL\x64 --with-openssl32=C:\Programs\OpenSSL\x32 --with-libcurl=C:\Programs\curl\x64 --with-libcurl32=C:\Programs\curl\x32 --with-Qt5=C:\Programs\Qt\5.6.3-x64 --with-libvpx=C:\Programs\libvpx --with-libopus=C:\Programs\libopus --with-python=C:/Programs/Python
   env.bat
   kmk
   kmk C:/Devel/VirtualBox-src/out/win.amd64/release/bin/VirtualBox-6.1.18_OSE-r142142-MultiArch.exe

   The configure.vbs script verifies the environment and generates configuration files (AutoConfig.kmk and env.bat). The first kmk command builds the binaries and collect them into out\win.amd64\bin\. And the last command packs them all into the intermediate MSI package, which is then wrapped into an executable installer. Important notes:
   • You must use forward slashes in the last command. With backslashes, kmk would fail to find the build rules.
   • If you've changed the branding suffix you need to change the name of the target installer by replacing the «_OSE» with what you specified in the variable VBOX_BUILD_PUBLISHER.
   • The revision number in the MSI package name (142142) can be found in the file Version.kmk inside the VBOX_SVN_REV_VERSION_FALLBACK variable definition.
2. Even if you have a paid certificate you will find that you cannot install this distribution into Windows 10 booted with Secure Boot enabled. This OS has stricter requirements, and the drivers must be signed by none other than Microsoft themselves. The procedure is explained in details on various Internet resources and is unrelated to this article, so I'm not going to go deep into this topic. Instead I'll outline the main steps you need to take for integrating this task with the VB build procedure.
   • The main requirement here is that you must have not just a normal code signing certificate, but an EV one (Extended Validation). Also you need to register an account at Hardware Dev Center and add your certificate there.
   • After all the binary components have been built (that is, after the first kmk run which was without arguments), you need to create a CAB archive with the drivers. There is a batch script template for that; the build system modifies it according to the current task and puts it into out\win.amd64\release\repack\. Go into this path and run the following command:

     PackDriversForSubmission.cmd -x

     When the script finishes, the file VBoxDrivers-6.1.18r142142-amd64.cab will appear in the same directory.
   • You need to sign this CAB archive with your EV certificate. Then go to the Microsoft Hardware Dev Center, create a new submission, upload your signed archive, select the desired target OS version (make sure it's 64-bit) and send the submission.
   • It will take several minutes to process, after which you will have a ZIP archive where all the drivers have been signed by Microsoft in addition to your signature, and all the CAT files generated anew. Download this archive and place it somewhere so that the build system had access to it.
   • Unpack the archive and put all the files from the subdirectories directly into out\win.amd64\release\bin\ overwriting the existing files. You can do it either manually or using another script from the same path out\win.amd64\release\repack\ by running the following commands:

     set _MY_SIGNTOOL=C:\Programs\DevKits\8.1\bin\x64\signtool.exe
     UnpackBlessedDrivers.cmd -n -i path\to\signed.zip

     Of course, you need to put here the correct paths to signtool.exe and the ZIP archive.
   • Now you can run the second kmk command which packs all the components into the installer. If you were doing all this in the same console window, don't forget to switch back to the base directory of the VB project.

If neither of us has messed anything up, all this long chain should bring you a shiny new VirtualBox installer which differs from the Oracle one only by its icon, the About picture, and, of course, the profound lack of hardening. It's not hard to replace also the icon and the picture, but I won't go into this topic here.

For convenience I've created a single batch file which automates the full build of VB. You can use it if you need to build the complete installer regularly.

Just a few more words about installing the resultant distribution if you are using a self-signed certificate. As it was suddenly discovered, in Windows 8/10 it's not enough to boot into test mode, installation still fails claiming that the driver signatures are invalid. You can work around this problem by adding the certificates into your CA root storage:

1. Open the Properties dialog of the VB installer by right clicking it and selecting Properties, then switch to the Digital Signatures tab. You'll find two signatures there, both named «Roga and Kopyta Ltd», one is sha1, the other sha256. Select the first one, click Details.
2. Another dialog appears, in that you need to click the View Certificate button.
3. One more dialog, click Install Certificate.
4. In the Import wizard, select the location «Local Machine», click Next. After UAC confirmation (if needed), there's the certificate store selection. Select «Place all certificates in the following store», then click Browse and choose the store named «Trusted Root Certification Authorities». Then Next, Finish. You're done, the certificate is installed.
5. Close all the dialogs but the very first one, select the second signature (sha256) and repeat the steps 2 to 4 for it too.
6. Close all the dialogs, run the installer. Now it should work fine.

 Epilogue

When I finished writing the article I was myself surprized by its volume. At first I was planning to go into many details on reasons for choosing this or that way of solving each problem, explain what the problem is, exactly, and what alternative solutions might be applicable. But very soon it became clear that, if loaded with all these particulars, the text would be monstrously huge. I have to apologize, then, for the recipes looking like «just do this and ask no questions». I have a strong dislike for those myself, but I just couldn't find any other way. Occasionally I still tried to outline what was going on, to soften the impression.

A vast amount of details about VB build system had to remain behind the scene; for I have both felt reluctant to bloat the text, and, sometimes, was too lazy to go look for a different, more efficient approach of solving a problem. After all, my main aim, initially, was to get myself a working build of the current VirtualBox version. 4.3.12 had already been way outdated by that time, but I could not risk updating one of my most actively used tool to the hardened version which might simply stop working at any moment. Although, from time to time I do get back and, having found something new, add it to this article.

I do hope this article will find its audience. If you are interested to see and analyze the final result but don't want to set up all the pile of programs, you can get my version of the distribution from here: version 6.1.18. All the drivers (and other files) are signed with an untrusted self-signed certificate, so for installing this in the 64-bit Windows version you will have to reboot into the test mode. If you have any questions, suggestions, recommendations, please, feel free to write to me either here in comments, or via personal messages. May the Open Source be with you!

 Amendments

• Update of 8.04.2019

1. The article has been translated into English for the first time. Also numerous minor changes were introduced into the Russian version.
2. Added information about drivers signing for Windows 10.
3. The article is updated for VB 6.0.4.
4. cURL, OpenSSL, gSOAP, and some tools were updated.
5. The full build batch no longer has to be placed at the specific path. Also, it now contains the basic template for automated Windows 10 drivers signing.

• Update of 17.02.2021

1. The article is updated for VB 6.0.24.
2. Some dead links were updated.

• Update of 19.02.2021

1. The article is updated for VB 6.1.18.
2. Removed the instructions about building 32-bit libraries.
3. cURL, OpenSSL, gSOAP, libopus.