How to build protobuf with zlib

I recently worked on a task to deliver protobuf messages through Keryx. I needed to cache these messages in memory and on disk and discovered that the messages were uncompressed. I was glad to find that the protobuf C++ API provides zlib-based compression, since this is a good enough way to quickly reduce storage size and delivery time.

Protobuf provides the GzipOutputStream and GzipInputStream classes in the google::protobuf::io namespace. These classes are defined in google/protobuf/io/gzip_stream.h. However, a few linker errors revealed zlib support was not enabled in my initial protobuf build.

Windows is the main development environment and target for Keryx, so this guide will explain how to enable zlib compression for protobuf 3.5.1 built from source using Visual Studio 2015. The process for building with Visual Studio 2017 should be similar. A simpler approach is to install protobuf with zlib support using vcpkg:

You can learn more about vcpkg at GitHub. This guide will instead explain how to manually clone and build protobuf source from GitHub. The following steps differ slightly from those provided by protobuf developers, but you will still need:

• Git to clone the protobuf and zlib repositories
• CMake and Visual Studio to build protobuf and zlib
  

Clone and build zlib

The build instructions for protobuf 3.5.1 at GitHub recommend using zlib 1.2.8. Open a command prompt in a workspace folder (e.g., C:\workspace).
Obtain the zlib 1.2.8 source by cloning the v1.2.8 tag from the zlib project at GitHub with the following command:

We can reduce the number of commits by cloning just the commits leading to the v1.2.8 tag by using the –single-branch argument:

and even further by omitting all commit history and cloning only the commit at the v1.2.8 tag with the –depth=1 argument:

The zlib repository will be inside the zlib directory. Now let’s generate a Visual Studio solution with cmake in a new build directory that installs the resulting files to an install directory:

We can build the solution by manually opening zlib\build\zlib.sln with Visual Studio, or instead simply build from the command prompt using cmake. The following commands build the ALL_BUILD Visual Studio project in Release and Debug modes:

Build the INSTALL Visual Studio project in Release and Debug modes to copy the headers, configuration files, and generated binaries to the installation directory:

The cmake-generated zlib INSTALL project does not install the .pdb file generated in Debug mode, so you may want to copy it manually just in case you need to debug into zlib code:

Clone and build protobuf

Change the command prompt directory back to the parent workspace and clone protobuf 3.5.1 from GitHub:

Create a directory inside the protobuf clone to hold your cmake-generated Visual Studio solution, e.g., builds (there is already a Bazel configuration file called BUILD at the root of the protobuf repository):

Go into the new directory and run cmake against the protobuf repository’s cmake directory with the following configuration flags:

• protobuf_BUILD_SHARED_LIBS=ON to enable building shared libraries, i.e., dynamic link library (DLL) files in Windows
  
• protobuf_BUILD_TESTS=OFF to disable protobuf unit tests and thus avoid introducing a gmock dependency
• protobuf_WITH_ZLIB=ON to enable the use of zlib in protobuf
  
• protobuf_DEBUG_POSTFIX=d to append the letter d to the base name of debug binary files generated by the build
• protobuf_VERBOSE=ON to enable detailed build logs
• protobuf_MSVC_STATIC_RUNTIME=OFF to use the dynamic runtime instead of a static runtime
  
• CMAKE_DEBUG_POSTFIX=d to append the letter d to the base name of debug binary files generated by the build, may be redundant or otherwise duplicate protobuf_DEBUG_POSTFIX
• CMAKE_INSTALL_PREFIX=../install to copy headers, configuration files, and generated binaries to an install directory when building the INSTALL project
• ZLIB_ROOT=../../zlib/install to hint to the
  FindZLIB cmake module where to search for zlib (the instructions provided in cmake/README.md seemed more cumbersome and did not work for me)

We can then build and install the protobuf Visual Studio solution using cmake, just as we did for zlib:

The .pdb files generated by the Debug build are not installed by the INSTALL project, but you can copy them manually to the install\bin directory:

I will explain in an upcoming post how to serialize and unserialize a protobuf message with zlib-backed streams in C++ , as well as unserialize a protobuf message from a byte array of zlib-compressed data with the official protobuf C# API.

This work by Simverge Software LLC is licensed under a Creative Commons Attribution 4.0 International License.