Building Debian Packages for PHP Extensions

Disclaimer: Initially, this document was intended for our internal documentation repository, but the topic turned out to be not-so-internal and possibly interesting to the broader community. So here we are—publishing the doc, and even the code repositories landed on our GitHub. That’s how it goes.

A small note: this article omits some important aspects of a Debian Maintainer’s responsibilities, such as proper release naming, lintian compliance, code signing, and generally doing everything by the book to publish a package publicly according to strict Debian guidelines. This is an internal build. But everything is in your hands. If you're a bearded Linux maintainer, this may all seem painfully naïve to you—please try not to burst out laughing.

We use PHP extensively in our projects—for better or worse, trendy or outdated, it works, doesn’t ask for much, and gets the job done. Also, all this runs in stateful containers within a Debian-like OS environment (in our case, Ubuntu 22.04 LTS, though the exact distro isn’t critical).

The problem: Historically, we’ve relied on some fairly obscure PHP extensions—poorly maintained, barely supported by their authors, and completely absent from standard Debian package sources. We follow the principle of “do it right or don’t do it at all,” so Slackware-style binary installs outside package managers are discouraged. Hence, we need to build proper .deb packages for PHP extensions without breaking compatibility with the surrounding environment.

The specific PHP extensions in question—where this all started—are Blitz (by Alexey Rybak) and php-rdkafka (by Arnaud Le Blanc). They have similar issues, though Blitz's are more severe: no packages in standard repos, last release was ages ago, and since then, the author and community have added a bunch of useful patches—some of which fix compatibility issues with newer PHP versions that didn’t exist when the last release was made. In short: no packages, outdated releases, and valuable patches in HEAD.

Let’s go get our homebrew Debian Package Maintainer badge.

Hint: some of the environment files can be generated automatically or created manually—but that’s not always convenient. Sometimes it’s easier to copy from an existing working solution, and there’s nothing wrong with that.

What does a typical PHP extension build process look like?

• Install the required PHP version and its dev package (phpM.N-dev);

• Install required libraries and their dev packages (e.g., librdkafka and librdkafka-dev for php-rdkafka);

• Clone the repository or download and extract the source code archive;

• Run phpize in the extension directory;

• Run ./configure;

• Run make;

• Run make install;

After that, find the resulting shared library and add it to php.ini so that the interpreter picks it up. Most people probably stop here—especially in minimalist or stateless environments like Docker. But we’re going to build packages.

In fact, there’s an entire handbook on building Debian packages—packed with useful information and highly recommended to avoid stumbling around in the dark or being surprised by “black magic.” I’ll reference it throughout, but generally, I assume the reader has at least a basic understanding of the concepts.

Package Structure

A typical Debian package is an archive containing binaries and other payloads, plus a bit of metadata and installer control scripts (for dpkg, apt, etc.). It's the final product, tied to a specific architecture and set of dependencies. There's little point in unpacking it—no source code inside. Technically, source code can be included in deb packages (the so-called deb-src), but we’re starting from scratch here.

Let’s prep a directory for the future package. It should be empty and clean. We’ll do all the work inside this directory, and its structure will be treated as root-relative. We'll name it php-something, as convention dictates—so, for us, php-rdkafka.

All the Debian magic requires a standardized set of files inside a debian subdirectory. It’s not complicated, especially with the help of the dh-php helper for the debhelper toolset. Why use it? Building a .deb package is a relatively standard step-by-step process, no matter what you're packaging—from documentation to office suites. It can be done manually without helpers, but it’s a titanic effort. Nix maintainers have created tons of helpers for various scenarios. Writing a Makefile by hand is now rare.

The build process for Debian packages has several overrideable steps. These overrides go in debian/rules, which is a Makefile (usually empty unless customization is needed). We’ll just include the dh-php helper.

debian/rules:

#!/usr/bin/make -f
include /usr/share/dh-php/pkg-pecl.mk

Next, we need a control file, which belongs at debian/control. It defines the metadata for the future package. The file follows a strict syntax described in the Debian handbook. The easiest approach is to take an existing control file from another package and tweak it to suit your needs.

The contents of this file determine:

• the package names and the type of software,

• the description, relevant URLs, and maintainer names,

• the list of dependencies required to build or install the package.

Since we’ve started dissecting php-rdkafka, let’s break down the debian/control file from that package:

Source: php-rdkafka
Section: php
Priority: optional
Maintainer: Habr Team <[e-mail]>
Uploaders: Vadim Rybalko <[e-mail]>
Build-Depends: debhelper (>= 10~),
               dh-php (>= 4~),
               liblz4-dev,
               libzstd-dev,
               librdkafka-dev (>= 0.11~),
               php-all-dev
Standards-Version: 4.5.1
Homepage: https://pecl.php.net/package/rdkafka

Package: php-rdkafka
Priority: optional
Section: php
Architecture: amd64
Pre-Depends: php-common (>= 2:69~)
Depends: ${misc:Depends},
         ${pecl:Depends},
         ${php:Depends},
         ${shlibs:Depends}
Breaks: ${pecl:Breaks}
Replaces: ${pecl:Replaces}
Suggests: ${pecl:Suggests}
Provides: ${pecl:Provides},
          ${php:Provides}
Description: PHP-rdkafka is a stable Kafka client for PHP based on librdkafka
 .

The Source section describes the source package, while Package defines the binary package. The metadata is pretty straightforward, and the dependencies include both general and package-specific ones.

There’s a small trick: this file will be regenerated during the build process, but if it’s missing, several stages will fail. That’s why we save it as debian/control.in and also copy it to the conventional debian/control. The debian/control.in file acts as a template for generating the actual debian/control.

Why? Because the modern style for PHP packages in Debian involves including the PHP version number in the package name. Luckily, dh-php takes care of regenerating debian/control with a long list of package sections—one for each supported PHP version—using debian/control.in as the template.

At this stage, we have three files

• debian/control

• debian/control.in

• debian/rules

Next, we need a compatibility file located at debian/compat. It's very simple and contains just the debhelper compatibility version—say, version 10:

debian/compat:

10

Then there's an optional file, debian/gbp.conf, used by the gbp tool suite (gbp-buildpackage, gbp-dch, etc.). We're not actually using these tools, but the file is there just in case—for example, to enable automatic builds from Git tags. Honestly, we haven’t mastered those utilities (maybe someday), but we still maintain a strict branch and tag structure in Git, as described here, for future use:

debian/gbp.conf:

[DEFAULT]
debian-branch = debian/main
debian-tag = debian/%(version)s
upstream-branch = upstream
upstream-tag = upstream/%(version)s
pristine-tar = True

[dch]
meta = 1

[import-orig]
filter = ['.gitignore','debian']

This defines that the main packaging branch is called debian/main, each release is tagged as debian/%(version)s, the branch for upstream source is upstream, and each upstream source set is tagged as upstream/%(version)s. We'll go over how this works in practice a bit later—Git isn't initialized in the directory yet anyway.

At this stage, we have five files

• debian/compat

• debian/control

• debian/control.in

• debian/gbp.conf

• debian/rules

Next are two dh-php specific files used to create an .ini file for automatic inclusion when the package is installed in the system. The first one, debian/php-rdkafka.php, references the second:

debian/php-rdkafka.php:

mod debian/rdkafka.ini

The second file, debian/rdkafka.ini, isn’t much more complex.

debian/rdkafka.ini:

extension=rdkafka.so

At this point, it's a good idea to define the patch format—since we’ll definitely need to apply patches. There’s not much choice here, and in 95% of cases, it’s quilt. So we’ll declare that in debian/source/format:

3.0 (quilt)

If you ever need to use lintian—for instance, if you're planning to publish your build on Launchpad—you’ll have to learn how to work with override files. That’s because the vast majority of PHP sources will generate warnings out of nowhere. For now, we can simply create a placeholder file to deal with later. It’ll be debian/source.lintian-overrides with minimal content:

# Override invalid PHP license problem for PHP extensions

At this stage, we have ten files

• debian/compat

• debian/control

• debian/control.in

• debian/gbp.conf

• debian/php-rdkafka.php

• debian/rdkafka.ini

• debian/rules

• debian/source/format

• debian/source.lintian-overrides

We’re almost done with the debian directory. Two items remain: the changelog file (debian/changelog) and the patch directory—but those will be covered in a separate section.

So, debian/changelog holds version numbers and a list of changes. It’s not just a feel-good file for meatbags like most release notes—it actually defines the version number used during package build. While this file can be generated from Git, I haven’t mastered that part, so I edit it manually or via dch.

It follows a strict syntax. Here's what a fragment might look like:

php-rdkafka (6.0.3+4.1.2-2) stable; urgency=medium

  * Release for version 6.0.3
   - Ability to provide custom `librdkafka` path during pecl install (#526, @Wirone)
  * Unreleased patches for 6.0.3
   - Automation at Sat Jul 2 15:10:53 2022 +0200
   - Update release notes at Sat Jul 2 15:15:04 2022 +0200
   - Add private constructor on Metadata classes (#531) at Tue Jul 26 20:44:00 2022 +0200
   - Test against PHP 8.3 and librdkafka 1.9, 2.3 (#545) at Mon Dec 4 15:01:53 2023 +0100
   - feat: implement oauthbearer token refresh cb setter (#546) at Fri Jan 5 14:11:25 2024 -0500
   - Update README.md at Sat Jun 1 13:35:04 2024 +0200
  * Release for version 4.1.2
   - Enabled features on windows build: headers, purge, murmur (#410, @nick-zh, @cmb69)

 -- Vadim Rybalko <[e-mail]>  Mon, 28 Jul 2024 01:08:0230 +0100

php-rdkafka (6.0.3+4.1.2-1) stable; urgency=medium

  * Release for version 6.0.3
   - Ability to provide custom `librdkafka` path during pecl install (#526, @Wirone)
  * Release for version 4.1.2
   - Enabled features on windows build: headers, purge, murmur (#410, @nick-zh, @cmb69)

 -- Vadim Rybalko <[e-mail]>  Fri, 25 Nov 2022 19:30:45 +0200

php-rdkafka (6.0.2+4.1.2-1) stable; urgency=medium

  * Release for version 6.0.2
   - Fixed signature of RdKafka\KafkaConsumer::getMetadata() (#521, @arnaud-lb)
  * Release for version 4.1.2
   - Enabled features on windows build: headers, purge, murmur (#410, @nick-zh, @cmb69)

 -- Vadim Rybalko <[e-mail]>  Fri, 25 Nov 2022 19:19:50 +0200

...

What should you pay attention to here? Each section consists of the package name, version (or versions—we’ll get to that in a bit), priority, a description block, and a signature. The version number follows a specific format.

If we’re taking a release from upstream and simply want to reflect that in the changelog, we specify the version as defined by the upstream maintainer. But if we’re modifying it (and we are, since we’re at the very least adding Debian packaging scaffolding), then we append a revision number after a dash.

In the example above, a plus sign is used in the version string—this is a common way to express that the package supports multiple PHP versions, including older ones no longer maintained upstream. A bit later, it’ll become clearer how this versioning trick works.

If we’ve added patches relative to the last upstream release, we increment the revision number after the dash again. This comes in handy when pulling in patches from merge requests in the original Git repository.

If you don’t have the source code at hand yet, you can leave debian/changelog mostly empty and populate it later.

Now that almost everything is ready, it’s time to pull in the source code and wrap this all up into a proper package. Head to the upstream repository, check out the latest release, and save it in a subdirectory named after its version. Alternatively, download the release archive and unpack it:

I’m not a fan of that approach—I prefer pulling the source from a local Git clone in a neighboring directory. That said, downloading the archive is definitely the easiest way.

You should now have a file structure that looks like this:

• debian/changelog

• debian/compat

• debian/control

• debian/control.in

• debian/gbp.conf

• debian/php-rdkafka.php

• debian/rdkafka.ini

• debian/rules

• debian/source/format

• debian/source.lintian-overrides

• rdkafka-6.0.3/...

The ellipsis represents the source code files. Now let’s dig into them and find the package.xml file. This file is pretty important—it contains key metadata used by dh-php. For example, it defines the range of compatible PHP versions:

...
 <dependencies>
  <required>
   <php>
    <min>7.0.0</min>
    <max>8.99.99</max>
   </php>
   <pearinstaller>
    <min>1.4.8</min>
   </pearinstaller>
  </required>
 </dependencies>
...

As you can see, this version of php-rdkafka is not compatible with PHP 5.6. That’s not a big deal for us here, but other extensions, like Blitz, may have stricter constraints—with hard version ceilings for each PHP major release, and sometimes even issues with minors.

So, let’s use rdkafka as a sandbox and add support for building a package for PHP 5.6 as well. By looking through the commit history, we find that the last version compatible with PHP 5.6 is 4.1.2. Download it and extract it alongside the newer one. Now we have two source directories: rdkafka-6.0.3 and rdkafka-4.1.2. Each of them contains a package.xml.

Copy both XML files into the root of your working directory: the current one as package.xml, and the one from version 4.1.2 as package-5.xml.

The naming convention is important—dh-php looks specifically for XML files with names matching the major or major.minor version suffix.

$ cp rdkafka-6.0.3/package.xml package.xml
$ cp rdkafka-4.1.2/package.xml package-5.xml

Let’s double-check the current directory structure:

• debian/changelog

• debian/compat

• debian/control

• debian/control.in

• debian/gbp.conf

• debian/php-rdkafka.php

• debian/rdkafka.ini

• debian/rules

• debian/source/format

• debian/source.lintian-overrides

• package-5.xml

• package.xml

• rdkafka-4.1.2/...

• rdkafka-6.0.3/...

NB! The package.xml file may not always be present in the sources. Technically, it’s a PECL manifest and isn’t strictly part of the extension source itself. If it’s missing, you’ll need to create one manually or generate it using a special utility (which is often more trouble than it’s worth). Fortunately, the file is fairly small and declarative—you can safely take one from a similar package and adjust the contents quickly: metadata, changelog, and file list (which is optional, by the way).

Now’s a good time to initialize Git, just in case. Run git init, then create the upstream branch and add the source files:

(our_repo) $ git init
(our_repo) $ git checkout -b upstream
(our_repo) $ git add rdkafka-4.1.2 rdkafka-6.0.3

Commit the changes with a standard message (I borrowed one from the dh-php author Ondřej Surý’s repositories), and create a tag:

(our_repo) $ git commit -m "New upstream version 6.0.3+4.1.2"
(our_repo) $ git tag upstream/6.0.3+4.1.2

Now create the debian/main branch, switch to it, and merge in the corresponding upstream tag:

(our_repo) $ git checkout -b debian/main
(our_repo) $ git merge upstream/6.0.3+4.1.2

Time to populate debian/changelog as planned earlier. Pull changelog highlights from package.xml or the release page in the upstream repo, and fill in a new section in the file. At this point, everything is nearly ready for building the release.

Let’s commit what we’ve got. Add everything that’s left—both XML files and the entire debian directory—then commit to the debian/main branch and create the tag:

(our_repo) $ git add *.xml debian
(our_repo) $ git commit -m "Release for 6.0.3+4.1.2-1"
(our_repo) $ git tag debian/6.0.3+4.1.2-1

We now have two branches and two tags:

(our_repo) $ git branch
* debian/main
  upstream
(our_repo) $ git tag
debian/6.0.3+4.1.2-1
upstream/6.0.3+4.1.2

Building

NB! Chances are it won’t build successfully on the first try—something will probably go wrong. But hey, it’s worth a shot.

First, install build-essential (odd if you don’t already have it), along with the packages listed under Build-Depends in debian/control. Then, install the dev packages for all PHP versions you want to target. We're going full house here:

# apt install php5.6-dev php7.0-dev php7.1-dev \
php7.2-dev php7.3-dev php7.4-dev php8.0-dev \
php8.1-dev php8.2-dev php8.3-dev

I’ve already got them installed:

php5.6-dev is already the newest version (5.6.40-77+ubuntu22.04.1+deb.sury.org+1).
php7.0-dev is already the newest version (7.0.33-75+ubuntu22.04.1+deb.sury.org+1).
php7.1-dev is already the newest version (7.1.33-63+ubuntu22.04.1+deb.sury.org+1).
php7.2-dev is already the newest version (7.2.34-50+ubuntu22.04.1+deb.sury.org+1).
php7.3-dev is already the newest version (7.3.33-19+ubuntu22.04.1+deb.sury.org+1).
php7.4-dev is already the newest version (1:7.4.33-13+ubuntu22.04.1+deb.sury.org+1).
php8.0-dev is already the newest version (1:8.0.30-7+ubuntu22.04.1+deb.sury.org+1).
php8.1-dev is already the newest version (8.1.29-1+ubuntu22.04.1+deb.sury.org+1).
php8.2-dev is already the newest version (8.2.21-1+ubuntu22.04.1+deb.sury.org+1).
php8.3-dev is already the newest version (8.3.9-1+ubuntu22.04.1+deb.sury.org+1).

Great—now we can build. We'll do this in a subdirectory so we don’t clutter the repository with build artifacts:

(our_repo) $ mkdir -p _build/src
(our_repo) $ cp -r rdkafka-* _build/src/
(our_repo) $ cp -r debian _build/src/
(our_repo) $ cp -r package*.xml _build/src/
(our_repo) $ cd _build/src && dpkg-buildpackage -b -rfakeroot -us -uc

You can wrap all of this into a simple Makefile right away.

SHELL := /bin/sh
.DEFAULT_GOAL := build

build:
	mkdir -p _build/src
	rm -rf _build/src/*
	cp -r rdkafka-* _build/src/
	cp -r debian _build/src/
	cp -r package*.xml _build/src/
	cd _build/src && dpkg-buildpackage -b -rfakeroot -us -uc
	cd ../../

clean:
	test -d _build || rm -rf _build

You’ll see a ton of output. Let’s try to make sense of the key steps:

• First, it prints out the package metadata—which we’ve already written ourselves.

• Then, the script cleans everything and creates a separate build directory for each PHP version.

• Inside each build directory, the source code is copied according to the compatibility rules from the XML files, and then phpize, ./configure, and make are run for each PHP version.

• Next come the tests. It’s important that they pass. If they fail but you’re sure it’s a known issue and not a real bug (say, it's mentioned in the upstream repo’s issue tracker), you can skip the test phase entirely using the environment variable DEB_BUILD_OPTIONS="nocheck" before calling dpkg-buildpackage.

• After that, controlled installation takes place, and the resulting binaries are wrapped with the standard Debian PHP package structure—along with the appropriate .ini file hooks for automatic inclusion.

• You’ll see an absurd number of warnings—ignore them. The build will eventually finish.

All the output files will land in the _build directory, something like this:

(our_repo) $ ll _build
total 2168
drwxrwxr-x  3 vadim vadim   4096 Jul 28 00:59 ./
drwxr-xr-x  8 vadim vadim   4096 Jul 28 00:54 ../
-rw-r--r--  1 vadim vadim   1634 Jul 28 00:59 php-rdkafka-all-dev_6.0.3+4.1.2-1_amd64.deb
-rw-rw-r--  1 vadim vadim  19702 Jul 28 00:59 php-rdkafka_6.0.3+4.1.2-1_amd64.buildinfo
-rw-rw-r--  1 vadim vadim   8780 Jul 28 00:59 php-rdkafka_6.0.3+4.1.2-1_amd64.changes
-rw-r--r--  1 vadim vadim   1580 Jul 28 00:59 php-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 148470 Jul 28 00:59 php5.6-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  33582 Jul 28 00:59 php5.6-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 171860 Jul 28 00:59 php7.0-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37500 Jul 28 00:59 php7.0-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 175604 Jul 28 00:59 php7.1-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37358 Jul 28 00:59 php7.1-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 171798 Jul 28 00:59 php7.2-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37250 Jul 28 00:59 php7.2-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 174434 Jul 28 00:59 php7.3-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  36804 Jul 28 00:59 php7.3-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 176770 Jul 28 00:59 php7.4-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  36780 Jul 28 00:59 php7.4-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 179426 Jul 28 00:59 php8.0-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37282 Jul 28 00:59 php8.0-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 182130 Jul 28 00:59 php8.1-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37518 Jul 28 00:59 php8.1-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 186640 Jul 28 00:59 php8.2-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37508 Jul 28 00:59 php8.2-rdkafka_6.0.3+4.1.2-1_amd64.deb
-rw-r--r--  1 vadim vadim 186568 Jul 28 00:59 php8.3-rdkafka-dbgsym_6.0.3+4.1.2-1_amd64.ddeb
-rw-r--r--  1 vadim vadim  37688 Jul 28 00:59 php8.3-rdkafka_6.0.3+4.1.2-1_amd64.deb
drwxrwxr-x 15 vadim vadim   4096 Jul 28 00:59 src/

And there you have it—your packages, ready to be installed manually or published to your own repository. As you wish.

New level unlocked

Thought we were done? Not quite. There's a decent chance the build won’t succeed and will crash somewhere along the way. Most likely, after a two-year-old release, the build is simply broken for current PHP versions—and you’ll need to apply patches that the maintainer never bothered to wrap into a new release. Or maybe you’ll have to write the patches yourself.

Let’s start by inspecting the upstream repository with git log:

(original_repo) $ git log
commit 9cafbba8808963a373374524b0030f63d1b4c471 (HEAD -> 6.x, origin/HEAD, origin/6.x)
Author: Arnaud Le Blanc <[e-mail]>
Date:   Sat Jun 1 13:35:04 2024 +0200

    Update README.md

commit bcd5004f461d1d3a5f879bb21280bdde6f6800c2
Author: cb-freddysart <[e-mail]>
Date:   Fri Jan 5 14:11:25 2024 -0500

    feat: implement oauthbearer token refresh cb setter (#546)

commit b21a905832202e9051d0627036a96135f9c34da5
Author: Arnaud Le Blanc <[e-mail]>
Date:   Mon Dec 4 15:01:53 2023 +0100

    Test against PHP 8.3 and librdkafka 1.9, 2.3 (#545)
    
    
    
    Co-authored-by: Dirk Adler <[e-mail]>

commit 0aee7cf70a287d3e901787be144b7ff5a1441701
Author: Arnaud Le Blanc <[e-mail]>
Date:   Tue Jul 26 20:44:00 2022 +0200

    Add private constructor on Metadata classes (#531)

commit 428a552c5219120ca456b462fd67cedd53b55325
Author: Arnaud Le Blanc <[e-mail]>
Date:   Sat Jul 2 15:15:04 2022 +0200

    Update release notes

commit 413f7cce13d9456bd9bbc30402da86143574cc10
Author: Arnaud Le Blanc <[e-mail]>
Date:   Sat Jul 2 15:10:53 2022 +0200

    Automation

commit 9fca57149805f0d07c0cad9a8fd0155da455f2ae (tag: 6.0.3)
Author: Arnaud Le Blanc <[e-mail]>
Date:   Sat Jul 2 15:09:30 2022 +0200

    release/6.0.3 (#528)

Aha, there they are. Turns out there were 6 commits after the last release—two of them pretty important. We’ll need to grab those. Earlier, we checked out the upstream release tag into the upstream branch of our repo. We could just check out their latest HEAD, but that wouldn’t be the True Way™. We’ll do it properly—with patches.

The Debian Maintainer’s Handbook has a solid section on how to patch third-party code for compatibility. That’s what we’ll follow. First, we need a handy alias for dquilt, which we'll use to apply our life-saving patches. Add the following to your ~/.bashrc (or equivalent shell config—zsh folks know the drill):

alias dquilt="quilt --quiltrc=${HOME}/.quiltrc-dpkg"
. /usr/share/bash-completion/completions/quilt
complete -F _quilt_completion -o filenames dquilt

Then create the file ~/.quiltrc-dpkg with the following contents:

d=. ; while [ ! -d $d/debian -a $(readlink -e $d) != / ]; do d=$d/..; done
if [ -d $d/debian ] && [ -z $QUILT_PATCHES ]; then
    # if in Debian packaging tree with unset $QUILT_PATCHES
    QUILT_PATCHES="debian/patches"
    QUILT_PATCH_OPTS="--reject-format=unified"
    QUILT_DIFF_ARGS="-p ab --no-timestamps --no-index --color=auto"
    QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
    QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
    if ! [ -d $d/debian/patches ]; then mkdir $d/debian/patches; fi
fi

Next, add a .gitignore file to your repo so intermediate patching or build results don’t get in your way:

.pc/
_build/

The .pc directory is where all the patching magic happens. Once created, the patches will be saved to the debian/patches directory—which we don’t have yet, but dquilt will handle that for us.

Almost ready. For convenience, clone the upstream repository into a nearby directory so you can easily merge or copy from it.

Before patching, let’s define a strategy. I prefer creating a separate patch for each commit in the upstream repo, but you can also make a single patch covering a range of commits. Looking at the log, I see six commits since the last release. Let’s pull the list of files changed in each one:

(original_repo) $ git diff --name-only 9fca57149805f0d07c0cad9a8fd0155da455f2ae 413f7cce13d9456bd9bbc30402da86143574cc10
.github/workflows/release.yml
tools/extract-release-notes.php
tools/prepare-release.sh
(original_repo) $ git diff --name-only 413f7cce13d9456bd9bbc30402da86143574cc10 428a552c5219120ca456b462fd67cedd53b55325
package.xml
(original_repo) $ git diff --name-only 428a552c5219120ca456b462fd67cedd53b55325 0aee7cf70a287d3e901787be144b7ff5a1441701
metadata.c
metadata.stub.php
metadata_arginfo.h
metadata_broker.c
metadata_broker.stub.php
metadata_broker_arginfo.h
metadata_broker_legacy_arginfo.h
metadata_collection.c
metadata_collection.stub.php
metadata_collection_arginfo.h
metadata_collection_legacy_arginfo.h
metadata_legacy_arginfo.h
metadata_partition.c
metadata_partition.stub.php
metadata_partition_arginfo.h
metadata_partition_legacy_arginfo.h
metadata_topic.c
metadata_topic.stub.php
metadata_topic_arginfo.h
metadata_topic_legacy_arginfo.h
package.xml
php_rdkafka_priv.h
tests/metadata_001.phpt
tests/metadata_broker_001.phpt
tests/metadata_collection_001.phpt
tests/metadata_partition_001.phpt
tests/metadata_topic_001.phpt
tests/topic_partition_001.phpt
tests/topic_partition_002.phpt
topic_partition.c
(original_repo) $ git diff --name-only 0aee7cf70a287d3e901787be144b7ff5a1441701 b21a905832202e9051d0627036a96135f9c34da5
.github/workflows/test.yml
(original_repo) $ git diff --name-only b21a905832202e9051d0627036a96135f9c34da5 bcd5004f461d1d3a5f879bb21280bdde6f6800c2
.github/workflows/test/build-librdkafka.sh
conf.c
conf.h
conf.stub.php
conf_arginfo.h
conf_legacy_arginfo.h
config.m4
package.xml
tests/conf_callbacks.phpt
tests/conf_callbacks_rdkafka11.phpt
(original_repo) $ git diff --name-only bcd5004f461d1d3a5f879bb21280bdde6f6800c2 9cafbba8808963a373374524b0030f63d1b4c471
README.md

Commit 9fca57149805f0d07c0cad9a8fd0155da455f2ae was the release tag—we’ll start from the commit right after it.

In our local repo, let’s enter patching mode and create the first patch, then specify which files will be modified:

(our_repo) $ dquilt new v6_0_3_413f7cce13d9456bd9bbc30402da86143574cc10.patch
(our_repo) $ dquilt add rdkafka-6.0.3/.github/workflows/release.yml \
rdkafka-6.0.3/tools/extract-release-notes.php \
rdkafka-6.0.3/tools/prepare-release.sh 

You can name the patch file however you want, but it’s a good idea to give it something meaningful—a brief summary or the commit hash, for example.

Note: you need to prefix the file paths with the name of the directory containing the source files. This way, dquilt will track changes to the correct files.

Now, check out the corresponding commit in the upstream repo, and rsync the updated files into the appropriate version directory in your repo:

(original_repo) $ git checkout 413f7cce13d9456bd9bbc30402da86143574cc10
(original_repo) $ rsync -av --delete ./ ../our_repo/rdkafka-6.0.3/

If everything went smoothly, git status should show three modified files. Let’s generate the patch and add a description (you can use the commit message from git log):

(our_repo) $ dquilt refresh
(our_repo) $ dquilt header -e

This results in two new files in the debian directory:

• debian/patches/series — lists all patches in the order they should be applied

• debian/patches/v6_0_3_413f7cce13d9456bd9bbc30402da86143574cc10.patch — the actual patch file

Quick sanity check: if you compare the contents of the patch file with the output of git diff between the two commits, they should match. So yes, you can generate patches directly from Git—but I find dquilt more convenient.

Repeat this process for however many patches you need. One thing to watch out for: if the patch modifies package.xml, make sure to add both paths for the file (if it exists in multiple directories), and copy the updated XML file from the upstream source into the root of your working directory after applying it.

...
(our_repo) $ dquilt add package.xml rdkafka-6.0.3/package.xml
...
копирование в rdkafka-6.0.3/ из внешнего источника
...
(our_repo) $ cp rdkafka-6.0.3/package.xml package.xml
...

Once all patches are in place, you’ll have several new files in debian/patches. Time for a new release: add a new section to debian/changelog, and bump the revision number (the part after the dash). Then commit the changes:

(our_repo) $ git add debian/patches
(our_repo) $ git add debian/changelog
(our_repo) $ git commit -m "Release for 6.0.3+4.1.2-2"
(our_repo) $ git tag debian/6.0.3+4.1.2-2

Looks good—but now the repo’s a mess. Because of patching, we’ve got uncommitted changes in the upstream source files. We don’t need those, since they’re already included in the patches. Clean them up using standard methods (git checkout, rm -r, etc.).

Surprises

Expect plenty of them—we’re working with someone else’s code, often abandoned.

For example, there’s a quirk with dh-php: you can’t reliably add your own logic to debian/rules, because override targets get intercepted by dh-php. In my case, I wanted to tweak the override_dh_auto_test target to allow tests to run, but ignore failures (a pretty common pattern). However, my changes were stubbornly ignored. I eventually concluded that dh-php was the culprit and ended up disabling the test stage entirely for the php-blitz package using the more heavy-handed DEB_BUILD_OPTIONS="nocheck".

In another repository, the author had added some files to .gitignore after they were already committed. As a result, those files were only present in my working copy, didn’t make it into my repo because of the nested .gitignore, and that broke the build.

Some upstream patches even broke compatibility with older PHP versions. In these cases, you either need to split your releases by PHP version or carefully apply patches by hand—if it’s obvious what needs to be done.

In the End

So, what are our options once the build is done?

• You can grab the .deb file you need from the build output and install it directly via dpkg on your target system.

• You can sign the release and push it to your private package repository.

• You can polish the whole setup to full Debian guideline compliance and submit it to a public repo, like Launchpad.

As for us—we just build the packages via CI and store the results in GitLab artifacts. Everything is deployed via Ansible anyway. Still, it's worth wrapping things up into a proper repository—because it’s really not that hard.