Overview

Once every true-linux engineer gets a trouble: there is no any software in his distro or it's built without needed options. I am keen on the phrase: "Only source control gives you freedom".

Of course, you can build this software on your computer without any src-packages, directly (with simplification: configure, make, make install). But it's a non-reproducible solution, also hard for distribution.

The better way is to make distro-aligned package that can be built if needed and that produces lightly distributed binary-packages. It's about debian-source packages(debian,ubuntu,etc), pkgbuild (for arch), ebuild for gentoo, src-rpm for red hat-based, and many others.

I will use cri-o like a specimen.

Before reading the text below I strongly recommend to get familiarized with the official Debian policy manual placed here and debhelper manpage.

Also you will be required to setup some variables like DEBMAIL and DEBFULLNAME for proper data in changelog and other places.

Theory

Everything looks pretty simple. You should make a source-package with a source code, instructions how to build, patches, control files and some additional files that you may need.

I will not try to describe all potential variants, you can find them in debian policy. I see no pros to retype debian manuals.
I would like to highlight only crucial files:

control: file with mandatory information like package name, version, source, checksums, other data
rules: make-file with instructions how to build software
patches — directory with your patches for software. According to using quilt patch management system, you must have this directory with series file or build will fail.

Git interaction schema

Ok, this part is important. Usually in articles authors write how-to build package only once. But information on how-to maintain it, how to produce new versions again, again and again is missing.

I prefer to use such tips:

store ready src-deb and binary-deb in repo (everybody does it)
make separate git for your debian-dir with files for source-package
create and use additional target in rules:
- src-clean to clean "vanilla" source codes
- src-get to get "vanilla" source codes from original package git
- build-clean, to clean binary packages(useful for development and testing process)

Therefore there are 2 gits:

original software git, called vanilla
your git with debian dir for package creation

Environment preparation

First of all, you need to prepare your DEBMAIL and DEBFULLNAME vars. You can do it in ~/.bashrc

sed -i '/DEBEMAIL/d' ~/.bashrc
sed -i '/DEBFULLNAME/d' ~/.bashrc

cat << EOF | tee -a ~/.bashrc 2>/dev/null
DEBEMAIL="skif@skif-web.ru"
DEBFULLNAME="Alexey Lukyanchuk"
export DEBEMAIL DEBFULLNAME
EOF

source ~/.bashrc

Second step is config for quilt:

cat << EOF |tee  ~/.quiltrc 2>/dev/null
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using `quilt diff`.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
EOF

And I detest visual mode in vim, so

touch ~/.vimrc
sed -i '/^set mouse/d' ~/.vimrc
echo "set mouse-=a" >> ~/.vimrc

And don't forget to setup your git variables. Full explanation may be found here, I will do it in fast way:

git config --global user.email "skif@skif-web.ru"
git config --global user.name "Alexey Lukyanchuk"

If we talk about Debian, you need to install some dependencies:

build-essential
debmake
quilt
devscripts

I will do all work in podman container because it's comfortable and provides clear environment. Thus /volume dir is a dir with my project.

Prepare src package

I will split creation process into some steps. I will use "debmake" program with some arguments.

Skeleton

Ok, now I need to create skeleton of my future source package. The simple way to do it is "debmake" program.

First of all you need to download source code of software and and make archive with original source code. Usually you can find recommendations to download archive and rename it. I prefer to use git directly because sometimes you need to make packages for internal software of your company (which means accessible git but optional http archive):

cd /volume
git clone -b v1.26.0 --single-branch https://github.com/cri-o/cri-o.git ./cri-o

This nice command will clone only single version of sources instead of cloning all git.

Now I will create skeleton by using debmake tool. It can recognize package name version, but sometime something goes wrong. So, I will help a little bit:

cd cri-o
debmake -u 1.26.0 -r 1 -t

This command will produce some files:

ls -1 ../
cri-o
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_v1.26.0.orig.tar.gz

cri-o-v1.26.0 it's a dir with source code and debian dir.
cri-o-v1.26.0.tar.gz is a arch with software source code only.
cri-o_v1.26.0.orig.tar.gz is a symlink to cri-o-v1.26.0.tar.gz

I want to continue work in cri-o dir because it's more effective in feature, so, I will move debian dir from cri-o-1.26 to cri-o:

cp -r ../cri-o-v1.26.0/debian ./

Now you can see minimum and sufficient set of files. I don't want to plunge into theory (everything is descried in debian policy document), so, let's work.

ls debian/
README.Debian  changelog  control  copyright  patches  rules  source  tests  upstream  watch

Control

First of all, I want transform control file to proper state. Initial version looks pretty good as an example but not for production:

Source: cri-o
Section: unknown
Priority: optional
Maintainer: Alexey Lukyanchuk <skif@skif-web.ru>
Build-Depends: debhelper-compat (= 13)
Standards-Version: 4.5.1
Homepage: <insert the upstream URL, if relevant>
Rules-Requires-Root: no

Package: cri-o
Architecture: any
Multi-Arch: foreign
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: auto-generated package by debmake
 This Debian binary package was auto-generated by the
 debmake(1) command provided by the debmake package.

You can find list of Sections here. I want to use "utils".

Let's make proper control file:

Source: cri-o
Section: utils
Priority: optional
Maintainer: Alexey Lukyanchuk <skif@skif-web.ru>
Build-Depends: debhelper-compat (= 13),
               golang (>=1.19),
               golang-github-containers-common,
               libbtrfs-dev,
               libgpgme-dev,
               libseccomp-dev,
               pkg-config
Standards-Version: 4.5.1
Homepage: https://github.com/cri-o/cri-o
Vcs-Git: https://github.com/skif-web/cri-o-deb.git
Vcs-Browser: https://github.com/skif-web/cri-o-deb
Rules-Requires-Root: no

Package: cri-o
Architecture: any
Multi-Arch: foreign
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: auto-generated package by debmake
 This Debian binary package was auto-generated by the
 debmake(1) command provided by the debmake package.

Patches

In process of build I got trouble with btrfs. I found solution here.
Full usage of quilt is well described in debian article here.
Ok, let's see, how to add patch to source code in debian quilt style:

# apply existing patches
$ quilt push -a
No series file found
# make new patch
$ quilt new 0001-fix-btrfs-build-error.patch
Patch debian/patches/0001-fix-btrfs-build-error.patch is now on top
# add file to patch
$ quilt add vendor/github.com/containers/storage/drivers/btrfs/btrfs.go 
File ca added to patch debian/patches/0001-fix-btrfs-build-error.patch
$ vim vendor/github.com/containers/storage/drivers/btrfs/btrfs.go
# update patch from changed sources
$ quilt refresh
Refreshed patch debian/patches/0001-fix-btrfs-build-error.patch
# make proper header for patch
$ quilt header -e --dep3
Replaced header of patch debian/patches/0001-fix-btrfs-build-error.patch
# un-apply changes from source code
$ quilt pop -a
Removing patch debian/patches/0001-fix-btrfs-build-error.patch
Restoring vendor/github.com/containers/storage/drivers/btrfs/btrfs.go

No patches applied

And now I can show result — ready patch:

$cat debian/patches/0001-fix-btrfs-build-error.patch
Description: build fix (btrfs driver error)
Author: Shengjing Zhu <zhsj@debian.org>
---
This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
Index: containers-common/vendor/github.com/containers/storage/drivers/btrfs/btrfs.go
===================================================================
--- containers-common.orig/vendor/github.com/containers/storage/drivers/btrfs/btrfs.go
+++ containers-common/vendor/github.com/containers/storage/drivers/btrfs/btrfs.go
@@ -383,7 +383,7 @@ func subvolLimitQgroup(path string, size
        defer closeDir(dir)

        var args C.struct_btrfs_ioctl_qgroup_limit_args
-       args.lim.max_referenced = C.__u64(size)
+       args.lim.max_rfer = C.__u64(size)
        args.lim.flags = C.BTRFS_QGROUP_LIMIT_MAX_RFER
        _, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_QGROUP_LIMIT,
                uintptr(unsafe.Pointer(&args)))
$cat debian/patches/series 
# You must remove unused comment lines for the released package.
0001-fix-btrfs-build-error.patch

Additional files to clean

In this package I got a surprise: Makefile of cri-o produce crio.conf. It breaks build process because it's assessed like local source changes:

dpkg-source: info: local changes detected, the modified files are:
 cri-o/crio.conf

The simplest way to resolve it — add this file to remove it on clean target in debian/rules file. Create file debian/clean and add any path that must be removed by dh_clean:

$ cat debian/clean 
crio.conf

Add extra targets for comfortable maintaining

Well, now we come to my favorite part — add functions to get vanilla sources new versions. Let's see my rules:

#!/usr/bin/make -f
# You must remove unused comment lines for the released package.
#export DH_VERBOSE = 1
#export DEB_BUILD_MAINT_OPTIONS = hardening=+all
#export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
#export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed

mkfile_path:= $(abspath $(lastword $(MAKEFILE_LIST)))
debdir_path:= $(shell dirname ${mkfile_path})
pkgdir_path:= $(shell dirname ${debdir_path})
version:=$(shell dpkg-parsechangelog -SVersion -l ${debdir_path}/changelog)
vanillaVersion:=$(shell echo ${version}|awk -F\- '{print $$1}')
tmp_dir:= $(shell mktemp -d --tmpdir=${pkgdir_path})
pkgrootdir_path:= $(shell dirname ${pkgdir_path})
pkgdir_selfname:=$(shell basename ${pkgdir_path})

%:
    PREFIX=/usr dh $@

# override_dh_auto_install:
#   dh_auto_install -- prefix=/usr

#override_dh_install:
#   dh_install --list-missing -X.pyc -X.pyo

src-clean:
    find ${pkgdir_path} -maxdepth 1 -mindepth 1 -not -name debian -a -not -name .git -exec rm -rf  {} \;

src-get: src-clean
    $(eval tmp_dir := $(shell mktemp -d --tmpdir=${pkgdir_path}))
    git clone -b v${vanillaVersion} --single-branch https://github.com/cri-o/cri-o.git ${tmp_dir}/
    rm -rf ${tmp_dir}/.git
    mv ${tmp_dir}/* ${pkgdir_path}/
    rm -rf ${tmp_dir}
    debmake -t

build-clean:
    find ${pkgrootdir_path} -name ${pkgdir_selfname}_* -o -name ${pkgdir_selfname}-*|xargs rm -rf 

.PHONY: get-src src-clean build-clean

First of all, I have added vars to get version of packages and software from changelog. It's beautiful practice that guarantees alignment between sources and packages. And my targets src-clean and src-get remove downloaded sources and get new version from git.

And now to get new version I should make 3 steps:

make -f debian/rules src-clean — to clean old sources
add new version to changelog (with dch tool)
make -f debian/riles src-get — to get software sources new version
make -f debian/riles build-clean — to remove artifacts like bin packages, support files.

Add tests

After that lintian reveals next problem: no tests. I have writtten smoke test.
More complete explanation of tests may be found here.

There are 2 files that provide test suite:

debian/tests/control enumerates metadata
debian/tests/smoke is one of the tests

$ cat debian/tests/control 
Tests: smoke
$ cat debian/tests/smoke 
Test-Command: /usr/local/bin/crio -v

Copyright

Now we need to edit copyright file in debian dir. Debmake-tool creates it by scanning sources, but we need to change some lines in the beginning. I will change header to this state:

Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: cri-o
Upstream-Contact: https://github.com/cri-o/cri-o
Source: https://github.com/cri-o/cri-o.git

Explanation of this file you can find here

Readme.debian

This file provides any extra difference between original package and your Debian version. I have not done any changes, so, I have removed it.

Lintian

Lintian is a tool to check your package. I will add small fix to avoid "initial-upload-closes-no-bugs" error. This error says that initial upload does not close bugs, but it looks very logic, does not it?
So, I have added debian/cri-o.lintian-overrides file:

$ cat debian/cri-o.lintian-overrides
cri-o: initial-upload-closes-no-bugs

Build src package

Ok, now we have ready control file, build instructions in rules file. Now we need to make release in changelog.

dch -r --distribution testing ignored
vim debian/changelog
$ cat debian/changelog
cri-o (1.26.0-1) testing; urgency=low

  * Initial release. 

 -- Alexey Lukyanchuk <skif@skif-web.ru>  Wed, 04 Jan 2023 11:10:46 +0300

We are ready to build source package. I prefer to use debuild tool:

debuild -us -uc -S

And after some magic we will see ready files:

$ ls -1 ../
cri-o
cri-o-1.26.0
cri-o-1.26.0.tar.gz
cri-o-v1.26.0
cri-o-v1.26.0.tar.gz
cri-o_1.26.0-1.debian.tar.xz
cri-o_1.26.0-1.dsc
cri-o_1.26.0-1_amd64.build
cri-o_1.26.0-1_amd64.buildinfo
cri-o_1.26.0-1_amd64.changes
cri-o_1.26.0-1_amd64.deb
cri-o_1.26.0-1_source.build
cri-o_1.26.0-1_source.buildinfo
cri-o_1.26.0-1_source.changes
cri-o_1.26.0.orig.tar.gz
cri-o_v1.26.0.orig.tar.gz

Build binary package and check

Ok, now we can build binary package. Simple way to do it — use debuild again:

debuild -us -uc

And now you can find cri-o_1.26.0-1_amd64.deb in parent dir (/volume);

$ ls -1 /volume/*.deb
/volume/cri-o_1.26.0-1_amd64.deb

Store changes in git

Now we come to important thing — how to compare source version and debian package version. I mean — how to be sure that you have cloned proper version of debian-dir from your repo?
I prefer to use tags. It's universal way that can find needed commit in branch, by commit, elsewhere.

Also don't forget to run dh_clean before commit, it will clean temporary files of build system and debhelper:

make -f debian/rules src-clean
dh_clean

Now I can make commit (git add, git commit,git push...) and set tag. I will set tag RC because I am not sure that it's release version. I am going to use such naming convince:

version before release: v${version}-rc${number}
release version: v$(version)-release

$ git tag -a v1.26.0-rc0 
$ git tag                 
v1.26.0-rc0

Tips and tricks

Make your own podman image, it's faster
Use this page. Here you can find large amount of information:
- does package exist in Debian
- what is source package for this binary package
- which package provides file
- etc
If you want to add new package to Debian, check this list if requests here
General "Developer corner" of Debian community

Chapter and verses

Cri-o homepage
debian policy
Debhelper manual
Quilt usage
Debian package test suite
Copyright format
Useful information about debian pakages
List of requests for packages to add in Debian
Debian devloper corner
Debian developer reference
Debian maintainers guide
Git first time setup
Git basic tagging

Make first deb-src package by example cri-o