buildroot — my own experience with multi-platform distro creation

    Introduction


    In my previous article (Monitor linux) I wrote, what is this distro and how it works. Now i will write how to do it. It's may be interesting for everyone, who want to study buildroot.


    Target goals


    The result we get from article is the following:


    • Firmware (non-volatile image with restorable config)
    • Easy management via web-interface
    • Cross-platform (qemu x86_64, arm-based SBC like rasberry 4, beagle bone black and asus tinker board)
    • Support without extra effort

    Buildroot


    There are many different ways to build custom linux-based firmware:


    • Use other distro as base ( gentoo is cool for it )
    • Linux from scratch
    • Use build system like buildroot or yocto

    In my work way I have used all this ways in different projects. But buildroot is the best way for cross-platform firmware development. It's a simple and powerful system, based on Makefile's. You can customize everything, but is fully functional out of the box.
    I mean, that you don't need a lot of time to do something. There are many ready configs for a lot of SBC. Of course, you can customize them: from hostname change to giant high-customizable project.


    External-tree organization


    Now, why i use external tree? Of course, it's possible to store all you changes (board and kernel configs, packages, files, etc) in buildroot main directory. But it's not comfortable. External tree is more progressive way.
    Strongly recommend to read previous article to undestand external-tree work.


    My philosophy: only board-specific files (like board/kernel config file, bootloader files and genimage config) are stored in board separate directory.


    But some data must be ALWAYS identical for all boards:


    • data partitions
    • systemd my custom start/shutdown scripts
    • Post-build scripts
    • rootfs overlay
    • user list

    Ok, let's look onto external-tree directory:


    [alexey@comp overlay]$ ls -ll
    total 56
    -rw-rw-r-- 1 alexey alexey  203 Dec 11 22:59 Config.in
    drwxrwxr-x 6 alexey alexey 4096 Jan  5 19:07 board
    drwxrwxr-x 2 alexey alexey 4096 Jan  6 15:04 buildroot_patches
    drwxrwxr-x 2 alexey alexey 4096 Jan  6 20:09 configs
    -rw-rw-r-- 1 alexey alexey   52 Dec 11 22:59 external.desc
    -rw-rw-r-- 1 alexey alexey   79 Dec 11 22:59 external.mk
    -rw-rw-r-- 1 alexey alexey   91 Jan  5 13:19 genDataImage.cfg
    drwxrwxr-x 5 alexey alexey 4096 Dec 11 22:59 package
    drwxrwxr-x 2 alexey alexey 4096 Jan  5 23:23 post-build
    -rwxrwxr-x 1 alexey alexey  294 Jan  5 20:03 post-build.sh
    drwxrwxr-x 5 alexey alexey 4096 Dec 11 22:59 rootfs_overlay
    drwxrwxr-x 2 alexey alexey 4096 Jan  5 13:19 systemd
    -rw-rw-r-- 1 alexey alexey   63 Dec 11 22:59 users.txt
    drwxrwxr-x 2 alexey alexey 4096 Jan  5 18:27 utils

    external.desc, external.mk, Config.in — this files define external-tree and it's packages.


    Board,configs — it's directores with board-specific files.
    All other dirs and files IDENTICAL for all target boards.
    buildroot_patches — patches applied to buildroot
    package — directory with my packages(zabbix and rodos5_6 for thermometer)


    [alexey@comp overlay]$ tree  package/
    package/
    ├── manuals
    │   ├── Config.in
    │   └── manuals.mk
    ├── rodos
    │   ├── 0001-MAKEFILE-FIX.patch
    │   ├── Config.in
    │   └── rodos.mk
    └── zabbix
        ├── Config.in
        ├── zabbix-agent.service
        ├── zabbix.hash
        ├── zabbix.mk
        └── zabbix-server.service

    post-build.sh — run scripts from post-build directory on post-build stage. Of course, i can point all this scripts directory in board defconfig, but it's not jedi way. More comfortable is to point one script in board config. This script run scripts from post-build dir in alphanumeric order.


    rootfs_overlay — Buildroot can overlay your directory below target filesystem. It's good way to put custom files in firmware.


    [alexey@comp overlay]$ ls -1 rootfs_overlay/
    etc
    usr
    var

    systemd — It's storage with my custom systemd unit files and them scripts.


    [alexey@comp overlay]$ ls -1 systemd/
    logrotateTimer.service
    logrotateTimer.timer
    prepare.service
    prepare.sh
    rodos.sh
    settings_restore.service
    settings_restore.sh

    Yes, i can store them in rootfs_overlay, but I am a couch potato. I wrote post-build scripts, that do this work for me (post-build/0010-system-customization.sh):


    #########################################################
    #
    # install systemd files
    #
    #########################################################
    
    cp $BR2_EXTERNAL_monitorOverlay_PATH/systemd/*sh ${TARGET_DIR}/usr/bin/
    chmod +x ${TARGET_DIR}/usr/bin/*sh
    
    # services
    for service in $BR2_EXTERNAL_monitorOverlay_PATH//systemd/*service
    do
        serviceName=`echo $service|awk -F\/ '{print $NF}'`
        cp $service ${TARGET_DIR}/etc/systemd/system/
        cd ${TARGET_DIR}/etc/systemd/system/
        ln -sfr ${TARGET_DIR}/etc/systemd/system/$serviceName  ${TARGET_DIR}/etc/systemd/system/multi-user.target.wants/
    done
    
    # timers
    for timer in $BR2_EXTERNAL_monitorOverlay_PATH/systemd/*timer
    do
        timerName=`echo $timer|awk -F\/ '{print $NF}'`
        cp $timer ${TARGET_DIR}/etc/systemd/system/
        cd ${TARGET_DIR}/etc/systemd/system/
        ln -sfr ${TARGET_DIR}/etc/systemd/system/$timerName  ${TARGET_DIR}/usr/lib/systemd/system/timers.target.wants/
    done

    post-image
    By analogy of post-build scripts, this scripts run AFTER building system image is complete. I use them to create usb-flash image and update arch.


    utils — directory with self-written scripts, doesn't used in firmware build. Now only one script — autobuild.


    genDataImage.cfg — config file for genimage util. Generate data-partition, identical for all boards.


    rootfs overlay


    Ok, let's look onto rootfs overlay. I am using it to save static scripts like standalone scripts, logrotate configs(pgp-fpm and lighttpd need it), web-interface files.


    Now, a love to store bash-scritps functions in one big file that i use as library in my scripts. It's usr/lib/monitor_lib.sh file.


    Next, comfortable to use global variables in my scripts. I want to use $SETTINGS variable in all my scripts for settings-file path. Best way — use profile. Look to rootfs_overlay/etc/profile.d/monitorVariables.sh:


    DATA='/data'[alexey@comp overlay]$ cat rootfs_overlay/etc/profile.d/monitorVariables.sh 
    #!/bin/bash
    SETTINGS='/data/settings.txt'
    NETWORK_CONF='/etc/systemd/network/wired.network'
    MY_LIB='/usr/lib/monitor_lib.sh'

    Ok, next task — default config of system. I live to do firmware: unchangeable kernel+initramfs and changeable text file with settings. I save this file inside system image — it's a guarantee that i will always have correct default config.


    Next question — how to apply changes to all platform. If you use my overlay struct, this task is simple. I am using MELD for this. But you can use any file comparsion tool:



    post-build


    It's one of the powerful buildroot opportunities for me.


    On this stage, you can do EVERETHYNG with targer root filesystem, change any files!
    You can see that i use it for everything:


    • prepare fstab and other config files
    • run datavolume generation command
    • create update arch

    system prepare services


    I have to use 2-staged system restore system
    Why i do it? Because i don't radical rewrite unit dependencies.
    In first script i prepare system (mount,hostname,network,passwors,etc).
    In second i prepare zabbix (files,databases). Before i can do it, i should start postgresql.


    Postgresql need network, and etc dependencies.
    IMPORT SQL DUMP -> need running postgresql
    postgresql -> need ready network
    network -> need config file
    network config -> must be restored from system settings file.


    How it work:
    system start -> some targets -> multi-user.target
    multi-user.target -> settings_restore.service -> network.service,postgresql.service, systemd-networkd.service systemd-resolved.service -> prepare.service -> system start


    You can see, that possible to run scripts in strong order in systemd. But it's dont mean that i love systemd :-D


    Finale


    So, you can see how buildroot can create cross platform firmware without hard work. Important moments:


    • Overlay organization(minimum board-specific files)
    • post-build scripts for serious system customization
    • use custom packages (not necessary ported to main buildroot git)
    • True using of systemd opportunities
    AdBlock has stolen the banner, but banners are not teeth — they will be back

    More
    Ads

    Comments 2

      0
      Your buildroot articles really helped me in my own firmware creation. Thanks.
      Anyway I would like to see Your article about «Easy management via web-interface» (You declared).
        +1
        I glad to hear it!
        Web-interface really simple — some PHP for frontend and bash scripts in backend. Auth via lighttpd module.
        A don't like to develop any gui, including web). In addition, i don't like big interfaces. Bad way: write 100+ Mb web-interface for 60mb OS.

        But, if you need, i will write short for you.

      Only users with full accounts can post comments. Log in, please.