Notes on using expert_recipe in Debian/Ubuntu Preseed Files

When working with IaaS easily provisioning bare metal is always needed. So, Eucalyptus uses preseed files to setup Debian and Ubuntu servers for testing software, supporting customers, and education new users. At times there are complex needs for how the servers are setup and it is not always an easy task.

When first starting out with preseed with the need for a complex partition setup, partman-auto/expert_recipe can look daunting. There can be many questions with regard to the somewhat cryptic setup of the recipes. The Debian documentation isn’t very helpful upon first look either but after understanding how to setup a recipe, it becomes quite easy. When RAID and LVM are added, d-i partman-auto/expert_recipe can create more complex disk setups and is a very powerful feature of any preseed setup.

A basic partitioning scheme on /dev/sda using the preseed partman-auto/expert_recipe directive is below:

d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/expert_recipe string root :: 19000 50 20000 ext3 \
        $primary{ } $bootable{ } method{ format } \
        format{ } use_filesystem{ } filesystem{ ext3 } \
        mountpoint{ / } \
    . \
    2048 90 2048 linux-swap \
        $primary{ } method{ swap } format{ } \
    . \
    100 100 10000000000 ext3 \
        $primary{ } method{ format } format{ } \
        use_filesystem{ } filesystem{ ext3 } \
        mountpoint{ /srv/extra } \
    .
d-i partman-auto/choose_recipe select root
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true

The main piece that we’ll focus on is the partman-auto/expert_recipe line. (Note: It might look like this is multiple lines but it actually a single line with the newlines escaped.) In the above example three primary partitions for /, swap, and /srv/extra are created. The partman-auto/expert_recipe directive is broken down below.

d-i partman-auto/expert_recipe string root ::

the first part of this line tells the Debian installer that “expert_recipe” will be used with partman to partition the hard drive provided on the d-i partman-auto/disk line. Next the Debian installer is told that a string should be expected with the value for the directive. Finally, a recipe title of “root” is given to the recipe. The recipe title is used with the d-i partman-auto/choose_recipe select root directive to tell partman which recipe to use. The :: signals to the Debian installer that we are starting the recipe for the partitions.

Next well look at a single partition and how it is created.

19000 50 20000 ext3 \
        $primary{ } $bootable{ } method{ format } \
        format{ } use_filesystem{ } filesystem{ ext3 } \
        mountpoint{ / } \
    . \

The first piece of the above partition recipe consists of three numbers. The numbers refer to the minimum size of the partition in megabytes(19000), the priority that this partition gets it’s maximum size fulfilled (with lower numbers having a higher priority)(30), and the maximum size of partition being created. The two size values are in megabytes (20000), respectively. The next word refers to the format of the partition that will be created.

The next three lines tell partman that this partition should be primary, the partition should be flagged as bootable, this partition should be formated, the format of the filesystem should be ext3, and finally that the mountpoint for this partition will be “/”. The final line with a single “.” tells partman that this is the end of the definition for this partition. If more text follows then partman knows that more partitions are being defined but if a newline is read then it will know that the partition recipe is completed.

Unfortunately the expert_recipe part of partman can currently only handles a single disk for partition based recipes. There are some hacks with sfdisk that can be used with preseed/late_command that can add some basic functionality for other disks. If expert_recipe is used with a LVM setup then use multiple disks can be used as will be shown below.

Below is a more complicated setup utilizing a RAID 1 array on /dev/sda and /dev/sdb and a LVM on top of the created RAID array.

d-i     partman-auto/disk string /dev/sda /dev/sdb
d-i     partman-auto/method string raid
d-i     partman-lvm/device_remove_lvm boolean true
d-i     partman-md/device_remove_md boolean true
d-i     partman-lvm/confirm boolean true
d-i     partman-auto/choose_recipe select boot-root
d-i     partman-auto-lvm/new_vg_name string vg00
d-i     partman-auto/expert_recipe string        \
           boot-root ::                          \
             1024 30 1024 raid                   \
                $lvmignore{ }                    \
                $primary{ } method{ raid }       \
             .                                   \
             1000 35 100000000 raid              \
                $lvmignore{ }                    \
                $primary{ } method{ raid }       \
             .                                   \
             19000 50 20000 ext4                 \
                $defaultignore{ }                \
                $lvmok{ }                        \
                lv_name{ root }                  \
                method{ format }                 \
                format{ }                        \
                use_filesystem{ }                \
                filesystem{ ext4 }               \
                mountpoint{ / }                  \
             .                                   \
             2048 60 2048 swap                   \
                $defaultignore{ }                \
                $lvmok{ }                        \
                lv_name{ swap }                  \
                method{ swap }                   \
                format{ }                        \
            .                                    
d-i partman-auto-raid/recipe string \
    1 2 0 ext2 /boot                \
          /dev/sda1#/dev/sdb1       \
    .                               \
    1 2 0 lvm -                     \
          /dev/sda2#/dev/sdb2       \
    .                               
d-i     mdadm/boot_degraded boolean false
d-i     partman-md/confirm boolean true
d-i     partman-partitioning/confirm_write_new_label boolean true
d-i     partman/choose_partition select Finish partitioning and write changes to disk
d-i     partman/confirm boolean true
d-i     partman-md/confirm_nooverwrite  boolean true
d-i     partman/confirm_nooverwrite boolean true

The start of setting up the RAID array is signaled following lines:

d-i     partman-auto/method string raid
d-i     partman-md/confirm boolean true

The first part that partman will utilize is the partman-auto-raid/recipe directive. This string defines how the RAID array will be setup on /dev/sda and /dev/sdb. For example, we setup a RAID 1 array for an LVM using /dev/sda and /dev/sdb using the following:

1 2 0 lvm -                     \
          /dev/sda2#/dev/sdb2       \
    .

The first number represents the RAID level (1), the second number refers to the number of devices we are using in the RAID array (2), and the third number refers to the number of spares the RAID array will have available (0). Next the partition type of the RAID array is defined (lvm) and the “-” refers to the mount point of the array. Since a LVM on this RAID array is created there is no mount point but see the full example above for an example of this. The partitions on the disks that will be used for the array are referenced with each partition separated with a “#”. Similar to the partition example above, the array definition is ended with a “.” and any text that follows will be considered another array with a newline telling partman that this recipe is completed.

partman-auto/expert_recipe is used to define the partitions being created for the RAID arrays. Above two RAID arrays are being created for /boot and a LVM partition. The recipe then defines two logical volumes to be created on the LVM for “/” and swap.

Next, the definition for a RAID partition is given:

             1024 30 1024 raid                   \
                $lvmignore{ }                    \
                $primary{ } method{ raid }       \
             .                                   \

Above a RAID array of 1GB with the highest priority and a partition type of “raid” is setup. Since this partition is of the highest priority, it will be setup as /dev/sda1 and /dev/sda2 and will be utilized as “/boot” by the OS given the partman-auto-raid/recipe directive explained above. The $lvmignore{ } flag is used to make sure that when partman is creating LVM logical volumes that this partition is not created as a logical volume. Next the RAID partitions are defined to be primary and that the method for using this partition will be with a RAID array.

To start off the definition for the LVM partition the following is used to tell the Debian installer to setup a LVM:

d-i     partman-lvm/confirm boolean true
d-i     partman-auto-lvm/new_vg_name string vg00
d-i     partman-auto-lvm/guided_size string 30GB

Make sure that the “guide_size” value above is greater than or equal to the size of all logical volumes created. To define a logical volume to be created the following logical volume is defined in the recipe:

             19000 50 20000 ext4                 \
                $defaultignore{ }                \
                $lvmok{ }                        \
                lv_name{ root }                  \
                method{ format }                 \
                format{ }                        \
                use_filesystem{ }                \
                filesystem{ ext4 }               \
                mountpoint{ / }                  \
             .                                   \

Above a logical volume between 19GB and 20GB with an ext4 filesystem will be created. The $defaultignore{ } is used to keep partman from using this partition when it is creating physical partitions on the disks. Next, partman is directed that this part of the recipe should be used when creating logical volumes with $lvmok{ } and the logical volume is given the name “root” with $lv_name{ root }. The rest of the flags are the same as the earlier examples in that it tells partman that the logical volume should be formatted and what the mount point should be.

The above complete examples can be placed into preseed files and tweaked to give the desired results. Hopefully this helps with using partman-auto/expert_recipe in either a standalone mode or when utilizing RAID and LVM.

Update: I’ve added full preseed file examples on GitHub. Check them out in my Blog Scripts repo.

Another Update: I’ve added another post about using preseed on Ubuntu Lucid and Debian Squeeze where the disk uses GPT (2TB and over disk sizes). Check it out: Get expert_recipe, mdraid, LVM, GPT, and grub2 Playing Together on Ubuntu Lucid and Debian Squeeze