Skip to content

Latest commit

 

History

History
539 lines (436 loc) · 22.4 KB

old_and_new_proposal.md

File metadata and controls

539 lines (436 loc) · 22.4 KB
Raw

Storage proposal: current (yast-storage-ng) and old (yast-storage) approach

This document describes the storage proposals of both yast-storage and yast-storage-ng (mainly focusing on the latter), with a brief explanation of how they work and how both can/must be influenced by the settings coming from control.xml and other sources.

Current proposal (yast-storage-ng) - overview

The settings used to influence the behavior of the storage proposal are read from the partitioning section of control.xml. Based on those settings, the algorithm described below will setup the partitions and LVM volumes in the system disks.

Storage proposal settings

This chapter presents a brief summary of the elements in the mentioned partitioning section. More detailed information can be found at:

Most settings are grouped into two subsections of that partitioning section:

  • proposal Holds general settings for the proposal.
  • volumes A list of volume elements holding specific settings for each volume that should be created, including the root (/) file-system and any other separate mount point, like swap or /home.

How the proposal distributes the space

The current proposal uses the following two-steps approach when deciding which partitions should be created/deleted/resized and how much space to assign to every new partition or logical volume.

First step

A first step decides which volumes will be needed. Each volume will originate a partition or a LV in the second step.

The basic list of planned volumes will be taken from the volumes subsection in control.xml, except for those the user explicitly disables. In addition to those initial volumes, the proposal can plan more for extra partitions needed to boot the system.

For each volume, three sizes are specified - minimum, desired, and maximum. The maximum size can have the special value unlimited. In addition, every volume gets a weight.

Second Step

The second step creates the necessary partitions and logical volumes to make those planned volumes fit in the disk. It makes a first attempt targeting the "desired" size. If that fails, it tries again but aiming just for the "minimum" size. During each attempt, preexisting partitions are resized or deleted according to four settings that can be influenced by the user. See below.

If more space than the target is freed (or is available from the beginning), that extra space is distributed among all the volumes. The ratio of space is decided using the corresponding weights. No volume will grow beyond its maximum size, even if that means leaving unused space on the disk.

Using LVM doesn't make a big difference. The VG (with its required PVs) is created to accommodate the size of all the created LVs, although the exact behavior in that regard can be influenced. See below.

Making space

Every one of the explained attempts to allocate the partitions or LVM logical volumes may need to previously free the space used in the disks. Obviously, that's achieved deleting or resizing existing partitions. The selection of partitions to resize or delete can be influenced with four settings that are configurable by the user in every proposal run.

  • windows_delete_mode What to do regarding removal of existing partitions hosting a Windows system.
    • none: Never delete a Windows partition.
    • ondemand: Only delete the Windows partitions that must be removed in order to make the proposal possible.
    • all: Delete all Windows partitions, even if not needed.
  • linux_delete_mode The same but for partitions that are part of a Linux installation (partition id linux, swap, lvm or raid).
  • other_delete_mode For all other partitions that don't fit into the former two groups.
  • resize_windows Whether to resize Windows systems if needed.

Creation of LVM structures

As said before, using LVM doesn't make a big difference on how the proposal works. It simply allocates LVM logical volumes instead of partitions. To allocate such LVs, the proposal first needs to create (or reuse) a volume group that is big enough, which usually means creating one or several physical volumes.

If lvm_vg_strategy is set to "use_needed", the VG is created to perfectly fit the sizes of all the created LVs, with no extra unused space. If that setting is set to "use_available", the VG created by the proposal will use all the space that has become available after deleting partitions according to the settings explained above. A third value of the setting was planned in order to use a fixed size for the VG (which should be, of course, equal or bigger than the sum of the max sizes of all the volumes), but that has not been implemented so far because the original use-case seems to not be relevant nowadays.

Current proposal (yast-storage-ng) - by example

The configuration of the storage proposal for a given product or system role is much more powerful and flexible than it used to be with the old (yast-storage) system, but it's also more complex. Since explaining complex stuff is usually best addressed via examples, this section shows how the behavior of the old proposal can be reproduced in the current one by setting the right configuration in control.xml for several example products. It also illustrates how easy it is to accommodate changes and new use cases that were impossible to achieve with the old system.

For details that are not self-explanatory in the examples (or that are omitted for simplicity), remember to check the partitioning section of the main control.xml documentation.

Example: SLES

This partitioning section would emulate quite closely the behavior of SLES-12, proposing always / and swap partitions (or logical volumes, if the user decides to use LVM). In addition, it will give the user the opportunity to have a separate /home partition/volume. That option will be enabled by default if there is enough space to create a home of at least 5 GiB.

Some values are a direct translation of the legacy control.xml and others (like the relationship with Windows partitions) are inferred from the typical SLES use case.

<partitioning>
  <proposal>
    <lvm config:type="boolean">false</lvm>
    <windows_delete_mode config:type="symbol">all</windows_delete_mode>
    <linux_delete_mode config:type="symbol">ondemand</linux_delete_mode>
    <other_delete_mode config:type="symbol">ondemand</other_delete_mode>
    <lvm_vg_strategy config:type="symbol">use_available</lvm_vg_strategy>
  </proposal>

  <volumes config:type="list">
    <!-- The root filesystem -->
    <volume>
      <mount_point>/</mount_point>
      <fs_type>btrfs</fs_type>
      <desired_size config:type="disksize">5 GiB</desired_size>
      <min_size config:type="disksize">3 GiB</min_size>
      <max_size config:type="disksize">10 GiB</max_size>
      <weight config:type="integer">35</weight>

      <snapshots config:type="boolean">true</snapshots>
      <snapshots_configurable config:type="boolean">true</snapshots_configurable>
      <snapshots_percentage config:type="integer">300</snapshots_percentage>
      <!-- Disable snapshots for / if disabling /home and giving up on
           enlarged swap is not enough -->
      <disable_order config:type="integer">3</disable_order>

      <btrfs_default_subvolume>@</btrfs_default_subvolume>
      <subvolumes config:type="list">
        <!--
          This would be the same than the <subvolumes> list in the current
          (old) control.xml. Reproducing the whole list doesn't make much sense.
        -->
      </subvolumes>
    </volume>

    <!-- The home filesystem -->
    <volume>
      <mount_point>/home</mount_point>
      <fs_type>xfs</fs_type>

      <proposed config:type="boolean">true</proposed>
      <proposed_configurable config:type="boolean">true</proposed_configurable>
      <!-- Disable it in first place if we don't fit in the disk -->
      <disable_order config:type="integer">1</disable_order>

      <desired_size config:type="disksize">5 GiB</desired_size>
      <min_size config:type="disksize">5 GiB</min_size>
      <!-- Omitting max_size is also possible, but since we have an explicit
           max_size_lvm and fallback_for_max_size it's more clear to
           include it. -->
      <max_size config:type="disksize">unlimited</max_size>
      <max_size_lvm config:type="disksize">25 GiB</max_size_lvm>
      <weight config:type="integer">55</weight>
      <!-- If this volume is disabled and LVM is not being used, we want
           "/" to become greedy (unlimited max) -->
      <fallback_for_max_size>/</fallback_for_max_size>
    </volume>

    <!-- The swap volume -->
    <volume>
      <mount_point>swap</mount_point>
      <fs_type>swap</fs_type>

      <desired_size config:type="disksize">2 GiB</desired_size>
      <min_size config:type="disksize">1 GiB</min_size>
      <max_size config:type="disksize">2 GiB</max_size>
      <weight config:type="integer">10</weight>
      <adjust_by_ram config:type="boolean">true</adjust_by_ram>
      <adjust_by_ram_configurable config:type="boolean">true</adjust_by_ram_configurable>
      <!-- Give up on enlarging to RAM if we still don't fit in the disk
           after disabling separate home -->
      <disable_order config:type="integer">2</disable_order>
    </volume>
  </volumes>
</partitioning>

Example: openSUSE

In the case of emulating the openSUSE Leap 42 proposal, the volumes subsection wouldn't be much different from SLES-12, with the exception of some sizes. But the proposal one would probably look like this (more MS Windows friendly).

<proposal>
  <lvm config:type="boolean">false</lvm>
  <resize_windows config:type="boolean">true</resize_windows>
  <windows_delete_mode config:type="symbol">ondemand</windows_delete_mode>
  <linux_delete_mode config:type="symbol">ondemand</linux_delete_mode>
  <other_delete_mode config:type="symbol">ondemand</other_delete_mode>
  <lvm_vg_strategy config:type="symbol">use_available</lvm_vg_strategy>
</proposal>

Example: SLES4SAP

Just a tentative to show some possibilities, although maybe some aspects don't fit the current SAP requirements exactly.

Although the users would still be able to change the proposal settings, they wouldn't be able to change the filesystem type for / (Btrfs) or to disable the usage of snapshots on it. There is also no way (other than using the expert partitioner) to request a separate /home volume. The proposed swap would be much bigger than the typical SLES one.

Although is not part of this example, it would be easy to configure the proposal to suggest any other arbitrary set of separate data volumes instead of home (similar to /var/lib/docker in the CaaSP example below) that the user could enable or disable in the "Guided Setup" (the former "Proposal Settings").

<partitioning>
  <proposal>
    <lvm config:type="boolean">true</lvm>
    <windows_delete_mode config:type="symbol">all</windows_delete_mode>
    <linux_delete_mode config:type="symbol">ondemand</linux_delete_mode>
    <other_delete_mode config:type="symbol">ondemand</other_delete_mode>
    <lvm_vg_strategy config:type="symbol">use_available</lvm_vg_strategy>
  </proposal>

  <volumes config:type="list">
    <!-- The '/' filesystem -->
    <volume>
      <mount_point>/</mount_point>
      <!-- Enforce Btrfs for root by not offering any other option -->
      <fs_type>btrfs</fs_type>
      <fs_types>btrfs</fs_types>
      <desired_size config:type="disksize">40 GiB</desired_size>
      <min_size config:type="disksize">30 GiB</min_size>
      <max_size config:type="disksize">60 GiB</max_size>
      <weight config:type="integer">50</weight>
      <!-- Always use snapshots, no matter what -->
      <snapshots config:type="boolean">true</snapshots>
      <snapshots_configurable config:type="boolean">false</snapshots_configurable>

      <!-- You don't want to miss the / volume -->
      <proposed_configurable config:type="boolean">false</proposed_configurable>

      <btrfs_default_subvolume>@</btrfs_default_subvolume>
      <subvolumes config:type="list">
        <!--
          This would be the same than the <subvolumes> list in the current
          (old) control.xml. Reproducing the whole list doesn't make much sense.
        -->
      </subvolumes>
    </volume>

    <!-- The swap volume -->
    <volume>
      <mount_point>swap</mount_point>
      <fs_type>swap</fs_type>

      <desired_size config:type="disksize">6 GiB</desired_size>
      <min_size config:type="disksize">4 GiB</min_size>
      <max_size config:type="disksize">10 GiB</max_size>
      <weight config:type="integer">50</weight>
    </volume>

    <!--
      No home filesystem, so the option of a separate home is not even
      offered to the user.
      On the other hand, a separate data volume (optional or mandatory) could
      be defined.
    -->

  </volumes>
</partitioning>

Example: CaaSP

Once again, this example is just an approximation to show the possibilities. CaaSP doesn't allow the user to run the Guided Setup, so the result of the proposal cannot be influenced. This partitioning section would enforce the proposal to:

  • use the whole disk (deleting previous partitions),
  • use partitions (no LVM),
  • always use Btrfs with snapshots for / (failing to make a proposal if that's not possible instead of giving up on snapshots like the current proposal does),
  • never propose separate /home or swap partitions,
  • propose a separate /var/lib/docker partition if there is enough space for it (basically if the disk is bigger than 20 GiB, according to the sizes in the example).
<partitioning>
  <!-- Advise the user against using the expert partitioner -->
  <expert_partitioner_warning config:type="boolean">true</expert_partitioner_warning>

  <!-- All default settings will become final, since the user can't change them -->
  <proposal>
    <lvm config:type="boolean">false</lvm>
    <!-- Delete all previous partitions -->
    <windows_delete_mode config:type="symbol">all</windows_delete_mode>
    <linux_delete_mode config:type="symbol">all</linux_delete_mode>
    <other_delete_mode config:type="symbol">all</other_delete_mode>
  </proposal>

  <volumes config:type="list">
    <!-- The '/' filesystem -->
    <volume>
      <mount_point>/</mount_point>
      <fs_type>btrfs</fs_type>
      <desired_size config:type="disksize">15 GiB</desired_size>
      <min_size config:type="disksize">10 GiB</min_size>
      <max_size config:type="disksize">30 GiB</max_size>
      <weight config:type="integer">80</weight>
      <!-- Always use snapshots, no matter what -->
      <snapshots config:type="boolean">true</snapshots>
      <snapshots_configurable config:type="boolean">false</snapshots_configurable>

      <!-- You don't want to miss the / volume -->
      <proposed_configurable config:type="boolean">false</proposed_configurable>

      <btrfs_default_subvolume>@</btrfs_default_subvolume>

      <!-- Make '/' volume read-only -->
      <btrfs_read_only config:type="boolean">true</btrfs_read_only>

      <subvolumes config:type="list">
        <!--
          This would be the same than the <subvolumes> list in the current
          (old) control.xml. Reproducing the whole list doesn't make much sense.
        -->
      </subvolumes>
    </volume>

    <!-- Use /var/lib/docker as separate partition if 10+ GiB available -->
    <volume>
      <mount_point>/var/lib/docker</mount_point>
      <fs_type>btrfs</fs_type>
      <snapshots config:type="boolean">false</snapshots>
      <snapshots_configurable config:type="boolean">false</snapshots_configurable>

      <desired_size config:type="disksize">15 GiB</desired_size>
      <min_size config:type="disksize">10 GiB</min_size>
      <max_size config:type="disksize">unlimited</max_size>
      <weight config:type="integer">20</weight>

      <!-- Give up separate partition if it doesn't fit -->
      <disable_order config:type="integer">1</disable_order>

      <!-- If this volume is disabled, we want "/" to increase -->
      <!-- (don't increase min size as it would be pointless) -->
      <fallback_for_desired_size>/</fallback_for_desired_size>
      <fallback_for_max_size>/</fallback_for_max_size>
      <fallback_for_weight>/</fallback_for_weight>
    </volume>

    <!-- No swap partition is defined, so it's never created -->
  </volumes>
</partitioning>

The old proposal (yast-storage)

Most settings used to influence the behavior of the old yast-storage were read from the partitioning section of control.xml, with some settings having a fallback in /etc/sysconfig.

This chapter presents a brief summary of the structure of that old partitioning section.

Storage proposal settings

The explanation on how every option relates to the others (which is far from being straightforward) can be found below, in the section titled "How the old proposal distributed the space".

Proposal-specific options

  • try_separate_home
  • limit_try_home
  • home_path
  • home_fs
  • root_fs
  • root_space_percent
  • root_base_size
  • root_max_size
  • proposal_lvm
  • vm_desired_size
  • vm_home_max_size
  • vm_keep_unpartitioned_region
  • btrfs_increase_percentage

Proposal and partitioner options

In addition to the list above, there were two options, related to the usage of Btrfs for the root filesystem, that were used by the proposal and also by the expert partitioner when suggesting the default configuration for such filesystem.

  • subvolumes Optional list of Btrfs subvolumes. If it is missing, a hard-coded list is used.
  • btrfs_default_subvolume The default subvolume is not represented by an entry in the subvolumes list, but with a separate option that specifies only its path.

Other general options

The partitioning section of control.xml for yast-storage went beyond the configuration of the proposal's behavior. It also contained some options to influence other aspect of YaST, mainly the installer.

  • expert_partitioner_warning Whether an extra warning pop-up should be displayed if the user enters the expert partitioner during installation.
  • use_separate_multipath_module Whether to call the multipath client from the yast2-multipath package. If false, the multipath-simple client from yast2-storage is used.

Obsolete options

The following options were used by the so-called "flexible partitioning" feature, that was dropped in SLE-12 already:

  • prefer_remove
  • remove_special_partitions
  • partitions

How the old proposal distributed the space

This part of the document explains how the storage proposal works in yast-storage. The content of this section is inferred from the existing documentation, some inspection of StorageProposal.rb and quite some manual tests. It may not be 100% accurate, specially since many changes have been introduced over time.

The explanations are intentionally simplified to keep the focus on the main topic: the influence of the settings in the behavior of the proposal in the most common and simple scenarios.

For clarity reasons, the logic used to assign space to swap or the special partitions needed for booting is not covered in this section. Those values are decided and fixed at the beginning of the process, so they have relatively small influence on how the space is reclaimed or distributed.

There are two parameters to decide if a separate home partition or logical volume should be proposed: try_separate_home (boolean) and limit_try_home (size). A separate home will be proposed if try_separate_home is true and the available space is greater than or equal to limit_try_home.

That being said, the space is distributed in a different way with and without LVM.

Partition-based distribution

The old proposal tries to use all the available free space. There are settings to influence the minimum size of the partitions but not to limit the maximum free space reclaimed for the system.

Interestingly enough, if there is not enough free space, the old proposal tries to reformat the found Linux partitions instead of deleting them and creating new partitions with the desired sizes.

As mentioned, the proposal tries to use all the existing free space. If a separate home is proposed, the space is distributed among home and root, based on the ratio expressed by root_space_percent. There is a limitation to that distribution - if root reaches root_max_size, it will not grow further and the rest of the space is assigned to home.

If there is no separate home, root_max_size has no effect and root grows until filling all the available space.

LVM-based distribution

Again for clarity reasons, reusing preexisting VGs is left out of this section, that will focus on the situation in which the proposal needs to create a VG.

The base and maximum sizes for root or home are not taken in consideration when deciding the size of the created VG. There are independent settings for that.

If vm_keep_unpartitioned_region is false, the created VG will be as big as possible. That means the proposal will create PVs completely filling the available spaces. In addition all preexisting Linux partitions will be turned into PVs and added to the VG.

If vm_keep_unpartitioned_region is true, a PV of vm_desired_size will be created (this is the only case in which the proposal can be forced to leave some unused free space). Again, all preexisting Linux partitions will be turned into PVs and also added to the VG.

After creating the VG, its space is distributed among home and root following slightly different rules than in the partition-based approach. root_space_percent is also used to decide the ratio but root is never proposed to be bigger than root_max_size, no matter if there is a separate home or not. If a separate home is desired, it is never proposed to be bigger than vm_home_max_size.

As a result, it's very likely that the resulting VG will contain a lot of unassigned space, especially if vm_keep_unpartitioned_region is false (which is the default).

References

Some fate entries that influenced the old proposal behavior