Skip to content

Bringing AutoYaST to the 21th centry

Jump to bottom Edit New page
Imobach González Sosa edited this page Apr 27, 2020 · 16 revisions

Things to improve/fix

Tooling

Features

  • Templating mechanism (ERB, whatever). We need a mechanism to query the system (or run custom commands) and a set of directives to help the user (for instance, including a file from an URL). It could even replace classes/rules in the long term. Think about integrating questions too.
  • Check the current status of classes/rules mechanism. At least we need some testing.
  • Profile validation at runtime
  • Do not require a profile at all
  • Rework way AutoYaST sections relate to require packages. However, we may still need the .desktop files.

Fixes

Documentation

  • Deploy images? Are they deprecated? Otherwise, we should document them.

Miscellaneous

  • ‘Clone’ is not the appropriate term anymore. It leads to confusion.
  • Drop (a lot of) obsolete code (like this one).
  • Improve error handling (AutoinstIssue vs ad-hoc reporting)
  • When exporting registered add-ons, the URLs look wrong
  • [Configuration Management UI][]

Open questions

  • Identifying the root device
  • Minimal installation (no network, keyboard, etc. during 1st stage). See this card.

UI

Problems

  • The initial status is inconsistent: some modules are not configured, others are already cloned and others have some kind of default.
  • Clone is not always available. For instance, it is disabled for sysconfig without further explanation.
  • Apply to system is kind of broken because it is possible for some modules where it does not make sense. It is configurable in sysconfig, instead of being defined in a desktop file by each module.
  • The class editor looks completely broken: it is not possible to edit a class because it complains about some widget term and buttons do not react.
  • The validation ends up with a strange error (because yast2-audit-laf is missing):
/usr/bin/jing >&2 -c /usr/share/YaST2/schema/autoyast/rnc/profile.rnc /tmp/YaST2-01838-TjsqE9/valid.xml
fatal: file not found: /usr/share/YaST2/schema/autoyast/rnc/audit-laf.rnc (No such file or directory)
  • Many options are well hidden
  • No clue what reference profile should do
  • scripts edit allow Perl, Python and Shell (sh) scripts only (no Ruby or Bash)
  • pre-install script can be only shell, but it is mentioned only in help text, you can write it without issues
  • SSH keys can be cloned, but do nothing. Also not sure what apply to system does.
  • Cloning users does nothing
  • Edit software looks strange and, when I try to remove a package, it is not in the summary
  • CLI weird behavior. You can only specify one option and, additionally, there are too many unnecessary options.
# yast2 autoyast modname=bootloader
Unknown Command: modname=bootloader
Use 'yast2 autoyast2 help' for a complete list of available commands.
# yast2 autoyast2 help
No such client module autoyast2
# yast2 autoyast module modname=bootloader
## -> open yast UI and start given module
# yast2 autoyast file filename=/root/autoinst.xml
## -> open yast UI and load given xml
# yast2 autoyast file filename=/root/autoinst.xml module modname=bootloader
Unknown option for command 'file': modname

Technical Details

  • editing is done via auto client Export + Edit. If it is canceled or returns invalid data, it imports old data again (from initial Export).
  • It allows modules to modify menus in the wizard (if it is used, I never noticed it)
  • It applies to system -> calls Write for auto clients
  • Clones does auto client Reset and Read. Quite surprisingly, sets Modified.
  • Apply to system for a module is not the same as Apply whole profile (from the menu). It uses different code path. Apply whole profile emulates second stage of autoyast.

AutoYaST second stage

Why we would like to get rid of it ?

If we still use the second stage we need the complete YAST stack in order to run it. That’s often not wanted by the user/customer because this can blow up the system with an unneeded YAST installation. Are there other reasons for it?

Why it is a problem to get rid of it ?

  • 3rd party modules which have been defined in the AutoYaST configuration file are not in the inst-sys. So they have to be installed at first and have to be run in the installed system while the AY second installation stage.
  • ayast_setup https://documentation.suse.com/sles/15-SP1/html/SLES-all/InstalledSystem.html is using the complete second installation workflow in an installed system. It will be used by SAP products in order to set special hardware configuration for different SAP products.

Can it converge with firstboot?

Not sure because there are two different general concepts. Firstboot is designed for user action whereas AutoYaST will be lead by the AutoYaST configuration file. So if we would like to use firstboot instead of second stage installation defined in the AutoYaST configuration file, we would have to run firstboot in a kind of AutoYaST mode too. Doable but is there really a benefit in comparison of the second stage AY workflow ? Besides that we would still need the complete YAST environment in the installed system. (See “Why it is a problem to get rid of it ?”)

Can we move more stuff from 2nd stage to the 1st stage?

Yes, definitely. A normal SLES15-SP2 installation has produced a AutoYaST configuration file (with “yast clone_system”) with following entries which would be called again in the second stage while a new installation workflow:

lan, firewall, host, keyboard, language, nis, ntp-client, printer, proxy, tftp-server, timezone

As we are moving the network configuration completely to the first installation stage there should be no reason against configuring these modules in the first installation stage only. AutoYaST already evaluates if a second installation stage is needed at all. When we are managing to move these modules to first installation stage, the second installation stage will be not called. So no YAST environment would be needed in the installed system at all.

(Side note: We must not forget the post-script section)

Following modules have been available for the second stage installation at all while installing SLES15-SP2 with desktop environment:

“audit”, “audit-laf”, “auth-client”, “lan”, “ca_mgm”, “auth-server”, “proxy”, “customer_center”, “ldap”, “ldap-server”, “ldap-client”, “dhcp-server”, “dns-server”, “users”, “files”, “firewall”, “ftp-server”, “host”, “http-server”, “inetd”, “iscsi-client”, “isns”, “kdump”, “keyboard”, “ldapkrb”, “mail”, “nfs”, “nfs_server”, “nis”, “nis_server”, “ntp-client”, “openldap-mirrormode”, “printer”, “remote”, “samba-client”, “samba-server”, “security”, “services-manager”, “sound”, “squid”, “sysconfig”, “tftp-server”, “timezone”, “vpn”, “language”

The second installation stage will be activated only if we have YAST modules which still run in second stage only and modules which have been defined by 3rd parties.

BUT we must be careful with moving modules from second installation stage to first installation stage because we have to install these YAST modules in inst-sys. Will we run into resource problems ?

Complete different approach

If we do not manage to move all available modules to first installation stage (e.g. due performance/space problems in inst-sys) we could perhaps use another tool like Salt for this task when the system will be booted. We have already moved the complex stuff like storage, network,… to the first installation stage. Most of the still remaining modules like “nfs”, “ftp-server”, “printer” are more or less standard configuration stuff with which Salt should not have a problem. So what about extracting AY settings from still missed modules with https://github.com/openSUSE/yomi/blob/master/autoyast2yomi and calling YOMI after the system has booted. Just a thought. If it is an option we should talk with Alberto :-)

Network Status

Respective client suffers from having different behavior in the 1st stage only and/or second stage AY. In some cases, there is a slightly different behavior, code duplication across different clients (partly caused by switching to network-ng). Also, there is an option which is not clear whether it is still in use (start_immediately)

Hidden features

  • Post installation profile (see AutoInstall module) autoconf.xml
  • Support for finding and AutoYaST server via SLP
  • Support for relurl:// in the <file> sections

Metrics

directory Files lines of code lines per file cov %
modules 19 7028 370 48
clients 22 2576 117 0
include 18 5300 294 6
lib* 19 1444 76 88
78 16348 210 35.5
^ f l c

Modules

file lines flags
AutoInstall.rb 318
AutoInstallRules.rb 924
AutoinstClass.rb 160
AutoinstClone.rb 155
AutoinstCommon.rb 91 o
AutoinstConfig.rb 363
AutoinstDrive.rb 224 o
AutoinstFile.rb 254
AutoinstFunctions.rb 136
AutoinstGeneral.rb 420
AutoinstImage.rb 52
AutoinstPartition.rb 244 o
AutoinstPartPlan.rb 404 o
AutoinstScripts.rb 919
AutoinstSoftware.rb 933
AutoinstStorage.rb 171 o
ProfileLocation.rb 190
Profile.rb 728
Y2ModuleConfig.rb 342
7028
^ f

o: obsolete

AutoInstall
Generic functions.
AutoInstallRules
Rules/Classes handling.
AutoinstClass
Rules/Classes handling.
AutoinstClone
Cloning logic.
AutoinstCommon
Common partitioning functions!
AutoinstConfig
Misc functions about getting the profile (profile and classes paths, etc.).
AutoinstDrive
Partitioning functions.
AutoinstFile
Handle the <file> sections.
AutoinstFunctions
Misc functions (second_stage_required?, selected_product, etc.).
AutoinstGeneral
Handling of the <general> section.
AutoinstImage
Deploy images. It is documented but, anybody know how it works?
AutoinstPartition
Partitioning functions.
AutoinstPartPlan
Keeps the partition plan (old UI implementation).
AutoinstScripts
Handling of the scripts sections.
AutoinstSoftware
Handling of the <software> section.
AutoinstStorage
Import/Export the partitioning settings (storage-ng).
ProfileLocation
Code to fetch the profile. See the guide for the supported schemas.
Profile
Profile handling (parsing, saving, reporting about deprecating sections, etc.). Almost 1000 lines of code!
Y2ModuleConfig
Functions to deal with modules through *.desktop files.

Includes

File lines
io.rb 40
helps.rb 79
DriveDialog.rb 247 o
AdvancedPartitionDialog.rb 183 o
StorageDialog.rb 266 o
dialogs.rb 415
types.rb 13 o
script_dialogs.rb 482
PartitionDialog.rb 693 o
VolgroupDialog.rb 183 o
ask.rb 628
wizards.rb 65
xml.rb 149
conftree.rb 782
classes.rb 629
general_dialogs.rb 1097
common.rb 133
tree.rb 117
6201
^ f

Clients (autoyast2* packages)

client lines lib?
autoinst_scripts1_finish.rb 63
autoinst_scripts2_finish.rb 75
autoinst_test_clone.rb 39
autoinst_test_stage.rb 59
autoyast.rb 142
ayast_probe.rb 2 80
ayast_setup.rb 85 89
classes_auto.rb 78
clone_system.rb 138
files_auto.rb 352
general_auto.rb 71
inst_autoconfigure.rb 566
inst_autoimage.rb 73
inst_autoinit.rb 3 292
inst_autopost.rb 252
inst_autosetup.rb 509
inst_autosetup_upgrade.rb 3 273
inst_store_upgrade_software.rb 111
report_auto.rb 293
scripts_auto.rb 72
software_auto.rb 259
storage_auto.rb 89
3334 734
^ l l2
autoinst_scripts1_finish
chroot-scripts.
autoinst_scripts2_finish
chroot-scripts (again), download and prepare init and post scripts.
autoinst_test_clone
Client to clone a single client (yast autoinst_test_clone storage).
autoinst_test_stage
Run a custom autoinstall workflow using a stripped down control file.
autoyast
Start the AutoYaST UI.
ayast_probe
Dump storage data.
ayast_setup
Configures and already installed system (relies heavily in inst_autoconfigure).
classes_auto
UI to define classes.
clone_system
‘Clone’ the current system.
files_auto
Install the files described in the <files> section.
general_auto
UI for the general section. Import during installation is performed through in inst_autosetup* clients.
inst_autoinit
AutoYaST initialization (hardware probing, profile fetching and processing, iscsi-client, fcoe-client, registration!).
inst_autoconfigure
Configure a system. Basically used during 2nd stage and ayast_setup (Import and Write).
inst_autoimage
Installation using an image.
inst_autopost
Beginning of the 2nd stage. Gets the profile, imports the configuration and install the required packages (Import and Packages).
inst_autosetup
1st stage configuration: pre-scripts, add-ons, partitioning, networking, locales, etc
inst_autosetup_upgrade
1st stage configuration during upgrade: pre-scripts, add-ons, locales, etc.
inst_store_upgrade_software
Saves /root/autoupg_updated.xml including packages and patterns information.
report_auto
Import reporting settings.
software_auto
Set the software selection. In case of image-based installation, disable the kickoff and rpmcopy clients.
storage_auto
Partitioning.

Clients (other modules)

module client lines lib?
add-on/ add-on_auto.rb 205 x
audit-laf/ audit-laf_auto.rb 63
auth-client/ auth-client_auto.rb 4
bootloader/ bootloader_auto.rb 2
cluster/ cluster_auto.rb 63
configuration-management/ configuration_management_auto.rb 2
country/keyboard/ keyboard_auto.rb 44
country/language/ language_auto.rb 66
country/timezone/ timezone_auto.rb 57
dhcp-server/ dhcp-server_auto.rb 105
dns-server/ dns-server_auto.rb 62
drbd/ drbd_auto.rb 56
fcoe-client/ fcoe-client_auto.rb 193
firewall/ auto.rb 233 x
firstboot/ firstboot_auto.rb 84
ftp-server/ ftp_server_auto.rb 75 x
geo-cluster/ geo-cluster_auto.rb 63
http-server/ http-server_auto.rb 67
installation/ ssh_import_auto.rb 102 x
installation/ deploy_image_auto.rb 359 x
iplb/ iplb_auto.rb 63
iscsi-client/ iscsi-client_auto.rb 67
isns/ isns_auto.rb 51
kdump/ kdump_auto.rb 2
mail/ mail_auto.rb 70
network/ host_auto.rb 97
network/ lan_auto.rb 155
nfs-client/ nfs_auto.rb 73
nfs-server/ nfs_server_auto.rb 59
nis-client/ nis_auto.rb 60
nis-server/ nis_server_auto.rb 62
ntp-client/ auto.rb 57
online-update/ do_online_update_auto.rb 56
online-update-configuration/ online_update_configuration_auto.rb 231
printer/ printer_auto.rb 237
proxy/ auto_client.rb 107 x
registration/ scc_auto.rb 236 x
s390/ dasd_auto.rb 69
s390/ zfcp_auto.rb 69
samba-client/ samba-client_auto.rb 66
samba-server/ samba-server_auto.rb 62
security/ security_auto.rb 82
services-manager/ auto.rb 71 x
slp-server/ slp-server_auto.rb 63
sound/ sound_auto.rb 144
squid/ squid_auto.rb 65
support/ support_auto.rb 63
sysconfig/ sysconfig_auto.rb 57
tftp-server/ tftp-server_auto.rb 62
update/ inst_update_partition_auto.rb 67
users/ users_auto.rb 111
vpn/ auto_client.rb 81 x
Add a custom footer
Add a custom sidebar
Clone this wiki locally