Cockpit Podman redesign!

[garrett]

Hi all! I'm the designer on the Cockpit team and I've been working on a redesign of the Podman UI within Cockpit.

Intro

I've worked with Matej (@marusak), the main developer of the Cockpit-Podman page to come up with a few new concepts and would like to share a preview of what's probably going to be implemented in the not-too-distant future.

Disclaimers

These mockups serve as guidance on the direction of Cockpit-Podman. When they're implemented, it will happen in parts, so don't expect all the new features to land at once. And things may change while we're implementing them, of course. 😉

Goals

• Modernize the Podman UI.
  • Originally, the Podman UI in Cockpit started off as a fork of the old Docker UI. Things have changed a lot since then, for Podman (as we have user session based containers as well as system ones), in PatternFly (the old version was implemented with PatternFly 3 and custom widgets... but PatternFly 4 has new patterns), and Cockpit (a lot has changed over the years).
• Streamline the workflow.
  • Right now, you have to know to download an image, then click the play button to run the image as a container. It would be great if we could integrate the steps.
• Make the page's order flow with the conceptual model.
  • Images are at the bottom right now, but containers come from images. It's weird to start and the end and work backwards.

Main page

Here's the mockup of a collapsable images section at the top, a new containers list, and contextual menus to the side:

Highlights: Images list

I wanted to downplay the importance of images, and make them mainly an implementation detail as much as possible. You should be able to simply create a container with an image that exists on the network (either on a local server or on an Internet container registry like Fedora's, Quay.io, Docker Hub, etc.) or one that's already local (as it was built locally or already downloaded).

The image list would get quite long and push the container list down the page if it was simply at the top. This is a problem, as containers are actually what people are interested, not images for the containers.

The solution would be to either have a stub and put the full list on another page (similar to what Cockpit already does with the Firewall UI) or make the list itself expandable on the same page. If the images were on a sub page, however, then that would break the existing functionality of being able to start with an image and create a container from it.

Taking all that into consideration, I realized that we do need the list on the main page, but it needed to be in a collapsed state.

I've "bubbled up" as much important information as possible in the form of a summary. Most of the time, you care about images only when there are newer versions or if they're taking up too much space. In the unexpanded view, we intend to have a way to prune unused images (to free up space) and pull down new versions of images (to take up more space 😉, and more importantly, get new versions of the container images).

Here's a mockup what the Images list looks like when expanded:

The play buttons have been changed to much more explicit "create container" buttons and several new options have been added.

Highlights: Containers list

The main goal for the containers list is to make it more modern and similar to what we're doing elsewhere in Cockpit.

You might note that there's a big inconsistency with the new Machines page though, but with good reasons:

1. Container details are not editable. (In the Machines page, all sorts of aspects of virtual machines are editable. It's not the case for Containers.)
2. Pod groups are comprised of multiple containers. Splitting that apart into several sub-pages doesn't work so well.

I did discuss this with Matej and we did go back and forth a bit on this, but settled on the expandable lists as you see in the mockups due to the above reasons.

When containers are expanded, there are details on the main page. I have also mocked up how they would be grouped and how they would wrap on smaller size screens. The mockups above show it in full width, and this starts with a first wrap on a narrower screen and goes all the way down to mobile:

Create new container modal dialog

Instead of having a 2-step process to create new containers, I wanted to merge the download image step into the creation step. As the image download dialog was basically a source selection and a list, I moved them into a hybrid modal dialog using an autocomplete.

Here are wireframes showing how that would work, with a more featureful main view and integation with ports and such moved to another tab:

The wireframes above gloss over how the source and autocomplete dropdowns would work, but I have that covered in this wireframe:

As having tabs in modal dialogs is something we're not currently doing in Cockpit, I wanted to make sure this would look fine in an implementation, so I also made a high fidelity mockup to see what it would look like:

(We will probably consider if using tabs like this, which is consistent with the rest of Cockpit, or a box version of tabs makes sense visually. It's just a flag in React and we can determine this at the time of implementation.)

This modal is intentionally wider than most other dialogs in Cockpit, as there's a lot of stuff packed in there... especially in the integration view (see modal wireframes above). Because of how these widgets are already made in PatternFly, they will collapse down and work on smaller screens too (mainly by wrapping and getting taller). We'll make sure it works in smaller sizes, but should also optimize it for larger sizes so it doesn't feel too cramped. (Tabs are part of this.)

Also: The modal will have the container image pre-selected when creating a container from an image. As it will also have the pod specified when creating the container from a pod group.

Outro

I hope you like this behind the scenes view of what we're thinking about doing for the future of the Podman part of Cockpit! Please share any ideas you might have that could improve the Podman UI, especially if you're using containers (either Docker or Podman).

Additionally, there are a few new concepts in these mockups that we might use elsewhere in Cockpit:

• Expandable sections (I'm not sure where, but this could be setting some precedent)
• First suggested use of a toggle group
• Wider modal dialogs
• Tabs in modal dialogs

And we might consider something like the create new container dialog for the "Create VM" dialog Machines page.

Remember: Design is a process, and we'll be iterating as we go along too! (Everything you see here is subject to change, of course.)

[marusak]

Thank you @garrett !
One thing that is not clear to me is use of "Owner" and "Run as user" in "Create container" dialog. These two feel like they represent the same functionality. Also if you have "owner" of an image, that is the same "owner" of the container that is created from it. (One cannot mix and match owners of containers and images) Of course, with the new approach to images (download on demand from "Create container" dialog) it makes sense to let user to decide. But I think that only 'Owner" in the first line makes sense, as you need to commit to one user, and for that user pick image (either already downloaded or pulling a new image for this user) and then for the same user running the container.

So I would drop the "Run as user" line and renamed label of "Owner" to "Run as user". For now it would be "System" and current user (omitted when logged in as root) and if we once support running containers as arbitrary users (cockpit-project/cockpit-podman#716) then we can convert it into a dropdown.

Thoughts?

[martinpitt]

... but Im not sure how to enumerate the available users in an image. The default user is part of the metadata, but aside from this we can pretty much only assume "root" (uid 0), and any other would need to be a free-form input.

[marusak]

Oh right, I did not get that :) I consider myself an average user, so I think we need to make it more clear what it means.
Also fair point that we don't know users so we would have to have there input and not dropdown. But at the same time I am not sure how much use this one gets. Do we have requirements for this or do you have a specific use case in mind?

[garrett]

"Owner" and "Run as user" in "Create container" dialog

If you're running a system container, you can tell it to run as something like a wwwuser or whatever system user instead as root. The owner would still be root (in this case), but it'd run as someone else. At least that's my understanding.

But we probably need to be clearer; it threw me off a bit when thinking about what should go in the dialog.

If I'm not wrong, there are a minimum of 3 users involved with every container:

• owner of the container itself
• who it's running as (which is often the same, but can be different, especially system containers)
• which user is inside the container (could be root or specified as a different user)

[garrett]

The use case I'm thinking of is:

Someone sets up system containers for all their services. They want to run them as specific users, like one for the webserver, one for the SQL server. But they don't want to run the conatiners as root. If someone were to breach the container or if the container needs to read/write files outside of the container, then access is restricted to just that system user (not root).

From my understanding, a lot of people using Docker have the container drop from root privs down to a specific user. This is the same as what I have in mind here.

[martinpitt]

If you're running a system container, you can tell it to run as something like a wwwuser or whatever system user instead as root.

You can do that for user containers as well. The user inside the container is completely independent from the one who owns it (that's kind of the point of user namespaces and containers).

owner of the container itself
who it's running as (which is often the same, but can be different, especially system containers)

There is a technical difference, but it really doesn't matter for users. Presentation-wise, the second item here should not be shown at all IMHO. We should only expose these two:

• Owner: "you" (user container) or "system" (system container, aka sudo podman, needs Administrator privs in the UI)
• Container workload user (as you said). Containers normally come with a default user in their manifest (which we can parse), and root always exists, but there might be other defined users. And you can always specify a numeric UID if you know what you are doing (it might not be defined in the container's /etc/passwd, but most stuff will work just fine). In fact, this is exactly what OpenShift containers do -- they run containers with a random uid inside.

[martinpitt]

Thanks @garrett , this is really nice! Admittedly I squinted a bit when I saw the image list on the top (as it's not really the primary thing users think about), but the rationale makes sense -- you fear it would otherwise scroll off and not be discoverable?

If we can make image lookup in registries work fast enough for type-ahead search, that would be really nice. It does take a few seconds, so a wee bit too slow for typeahead auto-complete, but we can experiment with some spinners. If not, and that feels a bit clumsy, we still have the old full-blown dialog for downloading a new image. Overall it would still a net UX win, though.

[garrett]

In many cases, you'll have at least 1 image per container. And every time you update the image and replace the container with a new one, you'll have another image. If they're not pruned, the image list quickly gets longer than the container list. With a long list of images, someone won't be able to even start to see containers until they scroll down the page.

Of course, it could also be the case that many containers are spawned from a single image. It really depends on who's using it. But even in this case, people update images to the latest built one or they build a local image pretty frequently... and the list grows.

In typical usage, someone wants to see and sometimes alter statuses of their containers. But if they're not on the page without scrolling down, then there's an issue with the page's usefulness.

This is probably why the image list is under the container list right now... but that's also wrong UI-wise, as containers come from images, not the other way around... and interfaces are from left to right (in LTR languages) and top to bottom. Things that affect other things should come before and things that are affected by other things should come after. Hence, the current order is wrong and images should come before containers (without pushing containers off the page).

[garrett]

It does take a few seconds, so a wee bit too slow for typeahead auto-complete, but we can experiment with some spinners

I was thinking we chunk it in sections, so that we have an instant reply for local images and a spinner and pop up for later ones. If we could cache results (perhaps at least per session), that would be useful too. Therefore, it would react immediately even if it cannot be finished immediately.

In the cases where someone is starting a lot of containers from a single image without clicking on the create container from image button in the image list, this caching would certainly help. It also would help someone a lot if they're building or pulling images elsewhere, like at a command line. (Someone could build an image on the server and still use Cockpit to manage the containers. Or it could be more than one person doing these tasks.)

We could probably even optimize the loading further, like do a preliminary a search in the background based on current images as well, either after the page loads, or when someone opens the dialog. (The latter is probably better, as it would be acting on someone's direct action and intent.) But this is just an optimization idea.

[martinpitt]

In many cases, you'll have at least 1 image per container.

There's always exactly one image for a container. Correct that images need to be pruned from time to time to make disk space room. In most cases these don't clutter up the images list, though, as they get untagged. I.e. yesterday's quay.io/foo/bar:latest was image ID 12345, today it was refreshed and points to 1abcde, then 12345 does not appear any more in podman images (only in --all). Likewise, we should not show untagged images by default, but have an option for it.

I'm not too worried about the typeahead search. Of course local images are pretty much instant, it's just the remote registry searches which take a bit. But still, with a spinner this is still more convenient than separately having to download an image (and there the typeahead search is of course not any faster).

[garrett]

There's always exactly one image for a container.

In use, and not always, but often. If there aren't multiple containers from the same image.

A container will have multiple images when someone pulls down a new one with a different tag. (If someone is using tagged releases instead of just latest.)

Likewise, we should not show untagged images by default, but have an option for it.

We should show all the images when it's expanded, differentiate between the tagged versus untagged ones, and make it easy to clean up to recover space (in addition to showing how much space is used).

The designs show the space, make it easy to clean up, and hides the implementation details of the images by default. (It would probably get a bit complex to expand to see the images and then basically expand the list a second time to see the untagged images.)

---

I'm not too worried about the typeahead search

As for typeahead search, interactivity and responsiveness matters a lot for UX (just like for everything else). This is why I suggested instant results for local images and loading for the rest. If remote registries are < 500 ms, that's great and it's not a big issue. If they take longer than that (and they do in practice), then it's too long to wait to show things. But we do need to show something in < 100ms, else it'll feel sluggish. And we need to show some kind of progress (even indeterminate, such as a spinner and/or some kind of placeholder) > 500ms, else it'll feel slow.

(This applies to the rest of Cockpit too of course, and why I bring it up from time to time. I should probably bring it up even more and file a ton of annoying issues. 😉 And/or we could do a shallow+wide SPUR looking at response times across all of Cockpit to identify the problem points and then rank them in order of need of fixing.)

But this is critical for an interactive component, which the typeahead autocomplete is. And the typeahead widget is crucial for the design of the merged container creation modal dialog, which is why I'm emphasizing it.

Some excellent resources that explain this:

• Response time limits: https://www.nngroup.com/articles/response-times-3-important-limits/
• Nice summary of optimizations to decrease the feeling of slugishness: https://blog.placeit.net/ux-tactics-make-slow-things-seem-faster/

[KKoukiou]

I would like to focus on the main page design itself, not the dialogs.
We kept the expandable list rows in the design mockup so far. However, I believe this makes the UI be very dense, and of course it's harder on small devices to have an easy navigation with all that content.
I would actually prefer to see resource per page, at least for container details. Waiting for your opinions on that.

[garrett]

I mentioned it in the design above. The VMs are editable and have more properties overall, but containers are not editable and don't expose as many things. Also there are pod groups for containers, which complicates things a bit.

Additionally, when I talked with @marusak about it, he was vastly in favor of expandable lists and did not want sub-pages.

[marusak]

I am also in favor of inline expandable details. The main reasoning being that one wants to quickly iterate over containers and not spend too much time on them. Compared to VMs, user would just look at some detail about container and then do some action (delete, run, stop...) while on VM user might want to dig deeper into settings, tweak the machine... Jumping to separate page, just to check one detail and then remove the container seems too complicated really.

[garrett]

For the sake of discussion, I've mocked up what it would look like if containers would have a sub page:

If we did have sub-pages, then for Pods: We'd probably want to have a Pod view with all the related containers? Otherwise jumping back and forth between the view with all the containers to the specific container would probably make it easy to get lost in all the details of everything else. (You'd be able to jump to container details from the main page, and back, but with the breadcrumbs, you'd be able to jump back one level to show just the pod.)

You would be able to manage the container, including deletion (and run, stop, etc...), from the ⋮ kebab menu at the top.

[garrett]

look at some details about the container

Do we know what detail(s) would be likely looked at to determine if a machine would be stopped/started/deleted? We could consider showing that in the list by default if it's important enough.

[mrguitar]

The mock-ups looks amazing. Great work. Down the road I think it would be compelling to drive the process of podman generate systemd from cockpit. If users can get services running via cockpit, then once happy "commit" them to be persistent via a unit file w/ the option for auto-update. That would be killer IMO. Perhaps this belongs in a different github issue but it's what came to mind while looking at the mockups. Again, great job on this!

[garrett]

The mock-ups looks amazing. Great work.

Thanks for your kind words!

Down the road I think it would be compelling to drive the process of podman generate systemd from cockpit.

I fully agree. @marusak and I talked about this in our meetings when discussing these mockups, but the scope is already quite broad here. We should add the systemd integration functionality, though. It's rather dodgy if the containers don't automatically restart on a reboot, for example.

I'm manually doing this on my own server:

podman generate systemd -n NextCloud --restart-policy=always > /etc/systemd/system/podman-nextcloud.service
systemctl daemon-reload
systemctl enable --now podman-nextcloud

(After re-running podman generate systemd and updating the service file if it already exists, enabling it again is not needed, by-the-way.)

But we should totally manage this in the Cockpit-Podman UI.

It should probably be handled as a "restart policy" and Cockpit-Podman should probably call it cockpit-podman-SERVICENAME.service and place it in /etc/systemd/system/

Even better is if Podman itself could handle the management of the service files (rather than just creating the text for one), and then we could use that instead of managing the state ourselves.

[marusak]

Indeed, we already track it in cockpit-project/cockpit-podman#364 and https://bugzilla.redhat.com/show_bug.cgi?id=1820551

I believe that podman should implement this and just expose it through API. Cockpit has disadvantage, as users may use podman directly (or some other tool that communicates with podman) to delete containers, which would leave units that cockpit created behind. If podman would be implementing this, then it can always check if there is an autostart unit for the container that user is deleting and do proper cleanup.

[marusak]

There is Create container button but it does podman run, not podman create.
If my memory serves me right, we agreed with @garrett that we will name it this way and then add checkbox into the modal with Immediately start that would be by default in on state. With this behaviour it would still behave as podman run.

I don't see this checkbox in the modal, so stating it here, so we hopefully don't forget about it :)

[garrett]

Oh, right. We did talk about that. I think that's an oversight? We talked about so much. 😁

Something like this (although I'm not entirely happy with the bolded label):

Running ☑ Run after building

(It should be checked by default.)

Ideally, we'd eventually get to the point where we could have something like:

• Do not run
• Run once
• Run on every boot

Since it's 3 options, it should probably be a group of radio buttons.

Default should probably be run on every boot? Or at least run once.

[marusak]

Run after building

Building? That is surely not the right word, see podman build --help. Something like 'Start after creating' to keep with podman jargon?

[garrett]

Sure, "Start after creation" I guess? I was communicating the concept here more than the words. But you literally referenced "podman build --help". (Not podman create --help.)

[KKoukiou]

'Immediatelly start VM' is what we use for VMs page for the exactly the same functionality checkbox. Do we want to keep these two same?

[garrett]

We're working on an image autocomplete that combines the source and tags @ cockpit-project/cockpit-podman#773 (a current work in progress). This PR implements a create button, the combined image dropdown, and will probably include a download indicator when a container is created (or that will be in an immediate PR right after this one).

Combing and expanding the feature of the image dropdown will a little bit of a change for the create dialog, but will mainly impact the source, image, and tag dialogs. These changes should greatly help the container-oriented workflow.

@jelly and I had a few meetings about it and today's meeting notes should help illuminate the direction we're taking it. (Many will land in this PR, some are feature enhancements for the future).

---

Popover text

Info popover with something of the following (better explained):

(i) popover text for host[:port]/[user]/container[:tag]
Examples for:

• container search
• search a registry
• specify an image
• add an optional tag (defaults to :latest)

Placeholder in the dropdown: "Search string or container location"

Helper text

Below the dropdown

• "" (empty string at first, so form doesn't jump? create button disabled)
• Local image (from user/system?) will be used (success, create button enabled)
• Checking remote (validating that the image exists, button disabled while the check is happening)
• Image will downloaded (success, create button enabled )
• Remote image does not exist (error, create button disabled)
• Random error, like an API limit (enable create button anyway?)

Verification like this: curl -X GET -s -g --no-buffer --unix-socket /run/user/1000/podman/podman.sock 'http://localhost/v1.12/libpod/images/search?term=docker.io/library/redis&limit=10&listTags=true' | jq

Minimum for dropdown

• Show local images first
• Search icon and blue text for starting a search

User versus system images...

• Move system/user selector right above the image?
• Show both? Or an expander for others?

Future

• download progress indicator
• automatic search? (although it'd be nice to get in, if we can)
• filtering
• tag browsing for validated images (typing : would switch to tag mode)
• how do we show official images? ("label"/badge at the side? what about when selected?)
• option to pull the latest version (when there's a local image match)
• mockups for tag handling (later)
• mockup for "recommended" images during a search via shortnames (eventually)

[jelly]

I've talked with the podman team and they will remove the official images identifier in the podman cli in the future so while it stays available in the API I don't think we should implement this behaviour.

The download progress will be difficult to implement as far as I know we don't know and podman pull also shows no progress bar. I've made a ticket upstream to get this supported containers/podman#12341

[garrett]

I found some older notes I made while brainstorming for the design. A lot of this has been funneled into the redesign, but some of it was left in these notes.

There's a bunch of ideas in this list, and just because something is here, it doesn't mean we should implement it. But it does touch on some things like import/export, which are hinted at in the mockup (but not explicitly stated otherwise). Also, some of these concepts already now exist thanks to @marusak and @jelly working on the redesign and adding features outside of the redesign as well. 😉

For the sake of preserving some of these ideas, here they are:

• auto-start user service (if user containers exist); if user containers or images do not exist, start it upon creation of one
• pull latest version of an image
  • auto-update?
• view if image is used or not
  • select unused images (for deletion)
• show dockerfile details (if available)
• build new images based on a dockerfile???
• show used IP addresses & ports of running containers
• shortcut to console in the list's menu
• VM-like details page
• filesize of images and containers
• (podman system prune? but it's dangerous... perhaps volumes, which are ones unused by any container... there's also podman image prune / podman container prune)
• podman container as a systemd service
  • podman generate systemd
  • https://www.redhat.com/sysadmin/improved-systemd-podman
• user lingering of user services, for starting user services on boot: https://www.youtube.com/watch?v=AGkM2jGT61Y
• integrate systemd service enabled containers with system services?
• export and import container / pod configs?
  • podman generate kube POD_OR_CONTAINER
  • podman play kube YAML_FILE
• move images to a summary card (demoted from front page, since they're mainly an implementation detail); concentrate on relevant tasks in the summary
  • # of total images
  • combined size of images
  • # of unused images (ones unassociated with containers/pods)
  • combined size of unused images
  • if there are new upstream tags of any images
• images sub-page
  • all level (should also probably be available on the summary card, as well as at the top of the page)
    • "prune" unused images podman image prune -a
    • update all images (podman image pull for each image)
  • individual images
    • remove image (confirm when associated with an image)
    • update image
    • remove specific tags for each image?
• Show when there's a newer version of the image that the container isn't using yet
• Update container based on newer image, based on exact same config (but newer version of image)
• scaling moving between pods: a dialog to select the pod?
• scaling the lists of pods: should they be collapsable / expandable? should they use PF treeview?
• selinux issues
  • need :Z instead of :rw?
  • changing the storage location has a strange error
    • /bin/sh: error while loading shared libraries: libc.so.6: cannot change memory protections
    • fix is to semange fc ontext and restorecon?
      • https://github.com/containers/podman/blob/master/troubleshooting.md#11-changing-the-location-of-the-graphroot-leads-to-permission-denied
• BUG: problem with restarting images
  • in running only, they show up and disappear
  • the stop/restart/etc. actions go away and come back again and again
• "Recreate" (or "Rebuild"?) container
• containers aren't auto-starting?
  • podman generate systemd -n Caddy --restart-policy=always > /etc/systemd/system/podman-caddy.service
  • should expose this in the UI
  • should be on creation and in the ... menu, to change it afterward
  • should show that it's a service in the UI somehow
  • link to the services page for the service too?
• run as user in the creation dialog?
• re-create...
  • allow you to replace the container (delete and rebuild)
    • makes sense especially for customization
    • pull latest image
  • create a new one without removing the original (with a new name)
  • checkbox for removing old one
  • Edit? vs Clone/Copy? (different dialogs)
  • pull option
    • can it show up only when the image was pulled remotely (and not created locally)? RepoTags / RepoDigests?
• Create button, not "add"
  • Creation dialog should have a default checked run "Immediately start container"
• Port conflicts
  • show errors inline and show which container is doing 
  • Firewall issue where it was blocking access to podman containers

New UI in pods mockup: cockpit-project/cockpit-podman#238 (comment)
(Note: This is an old mockup and predates the redesign mockups above. But it's probably still worth looking at, especially for the discussion around pods.)

[garrett]

Note: We may want to consider the Docker-like restart options, as they now work in Podman, even after a reboot (as of the 3.3 release):

https://podman.io/releases/2021/08/31/podman-release-v3.3.0.html

The gotcha is that we'd need to enable podman-restart.service ...and I think it only works for system containers, not user containers.

Here's an excerpt from the release notes (of the above URL):

Podman now supports restarting containers created with the --restart option after the system is rebooted. Containers created with --restart=always can be automatically started when the system boots if the podman-restart.service systemd unit is enabled. Our main focus continues to be on managing containers directly with systemd via podman generate systemd, which has always allowed containers to be automatically started after boot and provides greater flexibility than the --restart option, but the addition of podman-restart.service will be useful for those seeking improved compatibility with Docker. The podman generate systemd command also saw several improvements, and will not default to using SDNotify instead of PID files, producing smaller and easier-to-understand unit files.

[jelly]

I discovered today there is also podman auto-update but this applies more for running containers (with a certain label) to auto-update them.