RFC: Overlay API v2

[Westbrook]

Table Of Contents

• Summary
• Basic Example
• Motivation
• Detailed Design
• Drawbacks/Constraints
• Alternatives
• Adoption strategy
• How do we educate people?
• Open Questions
• References
• Related Issues

Summary

New developments in the web platform provide an opportunity to streamline one of the most important APIs of Spectrum Web Components: overlays. The Overlay system powers every instance of content that visually obscures others, including tooltips, dialogs, pickers, and menus. It also incurs a high maintenance and support cost; overlay causes more issues than any other API.

Currently, Overlay implements a "user-space" solution by reparenting elements directly into the document's body. This has been the only option to guarantee correct visual presentation. However, new browser features like <dialog> and [popover] unlock the option of overlaying content visually without mutating the DOM.

This RFC proposes shifting both the public API and the internal implementation to seize this opportunity. It discusses these new APIs, how the Overlay API will leverage them, the use-cases they won't cover, and how consumers will be able to adopt this functionality.

Basic Example

The central thesis of this new API is that we should leverage browser primitives wherever possible, while also preparing for the presence of in-development browser primitives when they become available. In this way, we will focus on consumption of the <dialog> element and the popover attribute; APIs surfaced by the browser in a declarative manner. We will also focus on surfacing our orchestrations and speculative polyfilling of these capabilities in a declarative manner, which will take the form of the <sp-overlay> element. To support transition from our current imperative API, we will also surface this functionality imperatively. However, the imperative API will rely on injecting an <sp-overlay> element into the DOM tree from which you would like to overlay content, which has the possibility of negatively affecting style delivery or layout in your content. When an imperative API is preferred, consumption of the <sp-overlay> element will be more predictable if you leverage document.createElement(‘sp-overlay’) or new Overlay() directly. With these things in mind, we highly recommend declarative consumption of the new Overlay API and suggest that you partner with us early and often on ideating paths towards this usage.

Declarative modal dialog

<sp-overlay ?open=${this.isOpen} type=”modal”>
	#shadow-root
		<dialog><slot></slot></dialog>
	<div that i want to overlay></div>
</sp-overlay>

Declarative [popover] or “overlay” (attribute API)

<sp-overlay
	?open=${this.isOpen}
	?delayed=${}
	offset=${Number | [Number, Number]}
	placement=${placement}
	received-focus=${‘true’ | ‘false’ | ‘auto’ (default)}
	trigger=${idOfTrigger AND ${string}@${string}}  
	.triggerElement(s)=${trigger Element by reference} // match naming to aria element reference spec
	.triggerInteraction=${ ‘click’ | ‘longpress’ | ‘hover’ }
	type=${type}
>
	#shadow-root
		<dialog><slot></slot></dialog>
<div that i want to overlay></div>
</sp-overlay>

Imperative modal dialog

Overlay.open((overlayElement: HTMLElement)); // Modal-only API structure.

Imperative [popover]

Overlay.open(popover, {
	placement: 'left-start',
	trigger: this,
	type: “auto”
});

Motivation

Overlaid content in the browser is hard. Before the browser APIs that we will be leveraging to power this work, really solid ways of handling this content across all use cases were hard to find. While we’ve had quality success to date with our current approach, we believe that embracing new and pending browser APIs will go a long way in us and our consumers finding even more success going into the future. We see this being possible by the following motivations:

• Simplify usage of overlays & streamline the API
• Provide one clear, "right way" to accomplish overlay-related goals
• Improve the performance of overlays
• Reduce issues & support overhead
• Shrink the size & complexity of the overlay implementation
• Improve stability

The overlay system’s outsized number of issues & support is largely driven by its current need to reparent elements in the DOM. It has a high maintenance burden and low stability due to a large amount of user-space code that could now be deleted in favor of platform-provided functionality.

Detailed Design

Imperative API

The fill imperative API for opening an overlay is as follows:

Overlay.open(
	target: HTMLElement,
	options?: {
		delayed?: boolean;
		offset?: number | [number, number]; // supporting multi-axis
		placement?: Placement;
		receivesFocus: 'auto' | ‘true’ | ‘false’;
		trigger?: HTMLElement | VirtualTrigger;
		type?: defaults to “modal” | “page” | “hint” | “auto” | “manual”;
	}
);

Options

• delayed: {boolean} whether the Overlay participates in Spectrum design “delay” algorithm
• offset: {number | [number, number]} the main axis offset from the trigger to place the overlay or the main and cross axis offset from the trigger.
• placement: {Placement} the placement in relation to the trigger element
• receivesFocus: { ‘auto’ | ‘true’ | ‘false’ } whether to specifically pass focus into the overlay or not, defaults to ‘auto’ which follows the rules around focus defined by the overlay type
• trigger: {HTMLElement} the element that the overlay should be positioned in relation to.
• type: {‘modal’ | ‘page’ | ‘hint’ | ‘auto’ | ‘manual’ } Whether to be <dialog>.showModal() based (‘modal’ or ‘page’):, the ‘page’ value prevents the default escape functionality to close the overlay, or to be [popover].showPopover() based, and which popover value to use.

Return value

This API will return a reference to the <sp-overlay> element that it creates. This will give consuming applications full access to the API of the <sp-overlay> wrapping their content. To smooth adoption, we will be accepting the old Overlay.open() arguments for a short conversion period, and could see altering the return value when that call signature is leveraged in order to further reduce the changes needed to move, initially, to this new API.

element

<sp-overlay
	?open=${this.isOpen}
	?delayed=${}
	offset=${Number | [Number, Number]}
	placement=${placement}
	received-focus=${‘true’ | ‘false’ | ‘auto’ (default)}
	trigger=${idOfTrigger AND ${string}@${string}}  
	.triggerElement(s)=${trigger Element by reference} // match naming to aria element reference spec
	.triggerInteraction=${ ‘click’ | ‘longpress’ | ‘hover’ }
	type=${type}
>
	#shadow-root
		<dialog><slot></slot></dialog>
	<div that i want to overlay></div>
</sp-overlay>

Attributes

• open: {boolean} whether the overlay is visible
• delayed: {boolean} whether the Overlay participates in Spectrum design “delay” algorithm
• offset: {number | [number, number]} the main axis offset from the trigger to place the overlay or the main and cross axis offset from the trigger.
• placement: {Placement} the placement in relation to the trigger element
• receives-focus: { ‘auto’ | ‘true’ | ‘false’ } whether to specifically pass focus into the overlay or not, defaults to ‘auto’ which follows the rules around focus defined by the overlay type
• trigger: {${string}[@${string}]} this string accepts the ID of a trigger element that is within the same DOM tree and can be extended with the @ symbol to include the interaction (click | longpress | hover) to bind opening/closing the overlay on that element. This looks like the following:
  • trigger=”triggerID” would locate [id=”triggerID”] in the same DOM tree for use in positioning the overlaid content
  • trigger=”triggerID@click” would locate [id=”triggerID”] in the same DOM tree and then bind the open of the overlay to the click event
  • trigger=”triggerID@longpress” would locate [id=”triggerID”] in the same DOM tree and then bind the open of the overlay to the longpress event
  • trigger=”triggerID@hover” would locate [id=”triggerID”] in the same DOM tree and then bind the open/close of the overlay to the events that complete the “hover” contract
• type: {‘modal’ | ‘page’ | ‘hint’ | ‘auto’ | ‘manual’ } Whether to be <dialog>.showModal() based (‘modal’ or ‘page’):, the ‘page’ value prevents the default escape functionality to close the overlay, or to be [popover].showPopover() based, and which popover value to use.

Properties

• triggerElement: {HTMLElement} accepts a reference to a “trigger” element, this can be particularly useful when the trigger element is already known in a JS context or is not within the same DOM tree as the <sp-overlay> element
• triggerInteraction: {‘click’ | ‘longpress’ | ‘hover’} the event type to bind the overlay open/close to the trigger element

Events

• show: when the overlay opens
• hide: when the overlay closes

Drawbacks/Constraints

While the <dialog> element is rather well supported by browsers (https://caniuse.com/dialog), meaning that little to no polyfilling or fallback support will be needed, the [popover] attribute is still pre-agreement (https://open-ui.org/components/popup.research.explainer) and will feature contexts that it just cannot support correctly regardless of the fallback approach until it is supported natively in browsers. Some of these contexts have a “consumption workaround”, or things consumers can do to prevent them from affecting their applications, and others are hopefully minor edge cases, but they are as follows.

Complex Layering

This situation affects content that could be overlain that exists within ancestor content that has been provided a value for the CSS position and z-index properties that sits adjacent to content that has been provided a functioning z-index that is higher than the ancestor content.

<div style=”position: relative; z-index: 1;”>
	<element-to-be-overlaid></element-to-be-overlaid>
</div>
<div style=”position: relative: z-index: 1;”></div>

CSS Containment

This situation affects content with an ancestor that leverages the CSS property contain with a value that is or implies paint. In the case that the overlay should display within that ancestor, there is likely not to be an issue, but containing paint means that the overlay cannot appear outside of that ancestor, which may be unexpected.

<div style=”contain: content”>
	<element-to-be-overlaid-outside-of-parent></element-to-be-overlaid-outside-of-parent>
</div>

CSS Clip-path

Overlay content delivered within an ancestor element that leverages the CSS property clip-path has a chance of being clipped itself. This feels like the least likely situation as the clip-path property is helpful in only very specific situations.

<div style=”clip-path: inset(0 0);”>
	<element-to-be-overlaid-outside-of-parent></element-to-be-overlaid-outside-of-parent>
</div>

Working around CSS issues

A pragmatic workaround here is to draw a seam between what can really be contained in a given rect, vs. containers that require a permeable boundary. In the above examples, consider moving overflow, containment, or clip-path management into a separate child element’, to continue to benefit from performance optimizations without breaking the overlay presentation.

This would mean instead of:

<div with overflow/containment/click-path>
	Some content
	<element-to-be-overlaid-outside-of-parent></element-to-be-overlaid-outside-of-parent>
</div>

You could deliver your UI as:

<div>
	<div child with overflow/containment/click-path>
	Some content
</div>
<element-to-be-overlaid-outside-of-parent></element-to-be-overlaid-outside-of-parent>
</div>

Not all UIs will benefit from this approach, but in many cases, it will allow for mitigating the negative side effects of leveraging <dialog> and [popover] before they are fully supported in browsers.

Alternatives

Maintain & better document the current API

A subset of our objectives may be reachable through more complete documentation and examples of the current API. While enhanced documentation is a target of all of our work, we do realize that the depth of complexity in the current Overlay API leaves much to be desired and has disadvantaged our consumers in more ways than one.

Evolve the current API, maintaining core behavior

We’ve discussed ways that the current API could be evolved to achieve a different subset of our objectives.

The current Overlay API uses content reparenting to force overlay content to the end of the document to fully escape clipping and layering issues that can occur due to various CSS incantations. However, reparenting causes concentration as to the application of styles onto that content. See this documentation for more information, and follow these issue links for a better understanding of the issues that consumers have faced in the past.

Adoption strategy

In as many cases as possible, we’d like this update to the Overlay API to not be a breaking change for our consumers. When leveraging the imperative API, you will no longer be returned a single close() method, but instead, an <sp-overlay> element that has a close() method. We will be updating the internals of <overlay-trigger> to leverage the <sp-overlay> element and translating Overlay.open(owner: HTMLElement, interaction: TriggerInteractions, overlayElement: HTMLElement, options: OverlayOptions) calls to Overlay.open(target: HTMLElement, options: OverlayV2Options) under the covers. In some cases, this may subject consumers of this API to the Drawbacks/Constraints outlined above, however, we look to minimize the occurrence of this situation by working with our partners through a Canary release process to find ways to alleviate these issues in their applications.

While the initial focus will be one-for-one support of current consumption, we will be suggesting that consumers move towards direct consumption of the declarative <sp-overlay> element instead of <overlay-trigger> or the imperative API in new features immediately, and in their existing feature surfaces at their convenience. With the availability of <sp-overlay> we believe that taking greater control of imperative injection of this element into their application will benefit consumers greater than any future developments of the Overlay.open() API could and suggest that is they must deliver overlay content imperatively they leverage something like the following to do so:

const overlay = Object.assign(new Overlay(), {
	targetElement: HTMLElement,
	delayed: number,
	offset: number | [ number, number ],
	placement: Placement,
	type: ‘auto’ | ‘manual’ | ‘modal’ | ‘page’,
	trigger?: HTMLElement | VirtualElement,
});
el.append(overlay);
overlay.open = true;

Were any of our Client Contributors to have experience in tools like codemods or similar, we’d greatly appreciate their input on ways that we could leverage those capabilities to simplify this process for everyone involved.

How do we educate people?

The following will be made available to support consumers getting up to speed on the new APIs:

• updated SWC documentation site
• share before after snippets in the migration guide
• ongoing SWC office hours
• recordings of internal sessions refactoring code that consumes the current Overlay API

Open Questions

• Should we join the [popover] Origin Trial on Chromium browsers to reduce our dependency on polyfills and fallbacks? Is it possible to have an “SWC Origin Trial” or does each consuming product need to do this for themselves?
• Are any of our top-level consumers directly impacted by any of the drawbacks or constraints outlined above?
  • Particularly, the LRWeb team is interested in a closer examination of their consumption of this API.
• [popover=”hint”] has been pushed to V2 of the browser API here, is manual a fully acceptable path forward for “tooltip” like overlays?
• Is the consumption of sp-opened and sp-closed events wide enough that we should continue to dispatch these events for some time? Or possibly make them part of the new API?
• What consumers would prefer to be part of the pre-release program where we work to ensure the highest level of compatibility between current Overlay API usage and the usage that will be available in the new version?

References

• Dialogs, modality and popovers seem similar. How are they different?
• element
• Popover API
• Pop-ups: They're making a resurgence!

Related Issues

#424 #511 #518 #606 #617 #709 #710 #803 #848 #866 #1055 #1094 #1133 #1249 #1267 #1284 #1289 #1323 #1331 #1360 #1437 #1512 #1540 #1559 #1783 #1831 #1853 #1876 #1897 #2026 #2067 #2068 #2082 #2197 #2198 #2282 #2360 #2502 #2536 #2539 #2568 #2599 #2625 #2646 #2647 #2657 #2693

[nickschaap]

However, the imperative API will rely on injecting an element into the DOM tree from which you would like to overlay content, which has the possibility of negatively affecting style delivery or layout in your content.

I have found the imperative API to be pretty useful in development. I'm curious to know more about the style and layout delivery constraints mentioned here. I find the imperative Overlay API to be nice to use in situations where UI is overlaid that isn't necessarily coupled to a specific UI component such as toast type notifications/ popups that may be triggered by shared services. For example, a router might trigger a dialog to confirm the decision to leave a page.

With the current Overlay system, I also believe consumers must use the imperative approach if they want to leverage a VirtualTrigger. Are we planning to move away from VirtualTriggers?

If you do plan on keeping the Imperative API in the Overlay rewrite it would be nice if SWC could provide helpers for devs using Lit. For example for imperative modals:

Overlay.open(template: TemplateResult);

[Westbrook]

1. How to make a Toast?
   The way that a single toast works in a fully declarative manner should boil down to roughly (style API is yet to be set):

   <style>
     sp-overlay {
         --sp-overlay-left: 1em;
         --sp-overlay-bottom: 1em;
         --sp-overlay-top: auto;
     }
   </style>
   <sp-overlay ?open=${this.showToast} type="manual">
       <sp-toast>Toast message...</sp-toast>
   </sp-overlay>

   We'll discover over the course whether that's better managed via CSS Custom Properties or a CSS Part, but this is generally the approach. Hopefully, the <style> being outside of the overlay DOM makes it clear that you can style the overlay from the context in which it lives, rather than needing to position the styles so that it moves with the overlaid content. This means that those styles could be applied via static styles = ... in a Lit element, etc.

   There is a lot of nuance from there about how you would make a "toast system" as Spectrum suggests that only one Toast at a time, and there's a quality, yet incomplete discussion about that in Toast alert #2299 that it would be great to get your thoughts on.

2. How can a router trigger a dialog?
   This would likely work very similarly to what you see above, depending on whether the router is DOM-based or object-based. If it is object-based, then it would likely still callback to some sort of app element or the like that could leverage the APIs as outlined above.

3. How is Virtual Trigger used?
   There should be no changes in the ability to leverage a Virtual Trigger, only in the way that it would be applied. In the declarative API, this would be applied via the triggerElement property, e.g. <sp-overlay .triggerElement=${virtualTrigger}></sp-overlay> or via the trigger property on the options of the declarative API:

   Overlay.open(
       overlayElement,
       {
           trigger: virtualElement
       }
   );

   Context menus are a big part of what our Overlay API empowers, and we'd definitely not want to remove that.

4. What would you expect from that API you've suggested?

   Overlay.open(template: () => TemplateResult);

   What would you want this call to do? Note: I've made the template argument from above accept a method that returns a TemplateResult, but made no other changes.

   • Where would the TemplateResult be rendered in order to actually deliver it to the user?
   • How would you want to style that content?
   • What would you expect the TemplateResult and the data/event listeners therein to be bound to?
   • What other benefits do you see in this approach, beyond the simplicity of being tightly bound to lit?

[adixon-adobe]

@Westbrook it looks like LrW is primarily using sp-popover and overlay-trigger, not the Overlay class directly. I do expect us to run into some CSS layering bugs when popover migrates to this, but we should be able to fix those with the workarounds you mention. Outside of that, do you expect any API changes to popover with this, or will that largely be under the hood?

The other small concern I have here is the potential for stacking context bugs (particularly on mobile browsers where I've seen issues in the past, as they tend to take more heavy-handed approach to gpu optimization). This might mean using a lamer workaround beyond the ones you've listed for css layering like a css transform to force the z position to be higher. I'm confident we'll be able to work around and report any issues that come up though.

[Westbrook]

sp-popover isn't technically a part of the Overlay API, though it does seem like it should be, so we don't expect to make any changes to it as part of this work. overlay-trigger, sp-picker, sp-action-menu, sp-split-button, et al will have internal changes that that should not effect end users (other than a final deprecation of support for delivering sp-menu children to some of those elements).

The other small concern I have here is the potential for stacking context bugs (particularly on mobile browsers where I've seen issues in the past, as they tend to take more heavy-handed approach to gpu optimization).

You wouldn't happen to have any specific instances of issues that might be relevant to this that we could include in our testing process? Even if they happened to be "in app", we do play a rather rigorous RC phase where we hope to shake off as many app local issues as possible.

[Westbrook]

Ensure that the management of the open property in children of the <sp-overlay> element is documented clearly. This is particularly important when doing something like:

<sp-overlay>
    <some-non-swc-element>
        #shadow-root
            <sp-popover>...</sp-popover>
    </some-non-swc-element>
</sp-overlay>

Here the consumer needs to manage the relationship between the <sp-overlay> and the <sp-popover> on their own, while it would normally be managed directly by the library. While we should recommend decorating custom comment with SWC elements, it's not always possible and we need to prepare consumers for success.

[nickschaap]

It might be interesting to vend a ReactiveController or Mixin that helps manage this state.

[Westbrook]

What do think something like this might look like here?

[Westbrook]

Should we expand triggerInteraction: {‘click’ | ‘longpress’ | ‘hover’} to include context menu?

[nickschaap]

It would be nice if client applications could optionally specify a priority stacking order on their Spectrum overlays. This can be useful in cases where applications might trigger overlays to open programmatically and some overlays might have higher priority than others. For example, a blocking lost connection takeover modal always being placed on top. It can be tricky to orchestrate this at the application level especially when working on large apps being developed by multiple distributed teams.

I imagine this priority could come in the form of a simple number:

<sp-overlay priorityIndex="0" ...> ... </sp-overlay>

or perhaps it might be interesting to let client applications plugin to the Overlay system in some manner to define their own own overlay stacking strategy.

[Westbrook]

This poses an interesting problem with the architecture we're looking to leverage. There is definitely some room to "hack" this in a version that did not leverage [popover] or <dialog>, we'd have access to z-index to force "priority" in most cases. However, #top-layer, the browser API that is leveraged by <dialog> and [popover] (see this comment for more info on its pending availability) has a strict last in prioritization stack. This means that if you open an important overlay, then try to open a subsequent overlay with no self checking or self management, the important overlay will inherently be defeated as far as priority by the later overlay. The only way for the important overlay to win, assuming the later overlay is allowed to open to begin with, would be for the important overlay to close and open itself.

Off the top of my head, so things that an expansive Stacking Algorithm would need to know include:

• can a less important overlay be opened while a more important overlay is open?
• would a less important overlay immediately open when the more important overlay was closed?
• while a less important overlay is "queued" could it be "closed" at its call site and taken off of the queue stack?
• could a more important overlay have a side effect that cleared all subsequent overlays on the queue stack?
• should a queue be prepared to begin with? this implies that a visitor could get stuck in a series of messages preventing interaction with the application, e.g. https://how-i-experience-web-today.com/
• what else would need to be clear to be able to manage such a stack?

I'm fairly certain that this won't be come a day one feature of the Overlay API v2, as it wasn't a part of the previous. We're looking to open a lot of doors with this conversion, but won't be walking through all of them, just yet. However, that means that thinking through these things and gathering use cases and requirements today can support getting such an addition pushed up the priorities list, sooner. @nickschaap would you be interested in gathering an overview of how you might expect a Stacking Algorithm to address these concepts, particularly in support of your own product work, and for any other concepts that exist in your product work that we might not know about, to support starting that conversation? We could continue it here, and once it was more structured possibly start a new discussion thread to focus specifically on it.

[Westbrook]

More about [popover]'s pending availability in Chrome!! https://groups.google.com/a/chromium.org/g/blink-dev/c/uB_jxbRmjAM/m/Ona8hJ1BAQAJ

[joekukish]

Something that I think it is important to communicate, is to make sure that the events include the target that triggered the event. Currently if you have a page, several tooltips, popovers, etc, there is no way to identify which Overlay triggered the event. You can sometimes rely on the interaction type (if they are different), but if not there is no obvious way to recognize. Ideally event.target or event.relatedTarget would point to the Overlay that triggered the event.

[Westbrook]

Progress Update

[popover] in Chrome Beta

Keep an eye on Chrome Beta because it currently features [popover] without any flag. This means it should be available in Stable by the 24th or so. Visit the Chrome Status Roadmap for more details. 🎉

v2 in canary

We're actively working with canary releases on the overlay channel, specifically version 0.30.1-overlay.42. We're integrating the new API into mature and complex production code bases, which helps iron out subtle nuances and improves the supported feature surface.

Currently, the canary code is a moving target, making it challenging to update the API documentation. Before moving into an RC phase, we plan to integrate it into at least two more code bases.

RC phase

In the RC phase, the API will stabilize and become ready for broader consumer experimentation. The documentation will include a migration guide outlining the necessary breaking changes to adopt the new version of the Overlay API. Our goal is to minimize the required changes, and you can assist by making a few targeted updates to your Overlay API usage today:

1. Review the overlay content in your applications, particularly <sp-tooltip> and <sp-popover> elements. Ensure that control of their open attribute is delegated to the Overlay API.
2. If you leverage the event.detail.interaction values on our sp-opened and sp-closed events in a TypeScript context, make sure to type them as CustomEvent<OverlayOpenCloseDetail> for active checking. There will be expected changes in this area, which will be easier to follow with type-checking enabled.

Next steps

We expect to provide further updates by the end of this week. Feel free to join our Office Hours, leave comments below, or reach out on Slack to let us know what information you'd like to see.

[adixon-adobe]

It sounds like the new sp-overlay element coming with this work should help make it easier to have a custom theme for the overlay content (by just having a parent theme for the sp-overlay element).

[Westbrook]

Progress Update

A Release Candidate for this work has been released as of July 12th, 2023. Starting with version 0.34.1-rc.0 you can test what should be the fully stable implementation of this API.

Learn more about leveraging these updates via:

• our draft migration guide
• docs for:
  • sp-overlay
  • the imperative API
  • updated usage of overlay-trigger

Please share any feedback from your experience leveraging this RC release early and often so we can continue towards our current goal of a July release of this API.

[Westbrook]

This has shipped!