Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add file upload functionality #145

Closed
wants to merge 14 commits into from
Closed

Add file upload functionality #145

wants to merge 14 commits into from

Conversation

MahmoudHamdy02
Copy link
Collaborator

@MahmoudHamdy02 MahmoudHamdy02 commented Oct 7, 2023

Fixes #31
To do:

  • Upload multiple files
  • Upload large files (tested on 2.85G file)
  • Handle duplicate file names
  • Progress bar
  • Add cancel button
  • Hide upload option from localhost
  • Add error handling & remove temp file if error occurs
  • Add beforeunload event
  • Add tests - use b.upload_file()
  • Handle collisions (existing file name)
    • overwrite confirmation (for the download, edit locally, and upload to replace workflow)
    • a way to rename to a different file name (instead of overwriting)
  • Session inhibitor during uploads, to prevent timeouts
  • beforeunload handler during uploads, to prevent the page from being closed, reloaded, navigated away from, etc.

Copy link
Member

@martinpitt martinpitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Indeed this needs some work, please test it with large files. https://github.com/45Drives/cockpit-navigator/ does file uploads in chunks. It uses a spawn channel with .input() for that, instead of fsreplace -- the latter does not have a chunked API indeed, and is only meant for small files. That spawn based approach was recently fixed and properly supported in cockpit-project/cockpit#19325

src/header.jsx Outdated Show resolved Hide resolved
src/header.jsx Outdated Show resolved Hide resolved
src/header.jsx Outdated Show resolved Hide resolved
src/header.jsx Show resolved Hide resolved
@MahmoudHamdy02
Copy link
Collaborator Author

MahmoudHamdy02 commented Nov 2, 2023

Hi @garrett, I need some help with the design here :)

I've added a progress bar in the header, here is the initial implementation for now:
image

Where should it be located? Thanks!

Edit: Also, where should the cancel button be? :)

@garrett
Copy link
Member

garrett commented Nov 9, 2023

Do we want to only allow one upload at a time? (I see the disable upload button when uploading.) Because that would considerably change the how the UI would be handled if it supports multiple uploads or just one.

If multiple, we probably would want:

  • button always enabled
  • transitory progress indicator for all uploads, likely in a radial percentage manner, like a pie chart
  • progress at 100% would change to a done icon (probably a check in a circle?) and stay around for a little while
  • clicking the progress would show individual uploads and their own progress

This is similar to how GNOME Files handles long file operations (such as large files, network copying or moving, etc.):
GNOME Files mockups

In the actual implementation, they've dropped the number and show it like this:

image

image

It's similar to how both Firefox and now Chrome handle this too.

image

(Just like in Nautilus, there isn't a visible button unless you hover or click.)

We would probably want this upload indicator to go to the right of the download button (as it's conceptually tied to the upload action), after an upload is started. It'd cause a shift when it appears, but this is after uploading via a modal (or drag and drop).

@garrett
Copy link
Member

garrett commented Nov 9, 2023

If we do only want to allow only one upload, then we would replace the upload button with an indicator with the same width. It would change from

[ Upload ]

to

◕ Uploading...

Where ◕ is a circular progress indicator, and the button wouldn't be a button any longer. The downside (other than only supporting one upload at a time) is that this doesn't allow you to cancel the operation.

It would be similar to the secondary loading "progress" button, but with an actual state instead of just a spinner (so it would actually show progress):

image

@garrett
Copy link
Member

garrett commented Nov 9, 2023

The bad news for either implementation is that PatternFly does not provide all the widgets we need in either case.

There are donut utilization charts, but they're overkill for what we need. And if we went with the one file upload at a time method (as a first pass, I guess?) then we still wouldn't be able to properly implement it in the button. https://www.patternfly.org/charts/donut-utilization-chart

@garrett
Copy link
Member

garrett commented Nov 9, 2023

Meanwhile, I've made a custom HTML+CSS element that does what we want in this instance:

https://codepen.io/garrett/pen/oNmWxRp

image

It's all controlled by the --progress CSS custom property (aka: "CSS variable"), which can be set via HTML and/or JavaScript. We should have something that does a user-visible percentage string too, like that title. I think that makes it accessible to both screen readers and on hover. (But we'd probably want to use a PF tooltip instead of a title.)

So you'd want to use that element with that CSS. Set the colors to PF colors and adjust the --progress property (as shown in the JS side of things) and update the string (probably as a PF tooltip) to match the percentage.

I tried a number of different methods, including SVGs, using dash offset, and more... this seemed like the simplest way to implement it. The downside is as they're both at the same % (which is a string, BTW), they don't have anti-aliasing, so the edge looks a bit jagged when larger. It really is fine though at a smaller size... I've checked. The height @ 100% makes it fill up the vertical space. We could change that to a set height if we wanted... but I thought it would be most flexible like this, at least in this demo.

Copy link
Member

@garrett garrett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for working on this! I've added a few comments inline.

src/header.jsx Outdated Show resolved Hide resolved
Comment on lines +162 to +163
--progress-fill: grey;
--progress-empty: lightgrey;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to set these variables to PF colors! Thanks!

Instead of relying on dd and moving temporary files ourself the bridge
can already do this for us. This also uses the progress api to update
the progress bar and paves the way for multi file uploads.
@jelly
Copy link
Member

jelly commented Jan 15, 2024

I've rebased this PR and updated it to use the fsreplace1 channel instead of dd. This handles temporary files and moving for us. It also uses the progress event for the progress bar.

We can support multiple file uploads, the code in theory should already work except for the progress circle maybe.

This is how the progress circle looks like currently:

image

Some notable issues:

  • Upload completion clears the file list and after that it takes a while for the view to update. Say 0.5-1 second, long enough for a user to notice
  • Multiple file uploads seems to sometimes crash the bridge? Or take very very long for the files to show:
    image
  • We need to decide on the design, if we always show this circle it needs to be smaller, more clear that an complete was done and maybe even make it clickable. By using the progress API we now get progress per file upload (they are all separate so we should change the logic of the progress circle to show how many files are completed, except when one file is uploaded. Honestly I hardly see the circle progress on small / medium (100MB) uploads but this is all "local".

@jelly
Copy link
Member

jelly commented Jan 24, 2024

The plan is:

  • The upload button will reserve some more space to facilitate the pie chat progress indicator on both sides so the button is a bit bigger by default (with padding)
  • Disable the button when uploading.
  • When the file is being uploaded the progress will show in the pie chart in the button. (The text slides to the right and the button size overall does not change because we reserved space upfront).
  • When the upload is done it should show a check with a circle. ( CheckIcon and CheckCircleIcon ). The whole circle goes away after 2 seconds and the button goes to the initial state.

When the upload starts:

Add beforeUnloadNavigator

https://codepen.io/garrett/pen/ZEwJPdp (We want the middle one, not the pie chart)

let len = 0;
const content = readerEvent.target.result;
console.log(content);
len = content.byteLength;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://stackoverflow.com/a/25813769

This might read the whole file in memory.

@garrett
Copy link
Member

garrett commented Jan 30, 2024

I thought about this today when thinking of other icons in Navigator:

Something even easier is probably just using the upload icon (fa-upload / UploadIcon) by default, then swap to the spinner, then swap to the check when done, then swap back to the upload icon. We wouldn't have to special case padding, animation or anything else.

image

(We'd have to revisit when implementing multiple uploads though. But we'd have to anyway.)

@jelly
Copy link
Member

jelly commented Mar 8, 2024

Good news, I have a local branch with working upload with fsreplace1 chunked uploads and using the discussed upload button:

Screencast.from.2024-03-08.12-06-07.webm

@garrett I have two issues so far with implementing the discussed button:

  • We need to discuss the ability to cancel the upload, this should be possible to implement and potentially nice for big uploads.
  • Need some help to center the piechart progress (not urgent)

@allisonkarlitskaya we need to pick up our discussion on matrix:

the two big issues left to resolve, imho:

  • how to actually use this with the bridge (do we add 'ack' support there, do we enable receiving pongs in cockpit.js?)
  • making this bytes-based instead of frames-based

@jelly
Copy link
Member

jelly commented Mar 11, 2024

Good news, I have a local branch with working upload with fsreplace1 chunked uploads and using the discussed upload button:

Screencast.from.2024-03-08.12-06-07.webm
@garrett I have two issues so far with implementing the discussed button:

  • We need to discuss the ability to cancel the upload, this should be possible to implement and potentially nice for big uploads.
  • Need some help to center the piechart progress (not urgent)

@allisonkarlitskaya we need to pick up our discussion on matrix:

the two big issues left to resolve, imho:

  • how to actually use this with the bridge (do we add 'ack' support there, do we enable receiving pongs in cockpit.js?)
  • making this bytes-based instead of frames-based

Some updates from our discussion this morning:

  • @allisonkarlitskaya can we realistically support multiple uploads with multiple fsreplace1 channels?
  • If we are to go with show detailed information show it in a popover.

@allisonkarlitskaya
Copy link
Member

@allisonkarlitskaya can we realistically support multiple uploads with multiple fsreplace1 channels?

Yes. Why not?

@jelly
Copy link
Member

jelly commented Mar 11, 2024

@allisonkarlitskaya can we realistically support multiple uploads with multiple fsreplace1 channels?

Yes. Why not?

I was thinking that this might lead to some blocking / a lot of pings? Performance issues?

@garrett
Copy link
Member

garrett commented Mar 11, 2024

If we're thinking of adding cancelling, we should commit to the fully featured upload widget, as we'd have to completely change functionality of the button (simple would have to have click for canceling, but then it'd be repurposed to the near-opposite of adding another upload... so that would be confusing and would break expectations).

Here's a design for the full version. Let's see if the scope creep between this PR and the full thing would be too much or not to implement it all in one go.

Firstly, this assumes the default current mockup of:

Navigator, with inline breadcrumbs (6)

Note: There is no icon for upload in this version, as it would not need a placeholder as it wouldn't get a spinner inside the button.

Navigator, with inline breadcrumbs (7)

Uploading a file would cause a spinner to appear after a short while (to prevent uploading a tiny file to cause page shift... so probably like > 1s... we'll have to figure out what "feels" right). When clicking on it, you'd get a popover (https://www.patternfly.org/components/popover/) with the filename, progress, and remove actions.

They're trash icons for remove, so they're not confused with the typical (X) icon to close a popover.

Each entry has a border at the top to separate it from previous or the header. This makes it clearer which item the trash acts upon.

Unfortunately, this would cause a shift in the page for the filter and view actions. I don't think that can be helped, unless we swap the order of upload and the view. However, the convention is to have view first, then actions on the right. And the ⋮ menu is always on the right as part of the actions group, so this design is correct, even if it causes a shift in the view. But this is also why there's a small delay and we would only show the status for long-running uploads.

Additionally, the spinner is to the left of the upload button instead of the right, as we wouldn't want to shift the upload button when someone is uploading multiple files, as this could cause a misclick.

When the file is finished uploading, it should show up in the icon or details view with a yellow new highlight that fades out (our typical new animation). Additionally, when the file is added, the view should not scroll (this is probably the default anyway; I'm just clarifying it here).

We'd additionally want to support drag and drop, but that should be active on at least the view area (but probably actually on the full page) with an event handler and a custom overlay thing which has a scrim (either to dim the page or to hide it) and a message about it being a drop target. But I'd consider drag and drop to be fine for a follow-up, even if we're doing the upload implementation and widgetry in a "full" state in this PR.

@garrett
Copy link
Member

garrett commented Mar 11, 2024

Here's the current version of the Penpot file (in a ZIP, due to GitHub upload restrictions). Upload is on the upload page. (There are old mockups and new mockups in the file, so tread lightly if you're using this and don't expect any of the mockups to be final.)

Navigator.penpot.zip

@jelly
Copy link
Member

jelly commented Mar 12, 2024

Idea from this morning, if a user has IdleTimeout enabled we should probably use a session inhibitor so that uploads don't logout the users session. https://www.freedesktop.org/software/systemd/man/latest/org.freedesktop.login1.html

@garrett
Copy link
Member

garrett commented Mar 12, 2024

@jelly: Additionally, while uploads are happening, we should block tab/window closing using beforeunload https://developer.mozilla.org/en-US/docs/Web/API/Window/beforeunload_event (which should be reset when all of the the uploads are complete)

@garrett
Copy link
Member

garrett commented Apr 17, 2024

Latest snapshot of the Penpot document, BTW:
Files.zip

(There's a possibility that there are Penpot 2.0 related issues still; I had a few buttons that said "Secondary" instead of the actual text, for example. I think I fixed it all.)

@jelly
Copy link
Member

jelly commented May 22, 2024

Superseeded by #384

@jelly jelly closed this May 22, 2024
@jelly jelly deleted the file-upload branch May 22, 2024 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

File upload action from local to remote server
5 participants