Add a pair of flags forcing fsync during copy loop

Review Request #130084 - Created April 12, 2017 and updated

Information
KJ Tsanaktsidis
kio
master
6c5e6a8...
Reviewers
kdeframeworks
ossi, thiago

When copying a large-ish file (~1-2GB) from very fast storage to very slow storage (e.g. an NVME SSD to a cheap USB flash drive) on a machine with lots of RAM, Dolphin displays a progress bar which finishes in a fraction of a second (i.e. as fast as it takes to read the source file into the Linux page cache). Unmounting the drive then of course takes a long time, with only an indeterminate spinner.

This patch adds an option to force fsync during copy jobs, so that the copy progress bar measures how long it will take to actually copy the file to the destination.

I've added two flags - Fsync and FsyncCrossFilesystem - to the JobEnum flag. The former will cause all copy operations to fsync during the copy loop, whilst the latter will only fsync copies that are across different filesystems.

If this patch gets OK'd, I have another patch which adds support for this into the appropriate places in Dolphin. I would think that at least FsyncCrossFilesystem should be the default, but Fsync always might be a little heavy handed. At the least fsync'ing cross-filesystem copies ensures that the unmount won't take forever.

Tested the patch with KDE/Dolphin on Arch Linux, which is version 5.32.0. The diff applies cleanly to master so I assume there shouldn't be any issues there, but I've not actually checked that. As advertised, copying a file to USB flash storage now displays an accurate progress bar.

I experimented with how often fsync should be called on my hardware, and I found calling it every ~1M copied caused no decrease in copy performance whilst still providing accurate progress info. That is the setting I've gone with in this patch. I'm open to suggestions on how this could be tuned better though.

Issues

  • 5
  • 0
  • 0
  • 5
Description From Last Updated
typo Oswald Buddenhagen Oswald Buddenhagen
remove newline after "bool" and before function name (Qt/KF5 coding style) David Faure David Faure
&= doesn't do what you think, it's a bit-flag operation and not a boolean operation. 1=true 2=true 1&2=false You need ... David Faure David Faure
const auto... so that first() below doesn't detach. David Faure David Faure
const bool .. to make it clear it's not modified further down David Faure David Faure
KJ Tsanaktsidis
KJ Tsanaktsidis
David Faure
KJ Tsanaktsidis
Review request changed

Change Summary:

Removed the Job API changes; fsync is now always used when the target is on removeable media

Branch:

+master

Commit:

-07fed699d75cce72f8c8880d2c67848097e598d4
+6c5e6a8a06e7802533f8799b5a06e575b10beeef

Diff:

Revision 2 (+137 -1)

Show changes

Oswald Buddenhagen

   
src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

typo

src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

why are you testing hotpluggable? what's the exact definition in this context? technically, any sata drive is hotpluggable ...

src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

i'd use some parens here to make it more readable.

David Faure

   
src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

remove newline after "bool" and before function name
(Qt/KF5 coding style)

src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

&= doesn't do what you think, it's a bit-flag operation and not a boolean operation.

1=true
2=true
1&2=false

You need to use = hasMajorMinor &&, for lack of a &&= in the standard.

src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

const auto...

so that first() below doesn't detach.

src/ioslaves/file/file_unix.cpp (Diff revision 2)
 
 

const bool .. to make it clear it's not modified further down

Loading...