F3 speed at 50% of h2testw speed under Windows 7 SP1 64 bit #188
Comments
|
Try to adjust the parameter |
|
I am running and compiling F3 under linux. |
The write buffer passed to the operating system has previously been limited to MAX_BUFFER_SIZE. This commit makes it dynamically sized in order to reach maximum writing speeds. See details on issue #188.
|
Could you test pull request #190? I expect it to solve your problem. |
The write buffer passed to the operating system has previously been limited to MAX_BUFFER_SIZE. This commit makes it dynamically sized in order to reach maximum writing speeds. See details on issue #188.
|
I am running #190 using the same hardware/software setup as before. Now speed varies between 60 MByte/sec and 80 MByte/sec. The sound of the harddrive is not smooth yet unfortunately. |
|
You can try to comment everything in Notice that the implementations of NTFS in Linux and Windows are not identical, so that can also account for part of the difference. You can verify this possibility using another file system format. |
|
By commenting out everything in flush_chunk() and returning 0, the write speed more than doubled and varies between 150 and 170 MByte/sec. Is this version operating correctly despite having commented out the flush_chunk() function? |
|
Given your result, I think that The modified Anyway, your investigation has led to an improvement in the buffer that |
|
f3read has no speed issue: Validating file 3724.h2w ... 2097152/ 0/ 0/ 0 Data OK: 3.64 TB (7813657624 sectors) '=============================================== I will let f3write run till the end. Tomorrow I will invoke f3read to see how the failure to write the last bytes (kind of write hole) is reflected in the output of f3read. |
The write and read buffers passed to the operating system have previously been limited to MAX_BUFFER_SIZE. This commit makes them dynamically sized in order to reach maximum writing and reading speeds. See details on issue #188.
|
I updated |
|
I am testing the new f3read now. The previous f3read was already fast: Data OK: 3.64 TB (7813657624 sectors) I had run it to see if there were any errors shown as you said that the last bytes or buffers would not be written. The current test should be ready in 8h as the files written with f3write using a dynamic buffer and flushing disabled are still on my drive. |
|
Notice that the errors will only show up if you've removed the drive before the operating system had a chance to flush its cache onto the drive. If you ran My main point before was that |
|
There is no relevant change in speed by using dynamic buffers in f3read in my configuration: Data OK: 3.64 TB (7813657624 sectors) [... I think you mean to say: The operating system would not even need to read the drive since there is still a cached copy to return to f3read. Please let me know once the new version is out and I will retest it on another 4TB drive that I want to test before using it. |
|
I've merged pull request #190. |
|
I downloaded f3-master.zip from https://github.com/AltraMayor/f3 but there is no speed gain anymore when running f3write. The forced buffer flushing still seems to be active. Is that intended? |
|
Pull request #190 only includes the dynamic buffer. Your comment states that pull request #190 improves the write speed on your system. As I explained above, the write speeds that I ran As you can see, the average writing speed was 2.87 MB/s, and it took 5 minutes and 57 seconds to finish. If you multiply this speed rate and time, you obtain 1GB of written data. Then I prepared the environment to run And repeated my first measurement with an Notice that I added While the write speed of 540.37 MB/s is impressive, it is far from reality. Not only did the operation take 6 minutes and 40 seconds, but for this speed and time, |
I did not say so. Above I wrote: That was followed by: Thanks for preparing the long answer! I understand what you want to say. I wrote the answer as user "r2d3" to the following question: Regarding your example: 02:20:58
1,74e9 Byte / 400 sec =4,35 MByte/sec My assumption was that the latest average writing speed should be defined as (total bytes written) / (total time elapsed). Maybe I am using your program out of scope! My RAM is 4GB, the operating linux system resides on a 16 GB USB pen drive and the HDD has a size of 4 TB = 4000 GB. USB pen drives do not have any rotating parts so it seems to me that flushing caches do not degrade the write speed. |
In your opening post, you wrote "When I increased the buffer MAX_BUF from 2MB to 4 MB the write speed increased from <= 60 MByte/sec to between 60 and 70 MByte/sec." Therefore, I assumed that |
|
You are right. Compared to the initial situation #190 did improve write speed even in the synced environment, and it was less than 60 MB/s before. |
Thank you for helping others.
1GB = 2^30 = 1.074e9 not 1,74e9 1.074e9 / 400 = 2,685,000 B/s
I haven't measured it, but some of the time is the operating system pushing its buffers to the drive at command
What's confusing you is that |
Although flash drives do not have spinning discs, they have different write speeds for random and sequential writes analogous to hard discs. When a file is synced, the file system has to update the data on the disc and update the corresponding metadata of the file; the latter operation likely creates random accesses. So there may be a measurable degradation here. If you can make a valid argument for |
Using the same hardware configuration I am able to reach a write speed of roughly 120 MByte/sec under Windows 7 SP1 64 bit on a WD 4TB disk in a docking station (WD40EFZX) connected to one USB 3.0 port on a Thinkpad T440p.
f3write only yields 60 Mbyte/sec.
There is just one primary partition on the drive formated as NTFS.
As I am hearing a certain repeating pattern of head movement of the drive under Linuxmint 21 (Kernel 5.15, has Paragon driver support for NTFS) as opposed to the smooth continuous sound of the disk under Windows 7, I guessed it must be a buffer issue.
When I increased the buffer MAX_BUF from 2MB to 4 MB the write speed increased from <= 60 MByte/sec to between 60 and 70 MByte/sec. A further increase to 8 MB yielded a RAM access error.
What do i have to do to increase MAX_BUF to 16 MB or even 128 MB?
The text was updated successfully, but these errors were encountered: