Size limits ? Not all files are archived

[CouinCouin]

Hi :)

I'm near from achieve backup and restore feature, looks running well with Jingle Palette's sample files.
And I tried with my jingles set (folder size is 788 MB but not all files are used), and on restoring , wow, a lot of jingles missing.
Started to investigate, files are listed, but archive is corrupted, I can not read or extract any file in Windows explorer. Zip file size in 262 144 kB (in Size column of Explorer)
Tried with removing files to archive, zip is no long corrupted since its under 262 144 kB.

Tried with some files with the test/basic project to rid of an eventually wrong code in my project.

Here is the co, based on existing Command3_click :

Private Sub Command3_Click()
    Dim dblTimer        As Double
    Dim bResult         As Boolean
    Dim sLastError      As String
    Dim oOutput         As cMemoryStream
    Dim lLevel          As Long

    dblTimer = Timer
    Set m_oZip = New cZipArchive
    Set oOutput = New cMemoryStream
    m_bCancel = False
    lLevel = 8
    With m_oZip
        .AddFile pvInitMemStream("C:\TEMP\Ep.2 2017-06-04 (Export FB mono).mp3"), "Ep.2 2017-06-04 (Export FB mono).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.5 2017-06-25 (via E90).mp3"), "Ep.5 2017-06-25 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.7 2017-07-09 (via E90).mp3"), "Ep.7 2017-07-09 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.8 2017-07-16 (via E90).mp3"), "Ep.8 2017-07-16 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.10 2017-07-30 (via E90).mp3"), "Ep.10 2017-07-30 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.11 2017-08-13 (via E90).mp3"), "Ep.11 2017-08-13 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.12 2017-08-20 (via E90).mp3"), "Ep.12 2017-08-20 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.13 2017-08-27 (via E90).mp3"), "Ep.13 2017-08-27 (via E90).mp3"
        .AddFile pvInitMemStream("C:\TEMP\Ep.14 2017-09-03 (via E90).mp3"), "Ep.14 2017-09-03 (via E90).mp3"
        bResult = .CompressArchive(oOutput, Level:=lLevel)
        sLastError = .LastError
    End With
    Set m_oZip = Nothing
    WriteBinaryFile "C:\TEMP\Podcasts.zip", oOutput.Contents
    labProgress.Caption = IIf(bResult, "Done. ", sLastError & ". ") & Format(Timer - dblTimer, "0.000") & " elapsed"
End Sub

The result is that only the Ep.2 2017-06-04 (Export FB mono).mp3 is in the zip file.
About progress, I see that the other files are displayed for a short time than first , and percentage not goes to 100% (or near) for each file, but only around 30% max.

Some idea of what I do wrong ? (Of course, files are present and not bad).

Thanks :)
Couin

[wqweto]

This works here. Try getting the latest version of the whole repository (all files).

If still not working the you can ZIP your non-working project and attach it here so I can take a peek.

Btw, there is no reason to use memory streams for what you are doing. Just using plain filenames is more efficient in your case.

Instead of

.AddFile pvInitMemStream("C:\TEMP\Ep.5 2017-06-25 (via E90).mp3"), "Ep.5 2017-06-25 (via E90).mp3"

just call

.AddFile "C:\TEMP\Ep.5 2017-06-25 (via E90).mp3", "Ep.5 2017-06-25 (via E90).mp3"

or even shorter

.AddFile "C:\TEMP\Ep.5 2017-06-25 (via E90).mp3"

With CompressArchive call its the same -- no need to output to a memory stream at all, just call it with a string filename like this

bResult = .CompressArchive("C:\TEMP\Podcasts.zip", Level:=lLevel)

Don't fiddle with ReadBinaryFile, WriteBinaryFile and cMemoryStream unless you have good reason to use these i.e. the source of the data to be compressed is not in files but for instance coming from the internet request or you need the output of the compression not in a file but sent to a serial port for instance.

The use-case for in-memory operations (which is what you are using here) is very specific and compressing MP3 files is not one of these.

[CouinCouin]

Hi wqweto,

Thanks for help :)

I though that using memory was nrequiered to push the files to zip into a subfolder of my choice.

Just tested with changes you posted, it looks ok now , all files seems to be in the archive file, and this last is not corrupted.

(Also, mp3 files was just for testing with "big" files instead of several smalls).

Rest to implent creating subfolder when unzipping (in my project), but I think it will be also fine onece done.

Keep you informed :)

[CouinCouin]

Hi wqweto,

I also applied WriteBinaryFile on unzip script of my project, still look running well :)

Just noticed (but was the case before applying this changesà), that unziping is slower than ziping (Machine is i7-2600 + SSD).
Any idea or normal behavior ?

Thanks :)

[wqweto]

Extract must be faster than compressing in all cases.

Try using filenames directly instead of in-memory operations and WriteBinaryFile procedure.

[CouinCouin]

Oops I missed words in my previous message, I would say I applied WriteBinaryFile removal on unzip script :)

Here is the code I use :

Set m_oZip = New cZipArchive
m_bCancel = False
With m_oZip
.OpenArchive cmDlg.FileName
Open App.Path & "\FilesMap.ini" For Input As #F '(the file where is defined the original place of each file when built zip)
Do While Not EOF(1)
	'[...] Some code to define SourceFile (file in the archive) and destination file (where the archived file should be extracted)
	.Extract (destifile), SourceFile
End If
Loop
Close #1

It runs but it's slow (I find).

[wqweto]

It runs but it's slow (I find).

What do you compare it to?

You can decompress all files in a single call to Extract and compare speed.

You can extract your archive with 7-zip and compare time elapsed. ZipArchive should be at most twice as slow as 7-zip, not slower.

[CouinCouin]

I was comparing between compression (creating the zip file) and extraction (form the created zip file) with cZipArchive. Not with another archive software like 7-zip or winrar.
And in this test extract is slower than compress.

I will try to make a test with one extract call to compare (even if I will have to keep one extract per files as well as I extract the in folder depending of what I listed in the other thread (CSIDL etc etc).

[wqweto]

I see.

There is no point in comparing compression to decompression although generally ZIP decompression should always be faster.

It's best to compare compression to compression by other means/tools and decompression to decompression by other apps.

[CouinCouin]

Hi,

I revisited my code and look faster now :)

Thanks for your help !