-
Notifications
You must be signed in to change notification settings - Fork 0
Homework 3 - [LZW]
#3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
e1783eb
Homework 3 - LZW (wip)
ilya-krivtsov 3bce492
Added tests project (hw3 - lzw) (wip)
ilya-krivtsov fc6d5c4
Fixed writer and reader (hw3 - lzw) (wip)
ilya-krivtsov a797044
Fixed InternalsVisibleTo (hw3 - lzw) (wip)
ilya-krivtsov 686e43b
Added tests for writer and reader (hw3 - lzw) (wip)
ilya-krivtsov 3cccd8a
Added BWT (hw3 - lzw) (wip)
ilya-krivtsov 44a95c1
Switched writer and reader to uint (hw3 - lzw) (wip)
ilya-krivtsov f80bcc9
Added leaveOpen option and Flush() to writer (hw3 - lzw) (wip)
ilya-krivtsov e762911
Removed unnecessary casting to Span in Writer.Flush() (hw3 - lzw) (wip)
ilya-krivtsov 17fe1fa
Replaced stackalloc with ArrayPool (hw3 - lzw) (wip)
ilya-krivtsov 89a2749
Fixed sorting error in BWT.ForwardTransform() (hw3 - lzw) (wip)
ilya-krivtsov ae7ecc6
Moved random byte sequence generation for tests to its own class (hw3…
ilya-krivtsov 87ef171
Added trie (hw3 - lzw) (wip)
ilya-krivtsov ae3e24f
Rephrased api docs for Reader.ReadNext() for consistency (hw3 - lzw) …
ilya-krivtsov aa38e51
Fixed api docs for Reader.ReadNext() (hw3 - lzw) (wip)
ilya-krivtsov 80bb517
Removed unnecessary features in trie (hw3 - lzw) (wip)
ilya-krivtsov 5ef617b
Converted trie to dictionary (hw3 - lzw) (wip)
ilya-krivtsov 33afe78
Switched trie to store uint as value intstead of int (hw3 - lzw) (wip)
ilya-krivtsov d040fa1
Introduced constants for min and max width of numbers in writer and r…
ilya-krivtsov ccf6ce2
Added checks for stream read-/writeabiltiy to writer and reader (hw3 …
ilya-krivtsov 3b73ae8
Added more tests for writer and reader (hw3 - lzw) (wip)
ilya-krivtsov fc3f02d
Use ArrayPool in writer (hw3 - lzw) (wip)
ilya-krivtsov 0c4c56f
Use local ArrayPool instead of shared in BWT (hw3 - lzw) (wip)
ilya-krivtsov 1ce4bec
Switched trie to use generic type as value (hw3 - lzw) (wip)
ilya-krivtsov 8cb223b
Switched Trie to be internal instead of public (hw3 - lzw) (wip)
ilya-krivtsov 20a8e95
Removed Dispose(bool) from writer (hw3 - lzw) (wip)
ilya-krivtsov ce7a127
Changed trie to use char-by-char mode (hw3 - lzw) (wip)
ilya-krivtsov 0e34d48
Removed unnecessary test data from trie tests (hw3 - lzw) (wip)
ilya-krivtsov 7391840
Switched writer and reader back to int (hw3 - lzw) (wip)
ilya-krivtsov cfc13a1
LZW implementation (hw3 - lzw)
ilya-krivtsov 5ae229c
Switch input parameter in BWT.ForwardTransform to Span (hw3 - lzw)
ilya-krivtsov 3ec0ab3
Switch input parameter in forward and inverse transform in BWT to be …
ilya-krivtsov 032cc48
Disable debug logging in LZWWriter (hw3 - lzw)
ilya-krivtsov ca5ed63
Use non-nullable int in bit writer for good code coverage (hw3 - lzw)
ilya-krivtsov 8d99900
Updated api docs in LZWStream (hw3 - lzw)
ilya-krivtsov 4de2514
Swapped Read() and Write() in LZWStream for consistency (hw3 - lzw)
ilya-krivtsov 2a0d6ec
Made ZipperMode required parameter in LZWStream constructor (hw3 - lzw)
ilya-krivtsov 027102c
Updated 'uncompress' -> 'decompress' in api docs for consistency (hw3…
ilya-krivtsov 6fb71ea
Added checks for stream read-/writeabiltiy to LZW reader and writer (…
ilya-krivtsov 53ded80
Added BlockType.Flush and changed LZWStream.Flush behavior (hw3 - lzw)
ilya-krivtsov 16267da
Added tests for LZWStream (hw3 - lzw)
ilya-krivtsov e235467
Fixed api docs in LZWStream (hw3 - lzw)
ilya-krivtsov ef78f30
Added BWTStream and BWTMode (hw3 - lzw)
ilya-krivtsov 42b16d8
Overhauled LZWStream tests (hw3 - lzw)
ilya-krivtsov f0975ee
Added tests for BWTStream (hw3 - lzw)
ilya-krivtsov 0b63462
Added some checks in BWTStream (hw3 - lzw)
ilya-krivtsov 2e449db
Added image data as test source (hw3 - lzw)
ilya-krivtsov 907a0c5
Made BWTStream and BWTMode internal (hw3 - lzw)
ilya-krivtsov 7d8470f
Write block size as int instead of ushort (hw3 - lzw)
ilya-krivtsov 07240dd
Changed block sizes (hw3 - lzw)
ilya-krivtsov e21b964
Added ZipperStream and tests for it (hw3 - lzw)
ilya-krivtsov 2005b62
Added cli for zipper (hw3 - lzw)
ilya-krivtsov 20e49bf
Changed BWT to allow output to be longer than input (hw3 - lzw)
ilya-krivtsov aad99c1
Changed BWT.ForwardTransform to return positive index and added check…
ilya-krivtsov fb95049
Updated tests for BWT (hw3 - lzw)
ilya-krivtsov c74bca3
Made LZWStream internal (hw3 - lzw)
ilya-krivtsov 34215eb
Merge branch 'main' into hw-3-lzw
ilya-krivtsov 9d98eba
Merge branch 'hw-3-lzw' of https://github.com/ilya-krivtsov/ProgHomew…
ilya-krivtsov c8f6aed
Added BWT switch to ZipperStream.cs (hw3 - lzw)
ilya-krivtsov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| namespace Zipper.Cli; | ||
|
|
||
| using System.Buffers; | ||
| using System.Diagnostics; | ||
|
|
||
| /// <summary> | ||
| /// Provides methods and properties used to compress and decompress files. | ||
| /// </summary> | ||
| internal class FileZipper : IDisposable | ||
| { | ||
| private const int BufferSize = 512 * 1024; | ||
| private static readonly ArrayPool<byte> BufferPool = ArrayPool<byte>.Create(); | ||
|
|
||
| private readonly string? outputFileName; | ||
| private readonly string? outputFileNameTempA; | ||
| private readonly string? outputFileNameTempB; | ||
|
|
||
| private readonly Stream readFrom; | ||
| private readonly Stream writeTo; | ||
| private readonly Stream? writeToAlt; | ||
| private readonly long inputFileSize; | ||
| private readonly byte[] buffer; | ||
|
|
||
| private long bytesReadFromInput; | ||
|
|
||
| /// <summary> | ||
| /// Initializes a new instance of the <see cref="FileZipper"/> class. | ||
| /// </summary> | ||
| /// <param name="mode">Mode to use.</param> | ||
| /// <param name="inputFilePath">File to read data from.</param> | ||
| /// <param name="outputFilePath">File to write compressed/decompressed data to.</param> | ||
| public FileZipper(ZipperMode mode, string inputFilePath, string outputFilePath) | ||
| { | ||
| inputFileSize = new FileInfo(inputFilePath).Length; | ||
|
|
||
| var inputFile = File.OpenRead(inputFilePath); | ||
|
|
||
| if (mode == ZipperMode.Compress) | ||
| { | ||
| outputFileName = outputFilePath; | ||
| outputFileNameTempA = Path.GetTempFileName(); | ||
| outputFileNameTempB = Path.GetTempFileName(); | ||
|
|
||
| var outputFileA = File.Create(outputFileNameTempA); | ||
| var outputFileB = File.Create(outputFileNameTempB); | ||
|
|
||
| readFrom = inputFile; | ||
| writeTo = new ZipperStream(outputFileA, ZipperStream.MaxBlockSize, mode); | ||
| writeToAlt = new ZipperStream(outputFileB, ZipperStream.MaxBlockSize, mode, useBwt: true); | ||
| } | ||
| else | ||
| { | ||
| var outputFile = File.Create(outputFilePath); | ||
| readFrom = new ZipperStream(inputFile, ZipperStream.MaxBlockSize, mode); | ||
| writeTo = outputFile; | ||
| } | ||
|
|
||
| bytesReadFromInput = 0; | ||
| buffer = BufferPool.Rent(BufferSize); | ||
|
|
||
| EndOfFile = false; | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Gets progress as value between 0 and 1. | ||
| /// </summary> | ||
| public float Progress => (float)bytesReadFromInput / inputFileSize; | ||
|
|
||
| /// <summary> | ||
| /// Gets a value indicating whether end of file was reached. | ||
| /// </summary> | ||
| public bool EndOfFile { get; private set; } | ||
|
|
||
| /// <summary> | ||
| /// Compresses or decompresses part of input file. | ||
| /// </summary> | ||
| public void ReadAndWriteSingleBuffer() | ||
| { | ||
| int bytesRead = readFrom.Read(buffer, 0, BufferSize); | ||
|
|
||
| if (bytesRead == 0) | ||
| { | ||
| EndOfFile = true; | ||
| return; | ||
| } | ||
|
|
||
| bytesReadFromInput += bytesRead; | ||
|
|
||
| writeTo.Write(buffer, 0, bytesRead); | ||
| writeToAlt?.Write(buffer, 0, bytesRead); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Disposes all used files. | ||
| /// </summary> | ||
| public void Dispose() | ||
| { | ||
| BufferPool.Return(buffer); | ||
|
|
||
| readFrom.Dispose(); | ||
| writeTo.Dispose(); | ||
| writeToAlt?.Dispose(); | ||
|
|
||
| if (outputFileName != null) | ||
| { | ||
| Debug.Assert(outputFileNameTempA != null, $"{nameof(outputFileNameTempA)} is null"); | ||
| Debug.Assert(outputFileNameTempB != null, $"{nameof(outputFileNameTempB)} is null"); | ||
|
|
||
| var tempLengthA = new FileInfo(outputFileNameTempA).Length; | ||
| var tempLengthB = new FileInfo(outputFileNameTempB).Length; | ||
|
|
||
| if (tempLengthA < tempLengthB) | ||
| { | ||
| File.Move(outputFileNameTempA, outputFileName, true); | ||
| } | ||
| else | ||
| { | ||
| File.Move(outputFileNameTempB, outputFileName, true); | ||
| } | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| using System.Diagnostics; | ||
| using Zipper; | ||
| using Zipper.Cli; | ||
|
|
||
| const string helpMessage = | ||
| """ | ||
| Zipper - console tool for compressing and decompressing files | ||
|
|
||
| Usage: dotnet run -- <file> [options] | ||
| Options: | ||
| -h -? --help | Print this help message | ||
| ------------------------------------------------ | ||
| -c --compress | Compress specified file | ||
| ------------------------------------------------ | ||
| -u --uncompress | Decompress | ||
| -d --decompress | specified file | ||
| ------------------------------------------------ | ||
| -f --force | Overwrite files without asking | ||
|
|
||
| File path should be the first argument (unless --help specified) | ||
| Options can be specified in any order | ||
| Only either '--compress' or '--decompress' can be used at the same time | ||
| """; | ||
|
|
||
| args = [.. args.Select(x => x.Trim())]; | ||
|
|
||
| if (args.Length == 0 || (args.Length == 1 && args[0] is "-h" or "--help" or "-?")) | ||
| { | ||
| Console.WriteLine(helpMessage); | ||
|
|
||
| return 0; | ||
| } | ||
|
|
||
| string filePath = args[0]; | ||
| bool force = false; | ||
| ZipperMode? mode = null; | ||
|
|
||
| foreach (var arg in args.Skip(1)) | ||
| { | ||
| switch (arg) | ||
| { | ||
| case "-u" or "-d" or "--uncompress" or "--decompress": | ||
| if (mode != null) | ||
| { | ||
| Console.Error.WriteLine("Error: '--compress' or '--decompress' option can only be specified once"); | ||
| return 1; | ||
| } | ||
|
|
||
| mode = ZipperMode.Decompress; | ||
| break; | ||
|
|
||
| case "-c" or "--compress": | ||
| if (mode != null) | ||
| { | ||
| Console.Error.WriteLine("Error: '--compress' or '--decompress' option can only be specified once"); | ||
| return 1; | ||
| } | ||
|
|
||
| mode = ZipperMode.Compress; | ||
| break; | ||
|
|
||
| case "-f" or "--force": | ||
| force = true; | ||
| break; | ||
|
|
||
| default: | ||
| Console.Error.WriteLine("Error: unknown argument"); | ||
| return 1; | ||
| } | ||
| } | ||
|
|
||
| if (mode == null) | ||
| { | ||
| Console.Error.WriteLine("Error: neither '--compress' nor '--decompress' were specified"); | ||
| return 1; | ||
| } | ||
|
|
||
| if (!File.Exists(filePath)) | ||
| { | ||
| Console.Error.WriteLine($"Error: file '{filePath}' does not exist"); | ||
| return 1; | ||
| } | ||
|
|
||
| const string zippedExtension = ".zipped"; | ||
| string? newFilePath = null; | ||
| if (mode == ZipperMode.Compress) | ||
| { | ||
| newFilePath = $"{filePath}{zippedExtension}"; | ||
| } | ||
| else | ||
| { | ||
| if (!filePath.EndsWith(zippedExtension)) | ||
| { | ||
| Console.Error.WriteLine($"Error: extension of the specified file is not {zippedExtension}"); | ||
| return 1; | ||
| } | ||
|
|
||
| newFilePath = filePath[..^zippedExtension.Length]; | ||
| } | ||
|
|
||
| if (!force && File.Exists(newFilePath)) | ||
| { | ||
| Console.Write($"File '{newFilePath}' already exists, overwrite? (y/n): "); | ||
| if (Console.ReadLine()?.Trim() != "y") | ||
| { | ||
| Console.WriteLine("Cancelled"); | ||
| return 0; | ||
| } | ||
| } | ||
|
|
||
| const string hideCursorEscape = "\e[?25l"; | ||
| const string showCursorEscape = "\e[?25h"; | ||
| const string moveToLeftEscape = "\e[0G"; | ||
| const string clearLineEscape = "\e[2K"; | ||
| const string waitingSymblols = @"|/-\"; | ||
|
|
||
| Console.Write(hideCursorEscape); | ||
|
|
||
| using (var fileZipper = new FileZipper(mode.Value, filePath, newFilePath)) | ||
| { | ||
| var stopwatch = Stopwatch.StartNew(); | ||
| var lastLoggedTime = stopwatch.Elapsed; | ||
| int step = 0; | ||
| while (!fileZipper.EndOfFile) | ||
| { | ||
| fileZipper.ReadAndWriteSingleBuffer(); | ||
|
|
||
| if (stopwatch.Elapsed - lastLoggedTime > TimeSpan.FromMilliseconds(4)) | ||
| { | ||
| Console.Write(moveToLeftEscape); | ||
| RenderProgress(fileZipper.Progress, stopwatch.Elapsed, step, waitingSymblols); | ||
| lastLoggedTime = stopwatch.Elapsed; | ||
| } | ||
|
|
||
| step += 1; | ||
| } | ||
| } | ||
|
|
||
| Console.Write(clearLineEscape); | ||
| Console.Write(moveToLeftEscape); | ||
| Console.Write(showCursorEscape); | ||
|
|
||
| if (mode == ZipperMode.Compress) | ||
| { | ||
| var inputFileSize = new FileInfo(filePath).Length; | ||
| var outputFileSize = new FileInfo(newFilePath).Length; | ||
| var compressionRate = (float)inputFileSize / outputFileSize; | ||
|
|
||
| Console.WriteLine($"Compression rate: {compressionRate}"); | ||
| } | ||
|
|
||
| return 0; | ||
|
|
||
| static void RenderProgress(float progress, TimeSpan time, int step, string stepString) | ||
| { | ||
| Console.Write($" {stepString[step % stepString.Length]} "); | ||
| Console.Write("["); | ||
|
|
||
| for (int i = 0; i <= 100; i++) | ||
| { | ||
| Console.Write(progress >= i / 100f ? '=' : ' '); | ||
| } | ||
|
|
||
| Console.Write("]"); | ||
| Console.Write($" {progress * 100,5:0.0} %"); | ||
|
|
||
| if (time.TotalMinutes < 1) | ||
| { | ||
| Console.Write($" {time.Seconds} s"); | ||
| } | ||
| else if (time.TotalHours < 1) | ||
| { | ||
| Console.Write($" {time.Minutes} m {time.Seconds:00} s"); | ||
| } | ||
| else | ||
| { | ||
| Console.Write($" {time.Hours} h {time.Minutes:00} m {time.Seconds:00} s"); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>net9.0</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="../Zipper/Zipper.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.