Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JarReader #16

Open
wants to merge 8 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions src/main/java/net/fabricmc/mappingio/MappingReader.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import java.util.List;

import net.fabricmc.mappingio.format.EnigmaReader;
import net.fabricmc.mappingio.format.JarReader;
import net.fabricmc.mappingio.format.MappingFormat;
import net.fabricmc.mappingio.format.ProGuardReader;
import net.fabricmc.mappingio.format.SrgReader;
Expand Down Expand Up @@ -69,6 +70,8 @@ public static MappingFormat detectFormat(Reader reader) throws IOException {
case "MD:":
case "FD:":
return MappingFormat.SRG;
case "PK\u0003":
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correct when the jar is read as if it's utf-8?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This correctly identifies jar files when using something like

MappingReader.read(Path.of("minecraft-1.18.2-client.jar"), mappingTree);

return MappingFormat.JAR;
}

String headerStr = String.valueOf(buffer, 0, pos);
Expand Down Expand Up @@ -141,8 +144,12 @@ public static void read(Path file, MappingFormat format, MappingVisitor visitor)
}

if (format.hasSingleFile()) {
try (Reader reader = Files.newBufferedReader(file)) {
read(reader, format, visitor);
if (format == MappingFormat.JAR) {
JarReader.read(file, visitor);
} else {
try (Reader reader = Files.newBufferedReader(file)) {
read(reader, format, visitor);
}
}
} else {
switch (format) {
Expand Down
170 changes: 170 additions & 0 deletions src/main/java/net/fabricmc/mappingio/format/JarReader.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
/*
* Copyright (c) 2021 FabricMC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package net.fabricmc.mappingio.format;

import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.file.FileSystem;
import java.nio.file.FileSystemAlreadyExistsException;
import java.nio.file.FileSystems;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Locale;

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.FieldVisitor;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;

import net.fabricmc.mappingio.MappingVisitor;

public class JarReader {
public static void read(Path path, MappingVisitor mappingVisitor) throws IOException {
mappingVisitor.visitNamespaces("jar", new ArrayList<String>());
processFile(path, null, new AnalyzingVisitor(mappingVisitor));
}

private static final class DirVisitor extends SimpleFileVisitor<Path> {
DirVisitor(AnalyzingVisitor visitor) {
this.visitor = visitor;
}

@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
buffer = processFile(file, buffer, visitor);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In jars, some directories aren't valid package names, like ones with -, also the META_INF and the version-specific class files.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processFile ignores any files not ending with ".jar" or ".class", is that enough to skip non valid files?

Copy link

@liach liach Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't handle say classes in doc-files or multi-release classes in META-INF. Checking and skipping invalid directories may be needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't all classes be processed regardless of the directory they're in? processClass only uses the file content, not the file name or directory.

Copy link

@liach liach Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this isn't determinate for Multi-release jars, also doc files etc should be ignored

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, are there any directories besides "META-INF" and "doc-files" that need to be filtered out?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend skipping any class in any folder with - in name, as those are invalid java package names. However, I don't know how you want to deal with nested jars, and you might still consider jars in those directories.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It now skips any classes in folders with names containing "-". Jars are not affected by this but they should probably be skipped if they are in "doc-files" folders.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jars in "doc-files" folders are now skipped.


return FileVisitResult.CONTINUE;
}

private final AnalyzingVisitor visitor;
ByteBuffer buffer;
}

@SuppressWarnings("resource")
private static ByteBuffer processFile(Path file, ByteBuffer buffer, AnalyzingVisitor visitor) throws IOException {
String fileName = file.getFileName().toString().toLowerCase(Locale.ENGLISH);

if (fileName.endsWith(".jar")) {
URI uri = file.toUri();

try {
uri = new URI("jar:".concat(uri.getScheme()), uri.getHost(), uri.getPath(), uri.getFragment());
} catch (URISyntaxException e) {
throw new IOException(e);
}

FileSystem fs = null;
boolean closeFs = false;

try {
try {
fs = FileSystems.newFileSystem(uri, Collections.emptyMap());
closeFs = true;
} catch (FileSystemAlreadyExistsException e) {
fs = FileSystems.getFileSystem(uri);
}

DirVisitor dirVisitor = new DirVisitor(visitor);

for (Path rootDir : fs.getRootDirectories()) {
Files.walkFileTree(rootDir, dirVisitor);
}

buffer = dirVisitor.buffer;
} finally {
if (closeFs) fs.close();
}
} else if (fileName.endsWith(".class")) {
try (SeekableByteChannel channel = Files.newByteChannel(file)) {
if (buffer == null) buffer = ByteBuffer.allocate((int) Math.min(channel.size() + 1, 100_000_000));

while (channel.read(buffer) >= 0) {
if (!buffer.hasRemaining()) {
ByteBuffer newBuffer = ByteBuffer.allocate(buffer.capacity() * 2);
buffer.flip();
newBuffer.put(buffer);
buffer = newBuffer;
}
}
}

buffer.flip();
processClass(buffer.array(), buffer.arrayOffset() + buffer.position(), buffer.remaining(), visitor);
buffer.clear();
}

return buffer;
}

private static void processClass(byte[] classBytes, int offset, int length, AnalyzingVisitor visitor) {
ClassReader reader = new ClassReader(classBytes, offset, length);
reader.accept(visitor, ClassReader.SKIP_CODE | ClassReader.SKIP_DEBUG);
}

private static final class AnalyzingVisitor extends ClassVisitor {
AnalyzingVisitor(MappingVisitor mappingVisitor) {
super(Integer.getInteger("mappingIo.asmApiVersion", Opcodes.ASM9));

this.mappingVisitor = mappingVisitor;
}

@Override
public void visit(int version, int access, String name, String signature, String superName, String[] interfaces) {
try {
mappingVisitor.visitClass(name);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

@Override
public FieldVisitor visitField(int access, String name, String descriptor, String signature, Object value) {
try {
mappingVisitor.visitField(name, descriptor);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

return null;
}

@Override
public MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) {
try {
mappingVisitor.visitMethod(name, descriptor);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

return null;
}

private final MappingVisitor mappingVisitor;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ public enum MappingFormat {
SRG("SRG", "srg", false, false, false, false, false),
TSRG("TSRG", "tsrg", false, false, false, false, false),
TSRG2("TSRG2", "tsrg", true, false, false, true, false),
PROGUARD("ProGuard", "map", false, true, false, false, false);
PROGUARD("ProGuard", "map", false, true, false, false, false),
JAR("JAR", "jar", false, true, false, true, true);

MappingFormat(String name, String fileExt,
boolean hasNamespaces, boolean hasFieldDescriptors,
Expand Down