Exception while executing ic3.jar

[luzhujiu]

Hi @JordanSamhi,

I'm using this ic3.jar to generate the ICC model for IccTA with cmd

java -jar ic3.jar -a myapp.apk -cp ~/Library/Android/sdk/platforms/ -db cc.properties

But I get an exception as follows:

[Call Graph] For information on where the call graph may be incomplete, use the verbose option to the cg phase.
[Spark] Pointer Assignment Graph in 0.1 seconds.
[Spark] Type masks in 0.0 seconds.
[Spark] Pointer Graph simplified in 0.0 seconds.
[Spark] Propagation in 6.5 seconds.
[Spark] Solution found in 6.5 seconds.
Exception in thread "main" java.lang.RuntimeException: Could not find method
at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:364)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:388)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClass(AnalyzeJimpleClass.java:320)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeMethodForCallbackRegistrations(AnalyzeJimpleClass.java:245)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeRechableMethods(AnalyzeJimpleClass.java:184)
at soot.jimple.infoflow.android.AnalyzeJimpleClass.access$200(AnalyzeJimpleClass.java:64)
at soot.jimple.infoflow.android.AnalyzeJimpleClass$1.internalTransform(AnalyzeJimpleClass.java:127)
at soot.SceneTransformer.transform(SceneTransformer.java:39)
at soot.Transform.apply(Transform.java:90)
at soot.ScenePack.internalApply(ScenePack.java:40)
at soot.Pack.apply(Pack.java:116)
at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:144)
at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
at edu.psu.cse.siis.ic3.Main.main(Main.java:13)

I tried two apps with SDK 29&30 and both end up with this exception. Do you have any insight into the problem here? Thanks!

[JordanSamhi]

Hi @dongluzhujiu,

Could you please provide the APKs for which it generates this exception, I will try to replicate the issue and debug it.

[luzhujiu]

Hi @JordanSamhi ,

this is my toy app: sampleapp.apk
I hope the thing I missed is not so stupid :P

[yokotayokota]

Hello @JordanSamhi ,

I had been looking for ic3 that works without Dare and I eventually found it in your site.
I express my appreciation for your work.

It worked fine with a simple apk made by my self, but
with some apks downloaded from Google Play it failed and left some errors.
As an example, the error with YouTube apk (v14.25.57) is at the end of this post *1.
The error with YouTube apk (v16.37.36) is at the end of this post *2.

Do you have any insight into these errors?

Best regards,

*1 the error with YouTube apk (v14.25.57)
----------------------------
[Spark] Solution found in 24.1 seconds.
Exception in thread "main" java.lang.RuntimeException: Could not find method
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:364)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:388)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:382)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClass(AnalyzeJimpleClass.java:320)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeMethodForCallbackRegistrations(AnalyzeJimpleClass.java:245)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeRechableMethods(AnalyzeJimpleClass.java:184)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.access$200(AnalyzeJimpleClass.java:64)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass$1.internalTransform(AnalyzeJimpleClass.java:127)
        at soot.SceneTransformer.transform(SceneTransformer.java:39)
        at soot.Transform.apply(Transform.java:90)
        at soot.ScenePack.internalApply(ScenePack.java:40)
        at soot.Pack.apply(Pack.java:116)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:144)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
----------------------------

*2 the error with YouTube apk (v16.37.36)
----------------------------
Warning: malformed Manifest file: action at depth 3
Error when looking for XML resource files in apk /mnt/c/Users/s.yokota/ApkProjects/YouTube_v16.37.36/APK/YouTube.apk: java.lang.RuntimeException: Unknown entry type
java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
Exception in thread "main" java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
----------------------------

[JordanSamhi]

Hi @yokotayokota,

Could you share the apk you used ?

[yokotayokota]

Hello @JordanSamhi ,

Thank you for your quick reply.
I'm trying to use FlowDroid with ICC (i.e. -im parameter) for vulnerability asessment of Android apps.

These are apks that left errors:

[YouTube_v16.37.36.apk.tar.gz]
(https://github.com/JordanSamhi/Tools/files/7244903/YouTube_v16.37.36.apk.tar.gz)

Sorry, I can not attach the other apk (YouTube_v14.25.57.apk.tar.gz). I do not know why.
If you do not mind, I will send it to you by e-mail directly

[yokotayokota]

Hello @JordanSamhi,

The reason why I could not attach the apk yesterday is that the size of it is too large.
I split it into 2 pieces
I am sorry to bother you but could you follow the instruction below:

1. $ tar zxvf 00_YouTube_v14.25.57.apk.tar.gz
2. $ tar zxvf 01_YouTube_v14.25.57.apk.tar.gz
3. $ cat YouTube_v14.25.57.apk_00 YouTube_v14.25.57.apk_01 > YouTube_v14.25.57.apk

00_YouTube_v14.25.57.apk.tar.gz
01_YouTube_v14.25.57.apk.tar.gz

[yokotayokota]

Hello @JordanSamhi,

I made a tiny app test_simple.apk.tar.gz
and ic3 left the same error as YouTube_v16.37.36.apk.
(i.e. *2 error of my previous comment )

I hope this tiny app will help your investigation.

ic3 error of this app
-----------------------------
$ java -jar ic3.jar  -a /mnt/c/Users/s.yokota/ApkProjects/test_simple/APK/app-debug.apk  -cp /mnt/c/Users/s.yokota/AppData/Local/Android/Sdk/platforms -protobuf /mnt/c/Users/s.yokota/ApkProjects/test_simple/ic3ed/
Error when looking for XML resource files in apk /mnt/c/Users/s.yokota/ApkProjects/test_simple/APK/app-debug.apk: java.lang.RuntimeException: Unknown entry type
java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
Exception in thread "main" java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)

[JordanSamhi]

Dear @yokotayokota,

I stumbled upon the same error, after investigating, I found that the problem seems to come from Flowdroid (which IC3 relies on), I cannot devote too much time on this, I am sorry. I encourage you to debug and try to find a temporary fix until it is officially fixed.

[yokotayokota]

Dear @JordanSamhi,

Thank you for your investigation.
Thank you for your time.

Could you tell me who should I ask for help?
Is there anyone who uses ICC static analysis of Android apps recently?

In addition, I would like you to show me how you built your ic3.jar and its sources.

I am willing to wait for your reply.
Please do not overdo.

[JordanSamhi]

Concerning Flowdroid, you can continue the discussion in secure-software-engineering/FlowDroid#386 and show the error you had with IC3 above, you can see that it comes from Flowdroid (soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)).

Nevertheless, I have tried with apps that I have from other xps, I did not have your problem and was able to run ic3, get the model and launch flowdroid with the -im option.

If you want ready-to-use tools and scripts to perform ICC, you can have a look to this repository : https://github.com/JordanSamhi/RAICC (go to artefacts folder).

To build ic3 :
git clone https://github.com/siis/ic3.git
cd ic3
mvn clean package -P standalone

[yokotayokota]

Dear @JordanSamhi,

Thank you very much for your suggestions.
As you say, I will post the errors to FlowDroid #386.
In addition, I will try your RAICC.

May I ask you how you got apks that you succeeded ic3 + FlowDroid -im with?
I am wondering if these apks are built on the old environment?
If so, FlowDroid does not support apks that are built on recent environment(e.g. my Android Studio is 4.1.1).

Best regards,

[JordanSamhi]

I must say that the apks I tested are old apks, you can find it here: https://github.com/secure-software-engineering/DroidBench/tree/iccta/apk

[yokotayokota]

Good morning @JordanSamhi,
(Is it morning there now?)

They are too old. I am sorry.

Could you please allow me to ask you one more question?
Does ic3 detect registerReceiver and write out to a model file?
If not, is the information in the model file that we get from ic3
essentially as the same as the information in AndroidManifest.xml?

Best regards,

[JordanSamhi]

To allow ic3 to write to a model file, you have to use the -protobuf PATH_TO_MODEL_FILE option.
You will not have the same information as there is in the AndroidManifest, since it has to do with data propagation and control flow.

[yokotayokota]

Thank you @JordanSamhi,

Now I really understand.
First of all, I need to look at the results using these old apks.
Secondly, I make sure that FlowDroid possibly doesn't fit the modern apks.