Ghidra headless analysis failed

[Qynklee]

I use IDA pro 7.5, Ghidra 9.2.2 lastest, python 3.7.9
I'm trying to test decompile function in IDA with GhidraDec but not working any function :(

XML Exporter v5.0.1 : SDK 750
-----------------------------------------------------------
Exporting XML <PROGRAM> document ....
Processing PROGRAM                 CPU time: 0.0010
Processing DATATYPES               CPU time: 0.0010
Processing MEMORY_MAP              CPU time: 0.0560
Processing REGISTER_VALUES         CPU time: 0.0020
Processing CODE                    CPU time: 0.0020
Processing DATA                    CPU time: 0.0620
Processing COMMENTS                CPU time: 0.0050
Processing PROGRAM_ENTRY_POINTS    CPU time: 0.0000
Processing SYMBOL_TABLE            CPU time: 0.0030
Processing FUNCTIONS               CPU time: 0.0050
Processing MARKUP                  CPU time: 0.0080
                             Total CPU time: 0.6618
--------------------------------------
PROGRAM                           1
INFO_SOURCE                       1
PROCESSOR                         1
COMPILER                          1
DATATYPES                         1
STRUCTURE                         4
MEMBER                           13
MEMORY_MAP                        1
MEMORY_SECTION                    8
MEMORY_CONTENTS                   8
REGISTER_VALUES                   1
REGISTER_VALUE_RANGE             24
CODE                              1
CODE_BLOCK                       34
DATA                              1
DEFINED_DATA                    103
TYPEINFO_CMT                     92
COMMENTS                          1
COMMENT                          41
PROGRAM_ENTRY_POINTS              1
PROGRAM_ENTRY_POINT               1
SYMBOL_TABLE                      1
SYMBOL                           53
FUNCTIONS                         1
FUNCTION                         30
ADDRESS_RANGE                    30
STACK_FRAME                      16
STACK_VAR                         5
MARKUP                            1
MEMORY_REFERENCE                  1
--------------------------------------
Total XML Elements:             477
Database exported to: D:\C_Data\Desktop\TestJS\b47183de13c96177deeeb1983faf4172_OGHFm.xml
GhIDA:: [INFO] XML exporting completed
GhIDA:: [INFO] Ghidra headless (timeout: 300s)
GhIDA:: [INFO] Waiting Ghidra headless analysis to finish...
GhIDA:: [INFO] Ghidra analysis completed!
GhIDA:: [!] Expecting value: line 1 column 1 (char 0)
GhIDA:: [!] Ghidra headless analysis failed
GhIDA:: [!] Decompilation interrupted.

[Qynklee]

I dont know why but I downgrade Ghidra to 9.0.4. It will work :v But cant not sync code between IDA and Decompiled Function although I enabled it.
I think plugins need some fixs for lastest Ghidra.
Thanks you :D

[paxcut]

I have the same problem using Ida 7.5, regardless of Ghidra version used. I tried 9.0.4, 9.1.2 and 9.2.2 and I always get the same error (see below). I think it is an issue with Ida 7.5 and python3 (i use 3.8) because Ida 7.3 with python 2.7 don't have this problem. I looked for the error (Expecting value ...) but I couldn't see where the line originates and I looked at all the python code, the xml exporter and importer the loader and idaxml.py. I would appreciate it if somebody could help me figure out how to debug this issue because I am stumped.
Thanks

XML Exporter v5.0.1 : SDK 750

Exporting XML document ....
Processing PROGRAM CPU time: 0.0030
Processing DATATYPES CPU time: 1.8380
Processing MEMORY_MAP CPU time: 27.6796
Processing REGISTER_VALUES CPU time: 0.0030
Processing CODE CPU time: 13.3150
Processing DATA CPU time: 32.5685
Processing COMMENTS CPU time: 23.3866
Processing PROGRAM_ENTRY_POINTS CPU time: 0.0500
Processing SYMBOL_TABLE CPU time: 24.1321
Processing FUNCTIONS CPU time: 221.3125
Processing MARKUP CPU time: 119.3244
Total CPU time: 464.3966

PROGRAM 1
INFO_SOURCE 1
PROCESSOR 1
COMPILER 1
DATATYPES 1
STRUCTURE 3334
MEMBER 10323
UNION 80
REPEATABLE_CMT 5
ENUM 521
ENUM_ENTRY 6478
DISPLAY_SETTINGS 18
MEMORY_MAP 1
MEMORY_SECTION 5
MEMORY_CONTENTS 47
REGISTER_VALUES 1
REGISTER_VALUE_RANGE 29
CODE 1
CODE_BLOCK 92954
DATA 1
DEFINED_DATA 242312
TYPEINFO_CMT 224453
COMMENTS 1
COMMENT 260168
PROGRAM_ENTRY_POINTS 1
PROGRAM_ENTRY_POINT 1343
SYMBOL_TABLE 1
SYMBOL 224268
FUNCTIONS 1
FUNCTION 165034
ADDRESS_RANGE 165034
STACK_FRAME 163962
STACK_VAR 182727
MARKUP 1
MEMORY_REFERENCE 106566
EQUATE_REFERENCE 9

Total XML Elements: 1849684
Database exported to: **********\e20cc4c06bc4a3fb70137107d7d2479c_UvLsP.xml
GhIDA:: [INFO] XML exporting completed
GhIDA:: [INFO] Ghidra headless (timeout: 3000s)
GhIDA:: [INFO] Waiting Ghidra headless analysis to finish...
GhIDA:: [INFO] Ghidra analysis completed!
GhIDA:: [!] Expecting value: line 1 column 1 (char 0)
GhIDA:: [!] Ghidra headless analysis failed
GhIDA:: [!] Decompilation interrupted.

[jimmy-sonny]

@Qynklee: currently GhIDA only works with Ghidra up to v.9.1.2. The latest Ghidra version has a bug that has been already reported and fixed. I expect the patch to be included in one of the next releases of Ghidra.

[jimmy-sonny]

@paxcut I can confirm that the plugin works with the following configuration:

• Ghidra v. 9.1.2
• IDA Pro Version 7.5.200619 Linux x86_64 (64-bit address size)
• GhIDA 0.22 - IDA7.x-Python3 link
• Python 3.7.5
• IDAPython 64-bit v7.4.0

Please, check if you have correctly installed the GhIDA Python3 version (it is slightly different from the Python2 version) and let me know if the same problem happens with different binaries.

If the problem persists, there are [DEBUG] prints in the code that can be enabled and we can start debugging from there. If you need to modify the code, please be sure to select the dev-python3 branch.

[paxcut]

@jimmy-sonny Thank you for replying.
I don't have a linux Ida version and the one I use is slightly newer 7.5.201028. Also I use python 3.8.3.
My configuration is:
Ghidra 9.1.2
IDA Pro Version 7.5.201028 Windows x64 (64-bit address size)
GhIDA 0.22 - IDA7.x-Python3 link
Python 3.8.3
IDAPython 64-bit v7.4.0

Using this configuration the error persists. Should I try to switch to python 3.7.5? Do I need to try the earlier Ida version (7.5.200619)? I don't have a linux license for Ida so I can't try to see if it works there.
Also you mentioned [DEBUG] prints in the code but all the ones I found were in comments. I checked out the dev-python3 branch since I probably need to uncomment the [DEBUG] prints unless there is a better way to enable them. Please let me know.
Again, thank you for your help.

[jimmy-sonny]

I've to try with IDA 7.5 on Windows to check if everything works correctly. I'll let you know.
Regarding the [DEBUG] prints, unfortunately the only way is to manually enable them.

[paxcut]

I found some interesting facts.

Ida 7.5.200728 which is the previous release to the latest SP 2, can run GHida using any python 3 but with the following caveat. By observing the processes and looking at the java process created by the headless Ghidra run it is possible to see when the decompilation finishes because the process cpu will go to zero. At that point it is OK to cancel the Ghida plugin and the decompiled code will appear. If you don't cancel the plugin it will run until timeout is reached and then it will display the decompiled code. It appears that the code that checks to see if the headless run has finished no longer works for that version of Ida. Also when I use the clear function from cache option the XML exporter is run which seemed strange to me but it appears to be a consequence of the fact that the java process is not being killed when the plugin run is canceled.

If the above is tried in Ida SP2 (7.5.201028) the Ghida plugin will start by exporting to xml and when the headless Ghidra run is supposed to be invoked, the Ghida plugin stops and the error message shown in my first post is displayed. It appears that for this version of Ida the part that fails is the invocation of the headless Ghidra decompilation because no java process ever appears on task manager.

If you are going to test windows maybe this may help determine why it is not working on my end.

[paxcut]

I need to correct the previous post. the strange behavior that occurs when using the clear cache still occurs but it is not due to a failure to kill the java process. the java process is indeed killed but the xml export only occurs when I left click the disassembly to be decompiled. no "decompile function using Ghida " needs to be selected for this to occur. it happens as soon as the left click occurs which is what I found so strange.

[paxcut]

It appears that removing the folowwing arguments
stdout=subprocess.PIPE
stderr=subprocess.PIPE
in the call to subprocess.Popen fixes the problem in Ida sp2 but not in Ida sp3. I can't find any documentation in Ida that talks about changes to how subprocess is handled as it appears that most changes in sp3 are macos related. I'm not sure how to proceed but I am willing to help debug it if needed. (btw the DEBUG message on the p.returncode case gives an error about nontypes do not have read)

[paxcut]

I found a way to fix this in all versions of Ida 7.5 without removing the arguments mentioned in my previous post. First make sure there are no strange (illegal) characters in the path to Ida (I had '(' and ')' which are bad),but the error still occurs even if there are none.
I created PR #25 . Feel free to test it and merge it if it is satisfactory.
Thank you for all your help.

[paxcut]

I was using the wrong branch in the pull request but now it is fixed.

[Qynklee]

@Qynklee: currently GhIDA only works with Ghidra up to v.9.1.2. The latest Ghidra version has a bug that has been already reported and fixed. I expect the patch to be included in one of the next releases of Ghidra.

Thanks you for your answer :D Sorry for my delay. I will try yours very soon because now I'm working at the end of other project :D

[amarkovytch]

I am using IDA version 7.6 SP1 and Ghidra 10.0.1
Having the same issue

[paxcut]

did you try the fix i implemented? you can find them here

[IdanH101]

did you try the fix i implemented? you can find them here

It works! Thanks