EMEBSDDI version 5_0_20200714_ on Linux (Ubuntu 20.04) fails, apparently jumps over indexing step

[hakonanes]

Hello!

Running EMEBSDDI on Windows with binaries from Bluequartz (EMsoft-5.0.20200714.-Win64.zip) works nicely, but on Linux with the same pattern and NML file fails with the output:

> EMEBSDDI sic_4h_ebsddi.nml 


 Copyright (C) 2001-2019 Marc De Graef Research Group/CMU
 EMsoft comes with ABSOLUTELY NO WARRANTY.
 This is free software, and you are welcome to redistribute it
 under certain conditions; see License.txt file for details.


 Program name         : EMEBSDDI.f90
 Purpose              : Program to index EBSD patterns using a dynamically calculated pattern dictionary
 Platform             : Linux
 Source code version  : 5_0_20200714_
 Source code Revision : 
 Build Date/Time      : 2020-07-14 15:49:24Z

 See https://github.com/EMsoft-org/EMsoft/wiki for selected help pages.

 Jul 14 2020   6:56:34.695 PM


Number of command line arguments detected:           1
 Array size analysis
 ===================
 Ne           :         1024
 Nd           :         1024
 L            :         9216
 size result array  :      4194304
 size_in_bytes_dict :              37748736
 size_in_bytes_expt :              37748736
 Total allocations on GPU (Mb):                    76
 reading from xtalfile si_carbide_4h\si_carbide_4h.xtal
Orientation space sampling mode set to RFZ
Point group number and number of cubochoric sampling points :  25,  100
 Number of unique orientations sampled =        :  666187
--> Initializing OpenCL device
--> Allocating various arrays for indexing
Preprocessing experimental patterns
Creating temporary file :/home/hakon/.config/EMsoft/tmp/1_4h.data
 -> Number of threads set to  4
Starting processing of experimental patterns
 pattern size :  96 x   96
Completed row   5 of  350 rows
[...]
Completed row 350 of  350 rows
 -> experimental patterns preprocessed
Number of experimental patterns processed per second :    4819.7

 -> computing Average Dot Product map (ADP)

 -> Number of threads set to  4
 Jul 14 2020   6:57:43.601 PM
  actual number of OpenMP threads  =            4
Indexing duration (system_clock, s)                :
         3.514
Number of pattern comparisons per second           :
**************
Number of experimental patterns indexed per second :
     20916.336

At line 899 of file /home/hakon/kode/emsoft/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90
Fortran runtime error: Index '0' of dimension 2 of array 'eulerarray' outside of expected range (1:666624)

Error termination. Backtrace:
#0  0x7f8fd444ecd1 in ???
#1  0x7f8fd444f819 in ???
#2  0x7f8fd444fe96 in ???
#3  0x5599c41a5320 in __emh5ebsd_MOD_h5ebsd_writefile
	at /home/hakon/kode/emsoft/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90:899
#4  0x5599c411e868 in EBSDDIdriver
	at /home/hakon/kode/emsoft/EMsoft/Source/EMOpenCLLib/Indexingmod.f90:1288
#5  0x5599c40d22b9 in emebsddi
	at /home/hakon/kode/emsoft/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:86
#6  0x5599c40d22f6 in main
	at /home/hakon/kode/emsoft/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:54

I would have though I did something wrong, however it works fine on Windows... Anyone experiencing the same on Linux?

[hakonanes]

Also, don't know if it is related to this issue, but after a successful indexing on Windows, the dot product file dataset Scan 1/EBSD/Data/FZcnt is 666187 while the dataset Scan 1/EBSD/Data/DictionaryEulerAngles has the shape (666624, 3). The point group number is 25 (space group 186).

[marcdegraef]

Hi Håkon, FZcnt is the true number of Euler angle triplets in the fundamental zone for a given value of N, the cubochoric sampling number. Many of the arrays that are used for indexing must have dimensions that are a multiple of 16 when they are sent to  or used for the GPU computations, so those arrays will be over-dimensioned with respect to the FZcnt value. We typically pad them with zeros so you can ignore the actual values.   When reading the dot product file, I would first read FZcnt, and then user that as the maximum index into the other arrays. Marc.

…

On 7/15/20 4:51 AM, Håkon Wiik Ånes wrote: Also, don't know if it is related to this issue, but after a successful indexing on Windows, the dot product file dataset |Scan 1/EBSD/Data/FZcnt| is 666187 while the dataset |Scan 1/EBSD/Data/DictionaryEulerAngles| has the shape (666624, 3). The point group number is 25 (space group 186). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#120 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB26VWATQVBTGWOMKL57YR3R3VUZVANCNFSM4OZW26BA>.

[hakonanes]

Of course, thanks for the quick response!

[hakonanes]

Just to be 100% sure, am I correct in assuming that the indices in the TopMatchIndices dataset are 1-indices, and not 0-indices, into the DictionaryEulerAngles dataset? I.e. a TopMatchIndices element of value 1 points to the first element in DictionaryEulerAngles?

[hakonanes]

I still experience this issue. Apparently indexmain is 0 when it tries to index into the Euler angle dictionary array in various routines.

[aleslagter]

Hello Håkon,

I am seeing the exact same error that you report here. Running this on linux on a HPC I see:

At line 899 of file /projects/a9009/sbc538/tickets/197097/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90
Fortran runtime error: Index '0' of dimension 2 of array 'eulerarray' outside of expected range (1:999808)

Error termination. Backtrace:
#0  0x2b52e43b5960 in ???
#1  0x2b52e43b64d9 in ???
#2  0x2b52e43b6ad6 in ???
#3  0x5555cb096117 in __emh5ebsd_MOD_h5ebsd_writefile
	at /projects/a9009/sbc538/tickets/197097/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90:899
#4  0x5555cb071f33 in EBSDDIdriver
	at /projects/a9009/sbc538/tickets/197097/EMsoft/Source/EMOpenCLLib/Indexingmod.f90:1378
#5  0x5555caf0a137 in emebsddi
	at /projects/a9009/sbc538/tickets/197097/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:88
#6  0x5555caf0a568 in main
	at /projects/a9009/sbc538/tickets/197097/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:54

Did you find a solution for this?
Thank you!

Alejandra

[hakonanes]

Hi @aleslagter, sorry, but I cannot remember what the solution was in the end.

Which version of EMsoft are you using? Have you tried EMsoftOO? This is version 6, the newest one.

[aleslagter]

Hi @hakonanes, thank you for your reply!

I am running the develop version, since this is the one that was installed on the HPC that I am using (I am really new to dictionary indexing).
I might try the EMsoftOO version in the near future if it can be successfully installed in the HPC, and was considering trying kikuchipy in the meantime. I understand that kikuchipy should yield similar results to those of EMsoft, is this correct? I am in particular interested in resolving pseudo-symmetry.

Any advice is more than welcome!

For this particular error I tried different combinations of parameters in the .nml file and different format files and got always the same error. I am not even sure it is related to the input parameters or input file that is loaded, since the error seems to come from the index that is assigned when creating the grid of euler angles to map (if I got some of it right...).
I even tried indexing a single pattern an loading an euler.txt file with a single orientation, setting the numdictsingle and numexptsingle to 1, and got the exact same error.

[hakonanes]

Similar, although kikuchipy has re-implemented dictionary indexing without energy weighting of master patterns (just one available energy provided by the user). This may yield different results in some challenging cases.

Pseudo-symmetry may be resolved in the refinement step after dictionary indexing. For each pattern, the best-matching dictionary orientation and the pseudo-symmetrical equivalents, obtained by applying pseudo-symmetry operations defined by the user, are refined, and the best solution is the result. We can do this with EMsoft using the EMFitOrientation program: https://github.com/EMsoft-org/EMsoftOO/blob/5d9bd108a034365767e655b5a6817981ac1a27be/NamelistTemplates/EMFitOrientation.template#L45-L49. We have a similar option in kikuchipy (https://kikuchipy.org/en/stable/reference/generated/kikuchipy.signals.EBSD.refine_orientation.html). If you're not sure of the pseudo-symmetry operations, you may get some help by using the EMeqvPS program: https://github.com/EMsoft-org/EMsoftOO/wiki/EMeqvPS.

Good luck!