This code is a Python script that calculates Fleiss' Kappa, a statistical measure of inter-rater agreement, on data from an Excel file.
-Scoring Convention for Fleiss' Kappa
-Windows and Mac Installation instruction:
-GUI
Fleiss' kappa is a statistical measure for assessing the reliability of agreement between multiple raters when assigning categorical ratings to items. It compares the degree of agreement to what would be expected by chance. It can be used with binary or nominal-scale data, and can also be applied to ordinal data, but it is important to note that other measures such as Kendall's coefficients may be more appropriate for ordinal data. More information can be found on the Wikipedia page.
More information can be found here.
- < 0 Poor agreement
- 0.01 - 0.20 Slight agreement
- 0.21 - 0.40 Fair agreement
- 0.41 - 0.60 Moderate agreement
- 0.61 - 0.80 Substantial agreement
- 0.81 - 1.00 Almost perfect agreement
If you would like to run this program in the terminal Make sure to install the dependencies. The Main py file can be found here.
The Gui provides instructions on what types of files are accepted and how to use it. Here is a test file for the program: TestFile. These numbers are from the example on the WIKI.
- Click the 'Select File' button to choose an Excel file. Note only .xlsx,.xlsm,.xltx,.xltm are accepted.
- Make sure the sheet inside the excel sheet is called 'RawData'.
- The program should then automatically calculate Fleiss Kappa and display it in the window.
Once this file is selected the user should see a screen like this.
The Program Calculates the percentage and prints out a graph.
When Creating the excel sheet the sheet name must be "Raw Data" From there we can input data from the sheet. Note The categories are presented in the columns, while the subjects are presented in the rows.
- base64
- openpyxl
- webbrowser
- tkinter
- matplotlib
- The code has a button that opens the developer's website
- The code has a button that opens the Wikipedia page on Fleiss' Kappa
- The code has a button that opens the Github page of the developer
When you first try to open the app you will be prompted with this message.
To get around this open settings, then open "Privacy & Security", Lastly click "Open Anyway".
Click on "Use Password".
Enter your credentials and the application should now be able to open the app.
When You try to open the app on windows you will be promted with this message.
To Bypass this click on "More Info" then click "Run anyway"
