Non Functional Requirements for Machine Learning

Recently, there are more attention has been paid to certain qualities of ML solutions, particularly fairness and transparency, but also qualities such as privacy, security, and testability. From a requirements engineering (RE) perspective, such qualities are also known as non-functional requirements (NFRs). The ISO/IEC 25010 standard divides system/software product quality into eight categories, including performance efficiency,compatibility, usability, and security.

Qualities for Machine Learning:

1. Accuracy & Performance: Most ML work reports on algorithm accuracy (often precision and recall), i.e., how “correct” the output is compared to reality.
2. Fairness: Recent work has focused on technical solutions to make ML algorithms more fair, finding that the removal of sensitive features is not sufficient to ensure fair results, and considering the trade-off between fairness and other NFRs. Work in this area has attempted to find mathematical or formal definitions of fairness, e.g. statistical parity, individual fairness, and has found that the accurate implementation of fairness depends more on how fairness is defined and measured than how it is implemented.