(I. Showing the Bias) Learning to Rank (Rerank) the Recommended Team Members to Mitigate Popularity Bias

[hosseinfani]

@yogeswarl @Rounique
Please update me regarding your project in this issue page. Git does not allow me to assign this issue to more than 1 person :(

[hosseinfani]

[yogeswarl]

@hosseinfani
@Rounique and I's initial Idea involves around popularity bias in team formation: a case study.
As of now in our future work we will be working on re-ranking with metrics present in the Paper such as Average coverage of long tail (ACTL) , and average percentage of long tail items (APTL) also finding out the Group average popularity (GAP) . These are our targeted metric and we will be using the DBLP dataset as a starter to make this work.

Also we will be working on the re ranking found in the paper implementing it on team formation. We are in the verge of understanding how the algorithm works.

[hosseinfani]

@yogeswarl please add more details about the formulation of the metrics, and also, create a toy example for unit test your code to extract the metric values.

[yogeswarl]

Popularity bias in this case is defined as a discrepancy between the popular items and the Non popular items. A popular item in this scenario is defined as a team formed with authors having the most number of papers published with similar field of study(FOS). We create a toy(sample) data set to show this phenomenon.
We prove the hypothesis by following 3 metrics, namely:
**Average percentage of long tail Items (APLT):**https://arxiv.org/pdf/1901.07555.pdf

where |Ut| is the number of Teams in the test set and L(u) is the recommendation list of skill based author for each team.
This measure tells the number of authors belonging to the medium tail.

Average Coverage of Long Tail Items(ACTL): found in paper : https://arxiv.org/pdf/1901.07555.pdf

Measure the coverage of long tail items where I is present in T and 1(I E T) is equal to 1.

Group Average popularity(GAP): found in paper: https://arxiv.org/pdf/1907.13286.pdf

This measure the average popularity of the team with particular skill in that group.

[hosseinfani]

@yogeswarl @Rounique

1. I think it's better to put the definition of popularity over the members not the teams. We have popular researchers (dblp) or popular actors (movies).

2. What is \Phi in APLT?

3. I understand that we have to re-interpret the formulas for our own task. But I think the L(u) and U_t are not correctly reused. What is u such the L(u) is the recommendation list for u?

4. What is \Gamma? (it's not T) Also, what does this mean "if i present in \Gamma then it's one"? can you explain more base on our task?

5. when in the formula you have the "particular skill"? Also, what does group mean? same as team?

[yogeswarl]

Good Evening Dr. @hosseinfani,
Thank you very much for the extension, We believe we used it to our best possible measure.
We have posted our working in this GitHub link.
https://github.com/yogeswarl/fair-team-formation
We went with the Normalized Discounted Kullback-Leibler Divergence (NDLK) metric, an asymmetric measure of difference between probability distribution. The distribution was compared with our learning to rank model for Mitigating bias giving us a new ranked list of authors for a team.

Our readme file has more details on the working.
We can discuss further on our meeting tomorrow.

Thanks

[hosseinfani]

@yogeswarl
1- please answer my questions in the previous post.
2- you used part of opentf code without citation or reference!
3- you used a reranking package without further explanation.

[yogeswarl]

Thank you for

@yogeswarl
1- please answer my questions in the previous post.
2- you used part of opentf code without citation or reference!
3- you used a reranking package without further explanation.
Thank you for your response.

• We did use Popularity of members over teams.
• We did not use the APLT for the proposed method as of now. We wish to use those metrics for the experiment part and the final paper that we wish to publish.
• U is the author, and L(U) is the recommended list of authors. |Ut| are the teams formed.
• I will have to reaffirm on this when we work on the experiments and could really use your help in formulating! :)
• Group is same as team. say for instance, we can measure GAP for teams formed

2. We weren't sure how to cite a paper that wasn't published yet. Please help us in citing/referencing papers that are in draft. Should we mention the GitHub link instead?.
3. We made sure to explain it in our proposed method with a diagram. I will be sure to update the readme file with more explanation regarding.

[yogeswarl]

@hosseinfani, @Rounique and I have made the changes to the code. we are working on completing the document and will show our corrections tomorrow.

Thanks