Does calculating precision@k for one user makes sense?

[saba-zones]

Hi I am building a "product-product recommender system", where there is one main product, and it may be related to multiple addon products.

I am creating an interaction matrix based on relevance b/w my main prod and addon prod, hence a binary matrix where if the products are related in actual data, the value is 1, otherwise 0.

I have a sparse matrix situation where number of non-zero entries is much much less as compared to zero entries, which is fine because I only need few relevant items for every main product, to train my model.

Problem:
I am having difficulty in understanding what precision and recall value is good for me. I want my precision to be high in-terms of recommending most relevant top few products whenever a main product on the website is selected.

1. If I take a subset of 10 values from my interactions as well as main product features, and I try to check precision and recall for these 10 values against all addon product. Does that make sense?

val_interactions_subset = val_interactions.tocsr()[:10, :].tocoo()
product_features_subset = product_features[:10, :]
addon_product_features_subset = addon_product_features

For this I get a precision@k=3 of 0.2857 and recall@k=2 of 0.3785 for this. My understanding is, my model is right on avg 28% of the times it predicts addon product for these 10 main product. But this is very low, however my train interactions were of size (165480, 28576), from which I split only 15% into validation interactions.

2. Now, for analysis purpose: I want to check precision and recall for each main product in validation set individually, like how well my model is able to capture relevance b/w that one main product and its corresponding addon products. Is this the sound thing to do? I am not able to do this with lightfm.evaluations.precision@k function for one main product only.

I need help with this quickly. Please if someone can help me with understanding this .

Thanks in advance.

[saba-zones]

@maciejkula Can you help me with this situation?