Bug when scroll slowly in RecyclerView (scroll to top)

[icodeyou]

Hi everybody ! I really need you all 💃
When I scroll my recyclerview, it constantly scrolls to the top (everytime a new line appears at the bottom).
This bug makes the scrolling not fluid at all. If I can't find a way to fix it, I'll be forced to find another library.

Here is my code :

public class SuggestedTagsAdapter extends RecyclerView.Adapter<SuggestedTagsAdapter.SuggestedTagsViewHolder> {

    private Context context;
    private List<Tag> data;

    public SuggestedTagsAdapter(Context context) {
        data = new ArrayList<>();
        this.context = context;
    }

    @Override
    public SuggestedTagsViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
        View view = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.item_tag_suggested, parent, false);
        return new SuggestedTagsViewHolder(view);
    }

    @Override
    public void onBindViewHolder(SuggestedTagsViewHolder viewHolder, final int position) {
        final Tag tag = data.get(position);
        viewHolder.setTagText(tag.getName());
    }

    @Override
    public int getItemCount() {
        return data.size();
    }

    public void add(Tag tag) {
        this.data.add(tag);
        notifyItemInserted(data.size());
    }

    public void remove(Tag tag) {
        int index = data.indexOf(tag);
        if(this.data.remove(tag)) notifyItemRemoved(index);
    }

    public void setData(List<Tag> tags) {
        this.data.clear();
        this.data.addAll(tags);
        notifyDataSetChanged();
    }

    public List<Tag> getData() {
        return data;
    }

    public Tag get(int position) {
        return this.data.get(position);
    }

    public class SuggestedTagsViewHolder extends RecyclerView.ViewHolder {
        TextView tagTv;

        SuggestedTagsViewHolder(View itemView) {
            super(itemView);

            tagTv = (TextView) itemView.findViewById(R.id.item_siggested_tag);
        }

        private void setTagText(String tagName) {
            tagTv.setText(tagName);
        }
    }
}

I would really appreciate any help.
Thank you !