-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathST_review_metadata.tsv
We can make this file beautiful and searchable if this error is corrected: Illegal quoting in line 25.
99 lines (99 loc) · 41.6 KB
/
ST_review_metadata.tsv
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
OVERALL TASK QUESTIONS STYLE MEANING FLUENCY - GRAMMATICALITY - NATURALNESS OTHER
Key Paper Conference Year Task Human annotation? (Yes/No) Sentiment Formality Who are the annotators? <free text> Annotators payment? <free text> Quality Control? (Yes/No) Availability of annotations? (Yes/No) # Systems <number> # Sents/System <number> # Annotations/instance <number> IAA (Yes/No) IAA value <free text> Sampling method <free text> Other comments <free text> Present (Yes/No) Quality Criterion <free text> Absolute judgment (Yes/No) Relative judgment type pairwise | ranking | best selection Absolute rating scale <list> Lineage (Yes/No) Lineage Source <free text> Other <free text> Present (Yes/No) Quality Criterion <free text> Absolute judgment (Yes/No) Relative judgment type pairwise | ranking | best selection Absolute rating scale <list> Lineage (Yes/No) Lineage Source <free text> Other <free text> Present (Yes/No) Quality Criterion <free text> Absolute judgment (Yes/No) Relative judgment type pairwise | ranking | best selection Absolute rating scale <list> Lineage (Yes/No) Lineage Source <free text> Other <free text> Present (Yes/No) Quality Criterion <free text> Absolute judgment (Yes/No) Relative judgment type pairwise | ranking | best selection Absolute rating scale <list> Lineage (Yes/No) Lineage Source <free text> Other <free text> Notes
3 Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen ACL 2020 expertise Yes No No non-experts not available No No 5 1000 not available No n/a none n/a No n/a n/a n/a n/a n/a n/a n/a Yes content similarity Yes n/a [1, 2, 3, 4, 5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
4 Evaluating prose style transfer with the Bible Royal Society 2018 prose No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
5 Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation EMNLP 2020 simile Yes No No AMT not available No No 5 200 3 Yes Creativity: 0.34, R1: 0.43, R2: 0.51, Overall: 0.40 random n/a Yes creativity Yes* n/a [1, 2, 3, 4, 5] No n/a annotators are shown all system outputs but evaluate them in an absolute scale Yes relevance Yes* n/a [1, 2, 3, 4, 5] No n/a annotators are shown all system outputs but evaluate them in an absolute scale No n/a n/a n/a n/a n/a n/a n/a Yes overall quality Yes* n/a [1, 2, 3, 4, 5] No n/a annotators are shown all system outputs but evaluate them in an absolute scale the aspects of this paper do not perfectly fit under the 3 typical evaluation dimensions (e.g., two definitions of relevance)
6 Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation ACL 2019 sentiment Yes Yes No not available not available No No 3 100 1 No n/a random n/a Yes opposite sentiment No best selection n/a No n/a n/a Yes content preservation No best selection n/a No n/a n/a Yes fluency No best selection n/a No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
7 Plug and play language models: A simple approach to controlled text generation ICLR 2020 sentiment Yes Yes No not available not available No No 4 not available 3 No n/a not available n/a Yes sentiment strength No pairwise n/a No n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes fluency Yes n/a [1, 2, 3, 4, 5] Yes lample-multiple-attribute-2019 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
8 Controlling Linguistic Style Aspects in Neural Language Generation ACL (WS) 2017 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
9 Style Transfer in Text: Exploration and Evaluation AAAI 2018 sentiment Yes Yes No AMT not available No No 1 200 3 No n/a runs n/a No n/a n/a n/a n/a n/a n/a n/a Yes content preservation Yes n/a [0, 1, 2] n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
10 StyleNet: Generating Attractive Visual Captions with Styles IEEE 2017 attractive captions Yes No No AMT not available No No 4 not available not available No n/a not available n/a Yes attractive captions No best selection n/a No n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
11 IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation EMNLP 2019 sentiment Yes Yes No not available not available Yes No 5 200 not available No n/a random judges passed a qualification test Yes attribute change correctness Yes n/a [1, 2, 3, 4, 5] Yes li-2018; lample-2019 n/a Yes content preservation Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete, lample-2019 n/a Yes grammaticality Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete, lample-2019 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
12 Disentangled Representation Learning for Non-Parallel Text Style Transfer ACL 2019 sentiment Yes Yes No not available not available No No 5 not available 6 Yes Krippendorff’s alpha TS: 0.74; CP: 0.68, and LQ: 0.72 random n/a Yes transfer strength Yes n/a [1, 2, 3, 4, 5] Yes stent-2005 n/a Yes content preservation Yes n/a [1, 2, 3, 4, 5] Yes stent-2005 n/a Yes language quality Yes n/a [1, 2, 3, 4, 5] Yes stent-2005 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
13 Negative Lexically Constrained Decoding for Paraphrase Generation ACL 2019 formality, simplification No No Yes n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
15 Reformulating Unsupervised Style Transfer as Paraphrase Generation EMNLP 2020 formality, author imitation Yes No Yes AMT Yes Yes* No 4 150 3 Yes Fleiss Kappa: 0.13-0.45 not available n/a No n/a n/a n/a n/a n/a n/a n/a Yes similarity Yes n/a [0, 1, 2] Yes (Kok and Brockett, 2010; Iyyer et al., 2018) n/a Yes fluency Yes n/a [0, 1, 2] Yes (Kok and Brockett, 2010; Iyyer et al., 2018) n/a No n/a n/a n/a n/a n/a n/a n/a n/a
16 Multiple-attribute text rewriting ICLR 2019 multiple Yes No No AMT* not available No No 3 not available not available No n/a not available n/a Yes text attribute Yes n/a [positive, negative, relaxed, annoyed] No n/a n/a Yes Content preservation Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete n/a Yes Fluency Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete n/a Yes overall task No pairwise n/a No n/a n/a A/B + No preference
17 Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfer INLG 2020 sentiment Yes Yes No AMT not available No No 4 250 6 No n/a random unclear if 6 annotations per/instance or 6 annotators in total Yes style Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes Content Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes Fluency Yes n/a [1, 2, 3, 4, 5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
18 Domain adaptive text style transfer EMNLP 2019 sentiment, formality Yes No Yes AMT* not available No No 4 100 3 Yes Agreement with most common judgement: 0.759 random n/a Yes sentiment No pairwise n/a Yes mir-etal-2019-evaluating n/a Yes Content preservation No pairwise n/a Yes mir-etal-2019-evaluating n/a Yes Grammaticality & Fluency No pairwise n/a Yes mir-etal-2019-evaluating n/a Yes overall quality No pairwise n/a Yes mir-etal-2019-evaluating n/a A/B + No preference
19 A Persona-Based Neural Conversation Model ACL 2016 persona Yes No No AMT* not available Yes No 2 480 2 No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes consistency No pairwise n/a No n/a n/a 5 point zero sum scale [-2, 2]
20 Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer ACL 2018 sentiment Yes Yes No AMT not available No Yes 8 400 1 No n/a random n/a Yes similarity to the target attribute Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes preservation ofsource content Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes Grammaticality Yes n/a [1, 2, 3, 4, 5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
22 Complementary Auxiliary Classifiers for Label-Conditional Text Generation AAAI 2020 personality, sentiment, style captioning, topic-based No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
23 QuaSE: Sequence Editing under Quantifiable Guidance EMNLP 2018 sentiment Yes Yes No not available not available No No 3 500 5 No n/a not available n/a No n/a n/a n/a n/a n/a n/a n/a Yes Content preservation n/a n/a [2, 1, 0] No n/a n/a Yes fluency Yes n/a [1, 2, 3, 4] Yes Shen et al., 2017 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
24 Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning AAAI 2020 sentiment Yes Yes No experts not available No No 8 50 not available No n/a random n/a Yes style transfer accuracy Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete, Fu et al 2018: Style transfer in text: Exploration and evaluation. n/a Yes preservation of content Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete, Fu et al 2018: Style transfer in text: Exploration and evaluation. n/a Yes fluency Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete, Fu et al 2018: Style transfer in text: Exploration and evaluation. n/a No n/a n/a n/a n/a n/a n/a n/a n/a
25 Content preserving text generation with attribute controls NeurIPS 2018 sentiment Yes Yes No AMT not available No No 3 100 not available No n/a random n/a Yes attribute compatibility Yes n/a [positive, negative, neural] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes fluency/grammaticality Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes content compatibility: same semantic content as the reference sentence but have the opposite sentiment Yes n/a n/a No n/a pick the candidates that fit under the description (all that apply) n/a
26 A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer IJCAI 2019 sentiment, formality Yes Yes Yes experts not available No No 7 not available not available No n/a not available n/a Yes accuracy of the target style Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes preservation of original content Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes fluency Yes n/a [1, 2, 3, 4, 5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
27 POWERTRANSFORMER: Unsupervised Controllable Revision for Biased Language Correction EMNLP 2020 debiasing Yes No No AMT not available Yes No 4 100 not available Yes Krippendorf’s α=.52 (not mentioned on which task) Random n/a Yes highest agency No pairwise n/a No n/a n/a Yes closer in meaning to the original sentence No pairwise n/a No n/a n/a Yes gibberish Yes n/a ["easy to understand", "some grammar errors", "impossible to understand"] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
28 Politeness Transfer: A Tag and Generate Approach ACL 2020 politeness, gender, politics, sentiment Yes Yes No not available not available No No 2 100 not available No n/a not available n/a Yes target attribute match Yes* n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete for sentiment/formality only Yes Content preservation Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete n/a Yes grammaticality Yes n/a [1, 2, 3, 4, 5] Yes li-etal-2018-delete n/a No n/a n/a n/a n/a n/a n/a n/a n/a
30 Unsupervised Text Style Transfer with Padded Masked Language Models EMNLP 2020 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a also evaluate on sentence fusion (not a ST task)
31 Evaluating Style Transfer for Text NAACL 2019 sentiment Yes Yes No AMT not available No Yes* 3 244 3 Yes Fleiss’ kappa κ of content preservation based on style-masked texts is 0.297, 0.173 for unmasked texts, balanced IAA is Feiss' Kappa; instances are called "text" but look like sentences Yes style transfer intensity No pairwise n/a Yes evidence for relative over absolute rating: Neil Stewart, Gordon DA Brown, and Nick Chater. 2005. Absolute identification by relative judgment. Psychological review, 112(4):881–911. Tammo Bijmolt and Michel Wedel. 1995. The effects of alternative methods of collecting similarity data for multidimensional scaling. International Journal of Research in Marketing, 12(4):363–371. relative ranking on 1-5 scale (identical to completely different styles) Yes Content Preservation No pairwise n/a Yes evidence for relative over absolute rating: Neil Stewart, Gordon DA Brown, and Nick Chater. 2005. Absolute identification by relative judgment. Psychological review, 112(4):881–911. Tammo Bijmolt and Michel Wedel. 1995. The effects of alternative methods of collecting similarity data for multidimensional scaling. International Journal of Research in Marketing, 12(4):363–371. relative ranking on 1-5 scale Yes naturalness No pairwise n/a Yes evidence for relative over absolute rating: Neil Stewart, Gordon DA Brown, and Nick Chater. 2005. Absolute identification by relative judgment. Psychological review, 112(4):881–911. Tammo Bijmolt and Michel Wedel. 1995. The effects of alternative methods of collecting similarity data for multidimensional scaling. International Journal of Research in Marketing, 12(4):363–371. binary decision on which system is more natural No n/a n/a n/a n/a n/a n/a n/a n/a
32 Polite Dialogue Generation Without Parallel Data TACL 2018 polite dialogue generation Yes No No AMT not available No No 6 300 2 Yes the Kappa score 0.35 on Dialogue Quality and 0.46 (moderate) on Politeness. random Cohen's Kappa; generic QC but not task specific QC (location, worker approval rate); claims to release AMT interface and annotations on their webpage, but I can only find their code to train models: https://github.com/WolfNiu/polite-dialogue-generation Yes politeness level Yes* n/a [Polite, slightly polite, neutral, slightly rude, rude] Yes Wang et al. EMNLP 2017 https://www.aclweb.org/anthology/D17-1228/ eval is absolute ranking, but implicitly relative as annotators are presented with all 6 outputs when ranking No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes dialogue quality yes n/a [Very good, good, Acceptable, Poor, Very poor] No n/a absolute ratings are made in the context of all system outputs there is a second manual eval of style that evaluates politeness levels for 3 different model configs, I only described the main evaluation here
35 “My Way of Telling a Story”: Persona based Grounded Story Generation ACL (WS) 2019 persona based visual story generation No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
36 Style transfer through back-translation ACL 2018 sentiment, gender, political slant Yes Yes No not available not available No No 9 ( 3 per task, including source sentence) 100 (meaning preservation); 60 (fluency) not available No n/a random there are 11 human judges, but unclear how many annotations per samples No n/a n/a n/a n/a n/a n/a n/a Yes meaning/intent preservation Yes n/a [1,2,3,4,5]* Yes Christina L Bennett. 2005. Large scale evaluation of corpus-based synthesizers: Results and lessons from the blizzard challenge 2005. In Ninth European Conference on Speech Communication and Technol- ogy. they mark out stylistic words before giving them to annotators Yes fluency Yes No [1,2,3,4] Yes Shen et al., 2017 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
37 Automatically Neutralizing Subjective Bias in Text AAAI 2020 neutralizing subjective bias Yes No No AMT not available Yes No 9 200 out of domain, unclear how many in domain (Wikipedia) not available Yes Krippendorff’s alpha: 0.65 for fluency, 0.33 for meaning, and 0.51 for bias not available Krippendorff's alpha 0.65 for fluency, 0.33 for meaning, 0.51 for bias; they had 1800 raters total Yes bias Yes n/a [-2;-1;0;1;2] No n/a n/a Yes meaning Yes No [-2;-1;0;1;2] No n/a n/a Yes fluency Yes No [0, 1, 2, 3, 4] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
39 Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer NAACL 2018 formality Yes No Yes AMT not available Yes No 6 500 5 No n/a not available n/a Yes formality Yes n/a [-3,-2, -1, 0, 1, 2, 3] Yes PT16 n/a Yes meaning preservation Yes No [1;6] Yes Agirre et al. (2016) - SemEval Task 1 n/a Yes fluency Yes No [1,2,3,4,5] Yes Heilman et al. 2014 n/a Yes overall ranking (formality, taking into account fluency and meaning preservation) No ranking n/a No n/a n/a n/a
40 Obfuscating Gender in Social Media Writing EMNLP (WS) 2016 author gender No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
41 Adversarial Decomposition of Text Representation NAACL 2019 diachronic language change, social register No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
42 Reinforced Rewards Framework for Text Style Transfer ECIR 2020 formality, excitement, modern English to Shakespearean English Yes No Yes AMT not available No No 4 50 not available No n/a random n/a Yes transfer strength Yes No [1, 2, 3, 4, 5] Yes Rao & Tetreault 2018 n/a Yes content preservation Yes No Likert scale of 6 Yes Rao & Tetreault 2018 n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
43 Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer ACL 2018 offensive to non-offensive language No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
44 Semi-supervised Text Style Transfer: Cross Projection in Latent Space EMNLP 2019 ancient to modern Chinese formality Yes No Yes experts not available No No 2 400 not available No n/a random n/a Yes ancient to modern Chinese, modern to ancient Chinese, inf to formal Eng, formal to inf Eng Yes n/a unclear, it is likely {0, 1,2 } No n/a they write that their style eval is "similar" to meaning preservation but it is unclear if that means exactly the same or parts are the same Yes content Yes n/a [0, 1, 2] No n/a n/a Yes fluency Yes n/a unclear, it is likely {0, 1,2 } No n/a they write that their fluency eval is "similar" to meaning preservation but it is unclear if that means exactly the same or parts are the same No n/a n/a n/a n/a n/a n/a n/a n/a
45 Generation of Formal and Informal Sentences ENLG 2011 formality Yes No Yes experts not available No Yes* 1 100 2 No n/a not available n/a Yes formality Yes n/a binary No n/a I wasn't quite sure if the raters were doing the overall task or strictly formality. No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
46 Style Transfer from Non-Parallel Text by Cross-Alignment NeurIPS 2017 sentiment Yes Yes No not available not available No No 2 500 not available No n/a random n/a Yes sentiment Yes n/a [pos, neg, neither] No n/a n/a Yes overall transfer No ranking n/a No No they called this overall but the instructions really sounded like meaning preservation. "The annotator was shown a source sentence and the corresponding outputs of the systems in a random order, and was asked “Which transferred sentence is semantically equivalent to the source sentence with an opposite sentiment?”. They can be both satisfactory, A/B is better, or both unsatisfactory." More intructions are in the appendix. Yes fluency Yes n/a [1, 2, 3, 4] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
47 A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation Usenix Security Symposium 2018 anonymization Yes No No AMT not available No No 2 745 3 No n/a random seems like there are multiple similar evaluations conducted, was a little confusing No n/a n/a n/a n/a n/a n/a n/a Yes semantic similarity Yes n/a [0..5] Yes Agirre et al. (2016) - SemEval Task 1 it seems that the annotator sees both system outputs No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
48 Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer EMNLP 2019 gender, political slant Yes No No AMT not available No No 2 not available not available No n/a not available note that I split this paper up into two rows based on the tasks No n/a n/a n/a n/a n/a n/a n/a Yes content No ranking n/a No No there are only two systems so the choices are both good, both bad, or one better than the other Yes fluency No ranking n/a No No there are only two systems so the choices are both good, both bad, or one better than the other No n/a n/a n/a n/a n/a n/a n/a n/a
49 Adapting Language Models for Non-Parallel Author-Stylized Rewriting AAAI 2020 author stylization No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
50 Structured Content Preservation for Unsupervised Text Style Transfer arxiv 2018 sentiment, political slant transfer Yes Yes No not available not available No No 6 100 not available No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes not available No pairwise n/a No n/a I figured "Other" was the best category here based on: “Which sentence has an opposite sentiment of the original sentence and at the same time preserves the content of it?” They can choose: “A”, “B” or “the same”. n/a
51 Towards A Friendly Online Community: An Unsupervised Style Transfer Framework for Profanity Redaction COLING 2020 profanity Yes Yes No not available not available No No 4 100 3 No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a Yes content preservation Yes n/a [1..5] Yes li-etal-2018-delete n/a Yes grammaticality Yes n/a [1..5] Yes li-etal-2018-delete n/a No n/a n/a n/a n/a n/a n/a n/a n/a
52 Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation NeurIPS 2019 sentiment, romantic/humorous Yes No No AMT not available No No 3 200 total broken up into 4 tasks of 50 3 No n/a random note there are four tasks Yes attribute accuracy Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes content retainment Yes n/a [1..5] No n/a n/a Yes grammaticality Yes n/a [1..5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
53 Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer EMNLP 2019 formality No No Yes n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
54 Mask and Infill: Applying Masked Language Model to Sentiment Transfer IJCAI 2019 sentiment Yes Yes No not available not available No No 4 200 3 No n/a random n/a Yes target attribute match Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes content preservation Yes n/a [1..5] No n/a n/a Yes grammaticality Yes n/a [1..5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
55 A Dataset for Low-Resource Stylized Sequence-to-Sequence Generation AAAI 2020 formality Yes No Yes not available not available No No 5 300 not available No n/a random it is unclear on the #annotations/instance, they just mentioned 3 human annotators is asked to do the annotation. No IAA calculated. Github for the annotated data (but not human judgements): https://github.com/MarkWuNLP/Data4StylizedS2S Yes formality Yes n/a [-3,-2, -1, 0, 1, 2, 3] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes fluency Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes overall rank score No ranking n/a No n/a n/a the definition/naming of rating scale can also be different across papers. Paper data: https://github.com/MarkWuNLP/Data4StylizedS2S
56 Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach ACL 2018 sentiment Yes Yes No not available not available No No 3 200 not available No n/a random n/a Yes sentiment Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a Yes semantic similarity Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a code: https://github.com/lancopku/Unpaired-Sentiment-Translation
57 On Variational Learning of Controllable Representations for Text without Supervision ICML 2020 sentiment No Yes Yes n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a code: https://github.com/BorealisAI/CP-VAE
58 Formality Style Transfer with Hybrid Textual Annotations arxiv 2019 formality No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a https://github.com/xrc10/formal-sty-trans
59 Paraphrasing for Style COLING 2012 writing style Yes No No author n/a No No 3 100 2 No n/a random n/a Yes stylistic similarity Yes n/a not available No n/a does not tell the exact scale, but can roughly guess from Figure 2 Yes semantic adequacy Yes n/a not available No n/a scale inferred from Figure 2 Yes lexical dissimilarity Yes n/a not available No n/a scale inferred from Figure 2 Yes overall quality Yes n/a not available No n/a scale inferred from Figure 2 n/a
60 Decomposing Textual Information For Style Transfer NGT 2019 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
61 Unsupervised Text Style Transfer using Language Models as Discriminators NeurIPS 2018 sentiment No Yes Yes n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
62 Text Style Transfer via Learning Style Instance Supported Latent Space IJCAI 2020 sentiment, formality, poeticness Yes Yes Yes not available not available No No 5 50 3 No n/a random 5 systems evaluated for sentiment and formality, but only 2 for poeticness Yes sentiment, formality, poeticness Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes content preservation Yes n/a [1,2,3,4,5] No n/a n/a Yes fluency Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
63 Style Example-Guided Text Generation using Generative Adversarial Transformers arxiv 2020 multiple styles (e.g., Sciene, Politics ) Yes No No AMT not available Yes* No 2 900; 1764 not available No n/a random n/a Yes style correctness No pairwise n/a No n/a two systems are evaluated independently, show turkers generated text and reference and turker decide which one is more style correct No n/a n/a n/a n/a n/a n/a n/a Yes fluency No pairwise n/a No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
64 Parallel Data Augmentation for Formality Style Transfer ACL 2020 formality Yes No Yes experts not available No No 4 300 2 No n/a random n/a Yes formality Yes n/a [0,1,2] No n/a n/a Yes meaning preservation Yes n/a [0,1,2] No n/a n/a Yes fluency Yes n/a [0,1,2] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
65 Learning Sentiment Memories for Sentiment Modification without Parallel Data EMNLP 2018 sentiment Yes Yes No experts not available No No 3 200 2 No n/a random n/a Yes transformed sentiment degree Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a Yes content preservation degree Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a Yes fluency Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
66 Style Transfer as Unsupervised Machine Translation arxiv 2018 sentiment, romantic/humorous Yes Yes Yes not available not available No No 6 200 5 No n/a random n/a Yes style transfer accuracy Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes preservation of content Yes n/a [1,2,3,4,5] converted to binary No n/a n/a Yes fluency Yes n/a [1,2,3,4,5] converted to binary No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
67 STYLEPTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer NAACL 2021 sentiment, formality (fine-grained) Yes Yes Yes not available not available No No 3 210 2 No n/a not available n/a Yes style Yes n/a not available No n/a n/a Yes context Yes n/a not available No n/a n/a Yes clarity Yes n/a not available No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
68 On Learning Text Style Transfer with Direct Rewards NAACL 2021 sentiment, formality Yes Yes Yes AMT not available No No 3 50 3 No n/a random n/a Yes style transfer accuracy Yes n/a [1,2,3] No n/a n/a Yes content preservation Yes n/a [1,2,3] No n/a n/a Yes fluency Yes n/a [0,1] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
69 Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus NAACL 2021 sentiment, formality Yes Yes Yes not available not available No No 2 40 4 No n/a bin-based n/a Yes style control Yes n/a [1, 2, 3, 4, 5] No n/a n/a Yes content preservation Yes n/a [1,2,3,4,5] No n/a n/a Yes fluency Yes n/a [1,2,3,4,5] No n/a n/a Yes overall transfer quality Yes n/a [1,2,3,4,5] No n/a n/a n/a
21 The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement arxiv 2020 attractiveness Yes No No not available not available No No 4 100 3 No n/a random n/a Yes attractiveness Yes n/a [1,2,3] No n/a n/a Yes consistency Yes n/a [1,2,3] No n/a n/a Yes fluency Yes n/a [1,2,3] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
70 XFORMAL: A Benchmark for Multilingual Formality Style Transfer NAACL 2021 formality Yes No Yes AMT Yes Yes Yes 5 100 5 Yes Cohen’s κ coefficient: Formality 0.56-0.67, Fluency: 0.43-0.58, Meaning: 0.53-0.71, Overall: 0.41-0.44 random n/a Yes formality Yes n/a [-3,-2, -1, 0, 1, 2, 3] Yes PT16 n/a Yes meaning preservation Yes n/a [1,2,3,4,5] Yes Agirre et al. (2016) - SemEval Task 1 n/a Yes fluency Yes n/a [1,2,3,4,5] Yes Heilman et al. 2014 n/a Yes overall n/a ranking n/a Yes Rao & Tetreault 2018 n/a n/a
71 A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer ACL 2019 sentiment Yes Yes No not available not available No No 5 not available not available No n/a not available n/a Yes style polarity Yes n/a [1,2,3,4,5] No n/a n/a Yes content preservation Yes n/a [1,2,3,4,5] No n/a n/a Yes fluency Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
72 A Probabilistic Formulation of Unsupervised Text Style Transfer ICLR 2020 sentiment, formality, author imitation, word decipherment No Yes Yes n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
74 Adversarial Text Generation via Feature-Mover's Distance NeurIPS 2018 sentiment Yes Yes No not available volunteers No No 4 100 5 No n/a random n/a Yes sentiment Yes n/a [0,1,2,3,4,5] No n/a n/a Yes content preservation Yes n/a [0,1,2,3,4,5] No n/a n/a Yes fluency Yes n/a [0,1,2,3,4,5] n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
75 Adversarially Regularized Autoencoders ICML 2018 sentiment Yes Yes No AMT* not available No n/a 2 1000 not available No n/a stratified n/a Yes sentiment Yes n/a [positive,negative,neutral] No n/a n/a Yes similarity Yes n/a [1,2,3,4,5] No n/a n/a Yes naturalness Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
76 Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline ACL Workshop 2020 emotion Yes No No not available not available No No 4 100 2 Yes Spearman (1 and 0.8) random n/a Yes emotion No best-worst n/a Yes Louviere et al. 2015 n/a Yes similarity No best-worst n/a Yes Louviere et al. 2015 n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
77 Contextual Text Style Transfer EMNLP 2020 formality, offensiveness Yes No Yes AMT not available No No 3 200 3 No n/a random n/a Yes style control No best-selection n/a No n/a n/a Yes content preservation No best-selection n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes context consistency n/a best-selection n/a n/a n/a n/a n/a
78 Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer COLING 2020 sentiment Yes Yes No experts not available n/a No 3 200 4 No n/a stratified they say the annotators "are proficient in English and have sufficient background about this evaluation task" so I count this as expert Yes style transfer Yes n/a [1,2,3,4,5] No n/a n/a Yes content preservation Yes n/a [1,2,3,4,5] n/a n/a n/a Yes fluency Yes n/a [1,2,3,4,5] n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
79 DGST: a Dual-Generator Network for Text Style Transfer EMNLP 2020 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
80 Effective writing style imitation via combinatorial paraphrasing PETS 2020 sentiment, author, gender, age Yes Yes No authors n/a n/a No 4 200 not available No n/a stratified n/a No n/a n/a n/a n/a n/a n/a n/a Yes paraphrases Yes n/a not available n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
81 Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer ACL 2020 sentiment, formality Yes Yes Yes experts not available n/a No 5 not available 3 Yes Fleiss kappa (0.76-0.80) not available n/a Yes accuracy of style transfer Yes n/a [1,2,3,4,5] Yes Zhang et al. 2018 n/a Yes content preservation Yes n/a [1,2,3,4,5] Yes Zhang et al. 2018 n/a Yes fluency Yes n/a [1,2,3,4,5] Yes Zhang et al. 2018 n/a No n/a n/a n/a n/a n/a n/a n/a n/a
82 Formality Style Transfer with Shared Latent Space COLING 2020 formality Yes No Yes not available not available No No 5 200 4 No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes overall Yes n/a [perfect,good,fair,bad] n/a n/a n/a n/a
84 Generating sentences by editing prototypes TACL 2018 sentiment Yes Yes No AMT not available No No 3 545 not available No n/a no sampling n/a No n/a n/a n/a n/a n/a n/a n/a Yes similarity Yes n/a [1,2,3,4,5] Yes Aggiree n/a Yes grammaticality Yes n/a [1,2,3] No n/a n/a Yes plausibility Yes n/a [1,2,3] No n/a n/a n/a
85 Grammatical Error Correction and Style Transfer via Zero-shot Monolingual Translation arXiv 2019 GEC, formality Yes No Yes 3 people who are fluent but not native speakers not available No No 2 100 not available No n/a random n/a Yes formality Yes n/a [more formal, more informal, neither] No n/a n/a Yes meaning preservation Yes n/a [1,2,3,4] No n/a n/a Yes fluency Yes n/a [1,2,3,4] No n/a n/a Yes overall No best selection n/a No n/a which of the two outputs is better in general: alters style in the right direction, while at the same time preserving meaning of the original text and being fluent n/a
86 Hooks in the Headline: Learning to Generate Headlines with Controlled Styles ACL 2020 humor, romance, clickbait Yes No No 3 native-speakers not available No No 3 4 not available No n/a random n/a Yes style strength No best selection n/a No n/a n/a Yes relevance Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a Yes fluency Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a Yes attractiveness Yes n/a [1,2,3,4,5,6,7,8,9,10] No n/a n/a n/a
87 How Positive Are You: Text Style Transfer using Adaptive Style Embedding COLING 2020 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
88 Language Style Transfer from Sentences with Arbitrary Unknown Styles arXiv 2018 sentiment, romantic, shakespearean Yes Yes No hire 5 annotators not available No No 4 200 not available No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes overall Yes n/a [1,2,3,4] Yes fu-etal-2017 n/a n/a
90 Learning to Generate Multiple Style Transfer Outputs for an Input Sentence arXiv 2020 sentment Yes Yes No AMT not available No No 3 350 3 No n/a random n/a Yes diversity No pairwise n/a Yes Style transfer through back-translation. n/a No n/a n/a n/a n/a n/a n/a n/a Yes fluency No pairwise n/a Yes Style transfer through back-translation. n/a Yes overall No pairwise n/a Yes Style transfer through back-translation. n/a Two user study mentioned, used first one to fillin details
91 Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders ACL 2020 sentiment, topic generation Yes Yes No eight individual judges not available No No 4 200 8 No n/a not available n/a Yes conditionality Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes fluency Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
92 Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus NAACL 2019 sentiment, formality Yes Yes Yes not available not available No No 1 100 3 No n/a not available n/a Yes transfer strength No pairwise n/a No n/a compare original and transfered to judge which matches the target style better Yes content preservation Yes n/a [1,2,3,4,5,6] Yes Rao & Tetreault 2018 n/a Yes fluency Yes n/a [1,2,3,4,5,6] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
93 SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation NAACL 2018 headline specific to a publisher No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
94 ST2: Small-data Text Style Transfer via Multi-task Meta-Learning arXiv 2020 personal writing styles Yes No No two native English speakers not available No No 7 not available not available Yes kappa=0.769 not available n/a No n/a n/a n/a n/a n/a n/a n/a Yes content preservation Yes n/a [1,2,3,4,5] Yes A dual reinforcement learning framework for unsupervised text style transfer. n/a Yes fluency/naturalness Yes n/a [1,2,3,4,5] Yes A dual reinforcement learning framework for unsupervised text style transfer. n/a No n/a n/a n/a n/a n/a n/a n/a n/a
95 SentiInc: Incorporating Sentiment Information into Sentiment Transfer Without Parallel Data ECIR 2020 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
96 Sequence to Better Sequence: Continuous Revision of Combinatorial Structures ICML 2017 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
97 Shakespearizing Modern Language Using Copy-Enriched Sequence to Sequence Models EMNLP Workshop 2017 writing style No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
98 Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites EMNLP 2019 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
99 Toward Controlled Generation of Text ICML 2017 sentiment, tense No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
100 Towards Controlled Transformation of Sentiment in Sentences ICAART 2019 sentiment Yes Yes No not available not available No No 2 not available not available No n/a used a different dataset n/a Yes sentiment change Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes Grammatical and semantic correctness Yes n/a [1,2,3,4,5] No n/a n/a n/a
101 Unsupervised Automatic Text Style Transfer Using LSTM NLPCC 2017 writing style No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
102 Unsupervised Controllable Text Formalization AAAI 2019 readability Yes No Yes 3 language experts aware of the task not available No No 1 30 not available No n/a random n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a Yes readability No ranking n/a No n/a n/a n/a
103 Unsupervised Text Generation by Learning from Search NeurIPS 2020 paraphrase generation, formality Yes No Yes four annotators with linguistic background not available No No 2 120 2 Yes unclear random n/a No n/a n/a n/a n/a n/a n/a n/a Yes coherence and consistency Yes n/a [1,2,3,4,5] No n/a n/a Yes fluency/naturalness/grammaticality Yes n/a [1,2,3,4,5] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
104 Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles arXiv 2019 sentiment No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
105 Zero-Shot Style Transfer in Text Using Recurrent Neural Networks arXiv 2017 text rewrting for Bible No No No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
106 Rethinking Text Attribute Transfer: A Lexical Analysis INLG 2019 sentiment, gender No Yes No n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a
107 Style versus Content: A distinction without a (learnable) difference? COLING 2020 sentiment and opinion polarity transfer Yes Yes No not available not available No No 3 150 3-8 Yes krippendorff's alpha Style 0.752, content 0.772, Fluency 0.568 random n/a Yes style shift power Yes n/a ["positive", "negative", "neutral"] No n/a n/a Yes content preservation No ranking n/a No n/a n/a Yes fluency Yes n/a ["incorrect", "partly correct, "correct"] No n/a n/a No n/a n/a n/a n/a n/a n/a n/a n/a