Skip to content

Commit

Permalink
Changed 'Exploration' to 'Validation' (#25)
Browse files Browse the repository at this point in the history
Co-authored-by: Matthijs van Wijngaarden <Matthijsbbis@hotmail.com>
  • Loading branch information
Rensvandeschoot and Metais authored Apr 25, 2024
1 parent 0c47cf0 commit b8d8bf2
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 12 deletions.
10 changes: 5 additions & 5 deletions ASReviewLAB.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,13 @@ bottom right corner.

2. *Project name;*

Select Exploration Mode, fill out a project name and press ‘NEXT’. Note
Select Validation Mode, fill out a project name and press ‘NEXT’. Note
that you can fill out your name and a description as well.

![](images/ASReviewLAB/Interface3.png){width=95%}

For this exercise we are screening in the so-called ‘Exploration Mode’
of ASReview. By screening in the [Exploration
For this exercise we are screening in the so-called ‘Validation Mode’
of ASReview. By screening in the [Validation
Mode](https://asreview.readthedocs.io/en/latest/project_create.html#project-modes){target="_blank"},
we are going to make use of a [benchmark
dataset](https://asreview.readthedocs.io/en/latest/data_labeled.html#fully-labeled-data){target="_blank"}.
Expand Down Expand Up @@ -161,7 +161,7 @@ settings).

When performing a systematic review with your own data, you need to
provide the prior knowledge yourself (at least one relevant and one
irrelevant record). However, because you are using the Exploration Mode
irrelevant record). However, because you are using the Validation Mode
of ASReview, the relevant records are known; the original authors have
already read ALL records.

Expand Down Expand Up @@ -230,7 +230,7 @@ After the project is successfully initialized, you can start reviewing.

Everything is set up and ready to screen, well done!

Since we are in the Exploration Mode of ASReview, you can pretend to be an
Since we are in the Validation Mode of ASReview, you can pretend to be an
expert on the topic of PTSD and pretend you have all the knowledge of the
original screeners. All records in the dataset have been labeled as
relevant/irrelevant, which is indicated through a banner above each article.
Expand Down
8 changes: 4 additions & 4 deletions docs/ASReviewLAB.html
Original file line number Diff line number Diff line change
Expand Up @@ -279,9 +279,9 @@ <h3 class="anchored" data-anchor-id="step-3-creating-a-project">Step 3: Creating
<ol start="2" type="1">
<li><em>Project name;</em></li>
</ol>
<p>Select Exploration Mode, fill out a project name and press ‘NEXT’. Note that you can fill out your name and a description as well.</p>
<p>Select Validation Mode, fill out a project name and press ‘NEXT’. Note that you can fill out your name and a description as well.</p>
<p><img src="images/ASReviewLAB/Interface3.png" class="img-fluid" style="width:95.0%"></p>
<p>For this exercise we are screening in the so-called ‘Exploration Mode’ of ASReview. By screening in the <a href="https://asreview.readthedocs.io/en/latest/project_create.html#project-modes" target="_blank">Exploration Mode</a>, we are going to make use of a <a href="https://asreview.readthedocs.io/en/latest/data_labeled.html#fully-labeled-data" target="_blank">benchmark dataset</a>. This means that all records in the dataset have already been labeled as relevant or irrelevant. This is indicated to the user through a banner above each article. Note that in ‘Oracle Mode’ - when screening your own dataset - the relevant papers will not be marked; you, the oracle, have to make the decisions.</p>
<p>For this exercise we are screening in the so-called Validation Mode’ of ASReview. By screening in the <a href="https://asreview.readthedocs.io/en/latest/project_create.html#project-modes" target="_blank">Validation Mode</a>, we are going to make use of a <a href="https://asreview.readthedocs.io/en/latest/data_labeled.html#fully-labeled-data" target="_blank">benchmark dataset</a>. This means that all records in the dataset have already been labeled as relevant or irrelevant. This is indicated to the user through a banner above each article. Note that in ‘Oracle Mode’ - when screening your own dataset - the relevant papers will not be marked; you, the oracle, have to make the decisions.</p>
<p>More detailed information about setting up a project can be found on <a href="https://asreview.readthedocs.io/en/latest/project_create.html#project-information" target="_blank">ReadTheDocs</a>.</p>
<p><em>Have you started creating a new project? If so, you can proceed to step 4!</em></p>
</section>
Expand All @@ -299,7 +299,7 @@ <h3 class="anchored" data-anchor-id="step-4-the-dataset">Step 4: The dataset</h3
<section id="step-5-prior-knowledge" class="level3">
<h3 class="anchored" data-anchor-id="step-5-prior-knowledge">Step 5: Prior knowledge</h3>
<p>Before you can start screening the records, you need to tell ELAS what kind of records you <u>are</u> and what kind of records you <u>are not</u> looking for (i.e., relevant and irrelevant records, respectively). We call this <em>prior knowledge</em>. Based on the prior knowledge you provide, ELAS will reorder the stack of papers and provide you with the record that is most likely to be relevant (default settings).</p>
<p>When performing a systematic review with your own data, you need to provide the prior knowledge yourself (at least one relevant and one irrelevant record). However, because you are using the Exploration Mode of ASReview, the relevant records are known; the original authors have already read ALL records.</p>
<p>When performing a systematic review with your own data, you need to provide the prior knowledge yourself (at least one relevant and one irrelevant record). However, because you are using the Validation Mode of ASReview, the relevant records are known; the original authors have already read ALL records.</p>
<p>To select the prior knowledge you first need to click on the ‘ADD’ button next to ‘Add prior knowledge’; see also the documentation about the selection of <a href="https://asreview.readthedocs.io/en/latest/project_create.html#select-prior-knowledge" target="_blank">prior knowledge</a>. Now you will see a menu about selecting prior knowledge.</p>
<p>The following five papers are known to be relevant:</p>
<ul>
Expand Down Expand Up @@ -328,7 +328,7 @@ <h2 class="anchored" data-anchor-id="screening-phase">Screening phase</h2>
<section id="step-7-screening-the-records" class="level3">
<h3 class="anchored" data-anchor-id="step-7-screening-the-records">Step 7: Screening the records</h3>
<p>Everything is set up and ready to screen, well done!</p>
<p>Since we are in the Exploration Mode of ASReview, you can pretend to be an expert on the topic of PTSD and pretend you have all the knowledge of the original screeners. All records in the dataset have been labeled as relevant/irrelevant, which is indicated through a banner above each article. Click on the heart shaped Relevant button if the record is marked as relevant. If not, you can press the Irrelevant button.</p>
<p>Since we are in the Validation Mode of ASReview, you can pretend to be an expert on the topic of PTSD and pretend you have all the knowledge of the original screeners. All records in the dataset have been labeled as relevant/irrelevant, which is indicated through a banner above each article. Click on the heart shaped Relevant button if the record is marked as relevant. If not, you can press the Irrelevant button.</p>
<p>Now, all we need is a Stopping Rule to determine when you are confident that you have identified (almost) all relevant records in your dataset. For this exercise, continue screening records until you have marked <em>50 consecutive records as irrelevant</em>. You can check up on your progress in the <a href="https://asreview.readthedocs.io/en/latest/progress.html" target="_blank">Analytics page</a>.</p>
<p>When you have reached your Stopping Rule and you are done screening, go back to the Analytics page. Here you can see the summary statistics of your project such as the number of records you have labeled relevant or irrelevant. It also shows how many records are in your dataset and how many records you labeled irrelevant since you have screened the last relevant record. For more information about how to read these summary statistics and interpret the corresponding charts, check out the <a href="https://asreview.readthedocs.io/en/latest/progress.html#analytics" target="_blank">documentation</a> on the Analytics page.</p>
<p>The Van de Schoot (2017) dataset contained 43 relevant records in this particular example. Did you get to label all of them as relevant before you reached your Stopping Rule? If you did, great!</p>
Expand Down
Loading

0 comments on commit b8d8bf2

Please sign in to comment.