Topic: How is the familiarity of the expert calculated into the overall score of all reviews?  (Read 63567 times)

For the review scores, we want to take into account the familiarity of the experts with the topic of the submission.
How can we activate this option?
How is the overall review score calculated when the familiarity of the expert is considered?

You can include the familiarity of the expert in the calculation for the overall score of each submission. This means that a review that has been done by a reviewer with low familiarity will have less weight in comparison to a reviewer with high familiarity.

You can enable the option here:
Overview => Settings => Review Form Options and Factors
Go to the option “Familiarity of Expert” and enable it (see image 1).
When you click on “Edit Labels” directly in the option, you can define the labels and criteria of the form (see image 1).
When you do not enter any wording into a label, it will not be displayed to normal users.

While Reviewers enter their review, they will now be asked how familiar they are with the topic of the submission (see image 2).

Calculation Method

The familiarity of the experts is calculated into the reviews as shown by the following example:

A submission is reviewed by two experts. Expert A specifies her familiarity to be on a level of 10, while expert B states that his familiarity is only a level 4. The total review score of expert A will be multiplied by 10, the total review score of expert B will be multiplied by 4. Both results will be added up and divided by 14, thus creating the final review score.