Added wino and crow-s in Stereotype

JohnSnowLabs · Nov 13, 2023 · b6d06ef · b6d06ef
1 parent 059fd62
commit b6d06ef
Show file tree

Hide file tree

Showing 5 changed files with 18 additions and 15 deletions.
diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml
@@ -125,8 +125,8 @@ tests:
     url: /docs/pages/tests/sensitivity
   - title: Factuality
     url: /docs/pages/tests/factuality
-  - title: Wino Bias
-    url: /docs/pages/tests/wino-bias
+  - title: Stereotype
+    url: /docs/pages/tests/stereotype
   - title: Crows Pairs
     url: /docs/pages/tests/crows-pairs
   - title: StereoSet

diff --git a/...s/tests/crows-pairs/common_stereotypes.md → ...es/tests/stereotype/common_stereotypes.md b/...s/tests/crows-pairs/common_stereotypes.md → ...es/tests/stereotype/common_stereotypes.md
@@ -19,10 +19,11 @@ The test is passed if the absolute difference in the probability of masks is und
 
 #### Config
 ```yaml
-common-stereotypes:
-    min_pass_rate: 0.7
-    diff_threshold: 0.7
-    filter_threshold: 0.7
+stereotype:
+    common-stereotypes:
+      min_pass_rate: 0.70
+      diff_threshold: 0.10
+      filter_threshold: 0.15
 ```
 - **min_pass_rate (float):** Minimum pass rate to pass the test.
 - **diff_threshold (float):** The allowed difference in probabilities.

diff --git a/...ias/gender-occupational-stereotype-llm.md → ...ype/gender-occupational-stereotype-llm.md b/...ias/gender-occupational-stereotype-llm.md → ...ype/gender-occupational-stereotype-llm.md
@@ -18,7 +18,8 @@ This test is designed to assess gender-based occupational stereotypes specifical
 
 #### Config
 ```yaml
-gender-occupational-stereotype:
+stereotype:
+   wino-bias:
     min_pass_rate: 0.70
 ```
 - **min_pass_rate (float):** Minimum pass rate to pass the test.

diff --git a/...no-bias/gender-occupational-stereotype.md → ...reotype/gender-occupational-stereotype.md b/...no-bias/gender-occupational-stereotype.md → ...reotype/gender-occupational-stereotype.md
@@ -20,9 +20,10 @@ The test is passed if the absolute difference in the probability of male-pronoun
 
 #### Config
 ```yaml
-gender-occupational-stereotype:
-    min_pass_rate: 0.7
-    diff_threshold: 0.03
+stereotype:
+    wino-bias:
+      min_pass_rate: 0.70
+      diff_threshold: 0.03
 ```
 - **min_pass_rate (float):** Minimum pass rate to pass the test.
 - **diff_threshold (float):** The allowed difference in probabilities.

diff --git a/docs/pages/tests/wino-bias.md b/docs/pages/tests/wino-bias.md
@@ -1,10 +1,10 @@
 ---
 layout: docs
 header: true
-seotitle: Wino Bias | LangTest | John Snow Labs
-title:  Wino Bias 
+seotitle: Stereotype | LangTest | John Snow Labs
+title:  Stereotype 
 key: tests
-permalink: /docs/pages/tests/wino-bias
+permalink: /docs/pages/tests/stereotype
 aside:
     toc: true
 sidebar:
@@ -16,10 +16,10 @@ modify_date: "2019-05-16"
 
 <div class="main-docs" markdown="1">
 
-{% assign parent_path = "pages/tests/wino-bias" %}
+{% assign parent_path = "pages/tests/stereotype" %}
 {% for file in site.static_files %}
     {% if file.path contains parent_path %}
-        {% assign file_name = file.path | remove:  parent_path | remove:  "/" | prepend: "wino-bias/" %}
+        {% assign file_name = file.path | remove:  parent_path | remove:  "/" | prepend: "stereotype/" %}
         {% include_relative {{ file_name }} %}        
     {% endif %}
 {% endfor %}