Commit History

[Fix] change default models
b8a1ed4
Running

Joschka Strueber commited on

[Ref] back to underscore
42ed9bf

Joschka Strueber commited on

[Fix] subscript kappa
47b5af7

Joschka Strueber commited on

[Fix] alpha choices
b16e2d1

Joschka Strueber commited on

[Fix] replace latex with unicode and markdown
402b600

Joschka Strueber commited on

[Ref] back to markdown
bd1b20b

Joschka Strueber commited on

[Ref] switch to KaTeX Css in html
0d09d9a

Joschka Strueber commited on

[Ref] switch from mathjax in markdown to html block
5623280

Joschka Strueber commited on

[Fix] mathjax in metric explanation
69fd3ae

Joschka Strueber commited on

[Fix] change latex to mathjax
274c92e

Joschka Strueber commited on

[Ref, Fix] indentation error in answer key selection, longer explanation in demo, exclusion of broken dataset
c608f7f

Joschka Strueber commited on

[Fix] error in dataset name, error in digit check for str
3dfa66b

Joschka Strueber commited on

[Ref] list comprehensions in label filtering
64789e4

Joschka Strueber commited on

[Fix] error in label filtering
2d8352e

Joschka Strueber commited on

[Fix] error in filter_responses call
7c4f6b6

Joschka Strueber commited on

[Fix] import error
001064a

Joschka Strueber commited on

[Add] add bbh and gpqa benchmarks again with correct answer_index selection
0a42e99

Joschka Strueber commited on

[Ref] apply custom css to heatmap, increase size of images
4077e51

Joschka Strueber commited on

[Ref, Add] custom css for sizing, move demo utility to its own file
bd28414

Joschka Strueber commited on

[Ref] change table size
1b549fb

Joschka Strueber commited on

[Add, Ref] Add more info and table on metric, move model list to data/
b90e0d3

Joschka Strueber commited on

[Fix] removal of not working benchmarks
c24946e

Joschka Strueber commited on

[Fix] default heatmap
26c0eec

Joschka Strueber commited on

[Fix] error in default heatmap
c4145ee

Joschka Strueber commited on

[Add] ignore datasets that are not functional atm
bf2618d

Joschka Strueber commited on

[Add] default heatmap
45b2347

Joschka Strueber commited on

[Fix] key error for binary datasets
9e1c5ed

Joschka Strueber commited on

[Fix, Debug] wrong default model, check filter_labels
4b2993a

Joschka Strueber commited on

[Fix] error in deleting not-matching gt values
75132dc

Joschka Strueber commited on

[Ref] dataset selection
b1f98e1

Joschka Strueber commited on

[Fix] error in dataset default selection
58da8de

Joschka Strueber commited on

[Add, Fix] add list of ungated models
5c5dc6a

Joschka Strueber commited on

[Ref, Fix] use cached list of usable models, convert logits to OneHot for EC as well
64b132e

Joschka Strueber commited on

[Fix] type in default model name
cb7e104

Joschka Strueber commited on

[Debug] EC error
1f20712

Joschka Strueber commited on

[Ref, Add] change default models, remove sorting in plot
8be99c0

Joschka Strueber commited on

[Add] only load cached models
ec5f717

Joschka Strueber commited on

[Fix] wrong API calls
9f3c166

Joschka Strueber commited on

[Ref] check number of saved and loaded models
e604b65

Joschka Strueber commited on

[Ref] check number of saved models
715aed5

Joschka Strueber commited on

[Fix] add check if cached files have been saved
81438ca

Joschka Strueber commited on

[Fix, Add] check for #api calls, bug in warning
047f32f

Joschka Strueber commited on

[Add, Fix] add loading mechanism for cached models, change error to warning when computing heatmap
93d753c

Joschka Strueber commited on

[Add] saving unblocked models as file to read from
1e010df

Joschka Strueber commited on

[Fix, Add] fix bug with metric names
d2471f2

Joschka Strueber commited on

[Fix] catch all errors from API access
1072829

Joschka Strueber commited on

[Add] cache loading data from hf
e64ca4e

Joschka Strueber commited on

[Add] list of default models
5815cf9

Joschka Strueber commited on

[Add, Fix] change to CAPA, fix error in dataloading
ce6be70

Joschka Strueber commited on

[Add] filter gated models
5d4059c

Joschka Strueber commited on