Tweet Level


Codeswitched tweets :
2665 |
Monolingual tweets :
209 |
Total tweets :
2874
Team Accuracy Recall Precision F1-Score
JustAnEagerStudent 0.951 0.980 0.968 0.974
MSR-India 0.953 0.989 0.962 0.975
IUCL 0.912 0.956 0.949 0.952
dcu-uvt 0.958 0.994 0.961 0.977
CMU 0.930 0.945 0.979 0.962
A3-107 0.948 0.979 0.966 0.972
Baseline (LangID) 0.421 0.395 0.951 0.559
Baseline (Lexical) 0.904 0.949 0.947 0.948

Codeswitched tweets :
429 |
Monolingual tweets :
1197 |
Total tweets :
1626
Team Accuracy Recall Precision F1-Score
CMU 0.895 0.767 0.823 0.794
JustAnEagerStudent 0.713 0.946 0.478 0.635
IUCL 0.871 0.578 0.899 0.704
Baseline (LangID) 0.768 0.387 0.593 0.468
A3-107 0.873 0.872 0.711 0.783
dcu-uvt 0.838 0.944 0.629 0.755
Baseline (Lexical) 0.708 0.860 0.471 0.608
TAU 0.907 0.816 0.829 0.823
MSR-India 0.853 0.853 0.675 0.754

Codeswitched tweets :
247 |
Monolingual tweets :
66 |
Total tweets :
313
Team Accuracy Recall Precision F1-Score
CMU 0.815 0.931 0.849 0.888
IUCL 0.824 0.943 0.850 0.894
A3-107 0.751 0.814 0.863 0.838
MSR-India 0.818 0.955 0.837 0.892
Baseline (Lexical) 0.607 0.615 0.844 0.712

Codeswitched tweets :
32 |
Monolingual tweets :
2300 |
Total tweets :
2332
Team Accuracy Recall Precision F1-Score
MSR-India 0.947 0.344 0.097 0.152
IUCL 0.974 0.125 0.111 0.118
A3-107 0.605 0.719 0.025 0.048
CMU 0.861 0.531 0.052 0.095
Baseline (Lexical) 0.757 0.531 0.030 0.057

Codeswitched tweets :
293 |
Monolingual tweets :
1484 |
Total tweets :
1777
Team Accuracy Recall Precision F1-Score
CMU* 0.615 0.732 0.347 0.471
CMU 0.662 0.734 0.292 0.417
IUCL 0.766 0.249 0.271 0.260
GWU* 0.440 0.956 0.222 0.360
Baseline (Lexical) 0.513 0.775 0.221 0.344
MSR-INDIA 0.714 0.212 0.183 0.196
A3-107 0.469 0.823 0.213 0.338

Token Level

Nepali English


lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268

Team Recall Precision F1-Score
JustAnEagerStudent 0.944 0.949 0.947
MSR-India 0.967 0.929 0.948
IUCL 0.851 0.891 0.871
dcu-uvt 0.979 0.952 0.965
CMU 0.916 0.948 0.932
A3-107 0.954 0.932 0.943
Baseline (LangID) 0.571 0.765 0.654
Baseline (Lexical) 0.997 0.507 0.672

lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268
Team Recall Precision F1-Score
JustAnEagerStudent 0.965 0.964 0.965
MSR-India 0.980 0.959 0.969
IUCL 0.689 0.976 0.808
dcu-uvt 0.988 0.961 0.974
CMU 0.869 0.964 0.914
A3-107 0.984 0.955 0.969
Baseline (LangID) 0.923 0.628 0.747
Baseline (Lexical) 0.640 0.936 0.760

lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268
Team Recall Precision F1-Score
JustAnEagerStudent 0.000 1.000 0.000
MSR-India 0.000 0.000 0.000
IUCL 0.017 1.000 0.033
dcu-uvt 0.033 0.500 0.063
CMU 0.500 0.012 0.023
A3-107 0.000 1.000 0.000
Baseline (LangID) 0.000 1.000 0.000
Baseline (Lexical) 0.000 1.000 0.000

lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268
Team Recall Precision F1-Score
JustAnEagerStudent 0.510 0.657 0.574
MSR-India 0.350 0.644 0.454
IUCL 0.551 0.487 0.517
dcu-uvt 0.456 0.804 0.582
CMU 0.421 0.599 0.494
A3-107 0.390 0.791 0.522
Baseline (LangID) 0.000 1.000 0.000
Baseline (Lexical) 0.000 1.000 0.000

lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268
Team Recall Precision F1-Score
JustAnEagerStudent 0.968 0.935 0.951
MSR-India 0.955 0.990 0.972
IUCL 0.961 0.208 0.342
dcu-uvt 0.958 0.991 0.974
CMU 0.953 0.959 0.956
A3-107 0.935 0.957 0.946
Baseline (LangID) 0.549 0.915 0.686
Baseline (Lexical) 0.448 0.996 0.618

lang1 :
12286 |
lang2 :
17216 |
mixed :
60 |
ne :
1071 |
other :
9635 |
Total :
40268
Team Overall Accuracy
JustAnEagerStudent 0.946
MSR-India 0.952
IUCL 0.752
dcu-uvt 0.963
CMU 0.890
A3-107 0.946
Baseline (LangID) 0.700
Baseline (Lexical) 0.685

Spanish English


ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.024 0.048 0.032
JustAnEagerStudent 0.000 0.000 0.000
IUCL 0.000 0.000 0.000
Baseline (LangID) 0.000 1.000 0.000
A3-107 0.000 1.000 0.000
dcu-uvt 0.024 0.077 0.037
Baseline (Lexical) 0.000 1.000 0.000
TAU 0.024 0.200 0.043
MSR-India 0.000 1.000 0.000

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.942 0.923 0.933
JustAnEagerStudent 0.924 0.856 0.889
IUCL 0.952 0.930 0.941
Baseline (LangID) 0.891 0.747 0.812
A3-107 0.941 0.917 0.929
dcu-uvt 0.959 0.915 0.936
Baseline (Lexical) 0.991 0.621 0.764
TAU 0.966 0.939 0.952
MSR-India 0.961 0.924 0.942

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.920 0.951 0.936
JustAnEagerStudent 0.817 0.893 0.853
IUCL 0.930 0.935 0.932
Baseline (LangID) 0.853 0.757 0.802
A3-107 0.906 0.935 0.920
dcu-uvt 0.925 0.929 0.927
Baseline (Lexical) 0.424 0.898 0.576
TAU 0.952 0.952 0.952
MSR-India 0.944 0.932 0.938

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.000 0.000 0.000
JustAnEagerStudent 0.000 1.000 0.000
IUCL 0.000 1.000 0.000
Baseline (LangID) 0.000 1.000 0.000
A3-107 0.000 1.000 0.000
dcu-uvt 0.000 1.000 0.000
Baseline (Lexical) 0.000 1.000 0.000
TAU 0.000 1.000 0.000
MSR-India 0.000 1.000 0.000

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.241 0.481 0.321
JustAnEagerStudent 0.238 0.685 0.353
IUCL 0.487 0.625 0.547
Baseline (LangID) 0.000 1.000 0.000
A3-107 0.270 0.540 0.360
dcu-uvt 0.364 0.690 0.476
Baseline (Lexical) 0.000 1.000 0.000
TAU 0.481 0.672 0.561
MSR-India 0.195 0.525 0.285

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Recall Precision F1-Score
CMU 0.939 0.890 0.914
JustAnEagerStudent 0.942 0.911 0.926
IUCL 0.924 0.929 0.927
Baseline (LangID) 0.490 0.803 0.609
A3-107 0.924 0.880 0.902
dcu-uvt 0.926 0.953 0.939
Baseline (Lexical) 0.711 0.986 0.827
TAU 0.942 0.954 0.948
MSR-India 0.943 0.960 0.952

ambiguous :
41 |
lang1 :
7424 |
lang2 :
5278 |
mixed :
13 |
ne :
374 |
other :
4289 |
Total :
17419
Team Overall Accuracy
CMU 0.917
JustAnEagerStudent 0.879
IUCL 0.926
Baseline (LangID) 0.759
A3-107 0.909
dcu-uvt 0.925
Baseline (Lexical) 0.726
TAU 0.942
MSR-India 0.932

Mandarin English


lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Recall Precision F1-Score
CMU 0.980 0.979 0.980
IUCL 0.983 0.978 0.981
A3-107 0.977 0.979 0.978
MSR-India 0.984 0.976 0.980
Baseline (Lexical) 0.990 0.824 0.900

lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Recall Precision F1-Score
CMU 0.832 0.662 0.737
IUCL 0.839 0.666 0.742
A3-107 0.674 0.663 0.669
MSR-India 0.891 0.666 0.762
Baseline (Lexical) 0.381 0.613 0.470

lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Recall Precision F1-Score
CMU 0.000 0.000 0.000
IUCL 0.000 1.000 0.000
A3-107 0.000 1.000 0.000
MSR-India 0.000 1.000 0.000
Baseline (Lexical) 0.000 1.000 0.000

lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Recall Precision F1-Score
CMU 0.740 0.542 0.626
IUCL 0.701 0.503 0.586
A3-107 0.839 0.384 0.527
MSR-India 0.677 0.652 0.664
Baseline (Lexical) 0.000 1.000 0.000

lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Recall Precision F1-Score
CMU 0.215 0.812 0.340
IUCL 0.183 0.920 0.306
A3-107 0.215 0.714 0.330
MSR-India 0.210 0.949 0.344
Baseline (Lexical) 0.176 0.975 0.299

lang1 :
4703 |
lang2 :
881 |
mixed :
1 |
ne :
254 |
other :
442 |
Total :
6281
Team Overall Accuracy
MSR-India 0.904
IUCL 0.895
CMU 0.896
A3-107 0.875
Baseline (Lexical) 0.808

Modern Arabic Dialets


ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Recall Precision F1-Score
MSR-India 0.000 0.000 0.000
IUCL 0.000 0.000 0.000
A3-107 0.000 0.000 0.000
CMU 0.000 0.000 0.000
Baseline (Lexical) 0.000 1.000 0.000

ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Recall Precision F1-Score
MSR-India 0.965 0.921 0.942
IUCL 0.961 0.816 0.882
A3-107 0.924 0.953 0.938
CMU 0.922 0.970 0.946
Baseline (Lexical) 0.987 0.868 0.924

ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Recall Precision F1-Score
MSR-India 0.532 0.093 0.158
IUCL 0.348 0.089 0.142
A3-107 0.397 0.031 0.057
CMU 0.574 0.049 0.090
Baseline (Lexical) 0.206 0.041 0.068


ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Recall Precision F1-Score
MSR-India 0.470 0.748 0.577
IUCL 0.033 0.234 0.058
A3-107 0.702 0.770 0.734
CMU 0.778 0.706 0.740
Baseline (Lexical) 0.000 1.000 0.000

ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Recall Precision F1-Score
MSR-India 0.841 0.994 0.911
IUCL 0.004 0.019 0.006
A3-107 0.897 0.853 0.874
CMU 0.988 0.973 0.981
Baseline (Lexical) 0.821 0.992 0.898

ambiguous :
11 |
lang1 :
44134 |
lang2 :
141 |
ne :
5939 |
other :
3902 |
Total :
54127
Team Overall Accuracy
MSR-India 0.901
IUCL 0.788
CMU 0.910
A3-107 0.896
Baseline (Lexical) 0.864

Modern Arabic Dialets Test 2


ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.000 0.000 0.000
CMU 0.000 0.000 0.000
IUCL 0.000 0.000 0.000
GWU* 0.000 1.000 0.000
Baseline (Lexical) 0.000 1.000 0.000
MSR-INDIA 0.008 0.056 0.015
A3-107 0.000 0.000 0.000

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.844 0.611 0.709
CMU 0.854 0.690 0.763
IUCL 0.907 0.438 0.590
GWU* 0.899 0.620 0.734
Baseline (Lexical) 0.987 0.379 0.547
MSR-INDIA 0.962 0.422 0.587
A3-107 0.913 0.470 0.620

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.741 0.894 0.811
CMU 0.761 0.873 0.813
IUCL 0.477 0.783 0.593
GWU* 0.623 0.888 0.732
Baseline (Lexical) 0.163 0.892 0.276
MSR-INDIA 0.360 0.843 0.505
A3-107 0.344 0.879 0.494

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.000 1.000 0.000
CMU 0.000 1.000 0.000
IUCL 0.000 0.000 0.000
GWU* 0.500 0.005 0.010
Baseline (Lexical) 0.000 1.000 0.000
MSR-INDIA 0.000 1.000 0.000
A3-107 0.000 1.000 0.000

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.655 0.728 0.689
CMU 0.687 0.788 0.734
IUCL 0.085 0.286 0.131
GWU* 0.871 0.971 0.918
Baseline (Lexical) 0.000 1.000 0.000
MSR-INDIA 0.283 0.842 0.424
A3-107 0.585 0.797 0.675

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Recall Precision F1-Score
CMU* 0.986 0.987 0.986
CMU 0.982 0.985 0.984
IUCL 0.011 0.047 0.017
GWU* 0.993 0.968 0.981
Baseline (Lexical) 0.898 0.988 0.941
MSR-INDIA 0.295 0.851 0.438
A3-107 0.801 0.705 0.750

ambiguous :
119 |
lang1 :
10459 |
lang2 :
14800 |
mixed :
2 |
ne :
4321 |
other :
2940 |
Total :
32641
Team Overall Accuracy
CMU* 0.777
CMU 0.798
IUCL 0.519
GWU* 0.775
Baseline (Lexical) 0.471
MSR-INDIA 0.536
A3-107 0.598