ADAPTING THE FREIBURG MONOSYLLABIC WORD TEST FOR SLOVENIAN

hearing test. For this purpose, the Slovenian adaptation of the German tests developed by Hahlbrock (1953, 1960) – the Freiburg Monosyllabic Word Test and the Freiburg Number Test – are used in Slovenia (adapted in 1968 by Pompe). In this paper we focus on the Freiburg Monosyllabic Word Test for Slovenian, which has been criticized by patients as well as in the literature for the unequal difficulty and frequency of the words, with many of these being extremely rare or even obsolete. Since part of the patient’s communication function is retrieving the meaning of individual words by guessing, the less frequent and consequently less familiar words do not contribute to reliable testing results. We therefore adapt the test by identifying and removing such words and sup - plement them with phonetically similar words to preserve the phonetic balance of the list. The words used for replacement are extracted from the written corpus of Slovenian Gigafida and the spoken corpus of Slovenian GOS, while the optimal combinations of words are established by using computational algorithms.


INTRODUCTION
Speech audiometry is one of the standard methods used to diagnose the type of hearing loss and to assess the communication function of the patient by determining the level of the patient's ability to understand and repeat words presented to him or her in a hearing test.For this purpose, the Slovenian adaptation of the German tests developed by Hahlbrock (1953Hahlbrock ( , 1960) ) -the Freiburg Monosyllabic Word Test and the Freiburg Number Test -are used in Slovenia (adapted in 1968 by Pompe).These tests employ the use of phonetically balanced lists of existent monosyllabic words with the aim of determining the percentage of correctly repeated words at different sound intensity levels.
In this study we focus on the Freiburg Monosyllabic Word Test for Slovenian, which has been criticized by patients in personal communication during and after testing, as well in the literature for the unequal difficulty and frequency of the words, with many of extremely rare or even obsolete items (Podlesek et al. 2007;Podlesek et al. 2008). 1 As part of the patient's communication function is retrieving the meaning of individual words by guessing, the frequency of use of an individual word is crucial.The less frequent and consequently less familiar words (e.g. dac, golč, irh, lat, raš, sak) do not contribute to a reliable testing result, as they cannot be guessed to the same extent as more familiar words (e.g. bor, klop, pas, sin).We therefore propose that the test be adapted by identifying and removing less familiar words from the list and supplementing them with phonetically similar words so as to preserve the phonetic balance of the list.
The paper is organized as follows.In Section 2 we provide a general description of the Freiburg test.In Section 3 we proceed to identify less familiar words in the Freiburg test, while in Section 4 we describe the procedure of replacing the less familiar words with more familiar ones that we extract from various Slovenian corpora, the result being a new version of the Freiburg test.Section 5 discusses some remaining issues and Section 6 concludes the paper.

Speech Audiometry
Classic pure-tone audiometry assesses only basic deficits in auditory function.The audiometric curve determines the detected threshold levels (in dB) for selected frequencies.To evaluate the clinical impact of hearing loss, disorders affecting auditory pathways after the cochlea, and especially the rehabilitation of severe hearing loss and deafness with cochlear implants, it is necessary to use various other audiometric tests, such as sound localization, auditory discrimination, auditory pattern processing and speech audiometry.Speech audiometry assesses the understanding of words presented at a specified loudness in different conditions (Musiek et al. 2011).
A speech recognition test consists of the patient's listening and repeating words, with the clinician marking a tally of right and wrong responses.The percentage of test words correctly repeated by the patient is referred to as the speech recognition score (also the word recognition score or speech discrimination score).The percentage of the correctly repeated words depends on more than just the patient's speech recognition ability; it also depends on the patient's familiarity with the words and on the intensity at which the words are presented.The graph of performance -intensity function shows how the patient's speech recognition performance depends on the intensity of the test materials (Gelfand 2009).2Different diseases of the middle, inner ear and central auditory pathways result in different speech recognition scores and different performance -intensity curves, despite having similar pure tone audiometry curves (van Dijk et al. 2000).The speech recognition score is essential in evaluating and comparing the rehabilitation effects achieved with the use of classical hearing aids or cochlear implants (De Riuter 2015).
In the Department of Otorhinolaryngology, University Medical Centre Ljubljana, speech is normally assessed with the Slovenian adaptation of the German tests developed in Hahlbrock (1953Hahlbrock ( , 1960)), and the Freiburg Number Test.The Slovenian adaptations of these were developed in Pompe (1968).After the patient is fitted with a hearing aid, a 20% improvement of the speech recognition score represents a significant therapeutic effect.However, if the score of a patient fitted with a standard hearing aid is lower than 50% of the highest possible score, then this approach is not a satisfactory rehabilitation method, and a cochlear implant should be considered.Moreover, a constantly improving score following the rehabilitation of a cochlear implant user is proof of a well-selected speech rehabilitation method and the effective work of the related language specialists.

Freiburg Monosyllabic Word Test
In this paper we focus on the Slovenian adaptation of the Freiburg Monosyllabic Word Test (henceforth Freiburg Test-SLO-1968).This consists of 281 monosyllabic nouns in the nominative singular form (nine of them repeated).In the test, a patient listens to phonetically balanced columns of 28-29 monosyllabic Slovenian words in a quiet environment, with the stimulus intensity level increased in each consecutive column.A speech audiogram with the percentage of correctly repeated words at each level serves as the basis for estimating the patient's communication function.

IDENTIFYING LESS FAMILIAR WORDS IN THE FREIBURG TEST
The first step in our project is to identify and eliminate the less familiar words that appear in the Freiburg Test-SLO-1968.As there is no existing data based on the patient judgements of the words' frequencies, we refer to Podlesek et al. ( 2007), the corpus of written Slovenian Gigafida, and the corpus of spoken Slovenian GOS.

Podlesek et al. (2007)
Podlesek et al. (2007) gathered data on the frequency of the Freiburg test words in everyday spoken language, as judged by the native speakers of Slovenian.The frequency was assessed by a sample of 141 students who were given written lists of the Freiburg test words and asked to assess the frequency of occurrence of each in their everyday lives (i.e.how often they hear it on TV, radio, or use it in spoken language) by using a 5-point Likert scale (0 -never, 5 -very often).The information gathered is one of our criteria in providing a list of less familiar words.For a word to be considered less familiar, we set the threshold at the average score 1 or less (on a scale from 1 to 5) in the Podlesek et al. ( 2007) survey of native speakers.There are 65 such words, listed in (1).

Reference Corpora
Several reference corpora for the Slovenian language are freely accessible at the internet portal of the project "Communication in Slovene" (http://eng.slovenscina.eu/korpusi).A natural choice for determining the frequency of spoken words would be the corpus of spoken Slovenian GOS (for details on the corpus see Zemljarič Miklavčič et al. 2009;Verdonik et al. 2013).The corpus contains transcripts of approximately 120 hours of speech found in various situations: radio and TV shows, school lessons and lectures, private conversations between friends or within the family, work meetings, consultations, conversations in buying and selling situations.All speech is transcribed in two versions -with pronunciation-based spelling and with standardized spelling.The corpus contains around one million words.However, the drawback of the corpus is that it is still relatively small and many of the words that we would expect in a corpus of Slovenian are not found there, or have a very low number of hits (ceh "guild"-0, noj "ostrich"-0, volk "wolf"-4).
A much larger corpus than GOS is Gigafida (an upgrade of Fidaplus), which contains about 1.2 billion words (see Erjavec and Logar Berginc (2012), Logar Berginc and Krek (2012) and Logar Berginc et al. (2012) for more information on this corpus).The corpus has been automatically lemmatized and includes morphosyntactic descriptions (part-of-speech, gender, case, number).The option "advanced search" enables the user to determine the part-of-speech of the word (noun, verb, adjective) as well as choose whether to search only for a particular form or for all forms of a word.This search engine is to some extent successful in eliminating the erroneous hits when the part-of-speech is specified.For example, when searching for the word teč "runsupine", where the option noun is chosen, the search gives no hits, which is expected, as teč is a verb.Yet, with smuč -in an advanced search specified for noun -the hits are all the noun, verb and adjective occurrences, as the search engine provides hits with the meaning "pike perch" (nouns) as well as those with the meaning "ski" (verbs and adjectives, e.g.smuč.skoki "ski jumps").Even narrowing down the search to only the form smuč does not help -we still get the adjectival hits related to the meaning "ski".This is probably due to the incorrect connection between the lemma smuč "pike perch", present in the Slovar slovenskega knjižnega jezika (Dictionary of Standard Slovenian, hereafter SSKJ), and all the forms smuč -the ones relating to the meaning "pike perch" and those relating to the meaning "ski" (since the two are homonymous).
Reference corpora should thus be used with caution when establishing the frequency of individual words.Regardless of which corpus is used, special attention should be paid to the content of the results.There are numerous cases where an unlimited search in a corpus provides a very high number of hits for a certain word, but it then turns out that the vast majority of these are not for the word checked, but for some other, more familiar word that is homonymous with the word in question.

Final List of the Less Familiar Words in the Freiburg Test-SLO-1968
We now return to establishing the final list of the words to be eliminated from the Freiburg Test-SLO-1968, combining the results in Podlesek et al. (2007) and data from corpora available to us.First, there are some additional words that need to be considered for elimination from the Freiburg Test-SLO-1968, despite the fact that their frequency was not judged as below 1 in Podlesek et al.'s (2007) native speakers' test: (2) dna, hot, lišp, smuč, teč Let us begin with the words hot and smuč, which score 1.06 and 1.07, respectively, in the test for native speakers.According to SSKJ, the two words have the meanings hot "an interjection for a horse" and smuč "pike perch", which are words that are rarely used by speakers in their everyday lives.Both words were also erroneously recognized as frequent in the corpora, as a careful examination revealed that all or a great number of their hits are not for the actual dictionary meanings of the two words, but rather refer to the borrowed combination hot dog (for hot) and to the adjective smučarski, abbreviated as smuč "ski" (as described in the previous section) 4 .
A different problem occurs with the words dna and teč, which score 1.06 and 1.13, respectively.One of these is not a nominative singular noun, as is true of other nouns in the Freiburg Test- SLO-1968 (dna), while the other is not a noun at all (teč). 5An examination of the hits in the corpora reveals that the ones for dna are actually the plural nominative or accusative forms of the word dno "bottom" or the acronym DNA, and the hits for teč are the supine forms of the verb teči "to run".Finally, we decide to replace the word lišp, as it is marked archaic in SSKJ as well as in Slovenski pravopis (Slovenian Orthography 2001).

CONSTRUCTING THE FREIBURG TEST-SLO-2016
The next step is supplementing the words in (4) with more familiar words so as to preserve the letter frequencies in each column of the original test (Table 1).We begin by building the database of possible replacements (Section 4.1) and proceed to finding the optimal ones (Section 4.2).

Constructing the Database of Possible Replacement Words
To construct the database of possible replacements for the words in (4) we again refer to the corpora.For this purpose, we use the GOS and ccGigafida corpora, as they have XML sources available.The ccGigafida is ten times smaller than its base corpus Gigafida, and was made by random paragraph selection.The full Gigafida is unfortunately not available with its source, and ccGigafida is currently one of the largest freely available corpora of Slovenian (cc stands for the Creative Commons-Attribution-NonCommercial license).
where we specify the gender, thus eliminating the homonymous results (e.g.polk "regimentmasculine" is homonymous with polk "polka dance-feminine/plural/genitive").For others, where this approach does not work, we search for a particular form of the word.For example, with the word ar "are" we search for arov "are-plural/genitive", as ar "are-singular/nominative" mostly gives completely unrelated hits (acronyms, ara "downpayment", etc.).
The database of monosyllabic singular nouns from GOS and ccGigafida was constructed by first extracting the lemmas of all nouns (by searching for the lemmas with the XML msd tag "S*", where * represents any number of any characters).We then extract all nouns with one vowel and all those that contain the sonorant r, but no vowel (as in these the schwa appears in pronunciation, but not in writing, e.g.vrt "garden").The processing of the words in corpora was performed using Mathematica software (Wolfram Research), which has numerous built-in word-analysis tools.ccGigafida gives 21,942 hits and GOS 1,190.We then set the limit as to the number of hits for a word to be kept in the database; for the nouns from ccGigafida we set the limit at 200 hits, while we keep all the nouns from GOS, as this is a relatively small, but representative corpus of spoken language.We use both corpora because not all the words present in GOS have more than 200 hits in ccGigafida (e.g.jež "hedgehog" has eight hits in GOS and 187 in ccGigafida).This leaves us with 1,771 nouns from both corpora.
We then need to eliminate unsuitable words from this list.First, we exclude all the words that are acronyms or non-Slovenian words (mr, oš, fahr, boys, etc.).We then pass the words through another filter, as we need to eliminate colloquial or slang words (šiht, starš, ksiht, kšeft, baš, dec, etc.), the remaining English words (show, what, fan, bird, pub, etc.), acronyms (dag, kfor, kud, pef, sos, etc.), pronouns (jaz), proper names (Ptuj, Cres, Krim, Jan, etc.), words potentially uncomfortable for the speaker to say (seks), vulgar words (rit, fuk, drek, etc.) and the words that have one vowel, but are in fact bisyllabic (črka, brlog, črnec, prvak, etc.).When this is completed, the remaining words have to be checked against the list in the Freiburg Test-SLO-1968, as we have to make sure not to use words already present in the test as replacements.The final list contains 348 monosyllabic nouns that are suitable as replacements.

Finding Optimal Replacements
The goal was to replace the less familiar words in each column of the Freiburg Test-SLO-1968 (4) with words from the list of suitable replacements that was derived from the corpora, while preserving the letter frequencies in each column.We rely on computational algorithms to find the combinations of words that satisfy the letter frequency criterion.This is a computationally demanding task, since for large word sets it is impossible to search through all possible combinations of words and test them for the desired criterion (for example, the number of all possible combinations of 10 words from a set of 348 words is on the order of 10 20 ).We therefore employ a version of a recursive back-tracking algorithm (Knuth 2016), where the paths that cannot lead to a solution are discarded from the search tree (e.g. if one is looking for two words with a total of five letters, there is no need to search through combinations of words with a larger number of letters).
To further speed up the computation, the words were first vectorized in a 25-dimensional space of letters according to their letter count, and the task was then solved in Matlab, which is optimized for efficient vector computation.As an illustration: if the alphabet contained only three letters, the unit vectors would be "a"= (1,0,0), "b"=(0,1,0), and "c"=(0,0,1), and the word "aaac" would, for example, correspond to a vector (3,0,1).By using the optimized algorithm, we were able to find many combinations of the replacement words that exactly matched the letter count criterion for all columns except for column 5.For this column, where only two words had to be replaced (ril and šeh), the most suitable match differed by three letters.7Finally, the list of words that we propose for the Freiburg Test-SLO-2016 (shown in Table 4) was manually selected from the results returned by the algorithm.
Table 4: The proposed Freiburg Test-SLO-2016.The words that replaced less familiar words from the original test are marked with *.All the columns, except for column 5, exactly match the letter frequencies of the corresponding columns from the Freiburg Test-SLO-1968, while column 5 differs by three letters ("m," "v", "v").

Phonetic Balance
In this paper we follow Pompe and establish phonetic balance by referring to the letters in writing and not the actual sounds pronounced in the words.The phonetic balance achieved in this way is an approximation of the phonetic balance that takes into account the actual pronunciation.To explain this we need to refer to the notion of phoneme and allophone, and their relation to the letters in the alphabet.A phoneme is standardly defined as the smallest sound unit that can be segmented from the acoustic flow of speech and which functions as a semantically distinctive unit: if a sound unit is replaced by another sound unit in a word and the two words have a different meaning, we define the two sound units as phonemes, e.g. in the English pair pet -bet, /p/ and /b/ are phonemes.Phonemes are abstract units, each phoneme representing a class of phonetically similar sound variants, the allophones, which are in a complementary distribution, depending on the phonological environment they appear in.For example, in English, the phoneme /p/ has an aspirated variant [p h ] at the beginning of the syllable (as in pet), but a non-aspirated variant [p] elsewhere (e.g.loop).
The writing systems that use letters can be organized in different ways -some of them tend to use a letter to denote a phoneme, others are closer to using a letter for an allophone.In Slovenian, the tendency is for one letter to represent one phoneme.For example, the letter "n" stands for the phoneme /n/, which has three allophones: [ŋ] when followed by a velar consonant as in Anglija "England;"; [n j ] (for some speakers) when followed by [j#] or [jC] as in konj "horse," konjski "horse-adj" and [n] elsewhere, e.g.nos "nose".However, there is no one-to-one correspondence among phonemes and letters, as there are more phonemes than letters (29 versus 25).In fact, there are many cases in which a single letter stands for two or more phonemes, e.g. the letter "e" can denote [e] in led "ice", [ɛ] in žep "pocket" or [ə] in pes "dog".Finally, for some phonemes, no letter is used: in many words that contain the consonant [r] and the vowel [ə], the vowel is pronounced, but not expressed in writing: vrt "garden," smrt "death," etc.
Referring to letters instead of phonemes or allophones is thus an approximation on two levels.First, the letter-phoneme correspondence is not always one-to-one, and second, even if it were, the phonemes themselves can refer to different sounds in pronunciation, i.e. their allophones (see /n/ above).Referring to the allophones in the phonetic balance calculation would require a much more thorough linguistic analysis.It would, for example also require considering all the phonological rules that take place in Slovenian, such as the final devoicing of voiced obstruents (the word bog "god" is pronounced the same way as bok "hip"), the changes that occur at word boundaries, and the like.Moreover, the search engines of the corpora are organized according to written and not spoken language, with the exception of GOS, which is too small to be the only representative corpus of the language (see also Section 3.2).As corpora such as ccGigafida are crucial for establishing the word frequencies and building the database of words that we need for adapting the Freiburg test, we use the letters as approximations of the actual sounds, bearing in mind the limitations that come with this.
Another issue relating to phonetic balance that remains a challenge in our future research is balancing the occurrence of individual letters in the test with the occurrence of individual letters in the Slovenian language.We mentioned in Section 2 that the distribution of letters in the Freiburg Test-SLO-1968 only broadly reflects the distribution in the actual language (with the latter established in Jakopin (1999)).The same is true for the Freiburg Test-SLO-2016, as the new test has been designed in such a way that it preserves the phonetic balance of the older version.A more exact balance still remains to be achieved.

Syllable Structure
The Freiburg Test-SLO-1968 and the Freiburg Test-SLO-2016 are balanced with respect to the number of letters in individual columns consisting of 28 or 29 words.A possible balance to consider in future work is that with respect to the types of syllables that appear in the language, some of which are exemplified in (5) with the related notation shown in (6).
(5) um "mind" → Vs gol "goal" → oVs ples "dance" → osVo ring "ring" → sVso sklad "fund" → oosVo (6) Notation V for vowel; spelled a, e, i, o, u s for sonorant consonant; spelled m, n, v, j, l, r o for obstruent consonant spelled p, t, b, d, k, g, h, f, c, č, dž, s, š, z, ž The analysis of syllable structure in the test columns shows that 36 different syllable combinations are used in the test.Given the fact that one column consists of 28 or 29 monosyllabic nouns, not all combinations can be present in each individual column.The balance in terms of syllable structure appears a complex issue, and we thus leave it for future research.

CONCLUSION
In this paper we adapted the 1968 version of the Freiburg Monosyllabic Word Test for Slovenian by identifying and removing less familiar words from the list, supplementing them with phonetically similar words so as to preserve the phonetic balance of the list.The result is a new test, the Freiburg Test-SLO-2016, as well as a new database of monosyllabic nouns that are commonly used by native speakers of Slovenian.The new Freiburg test presents a great improvement in speech audiometry clinical practice in Slovenia, while the related database can be used for constructing new tests for diagnosing hearing loss in the future.The new test with its clinical implementation will provide a tool for better assessment of the patient's rehabilitation with different hearing aids.It will enable clinicians to select good candidates for cochlear implantation, and to distinguish different pathologies in central auditory pathways.
The adaptation crucially required the use of Slovenian corpora, the written corpus Gigafida and the spoken corpus GOS.These were of great help when determining the frequency of the words that appear in the test and in the extraction of new nouns needed as replacements, though we did encounter some problems with lemmatization and morphosyntactic tagging.In this light, we strongly encourage further funding and research of advanced algorithms of Slovenian corpora (e.g.accurate automatic lemmatization), as such work would greatly advance the application of the corpora in studies of the Slovenian language, resulting, among other things, in improved clinical practice with hearing impaired patients.hearing test.For this purpose, the Slovenian adaptation of the German tests developed by Hahlbrock (1953, 1960) -the Freiburg Monosyllabic Word Test and the Freiburg Number Test -are used in Slovenia (adapted in 1968 by Pompe).In this paper we focus on the Freiburg Monosyllabic Word Test for Slovenian, which has been criticized by patients as well as in the literature for the unequal difficulty and frequency of the words, with many of these being extremely rare or even obsolete.Since part of the patient's communication function is retrieving the meaning of individual words by guessing, the less frequent and consequently less familiar words do not contribute to reliable testing results.We therefore adapt the test by identifying and removing such words and supplement them with phonetically similar words to preserve the phonetic balance of the list.The words used for replacement are extracted from the written corpus of Slovenian Gigafida and the spoken corpus of Slovenian GOS, while the optimal combinations of words are established by using computational algorithms.

a
b c č d e f g h i j k l m n o p r s š t u v z ž

Table 1 :
Letter frequency in the Freiburg Test-SLO-1968 in test columns 1 through 10. 3

Table 2 :
Letter frequencies in Slovenian literature