Norwegian stemming algorithm

Links to resources

• Scandinavian language stemmers
• Javascript demo
• The stemmer in Snowball
• Sample Norwegian vocabulary
• Its stemmed equivalent
• Norwegian stop word list (UTF-8 encoding)

Here is a sample of Norwegian vocabulary, with the stemmed forms that will be generated by this algorithm:

The stemming algorithm

The Norwegian alphabet includes the following additional letters,

æ   å   ø

The following letters are vowels:

a   e   i   o   u   y   æ   å   ø

R2 is not used: R1 is defined in the same way as in the German stemmer. (See the note on R1 and R2.)

Define a valid s-ending as one of

b   c   d   f   g   h   j   l   m   n   o   p   r   t   v   y   z,
or k not preceded by a vowel.

Do each of steps 1, 2 and 3.

Step 1:

Search for the longest among the following suffixes in R1, and perform the action indicated.

(a) a   e   ede   ande   ende   ane   ene   hetene   en   heten   ar   er   heter   as   es   edes   endes   enes   hetenes   ens   hetens   ers   ets   et   het   ast

delete

(b) s

delete if preceded by a valid s-ending

(c) erte   ert

replace with er

(Note that only the suffix needs to be in R1, the letter of the valid s-ending is not required to be.)

Step 2:

If the word ends dt or vt in R1, delete the t.

(For example, meldt → meld, operativt → operativ)

Step 3:

Search for the longest among the following suffixes in R1, and if found, delete.

leg   eleg   ig   eig   lig   elig   els   lov   elov   slov   hetslov

The same algorithm in Snowball

routines (
           mark_regions
           main_suffix
           consonant_pair
           other_suffix
)

externals ( stem )

integers ( p1 x )

groupings ( v s_ending )

stringescapes {}

/* special characters */

stringdef ae   '{U+00E6}'
stringdef ao   '{U+00E5}'
stringdef o/   '{U+00F8}'

define v 'aeiouy{ae}{ao}{o/}'

define s_ending  'bcdfghjlmnoprtvyz'

define mark_regions as (

    $p1 = limit

    test ( hop 3 setmark x )
    goto v  gopast non-v  setmark p1
    try ( $p1 < x  $p1 = x )
)

backwardmode (

    define main_suffix as (
        setlimit tomark p1 for ([substring])
        among(

            'a' 'e' 'ede' 'ande' 'ende' 'ane' 'ene' 'hetene' 'en' 'heten' 'ar'
            'er' 'heter' 'as' 'es' 'edes' 'endes' 'enes' 'hetenes' 'ens'
            'hetens' 'ers' 'ets' 'et' 'het' 'ast'
                (delete)
            's'
                (s_ending or ('k' non-v) delete)
            'erte' 'ert'
                (<-'er')
        )
    )

    define consonant_pair as (
        test (
            setlimit tomark p1 for ([substring])
            among(
                'dt' 'vt'
            )
        )
        next] delete
    )

    define other_suffix as (
        setlimit tomark p1 for ([substring])
        among(
            'leg' 'eleg' 'ig' 'eig' 'lig' 'elig' 'els' 'lov' 'elov' 'slov'
            'hetslov'
                (delete)
        )
    )
)

define stem as (

    do mark_regions
    backwards (
        do main_suffix
        do consonant_pair
        do other_suffix
    )
)