The Snowball stemmer represents the Cyrillic alphabet with ASCII characters, following the standard Library of Congress transliteration scheme.
Here is a sample of Russian vocabulary, with the stemmed forms that will be generated by this algorithm:
| word | stem | word | stem | |||||||||
|
в вавиловка вагнера вагон вагона вагоне вагонов вагоном вагоны важная важнее важнейшие важнейшими важничал важно важного важное важной важном важному важности важностию важность важностью важную важны важные важный важным важных вазах вазы вакса вакханка вал валандался валентина валериановых валерию валетами вали валил валился валится валов вальдшнепа вальс вальса вальсе вальсишку вальтера валяется валялась валялись валялось валялся валять валяются вам вами |
⇒ |
в вавиловк вагнер вагон вагон вагон вагон вагон вагон важн важн важн важн важнича важн важн важн важн важн важн важност важност важност важност важн важн важн важн важн важн ваз ваз вакс вакханк вал валанда валентин валерианов валер валет вал вал вал вал вал вальдшнеп вальс вальс вальс вальсишк вальтер валя валя валя валя валя валя валя вам вам |
п па пава павел павильон павильонам павла павлиний павлиньи павлиньим павлович павловна павловне павловной павловну павловны павловцы павлыч павлыча пагубная падает падай падал падала падаль падать падаю падают падающего падающие падеж падение падением падении падений падения паденье паденья падет падут падучая падчерицей падчерицы падшая падшей падшему падший падшим падших падшую паек пазухи пазуху пай пакет пакетом пакеты пакостей пакостно пал |
⇒ |
п па пав павел павильон павильон павл павлин павлин павлин павлович павловн павловн павловн павловн павловн павловц павлыч павлыч пагубн пада пада пада пада падал пада пада пада пада пада падеж паден паден паден паден паден паден паден падет падут падуч падчериц падчериц падш падш падш падш падш падш падш паек пазух пазух па пакет пакет пакет пакост пакостн пал |
i-suffixes (*) of Russian tend to be quite regular, with irregularities of declension involving a change to the stem. Irregular forms therefore usually just generate two or more possible stems. Stems in Russian can be very short, and many of the suffixes are also particle words that make ‘natural stopwords’, so a tempting way of running the stemmer is to set a minimum stem length of zero, and thereby reduce to null all words which are made up entirely of suffix parts. We have been a little more cautious, and have insisted that a minimum stem contains one vowel.
The 32 letters of the Russian alphabet are as follows, with the transliterated forms that we will use here shown in brackets:
| а (a) | б (b) | в (v) | г (g) | д (d) | е (e) | ж (zh) | з (z) | |||||||
| и (i) | й (ì) | к (k) | л (l) | м (m) | н (n) | о (o) | п (p) | |||||||
| р (r) | с (s) | т (t) | у (u) | ф (f) | х (kh) | ц (ts) | ч (ch) | |||||||
| ш (sh) | щ (shch) | ъ (") | ы (y) | ь (') | э (è) | ю (iu) | я (ia) |
There is a 33rd letter, ё (e"), but it is rarely used and often replaced by е in informal writing. The original algorithm here assumed it had already been mapped to е (e); since 2018-03-16 the Snowball implementation we provide performs this mapping for you.
The following are vowels:
In any word, RV is the region after the first vowel, or the end of the word if it contains no vowel.
R1 is the region after the first non-vowel following a vowel, or the end of the word if there is no such non-vowel.
R2 is the region after the first non-vowel following a vowel in R1, or the end of the word if there is no such non-vowel.
For example:
p r o t i v o e s t e s t v e n n o m
|<------ RV ------>|
|<----- R1 ------>|
|<----- R2 ------>|
(See note on R1 and R2.)
We now define the following classes of ending:
PERFECTIVE GERUND:
group 1: в (v) вши (vshi) вшись (vshis')
group 2: ив (iv) ивши (ivshi) ившись (ivshis') ыв (yv) ывши (yvshi) ывшись (yvshis')
group 1 endings must follow а (a) or я (ia)
ADJECTIVE:
PARTICIPLE:
group 1: ем (em) нн (nn) вш (vsh) ющ (iushch) щ (shch)
group 2: ивш (ivsh) ывш (yvsh) ующ (uiushch)
group 1 endings must follow а (a) or я (ia)
REFLEXIVE:
ся (sia) сь (s')
VERB:
group 1: ла (la) на (na) ете (ete) йте (ìte) ли (li) й (ì) л (l) ем (em) н (n) ло (lo) но (no) ет (et) ют (iut) ны (ny) ть (t') ешь (esh') нно (nno)
group 2: ила (ila) ыла (yla) ена (ena) ейте (eìte) уйте (uìte) ите (ite) или (ili) ыли (yli) ей (eì) уй (uì) ил (il) ыл (yl) им (im) ым (ym) ен (en) ило (ilo) ыло (ylo) ено (eno) ят (iat) ует (uet) уют (uiut) ит (it) ыт (yt) ены (eny) ить (it') ыть (yt') ишь (ish') ую (uiu) ю (iu)
group 1 endings must follow а (a) or я (ia)
NOUN:
а (a) ев (ev) ов (ov) ие (ie) ье ('e) е (e) иями (iiami) ями (iami) ами (ami) еи (ei) ии (ii) и (i) ией (ieì) ей (eì) ой (oì) ий (iì) й (ì) иям (iiam) ям (iam) ием (iem) ем (em) ам (am) ом (om) о (o) у (u) ах (akh) иях (iiakh) ях (iakh) ы (y) ь (') ию (iiu) ью ('iu) ю (iu) ия (iia) ья ('ia) я (ia)
SUPERLATIVE:
ейш (eìsh) ейше (eìshe)
These are all i-suffixes. The list of d-suffixes is very short,
DERIVATIONAL:
ост (ost) ость (ost')
Define an ADJECTIVAL ending as an ADJECTIVE ending optionally preceded by a PARTICIPLE ending.
| бегавшая | = | бега | + | вш | + | ая | ||||||
| (begavshaia | = | bega | + | vsh | + | aia) |
In searching for an ending in a class, always choose the longest one from the class.
Undouble н (n) means, if the word ends нн (nn), remove the last letter.
Here now are the stemming rules.
All tests take place in the RV part of the word.
Do each of steps 1, 2, 3 and 4.
Step 1: Search for a PERFECTIVE GERUND ending. If one is found remove it, and that is then the end of step 1. Otherwise try and remove a REFLEXIVE ending, and then search in turn for (1) an ADJECTIVAL, (2) a VERB or (3) a NOUN ending. As soon as one of the endings (1) to (3) is found remove it, and terminate step 1.
Step 2: If the word ends with и (i), remove it.
Step 3: Search for a DERIVATIONAL ending in R2 (i.e. the entire ending must lie in R2), and if one is found, remove it.
Step 4: (1) Undouble н (n), or, (2) if the word ends with a SUPERLATIVE ending, remove it and undouble н (n), or (3) if the word ends ь (') (soft sign) remove it.
stringescapes {}
/* the 33 Cyrillic letters represented in ASCII characters following the
* conventions of the standard Library of Congress transliteration: */
stringdef a '{U+0430}'
stringdef b '{U+0431}'
stringdef v '{U+0432}'
stringdef g '{U+0433}'
stringdef d '{U+0434}'
stringdef e '{U+0435}'
stringdef e" '{U+0451}'
stringdef zh '{U+0436}'
stringdef z '{U+0437}'
stringdef i '{U+0438}'
stringdef i` '{U+0439}'
stringdef k '{U+043A}'
stringdef l '{U+043B}'
stringdef m '{U+043C}'
stringdef n '{U+043D}'
stringdef o '{U+043E}'
stringdef p '{U+043F}'
stringdef r '{U+0440}'
stringdef s '{U+0441}'
stringdef t '{U+0442}'
stringdef u '{U+0443}'
stringdef f '{U+0444}'
stringdef kh '{U+0445}'
stringdef ts '{U+0446}'
stringdef ch '{U+0447}'
stringdef sh '{U+0448}'
stringdef shch '{U+0449}'
stringdef " '{U+044A}'
stringdef y '{U+044B}'
stringdef ' '{U+044C}'
stringdef e` '{U+044D}'
stringdef iu '{U+044E}'
stringdef ia '{U+044F}'
routines ( mark_regions R2
perfective_gerund
adjective
adjectival
reflexive
verb
noun
derivational
tidy_up
)
externals ( stem )
integers ( pV p2 )
groupings ( v )
define v '{a}{e}{i}{o}{u}{y}{e`}{iu}{ia}'
define mark_regions as (
$pV = limit
$p2 = limit
do (
gopast v setmark pV gopast non-v
gopast v gopast non-v setmark p2
)
)
backwardmode (
define R2 as $p2 <= cursor
define perfective_gerund as (
[substring] among (
'{v}'
'{v}{sh}{i}'
'{v}{sh}{i}{s}{'}'
('{a}' or '{ia}' delete)
'{i}{v}'
'{i}{v}{sh}{i}'
'{i}{v}{sh}{i}{s}{'}'
'{y}{v}'
'{y}{v}{sh}{i}'
'{y}{v}{sh}{i}{s}{'}'
(delete)
)
)
define adjective as (
[substring] among (
'{e}{e}' '{i}{e}' '{y}{e}' '{o}{e}' '{i}{m}{i}' '{y}{m}{i}'
'{e}{i`}' '{i}{i`}' '{y}{i`}' '{o}{i`}' '{e}{m}' '{i}{m}'
'{y}{m}' '{o}{m}' '{e}{g}{o}' '{o}{g}{o}' '{e}{m}{u}'
'{o}{m}{u}' '{i}{kh}' '{y}{kh}' '{u}{iu}' '{iu}{iu}' '{a}{ia}'
'{ia}{ia}'
// and -
'{o}{iu}' // - which is somewhat archaic
'{e}{iu}' // - soft form of {o}{iu}
(delete)
)
)
define adjectival as (
adjective
/* of the participle forms, em, vsh, ivsh, yvsh are readily removable.
nn, {iu}shch, shch, u{iu}shch can be removed, with a small proportion of
errors. Removing im, uem, enn creates too many errors.
*/
try (
[substring] among (
'{e}{m}' // present passive participle
'{n}{n}' // adjective from past passive participle
'{v}{sh}' // past active participle
'{iu}{shch}' '{shch}' // present active participle
('{a}' or '{ia}' delete)
//but not '{i}{m}' '{u}{e}{m}' // present passive participle
//or '{e}{n}{n}' // adjective from past passive participle
'{i}{v}{sh}' '{y}{v}{sh}'// past active participle
'{u}{iu}{shch}' // present active participle
(delete)
)
)
)
define reflexive as (
[substring] among (
'{s}{ia}'
'{s}{'}'
(delete)
)
)
define verb as (
[substring] among (
'{l}{a}' '{n}{a}' '{e}{t}{e}' '{i`}{t}{e}' '{l}{i}' '{i`}'
'{l}' '{e}{m}' '{n}' '{l}{o}' '{n}{o}' '{e}{t}' '{iu}{t}'
'{n}{y}' '{t}{'}' '{e}{sh}{'}'
'{n}{n}{o}'
('{a}' or '{ia}' delete)
'{i}{l}{a}' '{y}{l}{a}' '{e}{n}{a}' '{e}{i`}{t}{e}'
'{u}{i`}{t}{e}' '{i}{t}{e}' '{i}{l}{i}' '{y}{l}{i}' '{e}{i`}'
'{u}{i`}' '{i}{l}' '{y}{l}' '{i}{m}' '{y}{m}' '{e}{n}'
'{i}{l}{o}' '{y}{l}{o}' '{e}{n}{o}' '{ia}{t}' '{u}{e}{t}'
'{u}{iu}{t}' '{i}{t}' '{y}{t}' '{e}{n}{y}' '{i}{t}{'}'
'{y}{t}{'}' '{i}{sh}{'}' '{u}{iu}' '{iu}'
(delete)
/* note the short passive participle tests:
'{n}{a}' '{n}' '{n}{o}' '{n}{y}'
'{e}{n}{a}' '{e}{n}' '{e}{n}{o}' '{e}{n}{y}'
*/
)
)
define noun as (
[substring] among (
'{a}' '{e}{v}' '{o}{v}' '{i}{e}' '{'}{e}' '{e}'
'{i}{ia}{m}{i}' '{ia}{m}{i}' '{a}{m}{i}' '{e}{i}' '{i}{i}'
'{i}' '{i}{e}{i`}' '{e}{i`}' '{o}{i`}' '{i}{i`}' '{i`}'
'{i}{ia}{m}' '{ia}{m}' '{i}{e}{m}' '{e}{m}' '{a}{m}' '{o}{m}'
'{o}' '{u}' '{a}{kh}' '{i}{ia}{kh}' '{ia}{kh}' '{y}' '{'}'
'{i}{iu}' '{'}{iu}' '{iu}' '{i}{ia}' '{'}{ia}' '{ia}'
(delete)
/* the small class of neuter forms '{e}{n}{i}' '{e}{n}{e}{m}'
'{e}{n}{a}' '{e}{n}' '{e}{n}{a}{m}' '{e}{n}{a}{m}{i}' '{e}{n}{a}{x}'
omitted - they only occur on 12 words.
*/
)
)
define derivational as (
[substring] R2 among (
'{o}{s}{t}'
'{o}{s}{t}{'}'
(delete)
)
)
define tidy_up as (
[substring] among (
'{e}{i`}{sh}'
'{e}{i`}{sh}{e}' // superlative forms
(delete
['{n}'] '{n}' delete
)
'{n}'
('{n}' delete) // e.g. -nno endings
'{'}'
(delete) // with some slight false conflations
)
)
)
define stem as (
// Normalise {e"} to {e}. The documentation has long suggested the user
// should do this before calling the stemmer - we now do it for them.
do repeat ( goto (['{e"}']) <- '{e}' )
do mark_regions
backwards setlimit tomark pV for (
do (
perfective_gerund or
( try reflexive
adjectival or verb or noun
)
)
try([ '{i}' ] delete)
// because noun ending -i{iu} is being treated as verb ending -{iu}
do derivational
do tidy_up
)
)