When you download Snowball, it already contains a make file to allow you to build it, like so:
make
You can confirm it's working with a simple test like so:
echo "running" | ./stemwords -l en
which should output: run
There's no built in way to install snowball currently - you can either copy the snowball binary to somewhere that's on your PATH (e.g. on a typical Linux machine: sudo cp snowball /usr/local/bin) or just run it from the source tree with ./snowball).
The snowball compiler has the following command line syntax,
Usage: snowball SOURCE_FILE... [OPTIONS]
Supported options:
-o, -output OUTPUT_BASE
-s, -syntax
-comments
-j, -java
-cs, -csharp
-c++
-pascal
-py, -python
-js[=TYPE] generate Javascript (TYPE values:
esm global, default: global)
-rust
-go
-ada
-w, -widechars
-u, -utf8
-n, -name CLASS_NAME
-ep, -eprefix EXTERNAL_PREFIX
-vp, -vprefix VARIABLE_PREFIX
-i, -include DIRECTORY
-r, -runtime DIRECTORY
-p, -parentclassname CLASS_NAME fully qualified parent class name
-P, -Package PACKAGE_NAME package name for stemmers
-S, -Stringclass STRING_CLASS StringBuffer-compatible class
-a, -amongclass AMONG_CLASS fully qualified name of the Among class
-gop, -gopackage PACKAGE_NAME Go package name for stemmers
-gor, -goruntime PACKAGE_NAME Go snowball runtime package
--help display this help and exit
--version output version information and exit
For example,
snowball danish.sbl -o q/danish
snowball danish.sbl -syntax
snowball danish.sbl -output q/danish -ep danish_
The first argument, SOURCE_FILE, is the name of the Snowball file to be compiled. Unless you specify a different programming language to
generate code for, the default is to generate ISO C which results in two output
files, a C source in OUTPUT_BASE.c and a corresponding header file in OUTPUT_BASE.h. This is similar for other
programming languages, e.g. if option -java is
present, Java output is produced in OUTPUT_BASE.java.
Some options are only valid when generating code for particular programming
languages. For example, the -widechars,
-utf8, -eprefix and
-vprefix options are specific to C and C++.
In the absence of the -eprefix and -vprefix options, the list of
declared externals in the Snowball program, for example,
externals ( stem_1 stem_2 moderate )
gives rise to a header file containing,
extern struct SN_env * create_env(void);
extern void close_env(struct SN_env * z);
extern int moderate(struct SN_env * z);
extern int stem_2(struct SN_env * z);
extern int stem_1(struct SN_env * z);
If -eprefix is used, its string, S1, is prefixed to each external
name, for example
-eprefix Khotanese_
would give rise to the header file,
extern struct SN_env * Khotanese_create_env(void);
extern void Khotanese_close_env(struct SN_env * z);
extern int Khotanese_moderate(struct SN_env * z);
extern int Khotanese_stem_2(struct SN_env * z);
extern int Khotanese_stem_1(struct SN_env * z);
If -vprefix is used, all Snowball strings, integers and booleans give
rise to a #define line in the header file. For example
-eprefix Khotanese_ -vprefix Khotanese_variable
would give rise the header file,
extern struct SN_env * Khotanese_create_env(void);
extern void Khotanese_close_env(struct SN_env * z);
#define Khotanese_variable_ch (S[0])
#define Khotanese_variable_Y_found (B[0])
#define Khotanese_variable_p2 (I[1])
#define Khotanese_variable_p1 (I[0])
extern int Khotanese_stem(struct SN_env * z);
The -utf8 and -widechars options affects how
the generated C/C++ code expects strings to be represented - UTF-8 or
wide-character Unicode (stored using 2 bytes per codepoint), or if neither is
specified, one byte per codepoint using either ISO-8859-1 or another encoding.
For other programming languages, one of these three options is effectively implicitly hard-coded (except wide-characters may be wider) - e.g. C#, Java, Javascript and Python use wide characters; Ada, Go and Rust use UTF-8; Pascal uses ISO-8859-1. Since Snowball 2.0 it's possible with a little care to write Snowball code that works regardless of how characters are represented. See section 12 of the Snowball manual for more details.
The -runtime option is used to prepend a path to any #include
lines in the generated code, and is useful when the runtime header files (i.e.
those files in the runtime directory in the standard distribution) are not
in the same location as the generated source files. It is used when
building the libstemmer library, and may be useful for other projects.
If -syntax is used the other options are ignored, and the syntax tree
of the Snowball program is directed to stdout. This can be a handy way
of checking that you have got the bracketing right in the program you have
written.
Any number of -include options may be present, for example,
snowball testfile -output test -ep danish_ \
-include /home/martin/Snowball/codesets \
-include extras
Each -include is followed by a directory name. With a chain of
directories D1, D2 ... Dn, a Snowball get directive,
get 'F'
causes F to be searched for in the successive locations,
F
D1/F
D2/F
...
Dn/F
— that is, the current directory, followed in turn by directories D1 to
Dn.
To access Snowball from C, include the header api.h, and any headers
generated from the Snowball scripts you wish to use. api.h declares
struct SN_env { /* ... */ };
extern void SN_set_current(struct SN_env * z, int size, char * s);
Continuing the previous example, you set up an environment to call the resources of the Khotanese module with
struct SN_env * z;
z = Khotanese_create_env();
Snowball has the concept of a ‘current string’. This can be set up by,
SN_set_current(z, i, b);
This defines the current string as the i bytes of data starting at
address b. The externals can then be called,
Khotanese_moderate(z);
/* ... */
Khotanese_stem_1(z);
They give a 1 or 0 result, corresponding to the t or f result of the Snowball routine.
And later,
Khotanese_close_env(z);
To release the space raised by z back to the system. You can do this for a
number of Snowball modules at the same time: you will need a separate
struct SN_env * z; for each module.
The current string is given by the z->l bytes of data starting at z->p.
The string is not zero-terminated, but you can zero terminate it yourself with
z->p[z->l] = 0;
(There is always room for this last zero byte.) For example,
SN_set_current(z, strlen(s), s);
Khotanese_stem_1(z);
z->p[z->l] = 0;
printf("Khotanese-1 stems '%s' to '%s'\n", s, z->p);
The values of the other variables can be accessed via the #define
settings that result from the -vprefix option, although this should not
usually be necessary:
printf("p1 is %d\n", z->Khotanese_variable_p1);
The stemming scripts on this Web site use Snowball very simply.
-vprefix is left unset, and -eprefix is set to the name of the
script (usually the language the script is for).
In the rare event that your Snowball script does not run perfectly the first time:
Remember that the option -syntax prints out the syntax tree. A question
mark can be included in Snowball as a command, and it will generate a call
debug(...). The defined debug in runtime/utilities.c (usually
commented out) can then be used. It causes the
current string to sent to stdout, with square brackets marking the
slice and vertical bar the position of c. Curly brackets mark the
end-limits of the string, which may be less than the whole string because
of the action of setlimit.
At present there is no way of reporting the value of an integer or boolean.
If desperate, you can put debugging lines into the generated C program.
You can pass -comments to the snowball compiler to get it to
generate comments showing the correspondence with the Snowball source which
makes it easier to find where to add such debugging code.
If you hit a snowball compiler bug, try to capture it in a small script before notifying us.
The main one is that it is possible to ‘pull the rug from under your own feet’ in constructions like this:
[ do something ]
do something_else
( C1 delete C2 ) or ( C3 )
Suppose C1 gives t, the delete removes the slice established on the first
line, and C2 gives f, so C3 is done with c set back to the value it had
before C1 was obeyed — but this old value does not take account of the byte shift
caused by the delete. This problem was foreseen from the beginning when designing
Snowball, and recognised as a minor issue because it is an unnatural thing to want to
do. (C3 should not be an alternative to something which has deletion as an
occasional side-effect.) It may be addressed in the future.