[Snowball-discuss] Possible issue with generated Java code

[Olly Betts]

On Tue, Nov 10, 2020 at 02:03:51AM +0000, Olly Betts wrote:
> I've tracked this down to a bug in the Java code Snowball generates for
> "goto" and "gopast" which matters inside "string-$" (I think this may be
> the only situation where it matters, but I haven't fully analysed yet).
> The failure string should get saved and cleared on entry to goto/gopast
> and restored on exit.  It is saved and restored, but the "clear" is
> missing.

It also affected "goto" and "gopast" with "setlimit", e.g. this generated
code which incorrectly increased the limit in two extra places:

    setlimit 'test' for ( gopast 'a' gopast 'a' )

That's fixed by the same change, and it doesn't affect the shipped
stemmers.

> Looking at the code I think there's a similar but with "try" inside
> "string-$" when used on a command which doesn't change the cursor, but
> I've not attempted to test that yet.

This is also a bug, but isn't in the shipped stemmers (because they
don't use "string-$".

> Also "try" seems to be missing a save and restore (unless this isn't
> needed for some reason I'm not seeing).

The failure state is saved and restored at a higher level (in
generate()) so this is OK.

> I suspect other languages may be affected, since most copied Java's
> approach (because the one we use for C doesn't work for most other
> languages).

All the other languages except C have the same bugs.

C only gets the case of goto/gopast inside setlimit wrong.

I've pushed a fix for all of these to git in commit
b7e1816375b376af9f1a0aa0f7b7ecde9e05dd3a.

> But features in the language should work as advertised in all languages
> (or failing that, at least fail noisily, ideally at compile time), so
> thanks for raising this.

None of these bugs affect stemmers we ship, but it's not good to have
lurking bugs waiting to catch people implemented new algorithms, so
thanks again for reporting this.

Cheers,
    Olly