Re: Malformed sav file - need help for investigation

[Ignácz , István]

Wow, that explains a lot!

Thanks for your detailed explanation!

On Wed, 10 Jun 2020 at 20:27, Ben Pfaff <blp@cs.stanford.edu> wrote:

> Ah. I can easily explain what the problem is.
>
> SPSS data files have a variable record for each variable. For string
> variables with a width greater than 8, they actually have a variable record
> for every 8 columns of width (rounding up). Lots of places in data files
> refer to variables by the index that includes such extra records. So, if
> you have a string variable with width 24, and it's the third variable in
> the file following two numeric variables, then it will have indexes 2, 3,
> and 4. When such a variable is referenced by index, only index 2 is
> supposed to be used. PSPP complains if index 3 or 4 is used instead, and
> SPSS (apparently) crashes instead.
>
> Maybe I should make PSPP more tolerant, just giving a warning instead of
> an error. If so, then it would be easy to fix up a file with this error by
> using pspp-convert.
>
> On Wed, Jun 10, 2020 at 10:24 AM Ignácz, István <istvan.ignacz@neticle.com>
> wrote:
>
>> Thanks, Ben for the quick response!
>>
>> Yes, the ultimate goal is to add a patch to the package which fixes the
>> problem or throws an error if the user of the package (in this case me)
>> provides invalid input. I think I'm facing the later.
>>
>> One significant difference between the test file and my case is that in my
>> case no variables and no cases are recognized so the reading process
>> basically crashes.
>> I think I've found the part where the PSPP ends up in a 'goto error'
>> (sys-file-reader.813) but I lack the context of what's going on here. I
>> guess the value labels could be misconfigured? Maybe there's a value in a
>> record that doesn't have a label in the variable settings?
>>
>> If I would know a little bit more about what happens in that part and why
>> that causes an error, maybe I could force my (and tiamo's) code to
>> reliably
>> produce the error.
>>
>> Thanks,
>> István
>>
>> On Wed, 10 Jun 2020 at 18:51, Ben Pfaff <blp@cs.stanford.edu> wrote:
>>
>> > I'm not sure exactly what you're looking for, but PSPP has a test case
>> > that intentionally provokes this error message. I'm attaching the file
>> it
>> > produces.
>> >
>> > I guess the ultimate goal here should be to fix tiamo's package?
>> >
>> > On Wed, Jun 10, 2020 at 9:42 AM Ignácz, István <
>> istvan.ignacz@neticle.com>
>> > wrote:
>> >
>> >> Greetings,
>> >>
>> >> I think this is not the usual topic on this list, excuse me if you
>> find it
>> >> off-topic.
>> >>
>> >> I'm facing a tough bug in my software and I'm looking for some help to
>> be
>> >> able to take the next steps.
>> >>
>> >> Our software generates sav files via tiamo's package:
>> >> https://packagist.org/packages/tiamo/spss
>> >> Usually, everything works fine, but in some cases, we generate
>> malformed
>> >> files. In this case, SPSS crashes without any notice, however, PSPP at
>> >> least gives a useful error message:
>> >> Variable index n refers to long string continuation.
>> >>
>> >> I cannot reproduce this issue on my side, so I tried to reverse
>> engineer
>> >> the PSPP codebase to find any cases when it could occur. So far not
>> with
>> >> much luck.
>> >>
>> >> Could you help me with some example cases?
>> >>
>> >> Thanks for your help,
>> >> István
>> >>
>> >
>>
>> --
>>
>> *István Ignácz*
>> Software Engineer @ Neticle
>>
>> +36 30 460 5356
>> istvan.ignacz@neticle.com
>> neticle.com
>>
>> <https://neticle.com/mediaintelligence/en/>  <https://zurvey.io/>
>> <https://api.neticle.com/>  <https://data.neticle.com/documentation>
>>
>> Neticle Labs Ltd.
>> Address: H-1082 Budapest, Leonardo da Vinci  41. (2nd floor/12)
>> Billing address: H-1213 Budapest, Veto street 10.
>
>