View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
16723 | Bug reports | Response browsing | public | 2020-10-07 10:57 | 2021-03-08 19:38 |
Reporter | Selcal | Assigned To | c_schmitz | ||
Priority | none | Severity | partial_block | ||
Status | closed | Resolution | fixed | ||
Product Version | 4.3.19 | ||||
Fixed in Version | 4.4.12 | ||||
Summary | 16723: SPSS data export does not include any delimiters after lastpage field | ||||
Description | When exporting response data to SPSS (v16 or newer, all responses), the SPSS_data_file does not correctly export the delimiters. I also have a slightly more complex survey, where the delimiters are included for more fields, but not trailing delimeters (ie, when the last questions on the export line are empty, nothing is exported leading to insufficient fields in the export). I can't include that example as it contains sensitive data. The attached simple sample demonstrated the problem well enough though. It already includes test answers. | ||||
Steps To Reproduce | -Open attached survey (two attached, one English, one Dutch, just to see if it was regional). | ||||
Additional Information |
| ||||
Tags | No tags attached. | ||||
Attached Files | survey_V4319_668368_SPSS_data_file(4).dat (240 bytes)
,,,,,,,,,1,'07-10-2020 09:57:15',1'nl''202616330''J63iJu1zErrCRLF''07-10-2020 09:57:05''07-10-2020 09:57:15''1''2''3''4' ,,,,,,,,,2,'07-10-2020 09:58:29',1'nl''1670669681''FJr538zX90txM8Y''07-10-2020 09:58:25''07-10-2020 09:58:29''1''2' survey_V4318_668368_SPSS_data_file(1).dat (240 bytes)
,,,,,,,,,1,'07-10-2020 09:57:15',1'nl''202616330''J63iJu1zErrCRLF''07-10-2020 09:57:05''07-10-2020 09:57:15''1''2''3''4' ,,,,,,,,,2,'07-10-2020 09:58:29',1'nl''1670669681''FJr538zX90txM8Y''07-10-2020 09:58:25''07-10-2020 09:58:29''1''2' survey_V433_668368_SPSS_data_file.dat (268 bytes)
,,,,,,,,,'1','07-10-2020 09:57:15','1','nl','202616330','J63iJu1zErrCRLF','07-10-2020 09:57:05','07-10-2020 09:57:15','1','2','3','4', ,,,,,,,,,'2','07-10-2020 09:58:29','1','nl','1670669681','FJr538zX90txM8Y','07-10-2020 09:58:25','07-10-2020 09:58:29','1','2',,, | ||||
Bug heat | 4 | ||||
Complete LimeSurvey version number (& build) | Version 4.3.19+201005 | ||||
I will donate to the project if issue is resolved | No | ||||
Browser | |||||
Database type & version | MariaDB, 10.3.23 | ||||
Server OS (if known) | Linux | ||||
Webserver software & version (if known) | Apache | ||||
PHP Version | 7.4.9 | ||||
I have now downgraded to 4.3.15 (prior to the updates to the SPSS export), and can confirm the export works properly on this version. |
|
This should be fixed in the latest version. Can check with that please? |
|
I've just run a check with 4.3.25, but unfortunately the export still isn't working properly. The issue seems to be with trailing string fields on each row (when no more numeric occurs on the row): no delimiters are inserted for those. To illustrate, the pattern I see, is if the fields to be exported are like this example 12 column row structure. (Calling numbers int, text str for simplicity): Then delimiters work up to field 8. However, the strings from position 9 (inclusive) have no delimiters anymore (and so if those fields are empty, nothing is shown at all). I can't publish the actual data from this survey, but I could fairly easily reproduce it before. I'll try to get an example up but that may be after the weekend. |
|
Couldn't help myself, so I set up a test now. The above is not correct, but I really can't find the pattern. I've attached an 2 example survey archives with answers (one with 2 groups, one with 1), and the SPSS export. Note the missing delimiters. In this case, it's for all the fields... I can't find why on some production surveys they partially come up. survey_841825_SPSS_data_file.dat (289 bytes)
,,,,,,,,,1,'06-11-2020 10:59:02',1,'nl','1178055172','jZT2XaO7VQAeK6n','06-11-2020 10:58:36','06-11-2020 10:59:02''foo''bar''foo''bar'111'foo'222'bar''foo' ,,,,,,,,,2,'06-11-2020 10:59:32',1,'nl','617589883','3WInDFdTu0CHBXZ','06-11-2020 10:59:14','06-11-2020 10:59:32''foo'111'bar'222 survey_814171_SPSS_data_file.dat (292 bytes)
,,,,,,,,,1,'06-11-2020 11:03:53',1,'nl','1932795011','hqV7jjuAtVf8cYj','06-11-2020 11:03:34','06-11-2020 11:03:53''foo''bar''foo''bar'111'foo' ,,,,,,,,,2,'06-11-2020 11:04:35',2,'nl','1290995659','YvotzLHiViJRfR3','06-11-2020 11:04:11','06-11-2020 11:04:35''foo''bar'111'foo'222'foo''bar' |
|
Ran a test update to 4.3.28, unfortunately still missing delimiters: So far, 4.3.15 is the last to work correctly for me. |
|
Actually I think this is to do with the setting of $num_fields in application/helpers/export_helper.php Line 137: Seems, at first glance, to fix the problem. It seems to work here, but I do notice that the newer version exports numerics within quotes. This may be correct; it's rather late now but I'll try to get my SPSS users to check. The number of delimiters, at least, is correct. |
|
I just received user feedback; with the above fix the issue seems fixed. |
|
I have applied the above fix to the current 4.3.31 update, and have used it in a live environment. There seem to be no adverse effects, and the SPSS export results are now correct. |
|
Update: Now running 4.4.6, this issue is still present and the above fix still works for me without issue. |
|
Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=31282 |
|
Thank you for your continuous feedback and not giving up on us. We have now patched this in LS3 and LS4 according to your suggestion. |
|
Fix committed to 3.x-LTS branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=31283 |
|
LimeSurvey: master 644abe69 2021-03-07 22:24 Details Diff |
Fixed issue 16723: SPSS data export does not include any delimiters after lastpage field |
Affected Issues 16723 |
|
mod - application/helpers/export_helper.php | Diff File | ||
LimeSurvey: 3.x-LTS b40ffbd1 2021-03-07 22:24 Details Diff |
Fixed issue 16723: SPSS data export does not include any delimiters after lastpage field |
Affected Issues 16723 |
|
mod - application/helpers/export_helper.php | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2020-10-07 10:57 | Selcal | New Issue | |
2020-10-07 10:57 | Selcal | File Added: spss_delimiters.jpg | |
2020-10-07 10:57 | Selcal | File Added: survey_archive_668368.lsa | |
2020-10-07 10:57 | Selcal | File Added: survey_archive_246634.lsa | |
2020-10-07 10:57 | Selcal | File Added: survey_V4319_668368_SPSS_data_file(4).dat | |
2020-10-07 10:57 | Selcal | File Added: survey_V4318_668368_SPSS_data_file(1).dat | |
2020-10-07 10:57 | Selcal | File Added: survey_V433_668368_SPSS_data_file.dat | |
2020-10-07 18:07 | cdorin | Assigned To | => c_schmitz |
2020-10-07 18:07 | cdorin | Status | new => assigned |
2020-11-05 11:56 | Selcal | Note Added: 60557 | |
2020-11-05 17:02 | c_schmitz | Status | assigned => feedback |
2020-11-05 17:02 | c_schmitz | Note Added: 60568 | |
2020-11-06 10:51 | Selcal | Note Added: 60579 | |
2020-11-06 10:51 | Selcal | Status | feedback => assigned |
2020-11-06 11:09 | Selcal | Note Added: 60580 | |
2020-11-06 11:09 | Selcal | File Added: survey_841825_SPSS_data_file.dat | |
2020-11-06 11:09 | Selcal | File Added: survey_archive_841825.lsa | |
2020-11-06 11:09 | Selcal | File Added: survey_814171_SPSS_data_file.dat | |
2020-11-06 11:09 | Selcal | File Added: survey_archive_814171.lsa | |
2020-11-25 00:01 | Selcal | Note Added: 60753 | |
2020-11-25 00:51 | Selcal | Note Added: 60754 | |
2020-11-25 10:51 | Selcal | Note Added: 60755 | |
2020-12-18 12:38 | Selcal | Note Added: 60998 | |
2021-02-11 12:59 | Selcal | Note Added: 62157 | |
2021-03-07 21:24 | c_schmitz | Changeset attached | => LimeSurvey master 644abe69 |
2021-03-07 21:24 | c_schmitz | Note Added: 62825 | |
2021-03-07 21:24 | c_schmitz | Resolution | open => fixed |
2021-03-07 21:25 | c_schmitz | Status | assigned => resolved |
2021-03-07 21:25 | c_schmitz | Note Added: 62826 | |
2021-03-08 09:28 | c_schmitz | Changeset attached | => LimeSurvey 3.x-LTS b40ffbd1 |
2021-03-08 09:28 | c_schmitz | Note Added: 62868 | |
2021-03-08 19:31 | c_schmitz | Fixed in Version | => 4.4.12 |
2021-03-08 19:38 | c_schmitz | Status | resolved => closed |