About LSA export : i think it's great if we have a "LSA decrypted" export, then user can choose if done for backup for same server then can be crypted. Or not.

:)

The worst one are Reloading response via token : crypted data

1) A simple test survey with one encrypted question of the same type (no other questions in survey) does NOT reproduce the problem - the fields in the question are decrypted correctly.

? Are you sure ?

Try to import included lsa : i think it broke with a 500 error (Wrong decryption key! Decryption key has changed since this data were last saved, so data can't be decrypted. Please consult our manual at https://manual.limesurvey.org/Data_encryption#Errors.)

Hi - I think I may have diverted attention away from my main issue here by talking about exporting / importing survey archives with encrypted fields.
I did spot issues 15668 and 15669 before posting, and have verified in a one-question survey with encryption that in this case, my export is decrypted - obviously that will be necessary to migrate it to another server with different encryption keys. That may not be so wise for a server backup but that's another question for debate / feature request to select as part of the export.

I did import your test survey and that had an error as expected when I tried to display a response:
"500: Internal Server Error
Call to a member function decrypt() on null"

This is my issue:
With the survey I uploaded here, I get NO errors when I activate the survey, enter one response including 'First Name, Surname', and then display the response.
In the DB I can see that the relevant fields are encrypted, but when displaying the response I see:

[G01Q07_SQ01] (First name) Ys1jwmKfS1+rYB274j2gO06GPrYdL...
[G01Q07_SQ02] (Surname) 5PORCDiCzCidyf4hN0LbX0wdG74d...

(I've shortened the strings above). No error is shown but the decryption is incorrect (the above strings are different from the DB fields for those questions)
Are you able to replicate the problem with my survey?
There seems to be something specific in my uploaded survey that's stopping the correct decryption, though the simple one-question survey with encryption DOES work.
Thanks!

I can confirm i have encrypted field in browse response.

The strange part : « though the simple one-question survey with encryption DOES work.» : you mean a survey with only ONE question (encrypoted) ?

Because a survey with 2 question : one crypted + on not crypted show (and export) only crypted data.

OK : database are not the same, but export as LSA, import as LSA : work and show same data.

Capture d’écran du 2020-05-26 11-51-21.png (16,884 bytes)   

Capture d’écran du 2020-05-26 11-51-21.png (16,884 bytes)   

Capture d’écran du 2020-05-26 11-50-34.png (9,461 bytes)   

Capture d’écran du 2020-05-26 11-50-34.png (9,461 bytes)   

Yes that seems to replicate what I'm seeing - both web UI and exported LSA are showing the same data (WRONGLY decrypted version of the DB contents).

Re your previous note - I have made two test surveys - one with only one question that was encrypted, the second survey with two questions, one of which was encrypted. BOTH behaved as expected with the DB fields being encrypted, but shown correctly in the Web UI and being exported unencrypted (as in the attached LSA for the two-question survey).
It is strange that the problem seems to be present in some surveys but not others. If if you are seeing the problem in your survey with only two questions (one encrypted) that may make debugging easier hopefully.

It's not really wrongly encryted since we can export as lsa and import (in same instance) without issue.

If there are a bad system here : we can not do it :)

Yes I understand what you are saying, the decryption on the 'bad' surveys / fields isn't showing an error so must the actual decryption must be happening correctly, but if so, then the correct plaintext string resulting from that decryption isn't being shown in the UI or the LSA. I have checked to see if a small sub-string of what is displayed in the UI / LSA field is present in the encrypted DB field, and it isn't. So I'm not sure where the output shown in UI / LSA is coming from.

Further information:
When a person drops out of the survey without a submit the encrypted data appears fine in the show responses for that record.
However, when the person completes the survey with a submit at the end the encrypted data is not properly displayed in the UI (show responses) for that record.

To follow with more examples - I have a short survey done on Version 4.3.3+200707. An export is attached.
The two questions "appear" to be the same and both are encrypted. The email that is sent to the admin shows with answers without clear text for either (attached)
The Admin "show responses" shows the first question garbled, but the second question is properly decrypted in the UI (response1)

You can create multiple questions of different types and some will be correct and others are not.
IMPORTANT - changing the ORDER of the questions has an impact. (response2) as the first is wrong and the second correct.
Adding another question appears to work properly. (response3)

Email that gets sent to admin  - encryted fields are "hidden"


Hello,

A new response was submitted for your survey 'testing'.

Click the following link to see the individual response:
https://guernseyresearch.com/survey/index.php/admin/responses/sa/view/surveyid/274463/id/11

Click the following link to edit the individual response:
https://guernseyresearch.com/survey/index.php/admin/dataentry/sa/editdata/subaction/edit/surveyid/274463/id/11

View statistics by clicking here:
https://guernseyresearch.com/survey/index.php/admin/statistics/sa/index/surveyid/274463


The following answers were given by the participant:


     Response ID : 11
     Date submitted : 01-01-1980 00:00:00
     Last page : 3
     Start language : en
     Seed : 1238572615

My first question group

     <p>What color</p> : Red [AO01]
     <p>email</p> : aKcreBSO7Gov58aIL6MkyrOn8ECq3BWpSsxBYIyR4i6UBq2DjkXV50+Z7PLLHwihHxXLwCATrHvuE7LzycJiBWQvblMxbUJzZEtVaDE3WktaV2M3TGZ6V1dLRThxM3ZDRWhMZnJQUVBpRTJ0bEp5WmkzNllCeEh5NGxsWkNUaThnTmNNaGVqWmsremFVR0RYZzNGRkRXUmhkbVZBWldVPQ==
     <p>name</p> : 1su1M7yg9GwdTt2P4mSCWTCYnJIwECbYcWUkX3CLd0y4mNPSI5IF8DhdxODp1KdrXiDilOlWAbPSO7OzBfmtBURhdmU=

Response2.jpg (91,899 bytes)   

Response2.jpg (91,899 bytes)   

Response3.jpg (94,834 bytes)   

Response3.jpg (94,834 bytes)   

Checked both : seems fixed now (export lsa from one system to another instance)

@GuernseyResearch : i confirm the 2 issue on last master.

Can you report separatly maybe ? @cdorin : need 2 issue or can be fixed here ?

This appears to be fixed in Version 4.4.0-RC3+210112

Fixed

However when responses are examined using the Web UI

Open again

No need token : just

1. Import
2. Activate
3. Launch
4. Submit
5. Browse, view and edit

Capture d’écran du 2021-03-05 08-06-07.png (33,050 bytes)   

Capture d’écran du 2021-03-05 08-06-07.png (33,050 bytes)   

Capture d’écran du 2021-03-05 08-06-14.png (43,605 bytes)   

Capture d’écran du 2021-03-05 08-06-14.png (43,605 bytes)   

Having the same problem with a few forms that were conducted before 4.4.0 was out (some time last month, I believe). The encryption was only enabled on some fields, probably by accident, and only these fields are being shown non-decrypted. I tried copying the database and setting up a new LimeSurvey instance pulled directly from master with the old security.php - no luck. Issue persists across Web UI and all exports, which makes it impossible for us to extract any of the answers. As some of those surveys were pretty critical and the results are going to be needed soon, is there any way that they could possibly be decrypted "manually"? Do I understand correctly that AES-256-GCM is used for encryption? Is there any other information I can provide or something I should try?

Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=31818

@GuernseyResearch, can you confirm that this problem is fixed for you at the latest version?

Did you notice any performance issues when exporting encrypted data of a larger survey (our survey has ~300 questions and 1000 responses)?

Our key survey that uses encryption has 40 questions (some are skipped) and TWO fields near the end that are encrypted because we ask for an optional name and email address if they want to participate in another survey. We have close to 1000 responses and don't appear to have a problem. When we do an export of the results we don't need to see the encrypted data (that is done as we process each response and communicate individually with the people responding). As a result, we have not seen any performance issues in the actual survey data.

Thanks for sharing your feedback!

We have many encrypted fields (~300 questions) and 1600 responses and the screen just freezes on export.

When exporting a smaller amount of responses, we get empty or "n/a" at the responses, though there IS some data. But that may be related to testing at a different system and not using the correct key for decryption... not sure. Maybe anyone has seen those emtpy responses at the export as well?

Do you need ALL those fields to be encrypted?
We are trying to follow privacy rules and protect personal information (name, email). As long as those are blocked the other data is not associated with a person. The database itself is protected so we feel that we are adequately following the requirements.
What causes you to encrypt so much of the survey?

not using the correct key for decryption...

Sure it broke somewhere here …

Release done.