Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate odk compatible surveys using enketo #832

Merged
merged 95 commits into from
May 27, 2022

Conversation

shankari
Copy link
Contributor

@shankari shankari commented May 27, 2022

Finally integrate the ODK compatible surveys using the enketo core library into master.
Concretely:

Now we are ready to merge to master.
Gory details of the merge.
e-mission/e-mission-docs#727

asiripanich and others added 30 commits August 10, 2021 17:38
Customise the intro steps:
- Consent form
- Random token login
- Answer Kobo survey
Customise RCITI branch
- toggleDarkMode
- toggleLowAccuracy
- chanegCarbonDataset
- This screen is quite concerning in term of privacy.
- Date range filter doesn't seem to work properly. (e-mission/e-mission-docs#664)
Shorten trip confirm options with emojis 🙉
Add support for enketo survey.
Refactor the code to be easier to read and maintain.
Update the code to be compatible with breaking changes from the master branch.
Simplify surver launch method.
Generalize the survey launch and label resolution process.
This should allow us to load and store the demographic survey as well
- Restore a sample user profile.json
- Create a new demographic button directive
    - the control launches the survey in a popup
    - the service reads all surveys and picks the most recent one
- change the control screen to use the new button instead of a one-off launch command
- remove the callback for the one-off launch from the control code
Concretely:
- add a new survey.html which just has the inline function
- once the account is registered, move to the next slide instead of popping up the survey
- add a new `inline` directive which has all the information from the `<ion-content>` in the popup
- have the directive call `initSurvey`
    - which involved having `initSurvey` exported from launch

There are still many weirdnesses that need to be fixed, but let's first get this committed
e-mission/e-mission-docs#727 (comment)
If the user "logs out" and goes back to the intro screen, it is re-created.
This ensures that we don't have two form fields sitting around right after we
have finished the onboarding, and that opening the popup will not add a new
form to the wrong form field

This fixes e-mission/e-mission-docs#727 (comment)
e-mission/e-mission-docs#727 (comment)
e-mission/e-mission-docs#727 (comment)
e-mission/e-mission-docs#727 (comment)

We tried to remove only the form directive but in that case, the form is not
re-created if we go back to the intro screen.
e-mission/e-mission-docs#727 (comment)

Deleting the entire view does work
e-mission/e-mission-docs#727 (comment)
To avoid duplication of content. We can theoretically generalize the
demographic button as well, but running out of time for additional refactoring
now.
The UI is still clunky but the functionality works:
- if no prior response, show the survey
- if prior response:
    - by default show a message asking the user whether they want to edit or skip
    - if editing, load the survey and let them edit
So people can determine whether they want to edit it or not.
Unfortunately, it doesn't work very well since it displays the code versions of
the name and value.

Concretely:
- this means that the names are not very meaningful (e.g. `hh_size` instead of `household size`)
- this also means that they will not support i18nA

Checking this in anyway for now, but will remove it in the next commit
e-mission/e-mission-docs#727 (comment)
In the case that the prior survey response is found, make the text bigger and more centered
At this point, we are basically done with the phone functionality
This makes it easier to search through the documents in MongoDB and to access
the fields in javascript.

This is consistent with:
e-mission/e-mission-docs#727 (comment)
and is the last client-side change for this feature
In ff7863d and
df21f08 we pulled out the enketo form code
into a separate form-base.html which we `ng-include`d in both the popup and
inline versions.

This actually commits that form-base.html which we forgot to commit earlier.
Data model: e-mission/e-mission-docs#727 (comment)

This makes the format more consistent with the other entries stored on the server.
- data.ts as a numeric timestamp in seconds
- data.fmt_time as a formatted date

- data.start_ts/end_ts for trip-oriented objects
- data.ts for one time surveys that are not related to a particular trip
We can fill them in with real implementations if/when we turn on inferences for
the survey-based labels.
This ensures that it is displayed correctly
Integrate ODK compatible surveys using enketo core into master
* Revert "chore: remove unused json"

This reverts commit d825fc2.

* Revert "patch(Diary): update tripgj label directly with the new answer"

This reverts commit c5ead07.

* Revert "patch(Diary): get rid of isAnalyzed and migrate to isDraft"

This reverts commit 9e6a2cd.

* Revert "patch(Intro): use SurveyLaunch"

This reverts commit c82d60a.

* Revert "patch(Settings): refactor to use new startSurveyPrefilled options param"

This reverts commit b8a7ab0.

* Revert "patch(Diary): optimize isAnalyzed call"

This reverts commit b57742a.

* Revert "patch(Settings): change domain to fourstep.dev"

This reverts commit 2447ecb.

* Revert "patch(Login): login with anonymous token"

This reverts commit 4b3cbab.

* Revert "patch(Login): login with anonymous token"

This reverts commit 4b3cbab.

* Revert "patch(Intro): update consent and intro pages to match RCITI profile"

This reverts commit b38e5a4.

* Revert "patch(Profile): add edit User Profile and End Survey buttons that launch surveys"

This reverts commit 32425b9.

* Revert "patch(Enketo): fix answers not restore correctly"

This reverts commit 04eb99f.

* Revert "patch(Diary): use EnketoSurvey for trip confirmation"

This reverts commit e425caf.

* Revert "patch(Diary): Dynamic input confirm button sizing"

This reverts commit a7f43d1.

* Revert "feat(Enketo): Enketo survey support"

This reverts commit fb338f1.

* Revert "patch(Tabs): hide Label and Heatmap tabs"

This reverts commit 610fdab.

* Revert "Shorten trip confirm options with emojis 🙉"

This reverts commit 7e6672e.

* Revert "Modify trip confirm options"

This reverts commit 26173fb.

* Revert "Update the login instruction"

This reverts commit 75eb6db.

* Revert "Update the support line number"

This reverts commit 1ab1002.

* Revert "Make styling changes to consent-text.html"

This reverts commit 802d39a.

* Revert "Change consent text's text-align  from 'justify' to 'left'"

This reverts commit 8024fcb.

* Revert "Comment out onCurentTrip"

This reverts commit 32f3b07.

* Revert "Remove some options  from the control list"

This reverts commit 34980a7.

* Revert "Use the defined css classes in consent-text.html"

This reverts commit 9c90fd8.

* Revert "Modify the login screen to only accept token"

This reverts commit c1b2c6a.

* Revert "Increase text size in intro.css"

This reverts commit 714018d.

* Revert "Replace a missing function"

This reverts commit ed2b9c2.

* Revert "Add edit user profile and end survey buttons."

This reverts commit 5104cec.

* Revert "Use an indirect link to open the user profile survey"

This reverts commit cdeaf45.

* Revert "Use random token login and initial kobotoolbox survey"

This reverts commit d2028a3.

* Revert "Comment out the summary screen"

This reverts commit 25e8795.

* Restore the labels for the survey option

* Revert "Revert "feat(Enketo): Enketo survey support""

This reverts commit 28a0b63.

* Manually revert isAnalysed function

Manually revert parts of
fb338f1
that are not related to enabling the survey code and that we don't want to restore

* Revert other changes that we don't want to include just yet

* Remove external survey launch dependency from the control settings

Since we use the new button directive instead

* revert to multilabel + restore answers while switching between diary and label

+ whitespace fixes

* Change the default options to map to MULTILABEL instead of ENKETO

To be consistent with the list view

* Revert the description for the share button

Should change to NREL later

* One final change to metrics

* Remove merge conflict

* Remove merge conflict!

* Revert email address from Amarin to me

* Remove the overridden trip confirmation

To avoid future merge conflicts
+ fix the sample

* Remove this specific configuration

To avoid future conflicts
+ expand the sample

* Restore whitespace

* Fix whitespace

* Restore dashboard since we also restored the multilabel survey

* Ensure that we show the survey page after the login

So we replace `finish` with `next`
This should not have been reverted
So that it looks proper

It centers the button and ensures that it takes up all the width of the item.
e-mission/e-mission-docs#727 (comment)
and
e-mission/e-mission-docs#727 (comment)
Where if the user input list has a single entry, we can return it.
If we have one user input and multiple trips, we can't just return that one
input. That is the input for one trip, not the others!
e-mission/e-mission-docs#727 (comment)

wrt performance, we already handled the major performance issue by only having
to manually match unprocessed entries after the end of the pipeline

Testing done:
- loaded a day with a single user input
- before this change, all trips were flagged with that input
- after this change, only the first was
Before this, we used to return the Promises.all and fill in the map in its
then.

As part of the refactoring for the enketo trip labels,
aec0051

we changed it to return an array of the result and the Promise.all

This meant that the infinite scroll code, which was expecting a result map, got
a promise array instead, and so the mapping of user inputs in the infinite
scroll list = label list failed consistently.

Fixing this by re-returning `Promise.all` and the result and mapping from the `then`

Note that the labels from the diary were not visible on the label screen even for the classic multilabel UI
e-mission/e-mission-docs#727 (comment)

Original code:

```

            return Promise.all(manualPromises).then((manualResults) => {
                const manualConfirmResults = {};
                manualResults.forEach(function(mr, index) {
                  manualConfirmResults[ConfirmHelper.INPUTS[index]] = mr;
                });
                return [result, manualConfirmResults];
            });
```

Broken code:

```
            return [result, Promise.all(manualPromises).then((manualResults) =>
                manualInputFactory.processManualInputs(manualResults, manualConfirmResults))];
```

Fixed code:

```
            return Promise.all(manualPromises).then((manualResults) => {
                manualInputFactory.processManualInputs(manualResults, manualConfirmResults);
                return [result, manualConfirmResults];
            });
```
… and vice versa

- Replace `etbs.SINGLE_INPUT` by `etbs.SINGLE_KEY` for consistency
- While caching a newly added entry, include a `write_ts` in the entry
By handling the different trip structure formats in the diary and the label
view
e-mission/e-mission-docs#727 (comment)
Tried to remove the "To Label" but then the spacing on the top got all messed
up. No time or energy to deal with little fiddly labels.
return UnifiedDataLoader.getUnifiedMessagesForInterval(
ConfirmHelper.inputDetails[inp].key, pendingLabelQuery);
inp_key, pendingLabelQuery).then(manualInputFactory.extractResult);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: extractResult is primarily for enketo where we check for version number matching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants