Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Sheets export for form data #71

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open

Google Sheets export for form data #71

wants to merge 34 commits into from

Conversation

pitzer
Copy link
Contributor

@pitzer pitzer commented Aug 24, 2015

This pull request creates a new export option for Google Sheets (GSHEETS).

High level overview on how the Sheets export works:

  • The export requires a Google service account that is configured in (onadata/settings/common.py).
  • The export view (onadata/apps/viewer/views.py) checks for a valid OAuth token and optionally redirects to grant the service account write access to the user's Google Drive.
  • An export can be started with either default settings or advanced settings. Advanced settings allow optional flattening of repeated fields as well as export of the raw XLSForm as part of the export.
  • When started, a celery export task is started (onadata/apps/viewer/tasks.py)
  • The export task uses the SheetsExportBuilder (onadata/libs/utils/google_sheets.py) to dump all form data into a Google Sheet.
  • SheetsExportBuilder add the service account as an editor and uses this account to write data.

Misc:

  • Creates ExportBuilder classes for each export option. This refactors onadata/libs/utils/export_tools.py and moves a lot of code into individual classes/modules.
  • Adds a test for Google Sheets export (onadata/apps/main/tests/test_google_docs_export.py)
  • Removes old Google Docs export (GDOCS) that stopped working as the APIs have changed
  • The UI part of this feature is in Google Sheets export for form data kobocat-template#72

michaelolson and others added 30 commits July 24, 2015 14:26
@michaelolson
Avoid up to two additional saves when using unique_type_for_form.
71afe71
@sandizzle
Initial scaffolding for Google Sheets Sync
63225d7
@sandizzle
Merge branch 'google' of https://github.com/pitzer/kobocat into google
65726fb
@pitzer
Adding a stub for the Google Sheets API.
12146b5
@pitzer
Added kobo service user credentials.
9fde67c
@pitzer
Merge branch 'google' of github.com:pitzer/kobocat into google
6304bcb
@pitzer
Added a stub for the Google Sheets API.
4fbdf16
@pitzer
Adds a base class for export builders and a first implementation of a
8b63233 
Sheets builder.
@pitzer
A first implementation of an export build for xls.
0162aee
@pitzer
A first implementation of an export build for csv.
8651ad1
@pitzer
Add support for flat CSV export.
9b7caad
@pitzer
A first implementation of an export build for sav.
696ca84
@pitzer
Refactors export framework.
7186ea5 
This commit changes the export framework from using a single
ExportBuilder class and multiple export functions (e.g.
ExportBuilder.to_xls_export and ExportBuilder.to_zipped_csv) to
sub-classing ExportBuilder for specialized exports (e.g XlsExportBuilder
and CsvExportBuilder). This enables better separation of specialized
export logic and cleans up export_tools.py. 

The commit also enables Google Sheets export through the
SheetsExportBuilder and removes old Google Docs export related code.
@pitzer
Adds a unit test for google sheets export.
36075ea 
The test can be executed with:
python manage.py test onadata.apps.main.tests.test_google_sheets_export
@pitzer
Cleans up some code in CSV export builders.
8203b17
@pitzer
Cleans up some code in the XLS export builder.
07db418
@pitzer
Rolling back changes that break authentication.
bc6703d
@pitzer
Small bugfix in XLS export.
70379b7
@pitzer
Some cleanups and fixes for the Google Sheets export.
41f8338 
- removed filename extension. extensions are now optional
- renamed GSHEETS to Google Sheets
- added a sheet title to export builder
- cleanups in sheets export:
  - remove default worksheet (Sheet1)
  - reuse first row when inserting data
  - export a URL to open the sheet in a browser
@pitzer
Cleans up debug prints.
8d79314
@sandizzle
Add support for login with service account and adding the service acc…
475766c 
…ount as editor on the export spreadsheet
@sandizzle
adding oauth2client requirement
65cc3d8
@pitzer
Resolving merge conflicts.
6311e9f
@pitzer
Adds a fallback for non-existing google key file.
e5c1ddb
@pitzer
Small fix after merging with Sandip's changes.
697aa8f
@pitzer
Added UI options to configure Google Sheets Export.
082e698 
- Flattened or tabular output
- Export of XLSForm data
@pitzer
Fixes a small bug in create_async_export
a6706a9
@pitzer
Removes a bunch of debug print statements.
ec327c7
@pitzer
Fixes Google Sheets Export test.
67d17e0
@pitzer
Adds another test for flattened export.
b7632bb
@pitzer pitzer mentioned this pull request Aug 24, 2015
Merged
jnm pushed a commit that referenced this pull request Mar 28, 2019
Merge pull request #71 from hossein/xls_analyzer_better_errors
49aef23 
Template for errors related to Excel Analyzer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pitzer @michaelolson @sandizzle