Merge pull request #892 from kobotoolbox/block-api-form

Remove public access to project API endpoint

README.md

@@ -1,15 +1,15 @@
-# KoBoCAT
+# KoboCAT
 
 ## Important notice when upgrading from any release older than [`2.020.18`](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.18)
 
-Up to and including release [`2.020.18`](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.18), this project (KoBoCAT) and [KPI](https://github.com/kobotoolbox/kpi) both shared a common Postgres database. They now each have their own. **If you are upgrading an existing single-database installation, you must follow [these instructions](https://community.kobotoolbox.org/t/upgrading-to-separate-databases-for-kpi-and-kobocat/7202)** to migrate the KPI tables to a new database and adjust your configuration appropriately.
+Up to and including release [`2.020.18`](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.18), this project (KoboCAT) and [KPI](https://github.com/kobotoolbox/kpi) both shared a common Postgres database. They now each have their own. **If you are upgrading an existing single-database installation, you must follow [these instructions](https://community.kobotoolbox.org/t/upgrading-to-separate-databases-for-kpi-and-kobocat/7202)** to migrate the KPI tables to a new database and adjust your configuration appropriately.
 
 If you do not want to upgrade at this time, please use the [`shared-database-obsolete`](https://github.com/kobotoolbox/kobocat/tree/shared-database-obsolete) branch instead.
 
 ## Deprecation Notices
 
 Much of the user-facing features of this application are being migrated
-to <https://github.com/kobotoolbox/kpi>. KoBoCAT's data-access API and
+to <https://github.com/kobotoolbox/kpi>. KoboCAT's data-access API and
 OpenRosa functions will remain intact, and any plans to the contrary
 will be announced well in advance. For more details and discussion,
 please refer to
@@ -18,6 +18,15 @@ please refer to
 As features are migrated, we will list them here along with the last
 release where each was present:
 
+  - On 14 June 2021, the ability to upload forms directly to KoboCAT was
+    removed, and it was announced that the legacy KoboCAT user interface would
+    be preserved for "a few more months". After more than two years, we have
+    removed the user interface and related endpoints entirely in release
+    [2.023.37](https://github.com/kobotoolbox/kobocat/releases/tag/2.023.37).
+    **This includes the ability to upload XLSForms via the legacy KoboCAT API.**
+    Please use the KPI `v2` API for all form management. Other removed features
+    should already be available in KPI as well. Please see
+    [REMOVALS.md](REMOVALS.md) for a complete list.
   - To ensure security and stability, many endpoints that were already
     available in KPI, long-unsupported, or underutilized have been removed in
     release
@@ -28,21 +37,21 @@ release where each was present:
     available in the release
     [2.020.39](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.39).
   - REST Services - an improved version [has been added to
-    KPI](https://github.com/kobotoolbox/kpi/pull/1864). The last KoBoCAT
+    KPI](https://github.com/kobotoolbox/kpi/pull/1864). The last KoboCAT
     release to contain legacy REST services is
     [2.019.39](https://github.com/kobotoolbox/kobocat/releases/tag/2.019.39).
 
 ## About
 
-kobocat is the data collection platform used in KoBoToolbox. It is based
+kobocat is the data collection platform used in KoboToolbox. It is based
 on the excellent [onadata](http://github.com/onaio/onadata) platform
 developed by Ona LLC, which in itself is a redevelopment of the
 [formhub](http://github.com/SEL-Columbia/formhub) platform developed by
 the Sustainable Engineering Lab at Columbia University.
 
 Please refer to
 [kobo-install](https://github.com/kobotoolbox/kobo-install) for
-instructions on how to install KoBoToolbox.
+instructions on how to install KoboToolbox.
 
 ## Code Structure
 
@@ -76,7 +85,7 @@ To compile MO files and update live translations
 $ django-admin.py compilemessages ;
 $ for app in {main,viewer} ; do cd kobocat/apps/${app} && django-admin.py compilemessages && cd - ; done
 ```
-## Testing in KoBoCAT
+## Testing in KoboCAT
 
 For kobo-install users, enter the folder for kobo-install and run this command
 

REMOVALS.md

@@ -1,4 +1,67 @@
-# KoBoCAT endpoint removals as of release 2.021.22
+# KoboCAT endpoint removals as of release 2.023.37
+
+The entire KoboCAT user interface has been removed.
+The last release to contain a user interface or any of the endpoints listed below was [2.023.21](https://github.com/kobotoolbox/kpi/releases/tag/2.023.21).
+
+## Obsolete User Interface
+
+URL Pattern | View Class or Function | View Name
+-- | -- | --
+`/` | `onadata.apps.main.views.home`
+`/<username>/` | `onadata.apps.main.views.profile` | `user_profile`
+`/<username>/api-token` | `onadata.apps.main.views.api_token` | `api_token`
+
+## Obsolete KPI Integrations
+
+URL Pattern | View Class or Function | View Name
+-- | -- | --
+`/login_redirect/` | `onadata.apps.main.views.login_redirect`
+`/<str:username>/forms/<str:id_string>/map` | `onadata.apps.main.views.view_func` | `redirect_map_to_kpi`
+`/<str:username>/reports/<str:id_string>/digest.html` | `onadata.apps.main.views.view_func` | `redirect_analyze_data_to_kpi`
+`/<str:username>/reports/<str:id_string>/export.html` | `onadata.apps.main.views.view_func` | `redirect_view_data_in_table_to_kpi`
+
+## Obsolete Enketo Integration
+
+URL Pattern | View Class or Function | View Name
+-- | -- | --
+`/api/v1/forms/<pk>/enketo\.<format>/` | `onadata.apps.api.viewsets.xform_viewset.XFormViewSet` | `xform-enketo`
+`/api/v1/forms/<pk>/enketo` | `onadata.apps.api.viewsets.xform_viewset.XFormViewSet` | `xform-enketo`
+
+## Obsolete Form Management
+
+URL Pattern | View Class or Function | View Name
+-- | -- | --
+`/forms/<uuid>` | `onadata.apps.main.views.show` | `show_form`
+`/<username>/forms/<id_string>/api` | `onadata.apps.main.views.api` | `mongo_view_api`
+`/<username>/forms/<id_string>/data\.csv` | `onadata.apps.viewer.views.data_export` | `csv_export`
+`/<username>/forms/<id_string>/data\.kml` | `onadata.apps.viewer.views.kml_export`
+`/<username>/forms/<id_string>/data\.xls` | `onadata.apps.viewer.views.data_export` | `xls_export`
+`/<username>/forms/<id_string>/delete-doc/<data_id>` | `onadata.apps.main.views.delete_metadata` | `delete_metadata`
+`/<username>/forms/<id_string>/doc/<data_id>` | `onadata.apps.main.views.download_metadata` | `download_metadata`
+`/<username>/forms/<id_string>/edit` | `onadata.apps.main.views.edit` | `edit_form`
+`/<username>/forms/<id_string>/formid-media/<data_id>` | `onadata.apps.main.views.download_media_data` | `download_media_data`
+`/<username>/forms/<id_string>/form_settings` | `onadata.apps.main.views.show_form_settings` | `show_form_settings`
+`/<username>/forms/<id_string>` | `onadata.apps.main.views.show` | `show_form`
+`/<username>/forms/<id_string>/photos` | `onadata.apps.main.views.form_photos` | `form_photos`
+`/<username>/superuser_stats/<base_filename>` | `onadata.apps.logger.views.retrieve_superuser_stats`
+`/<username>/superuser_stats/` | `onadata.apps.logger.views.superuser_stats`
+
+## Obsolete Documentation
+
+URL Pattern | View Class or Function | View Name
+-- | -- | --
+`/admin/doc/bookmarklets/` | `django.contrib.admindocs.views.BookmarkletsView` | `django-admindocs-bookmarklets`
+`/admin/doc/` | `django.contrib.admindocs.views.BaseAdminDocsView` | `django-admindocs-docroot`
+`/admin/doc/filters/` | `django.contrib.admindocs.views.TemplateFilterIndexView` | `django-admindocs-filters`
+`/admin/doc/models/<app_label>\.<model_name>/` | `django.contrib.admindocs.views.ModelDetailView` | `django-admindocs-models-detail`
+`/admin/doc/models/` | `django.contrib.admindocs.views.ModelIndexView` | `django-admindocs-models-index`
+`/admin/doc/tags/` | `django.contrib.admindocs.views.TemplateTagIndexView` | `django-admindocs-tags`
+`/admin/doc/templates/<path:template>/` | `django.contrib.admindocs.views.TemplateDetailView` | `django-admindocs-templates`
+`/admin/doc/views/` | `django.contrib.admindocs.views.ViewIndexView` | `django-admindocs-views-index`
+`/admin/doc/views/<view>/` | `django.contrib.admindocs.views.ViewDetailView` | `django-admindocs-views-detail`
+`/api-docs/` | `django.views.generic.base.RedirectView`
+
+# KoboCAT endpoint removals as of release 2.021.22
 
 The last release to contain any of the endpoints listed below was [2.021.21](https://github.com/kobotoolbox/kpi/releases/tag/2.021.21).
 
@@ -85,7 +148,7 @@ URL Pattern | View Class or Function | Description | Available in KPI
 `/xls2xform/` | `onadata.apps.main.views.xls2xform` | Unused informational page | Yes (on kobotoolbox.org)
 
 
-# KoBoCAT endpoint removals as of release [2.020.40](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.40)
+# KoboCAT endpoint removals as of release [2.020.40](https://github.com/kobotoolbox/kobocat/releases/tag/2.020.40)
 
 The last release to contain any of the endpoints listed below was https://github.com/kobotoolbox/kobocat/releases/tag/2.020.39.
 

onadata/apps/api/permissions.py

@@ -1,6 +1,7 @@
 # coding: utf-8
 from django.http import Http404
 from kobo_service_account.models import ServiceAccountUser
+from kobo_service_account.utils import get_real_user
 from rest_framework.permissions import (
     BasePermission,
     DjangoObjectPermissions,
@@ -79,10 +80,21 @@ class XFormPermissions(ObjectPermissionsWithViewRestricted):
 
     def has_permission(self, request, view):
         # Allow anonymous users to access shared data
-        if request.method in SAFE_METHODS and \
-                view.action and view.action == 'retrieve':
+        if (
+            request.method in SAFE_METHODS
+            and view.action
+            and view.action == 'retrieve'
+        ):
             return True
 
+        if (
+            request.method not in SAFE_METHODS
+            and view.action
+            and view.action in ['create', 'update', 'partial_update', 'destroy']
+            and not isinstance(request.user, ServiceAccountUser)
+        ):
+            raise LegacyAPIException
+
         return super().has_permission(request, view)
 
     def has_object_permission(self, request, view, obj):

onadata/apps/api/tests/viewsets/test_abstract_viewset.py

@@ -8,8 +8,10 @@
     User
 )
 from django.test import TestCase
+from django.test.client import Client
 from django_digest.test import Client as DigestClient
 from django_digest.test import DigestAuth
+from kobo_service_account.utils import get_request_headers
 from rest_framework.reverse import reverse
 from rest_framework.test import APIRequestFactory
 
@@ -50,7 +52,9 @@ def setUp(self):
         self._add_permissions_to_user(AnonymousUser())
         self.maxDiff = None
 
-    def publish_xls_form(self, path=None, data=None, assert_=True):
+    def publish_xls_form(
+        self, path=None, data=None, assert_=True, use_service_account=True
+    ):
         if not data:
             data = {
                 'owner': self.user.username,
@@ -65,14 +69,35 @@ def publish_xls_form(self, path=None, data=None, assert_=True):
 
         if not path:
             path = os.path.join(
-                settings.ONADATA_DIR, "apps", "main", "tests", "fixtures",
-                "transportation", "transportation.xls")
+                settings.ONADATA_DIR,
+                'apps',
+                'main',
+                'tests',
+                'fixtures',
+                'transportation',
+                'transportation.xls',
+            )
 
         xform_list_url = reverse('xform-list')
 
+        if use_service_account:
+            # Only service account user is allowed to `POST` to XForm API
+            client = Client()
+            service_account_meta = self.get_meta_from_headers(
+                get_request_headers(self.user.username)
+            )
+            service_account_meta['HTTP_HOST'] = settings.TEST_HTTP_HOST
+        else:
+            # For test purposes we want to try to `POST` with current logged-in
+            # user
+            client = self.client
+            service_account_meta = self.extra
+
         with open(path, 'rb') as xls_file:
             post_data = {'xls_file': xls_file}
-            response = self.client.post(xform_list_url, data=post_data)
+            response = client.post(
+                xform_list_url, data=post_data, **service_account_meta
+            )
 
         if not assert_:
             return response