Merge conflicts

.gitignore

@@ -0,0 +1,2 @@
+__pycache__
+.history/

CHANGELOG.md

@@ -1,15 +1,3 @@
 # CHANGELOG
 
-TODO(brycew): back-port previous releases to this change log from GitHub release notes.
-
-## Version v0.19.1
-
-### Fixed
-
-Update sqlalchemy by @BryceStevenWilley in #75
-
-Docassemble 1.4.33 updated to SQL Alchemy 2.0, which broke the session viewing functionality in the Dashboard.
-This patch fixes things to work on the latest docassemble version, all the way back to docassemble version 1.2.68.
-
-Full Changelog: [v0.19.0...v0.19.1](https://github.com/SuffolkLITLab/docassemble-ALDashboard/compare/v0.19.0...v0.19.1)
-
+See https://github.com/SuffolkLITLab/docassemble-ALDashboard/releases

README.md

@@ -2,53 +2,66 @@
 
 [![PyPI version](https://badge.fury.io/py/docassemble.ALDashboard.svg)](https://badge.fury.io/py/docassemble.ALDashboard)
 
-A single pane of glass that centralizes some tedious Docassemble admin configuration tasks
+A single tool and interview to centralize some tedious Docassemble admin configuration tasks.
 
-![image](https://user-images.githubusercontent.com/7645641/123702117-bdd7d300-d830-11eb-8c0e-8e204d912ff8.png)
+![A screenshot of the ALDashboard menu with choices: "Admin only - manage users", "Admin only - stats", "Install assembly line", "Verify API Keys", "Install packages", "update packages", "Package scanner", "View Answer files", "generate review screen draft", "validate docx template", "validation translation files", "prepare translation files", "validate an attachment fields block", "PDF tools", and "Compile Bootstrap theme"](https://github.com/SuffolkLITLab/docassemble-ALDashboard/assets/6252212/29539eec-3891-476b-b248-dd3db986d899)
 
 1. Install the Document Assembly Line packages (support files for [Court Forms Online](https://courtformsonline.org))
 1. Searchable user management - reset passwords and change privileges.
 1. Installing or updating several packages at once.
 1. Listing and viewing the contents of an (unencrypted) interview to facilitate debugging errors on production servers.
-1. View analytics/stats captured with store_variable_snapshot.
+1. View analytics/stats captured with `store_variable_snapshot`.
 1. List the files inside a particular package installed on the server.
 1. Gather files from a user who left the organization/unknown username and password.
 1. Review screen generator
 1. validate DOCX Jinja2 templates
+1. Generate a [custom bootstrap theme](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/customization/overview#creating-a-custom-theme-from-source-instead-of-with-a-theme-generator) for your interviews.
 
 Ideas:
 1. Add a link to the dispatch directive for an existing file in an existing package.
-1. Generating translation files [TBD].
+1. Generate translation files [TBD].
 
 To use, you must create a docassemble API key and add it to your
 configuration, like this:
 
 `install packages api key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
 
+If you want the ALDashboard to be a dropdown option for admins and developers, add the following to the configuration before your `install packages api key`: 
+
+    administrative interviews:
+      - interview: docassemble.ALDashboard:data/questions/menu.yml
+        title: Dashboard
+        required privileges:
+          - admin
+          - developer
+
 ## Some screenshots
 
 ### Main page
-![image](https://user-images.githubusercontent.com/7645641/123702117-bdd7d300-d830-11eb-8c0e-8e204d912ff8.png)
+![A screenshot of the ALDashboard menu with choices: "Admin only - manage users", "Admin only - stats", "Install assembly line", "Verify API Keys", "Install packages", "update packages", "Package scanner", "View Answer files", "generate review screen draft", "validate docx template", "validation translation files", "prepare translation files", "validate an attachment fields block", "PDF tools", and "Compile Bootstrap theme"](https://github.com/SuffolkLITLab/docassemble-ALDashboard/assets/6252212/29539eec-3891-476b-b248-dd3db986d899)
 
 ### Manage users
 
-![image](https://user-images.githubusercontent.com/7645641/123702231-e069ec00-d830-11eb-94dc-5ec0abb86bc9.png)
+![A screenshot that says "Manage users" with the fields "User", "What do you want want to do? Reset password or Change user permissions", "New Password", and "Verify new Password"](https://user-images.githubusercontent.com/7645641/123702231-e069ec00-d830-11eb-94dc-5ec0abb86bc9.png)
 
 ### Bulk install packages from GitHub
 
-![image](https://user-images.githubusercontent.com/7645641/123702290-efe93500-d830-11eb-9fdf-a5935ff4078e.png)
+![A screenshot that says "What packages do you want to install?" The fields are for "Github URL", "YAML filename", and "Short name or alias (no spaces)"](https://user-images.githubusercontent.com/7645641/123702290-efe93500-d830-11eb-9fdf-a5935ff4078e.png)
 
 ### Bulk update packages
 
-![image](https://user-images.githubusercontent.com/7645641/123702362-068f8c00-d831-11eb-9ce4-df7a67ffcfeb.png)
+![A screenshot that says "What packages do you want to update?" followed by a list of packages. For example, "docassemble.209aPlaintiffMotionToModify", "docassemble.ALAffidavitOfIndigency", and more.](https://user-images.githubusercontent.com/7645641/123702362-068f8c00-d831-11eb-9ce4-df7a67ffcfeb.png)
 
 ### View / search sessions by user and interview name
 
-![image](https://user-images.githubusercontent.com/7645641/123702422-1d35e300-d831-11eb-84d5-5e7385deb901.png)
+![A screenshot that says "What interview do you want to view sessions for?" The fields are "File name" and "User (leave blank to view all sessions)"](https://user-images.githubusercontent.com/7645641/123702422-1d35e300-d831-11eb-84d5-5e7385deb901.png)
 
-![image](https://user-images.githubusercontent.com/7645641/123702464-2cb52c00-d831-11eb-80fc-f2291e824eae.png)
+![A screenshot that says "Recently generated sessions for docassemble.MA209AProtectiveOrder:data/questions/209a_package.yml" with 5 sessions below.](https://user-images.githubusercontent.com/7645641/123702464-2cb52c00-d831-11eb-80fc-f2291e824eae.png)
 
 ### View interview stats captured with `store_variables_snapshot()`
 
-![image](https://user-images.githubusercontent.com/7645641/123702623-5e2df780-d831-11eb-8937-6625df74ab22.png)
+![A screenshot with the title "Stats for Eviction Moratorium: 9". Below is the text "Total submissions: 9", "Group by: zip | state | modtime", and "Excel Download" followed by a map that can be filtered by state or by date.](https://user-images.githubusercontent.com/7645641/123702623-5e2df780-d831-11eb-8937-6625df74ab22.png)
+
+### Generate a bootstrap theme
 
+![A screenshot with the title "Your file is compiled!", below is the text "You can view and copy your file, or download it directly by right clicking the link to save it as a CSS file". Below that are examples of Bootstrap components like buttons and nav bars.](https://github.com/SuffolkLITLab/docassemble-ALDashboard/assets/6252212/079e428d-4cae-4f75-8b1b-227c28f32a44)

docassemble/ALDashboard/__init__.py

@@ -1 +1 @@
-__version__ = '0.20.1'
+__version__ = '0.21.1'

docassemble/ALDashboard/aldashboard.py

@@ -1,3 +1,6 @@
+import os
+import shutil
+import subprocess
 from docassemble.webapp.users.models import UserModel
 from docassemble.webapp.db_object import init_sqlalchemy
 from github import Github  # PyGithub
@@ -20,10 +23,20 @@
 )
 from docassemble.base.config import daconfig
 from docassemble.webapp.backend import cloud
-from docassemble.base.util import log, DAFile, DAObject, DAList, word
+from docassemble.base.util import (
+    log,
+    DAFile,
+    DAObject,
+    DAList,
+    word,
+    DAFileList,
+    get_config,
+    space_to_underscore,
+)
 from ruamel.yaml import YAML
 from ruamel.yaml.compat import StringIO
 import re
+import werkzeug
 
 db = init_sqlalchemy()
 
@@ -39,6 +52,8 @@
     "ALPackageInstaller",
     "get_package_info",
     "install_from_pypi",
+    "install_fonts",
+    "list_installed_fonts",
 ]
 
 
@@ -122,13 +137,13 @@ def da_write_config(data: Dict):
     restart_all()
 
 
-def speedy_get_users() -> List[Tuple[int, str]]:
+def speedy_get_users() -> List[Dict[int, str]]:
     """
     Return a list of all users in the database. Possibly faster than get_user_list().
     """
     the_users = UserModel.query.with_entities(UserModel.id, UserModel.email).all()
 
-    return [user for user in the_users]
+    return [{user[0]: user[1]} for user in the_users]
 
 
 def get_users_and_name() -> List[Tuple[int, str, str, str]]:
@@ -139,7 +154,9 @@ def get_users_and_name() -> List[Tuple[int, str, str, str]]:
     return users
 
 
-def speedy_get_sessions(user_id: Optional[int] = None, filename: Optional[str] = None) -> List[Tuple]:
+def speedy_get_sessions(
+    user_id: Optional[int] = None, filename: Optional[str] = None
+) -> List[Tuple]:
     """
     Return a lsit of the most recent 500 sessions, optionally tied to a specific user ID.
 
@@ -180,7 +197,7 @@ def speedy_get_sessions(user_id: Optional[int] = None, filename: Optional[str] =
         user_id = None
 
     with db.connect() as con:
-        rs = con.execute(get_sessions_query, {"user_id":user_id, "filename":filename})
+        rs = con.execute(get_sessions_query, {"user_id": user_id, "filename": filename})
     sessions = []
     for session in rs:
         sessions.append(session)
@@ -230,7 +247,7 @@ def init(self, *pargs, **kwargs):
 
 class ErrorLikeObject(DAObject):
     """
-    An object with a `template_name` that identifieds the DALazyTemplate that will
+    An object with a `template_name` that identifies the DALazyTemplate that will
     show its error. It can contain any other attributes so its template can access them
     as needed. DAObject doesn't seem to be enough to allow template definition.
     """
@@ -241,6 +258,52 @@ def init(self, *pargs, **kwargs):
         self.template_name = kwargs.get("template_name", "unknown_error")
 
 
+def install_fonts(the_font_files: DAFileList):
+    """
+    Install fonts to the server and restart both supervisor and unoconv.
+    """
+    # create the /var/www/.fonts directory if it doesn't exist
+    if not os.path.exists("/var/www/.fonts"):
+        os.makedirs("/var/www/.fonts")
+
+    # save the DAFile to /var/www/.fonts
+    for f in the_font_files:
+        shutil.copyfile(
+            f.path(), "/var/www/.fonts/" + werkzeug.utils.secure_filename(f.filename)
+        )
+
+    output = ""
+    output += subprocess.run(
+        ["fc-cache", "-f", "-v"], capture_output=True, text=True
+    ).stdout
+    output += subprocess.run(
+        ["supervisorctl", "restart", "uwsgi"], capture_output=True, text=True
+    ).stdout
+    output += subprocess.run(
+        ["supervisorctl", "start", "reset"], capture_output=True, text=True
+    ).stdout
+    if get_config("enable unoconv"):
+        output += subprocess.run(
+            ["supervisorctl", "-s", "http://localhost:9001", "restart", "unoconv"],
+            capture_output=True,
+            text=True,
+        ).stdout
+
+    return output
+
+
+def list_installed_fonts():
+    """
+    List the fonts installed on the server.
+    """
+    fc_list = subprocess.run(["fc-list"], stdout=subprocess.PIPE)
+    output = subprocess.run(
+        ["sort"], stdin=fc_list.stdout, capture_output=True, text=True
+    ).stdout
+    fc_list.stdout.close()
+    return output
+
+
 #  select userdict.filename, num_keys, userdictkeys.user_id, modtime, userdict.key from userdict natural join (select key, max(modtime) as modtime, count(key) as num_keys from userdict group by key) mostrecent left join userdictkeys on userdictkeys.key = userdict.key order by modtime desc;
 # db.session.query
 

docassemble/ALDashboard/create_package.py

@@ -145,7 +145,7 @@ def create_package_zip(pkgname: str, info: dict, author_info: dict, folders_and_
 """
   setupcfg = """\
 [metadata]
-description-file = README.md
+description_file = README.md
 """
   setuppy = """\
 import os

docassemble/ALDashboard/data/questions/api_test.yml

@@ -0,0 +1,95 @@
+---
+include:
+  - docassemble.ALToolbox:display_template.yml
+  - nav.yml
+---
+metadata:
+  title: Test API keys
+  sessions are unique: True
+  required privileges:
+    - admin
+    - developer
+  temporary session: True
+---
+objects:
+  - my_person: Individual
+---
+mandatory: True
+code: |
+  address_to_complete
+  if my_person.address.address:
+    my_person.address.geocode()
+  if my_person.email:
+    email_success = send_email(to=my_person, subject="Test message from Docassemble", body="Test message body. Thanks for testing with ALDashboard.")
+  if my_person.phone_number:
+    sms_success = send_sms(to=my_person, body="Test message from Docassemble. Thanks for testing with ALDashboard.")
+  api_results  
+---
+question: |
+  Test an API key
+fields:
+  - note: |
+      ---
+      Google Maps
+
+      Server API key: `${ get_config("google", {}).get("api key") }` [BR]
+      HTTP/S API key: `${ get_config("google", {}).get("google maps api key") }`
+  - Address auto complete: address_to_complete
+    address autocomplete: True
+    required: False
+  - "Address geocoding (enter at least a street address to trigger)": my_person.address.address
+    required: False
+  - Unit: my_person.address.unit
+    required: False
+  - City: my_person.address.city
+    required: False
+  - State: my_person.address.state
+    required: False
+  - Zip: my_person.address.zip
+    required: False
+  - County: my_person.address.county
+    required: False
+  - note: |
+      ---
+      Email sending
+
+      ```
+      ${ get_config("mail") }
+      ```
+  - Enter an email to get a test message: my_person.email
+    datatype: email
+    required: False
+  - note: |
+      ---
+      SMS sending
+
+      ```
+      ${ get_config("twilio") }
+      ```
+  - Enter a phone number to get a test message: my_person.phone_number
+    datatype: phone
+    required: False
+---
+event: api_results
+question: |
+  Results
+subquestion: |
+  % if my_person.address.address:
+  Geocoded: ${ my_person.address.was_geocoded_successfully() } [BR]
+  % if my_person.address.was_geocoded_successfully():
+  Long address: ${ my_person.address.norm_long.on_one_line() } [BR]
+  % endif
+  % if hasattr(my_person.address, "county"):
+  County: ${ my_person.address.county }
+  % endif
+  % endif
+
+  % if my_person.email:
+  Email success: ${ email_success }
+  % endif
+
+  % if my_person.phone_number:
+  SMS success: ${ sms_success }
+  % endif
+
+  Use the back button to try again