Skip to content

Commit

Permalink
1.9.0 (#502)
Browse files Browse the repository at this point in the history
  • Loading branch information
NeonDaniel authored Feb 28, 2024
2 parents 3e4485f + 0466b78 commit fa1b2d4
Show file tree
Hide file tree
Showing 13 changed files with 1,257 additions and 100 deletions.
9 changes: 9 additions & 0 deletions .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -224,3 +224,12 @@ jobs:
with:
name: language-util-test-results
path: tests/language-util-test-results.xml

- name: Test Hana Utils
run: |
pytest tests/hana_util_tests.py --doctest-modules --junitxml=tests/hana-util-test-results.xml
- name: Upload hana utils test results
uses: actions/upload-artifact@v2
with:
name: hana-util-test-results
path: tests/hana-util-test-results.xml
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@
/dist/
/.pytest_cache/
tests/test-results.xml
*.egg-info/
*.egg-info/
__pycache__
44 changes: 38 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,52 @@
# Changelog

## [1.8.2a2](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.2a2) (2023-12-27)
## [1.8.3a6](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a6) (2024-02-26)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.2a1...1.8.2a2)
[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.3a5...1.8.3a6)

**Merged pull requests:**

- Use OpenStreetMap geocoder to work around maps.co API changes [\#493](https://github.com/NeonGeckoCom/neon-utils/pull/493) ([NeonDaniel](https://github.com/NeonDaniel))
- Update Hana utils to prep for release [\#501](https://github.com/NeonGeckoCom/neon-utils/pull/501) ([NeonDaniel](https://github.com/NeonDaniel))

## [1.8.2a1](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.2a1) (2023-12-20)
## [1.8.3a5](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a5) (2024-02-14)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.1...1.8.2a1)
[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.3a4...1.8.3a5)

**Merged pull requests:**

- Better default profile handling [\#492](https://github.com/NeonGeckoCom/neon-utils/pull/492) ([NeonDaniel](https://github.com/NeonDaniel))
- feat: adds function for installing packages via pip [\#499](https://github.com/NeonGeckoCom/neon-utils/pull/499) ([dblencowe](https://github.com/dblencowe))

## [1.8.3a4](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a4) (2024-02-07)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.3a3...1.8.3a4)

**Merged pull requests:**

- Fix directory errors in hana\_utils [\#500](https://github.com/NeonGeckoCom/neon-utils/pull/500) ([NeonDaniel](https://github.com/NeonDaniel))

## [1.8.3a3](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a3) (2024-01-24)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.3a2...1.8.3a3)

**Merged pull requests:**

- Add utilities for interfacing with Neon HANA [\#497](https://github.com/NeonGeckoCom/neon-utils/pull/497) ([NeonDaniel](https://github.com/NeonDaniel))

## [1.8.3a2](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a2) (2024-01-02)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.3a1...1.8.3a2)

**Merged pull requests:**

- Refactor and Deprecate NeonFallbackSkill [\#496](https://github.com/NeonGeckoCom/neon-utils/pull/496) ([NeonDaniel](https://github.com/NeonDaniel))

## [1.8.3a1](https://github.com/NeonGeckoCom/neon-utils/tree/1.8.3a1) (2023-12-29)

[Full Changelog](https://github.com/NeonGeckoCom/neon-utils/compare/1.8.2...1.8.3a1)

**Merged pull requests:**

- Update for ovos-workshop compat [\#495](https://github.com/NeonGeckoCom/neon-utils/pull/495) ([NeonDaniel](https://github.com/NeonDaniel))



Expand Down
141 changes: 141 additions & 0 deletions neon_utils/hana_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# NEON AI (TM) SOFTWARE, Software Development Kit & Application Framework
# All trademark and other rights reserved by their respective owners
# Copyright 2008-2022 Neongecko.com Inc.
# Contributors: Daniel McKnight, Guy Daniels, Elon Gasper, Richard Leeds,
# Regina Bloomstine, Casimiro Ferreira, Andrii Pernatii, Kirill Hrymailo
# BSD-3 License
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from this
# software without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
# OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

import requests
import json

from os import makedirs
from os.path import join, isfile, isdir, dirname
from time import time
from ovos_utils.log import LOG
from ovos_utils.xdg_utils import xdg_cache_home

_DEFAULT_BACKEND_URL = "https://hana.neonaiservices.com"
_client_config = {}
_headers = {}


def _get_client_config_path(url: str = _DEFAULT_BACKEND_URL):
url_key = hash(url)
return join(xdg_cache_home(), "neon", f"hana_token_{url_key}.json")


class ServerException(Exception):
"""Exception class representing a backend server communication error"""


def _init_client(backend_address: str):
"""
Initialize request headers for making backend requests. If a local cache is
available it will be used, otherwise an auth request will be made to the
specified backend server
@param backend_address: Hana server URL to connect to
"""
global _client_config
global _headers

if not _client_config:
client_config_path = _get_client_config_path(backend_address)
if isfile(client_config_path):
with open(client_config_path) as f:
_client_config = json.load(f)
else:
_get_token(backend_address)

if not _headers:
_headers = {"Authorization": f"Bearer {_client_config['access_token']}"}


def _get_token(backend_address: str, username: str = "guest",
password: str = "password"):
"""
Get new auth tokens from the specified server. This will cache the returned
token, overwriting any previous data at the cache path.
@param backend_address: Hana server URL to connect to
@param username: Username to authorize
@param password: Password for specified username
"""
global _client_config
# TODO: username/password from configuration
resp = requests.post(f"{backend_address}/auth/login",
json={"username": username,
"password": password})
if not resp.ok:
raise ServerException(f"Error logging into {backend_address}. "
f"{resp.status_code}: {resp.text}")
_client_config = resp.json()
client_config_path = _get_client_config_path(backend_address)
if not isdir(dirname(client_config_path)):
makedirs(dirname(client_config_path))
with open(client_config_path, "w+") as f:
json.dump(_client_config, f, indent=2)


def _refresh_token(backend_address: str):
"""
Get new tokens from the specified server using an existing refresh token
(if it exists). This will update the cached tokens and associated metadata.
@param backend_address: Hana server URL to connect to
"""
global _client_config
_init_client(backend_address)
update = requests.post(f"{backend_address}/auth/refresh", json={
"access_token": _client_config.get("access_token"),
"refresh_token": _client_config.get("refresh_token"),
"client_id": _client_config.get("client_id")})
if not update.ok:
raise ServerException(f"Error updating token from {backend_address}. "
f"{update.status_code}: {update.text}")
_client_config = update.json()
client_config_path = _get_client_config_path(backend_address)
with open(client_config_path, "w+") as f:
json.dump(_client_config, f, indent=2)


def request_backend(endpoint: str, request_data: dict,
server_url: str = _DEFAULT_BACKEND_URL) -> dict:
"""
Make a request to a Hana backend server and return the json response
@param endpoint: server endpoint to query
@param request_data: dict data to send in request body
@param server_url: Base URL of Hana server to query
@returns: dict response
"""
_init_client(server_url)
if time() >= _client_config.get("expiration", 0):
try:
_refresh_token(server_url)
except ServerException as e:
LOG.error(e)
_get_token(server_url)
resp = requests.post(f"{server_url}/{endpoint.lstrip('/')}",
json=request_data, headers=_headers)
if resp.ok:
return resp.json()
else:
raise ServerException(f"Error response {resp.status_code}: {resp.text}")
33 changes: 32 additions & 1 deletion neon_utils/packaging_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
import sys
import re
import importlib.util
from typing import Tuple, Optional
from typing import Tuple, Optional, List
from tempfile import mkstemp

import pkg_resources
import sysconfig
Expand Down Expand Up @@ -265,3 +266,33 @@ def get_skill_license(): # TODO: Implement OSM version of this
skill_data = dict_merge(default_skill, skill_data)
skill_data["requirements"]["python"].sort()
return dict(dict_merge(skill_data, readme_data))

def install_packages_from_pip(core_module: str, packages: List[str]) -> int:
"""
Install a Python package using pip
:param core_module: string neon core module to install dependency for
:param packages: List(string) list of packages to install
:returns: int pip exit code
"""
import pip
_, tmp_constraints_file = mkstemp()
_, tmp_requirements_file = mkstemp()

install_str = " ".join(packages)

with open(tmp_constraints_file, 'w', encoding="utf8") as f:
constraints = '\n'.join(get_package_dependencies(core_module))
f.write(constraints)
LOG.info(f"Constraints={constraints}")

with open(tmp_requirements_file, "w", encoding="utf8") as f:
for pkg in packages:
f.write(f"{pkg}\n")

LOG.info(f"Requested installation of plugins: {install_str}")
pip_args = ['install', '-r', tmp_requirements_file, '-c', tmp_constraints_file]
result = pip.main(pip_args) if hasattr(pip, 'main') else pip._internal.main(pip_args)

if result != 0:
return result
return 0
15 changes: 1 addition & 14 deletions neon_utils/skills/mycroft_skill.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
import os.path
import yaml

from threading import Event, Thread
from threading import Event
from typing import Optional
from json_database import JsonStorage
from ovos_bus_client.message import Message
Expand All @@ -57,14 +57,6 @@ def __init__(self, name=None, bus=None, *args, **kwargs):
self._speak_timeout = 30
self._get_response_timeout = 15 # 10 for listener, 5 for STT, then timeout

@property
def settings_path(self):
# TODO: Deprecate backwards-compat. wrapper after ovos-workshop 0.0.13
try:
return super().settings_path
except AttributeError:
return super()._settings_path

@property
def location(self):
"""
Expand All @@ -90,11 +82,6 @@ def _init_settings(self):
json.dump(self.settings, f, indent=4)
self._initial_settings = dict(self.settings)

def _init_settings_manager(self):
# TODO: Same as upstream implementation?
from ovos_workshop.settings import SkillSettingsManager
self.settings_manager = SkillSettingsManager(self)

def _read_default_settings(self):
yaml_path = os.path.join(self.root_dir, "settingsmeta.yml")
json_path = os.path.join(self.root_dir, "settingsmeta.json")
Expand Down
Loading

0 comments on commit fa1b2d4

Please sign in to comment.