Initial commit

.dockerignore

@@ -0,0 +1,4 @@
+tmp
+.env
+.git*
+Dockerfile

.gitattributes

@@ -0,0 +1,42 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+embeddings/mfcom_aws.kb/index.pkl filter=lfs diff=lfs merge=lfs -text
+embeddings/mfcom_aws.kb/index.faiss filter=lfs diff=lfs merge=lfs -text
+embeddings/mfcom_azure.kb/index.pkl filter=lfs diff=lfs merge=lfs -text
+embeddings/mfcom_azure.kb/index.faiss filter=lfs diff=lfs merge=lfs -text
+embeddings/blips_vol_30_azure.kb/index.pkl filter=lfs diff=lfs merge=lfs -text
+embeddings/blips_vol_30_azure.kb/index.faiss filter=lfs diff=lfs merge=lfs -text
+embeddings/blips_vol_30_aws.kb/index.pkl filter=lfs diff=lfs merge=lfs -text
+embeddings/blips_vol_30_aws.kb/index.faiss filter=lfs diff=lfs merge=lfs -text

.github/workflows/main.yml

@@ -0,0 +1,53 @@
+name: Haiven Community Knowledge Pack Workflow
+run-name: Build Docker Image
+
+env:
+  REGISTRY: "ghcr.io"
+
+on:
+  push:
+  workflow_dispatch: # allows for manual invocation
+  repository_dispatch:
+    types: [haiven-base-new]
+
+jobs:
+
+  Build:
+    if: github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: 'read'
+      packages: 'write'
+    steps:
+      - uses: 'actions/checkout@v4.1.1'
+        with:
+          lfs: true
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          # TODO: Can we switch to GITHUB_TOKEN once haiven repo is public?
+          # GH_REGISTRY_TOKEN needs read and write access to packages, to both pull the base image and then push the new image later
+          password: ${{ secrets.GH_REGISTRY_TOKEN }}
+
+      - uses: docker/metadata-action@v5
+        id: metadata
+        with:
+          images: ${{ env.REGISTRY }}/${{ github.repository }}
+
+      - name: Kaniko build community instance image
+        # if: steps.changes.outputs.build == 'true'
+        id: 'kaniko-community-instance'
+        uses: int128/kaniko-action@v1
+        with:
+          tags: ${{ steps.metadata.outputs.tags }}
+          labels: ${{ steps.metadata.outputs.labels }}
+          cache: true
+          cache-repository: ${{ env.REGISTRY }}/${{ github.repository }}/cache
+          push: true
+          context: ./
+          build-args: |
+            REGISTRY_URL=${{ env.REGISTRY }}/tw-haiven/
+            TAG=main

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+

Dockerfile

@@ -0,0 +1,8 @@
+ARG REGISTRY_URL
+ARG TAG=main
+
+FROM ${REGISTRY_URL}haiven:${TAG}
+
+COPY . /app/teams
+
+CMD ["python", "-u", "main.py"]

LICENSE.md

@@ -0,0 +1,3 @@
+
+
+<p xmlns:cc="http://creativecommons.org/ns#" xmlns:dct="http://purl.org/dc/terms/"><a property="dct:title" rel="cc:attributionURL" href="https://github.com/tw-haiven">Haiven knowledge pack</a> by <a rel="cc:attributionURL dct:creator" property="cc:attributionName" href="https://thoughtworks.com">Thoughtworks</a> is licensed under <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1" target="_blank" rel="license noopener noreferrer" style="display:inline-block;">CC BY-NC-SA 4.0<img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1" alt=""><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1" alt=""><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/nc.svg?ref=chooser-v1" alt=""><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1" alt=""></a></p>

README.md

@@ -0,0 +1,3 @@
+# Community Knowledge Pack
+
+This repository is a sample "knowledge pack" for the Haiven team assistance sandbox. You will find more information in the [main Haiven repository](https://github.com/tw-haiven/haiven).

contexts/demo_crm/README.md

@@ -0,0 +1,4 @@
+
+Example of some knowledge context snippets for a fictitious team that is implementing a CRM (Customer Relationship Management). 
+
+A knowledge context can be created for a domain, a specific application or service, a team, ... Haiven users can switch between contexts in the UI, making the prompts reusable while these snippets inject contextual information per context.

contexts/demo_crm/architecture.md

@@ -0,0 +1,21 @@
+---
+key: architecture
+title: Architecture
+---
+
+The application consists of a web frontend and a backend with HTTP RESTful APIs, and has multiple integration points with other systems
+
+# Core technology stack
+* Frontend: JavaScript, ReactJS, Gulp, Browserify, Jest
+* Backend: Java, Gradle, Guice, Jersey
+* Persistence: Postgres, Solr for search indexing
+* Deployment: Docker, Kubernetes
+
+# Integration points with other systems
+INTEGRATION: Existing CRM interface: customer data and activities, flat file exports in CSV, to provide master data for the Sales Force Application
+* Customer data (export)
+* Contacts data (export)
+* Address data (export)
+* Activities data (export and import) > visits, calls, …
+
+INTEGRATION: Employee Active Directory login, to integrate for login functionality

contexts/demo_crm/domain.md

@@ -0,0 +1,16 @@
+---
+key: domain
+title: Domain
+---
+
+You're part of a team building a web application that will allow salespeople in a company to organise their work and document their customer relationships.
+
+Users: Sales Managers who are looking after a portfolio of restaurants/hotels/caterers/traders, looking to build a personal relationship and increase the revenue their respective customers are generating.
+
+Cross-functional requirements: 
+- It needs to work on mobile phones and with unstable internet connection (aka “offline”)
+- Needs to support internationalisation and localisation
+
+Domain terminology:
+- Contact: a visit of a customer, or a phone call, or an email. A sales manager usually takes notes of each contact
+- Activity: synonym for "Contact"

contexts/demo_crm/frontend_coding_patterns.md

@@ -0,0 +1,24 @@
+---
+key: frontend_coding_patterns
+title: Frontend coding patterns
+---
+
+Tech stack: typescript, react, redux, konvajs and react-konva. And vitest, react testing library for model, view model and related hooks, cypress component tests for view.
+
+All code should be written in the tech stack mentioned above. Requirements should be implemented as React components in the MVVM architecture pattern.
+
+2 types of view models:
+* Shared view model. View model that represents states shared among local and remote users.
+* Local view model. View model that represents states only applicable to local user
+
+Common implementation strategies:
+* Shared view model is implemented as Redux store slice. Tested in vitest.
+* Local view model is implemented as React component props or states(by useState hook), unless for global local view model, which is also implemented as Redux store slice. Tested in vitest.
+* Hooks are used as the major view helpers to retrieve data from shared view model. For most the case, it will use ‘createSelector’ and ‘useSelector’ for memorization. Tested in vitest and react testing library.
+* Don’t dispatch action directly to change the states of shared view model, use an encapsulated view model interface instead. In the interface, each redux action is mapped to a method. Tested in vitest.
+* View consists of konva shapes, implemented as React component via react-konva. Tested in cypress component tests
+
+Patterns to be followed when testing the component:
+* When writing a test, use describe instead of test
+* Data-driven tests are preferred.
+* When testing the view component, use a fake view model via the view model interface

contexts/demo_ev_charging_stations/README.md

@@ -0,0 +1,4 @@
+
+Example of some knowledge context snippets for a fictitious team that is implementing software for EV charging stations. 
+
+A knowledge context can be created for a domain, a specific application or service, a team, ... Haiven users can switch between contexts in the UI, making the prompts reusable while these snippets inject contextual information per context.

contexts/demo_ev_charging_stations/architecture.md

@@ -0,0 +1,23 @@
+---
+key: architecture
+title: Architecture
+---
+
+You're part of a team building the software for Electrical Vehicle (EV) charging stations in a large charging stations network.
+
+Components of the architecture are:
+
+* Embedded Software/Firmware, Written in C/C++ and runs on a microcontroller.
+* User Interface - built with "Flutter for embedded"
+* Authentication and Security
+* Payment Processing Module - handles billing and payment transactions. can integrate with external payment gateways and support various payment methods, including credit cards, mobile payments, and subscription models.
+*  Data Management - The station collects data on usage, power consumption, and operational status for billing, maintenance, and analytics; stored in memory, regularly synced to the Cloud
+*  Remote Management and Monitoring, Over-the-air updates - Updating station firmware, configuring settings, troubleshooting issues, and generating reports on usage and operational status.
+
+### Domain Glossary
+- OCPP (Open Charge Point Protocol) communication standard used between charging stations and central management systems
+- EVSE (Electric Vehicle Supply Equipment) Refers to the components and infrastructure that deliver electrical energy to recharge electric vehicles
+- AC (Alternating Current) and DC (Direct Current) Charging; AC charging is slower and typically used for residential or light commercial use. DC charging is faster
+- Connector Types: Type 1 (North America, Japan), Type 2 (Europe), CCS (North America, Europe)
+- V2G (Vehicle-to-Grid) Technology that allows electric vehicles to communicate with the power grid
+- SoC (State of Charge)

contexts/demo_ev_charging_stations/domain.md

@@ -0,0 +1,37 @@
+---
+key: domain
+title: Domain
+---
+
+You're part of a team building the software for Electrical Vehicle (EV) charging stations in a large charging stations network.
+
+Features are:
+
+* Embedded Software/Firmware
+This is the low-level software running directly on the hardware of the charging station. It controls the physical components such as connectors, cables, and power management systems. It ensures that the charging process starts, monitors it for any issues, and stops the charge at the appropriate time or in case of an error.
+
+* User Interface
+Touchscreen interface for the user to interact with. Used too start or stop a charge, authenticate (via a mobile app), or view charging status and pricing information.
+
+* Authentication and Security
+Happens via mobile app authentication
+
+* Payment Processing Module
+handles billing and payment transactions. can integrate with external payment gateways and support various payment methods, including credit cards, mobile payments, and subscription models.
+
+*  Data Management
+The station collects data on usage, power consumption, and operational status for billing, maintenance, and analytics
+
+*  Remote Management and Monitoring
+Updating station firmware, configuring settings, troubleshooting issues, and generating reports on usage and operational status.
+
+*  Energy Management
+helps optimize the power load, reduce operational costs, and ensure stability of the electrical grid
+
+### Domain Glossary
+- OCPP (Open Charge Point Protocol) communication standard used between charging stations and central management systems
+- EVSE (Electric Vehicle Supply Equipment) Refers to the components and infrastructure that deliver electrical energy to recharge electric vehicles
+- AC (Alternating Current) and DC (Direct Current) Charging; AC charging is slower and typically used for residential or light commercial use. DC charging is faster
+- Connector Types: Type 1 (North America, Japan), Type 2 (Europe), CCS (North America, Europe)
+- V2G (Vehicle-to-Grid) Technology that allows electric vehicles to communicate with the power grid
+- SoC (State of Charge)

embeddings/README.md

@@ -0,0 +1,6 @@
+
+These are sample knowledge bases that can be questioned in the "Knowledge Chat" tab of the Haiven UI, and also pulled into chat conversations as additional resources.
+
+For this to work in the application, you need to run Haiven with access to the embeddings model that created these, so that the user's questions can be matched. The markdown files mention the "provider" used to create the embeddings, you'll find the corresponding embeddings model in [Haiven's `config.yaml`](https://github.com/tw-haiven/haiven/blob/main/app/config.yaml).
+
+Note that the Retrieval-Augmented Generation (RAG) implementation in Haiven is VERY basic, as everything happens in memory only. We currently prioritise simple deployment over sophisticated RAG. We want it to be "just good enough" to give experimenters an indication of the potential. If you like the Haiven sandbox as a starting point, but would like more powerful RAG capabilities, you would have to change the [Haiven code](https://github.com/tw-haiven) to connect to a proper information retrieval setup, like e.g. one of the big cloud providers' AI studios. 

embeddings/blips_vol_30_aws.md

@@ -0,0 +1,9 @@
+---
+key: radar-vol-30_aws
+title: Technology Radar vol 30
+description: Thoughtworks Technology Radar, volume 30
+source: https://thoughtworks.com/radar
+path: blips_vol_30_aws.kb
+provider: aws
+sample_question: "I build a lot of Microservices, what's interesting for me?"
+---

embeddings/blips_vol_30_azure.md

@@ -0,0 +1,9 @@
+---
+key: radar-vol-30_azure
+title: Technology Radar vol 30
+description: Thoughtworks Technology Radar, volume 30
+source: https://thoughtworks.com/radar
+path: blips_vol_30_azure.kb
+provider: azure
+sample_question: "I build a lot of Microservices, what's interesting for me?"
+---

embeddings/mfcom_aws.md

@@ -0,0 +1,10 @@
+---
+key: mfcom_aws
+title: Martin Fowler's Website
+description: Content from martinfowler.com
+
+source: https://martinfowler.com
+path: mfcom_aws.kb
+provider: aws
+sample_question: "We're about to start a legacy migration, what should we consider?"
+---

embeddings/mfcom_azure.md

@@ -0,0 +1,10 @@
+---
+key: mfcom_azure
+title: Martin Fowler's Website
+description: Content from martinfowler.com
+
+source: https://martinfowler.com
+path: mfcom_azure.kb
+provider: azure
+sample_question: "We're about to start a legacy migration, what should we consider?"
+---

prompts/brainstorming/README.md

@@ -0,0 +1,5 @@
+## Brainstorming prompts
+
+These are prompts that get the AI to ask the user questions. If you want to add your own, use the [user_story.md](user_story.md) one as an example, in particular use the `<Thought>`, `<Question>` and `<Answer>` pseudo-tags, as the application will be using them to help parse the results.
+
+Note that these prompts require models that support stop sequences in their API. Also, not all models are great at this type of "reasoning", so you may get mixed results depending on which model you use. For more information about the implementation of the "Brainstorming" tab, refer to `[chats.py](https://github.com/tw-haiven/haiven/blob/main/app/shared/chats.py)` and look for `Q_A_Chat`.

prompts/brainstorming/user_story.md

@@ -0,0 +1,37 @@
+---
+identifier: user-story-brainstorming-0cc383e56743
+title: "User Story"
+system: "You are a business analyst who is familiar with specification by example. I'm the domain expert."
+
+categories: ["analysis"]
+
+help_prompt_description: "Brainstorm about a user story: The AI will ask probing questions to help you discover gaps in your thinking. It will then generate some given/when/then scenarios based on your answers"
+help_user_input: "Provide the high level description of the user story that you have so far"
+help_sample_input: "Plan activity for a specific date: The user should be able to add a new planned activity to the application, for a specific customer, with a date and some notes describing what they want to do"
+---
+
+Help me explain a user story as scenarios.
+
+------
+CONTEXT:
+{domain}
+
+------
+
+USER STORY:
+{user_input}
+
+------
+
+Explain the user story as scenarios. Use the following format:
+
+<Thought>Think about what is still uncertain about defining the user story. Ignore technical concerns and purpose, only focus on defining scenarios. </Thought>
+<Question>the question to ask to clarify the user story</Question>
+<Answer>The answer you suggest for the question</Answer>
+
+... (this Thought/Question/Answer repeat at least 3 times, at most 10 times)  
+
+<Thought>You know enough to explain the user story</Thought>
+Scenarios: List all possible scenarios with concrete example in Given/When/Then style
+
+If the user answers "stop" to a question, stop asking questions, have the Though "I know enough to explain the user story"