Remote deployment

[ItzCrazyKns]

Get help from the community regarding deployment on a remote server or exposing it locally to your network.

[spammenotinoz]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free.
Secondly the backend and search API's are exposed to the client.
So really intended for a trusted audience, otherwise Architectual changes are required.

[spammenotinoz]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

I have not YET reviewed the code, but I see this project uses langchain, so in theory if you used something like LiteLLM Proxy in the same docker network, just set the baseUrl to LiteLLM Proxy (don't expose LiteLLM), then simply reference lite LiteLLM by the internal container name, so you don't need to define an OpenAI key at all, just use a dummy key.
Remove the settings \ section from the GUI.
Add some authentication.
You do not need to expose SearchNG, just reference it by the internal docker name.
ie: in the config.toml
[API_ENDPOINTS]
SEARXNG = "http://searxng:8080" #use the internal port not the external port
And in docker compose rem out the ports;
In docker args for Perplexica revise to - SEARXNG_API_URL=http://searxng:8080

So that leave the issue of having to expose the perplexica-backend (which makes all the API calls exposed).
Wouldn't be so much of a problem if a secret key is being used, but I didn't pickup on one.

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

[delfireinoso]

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

For my experience Perplexity is ahead the free version of Perpelxity, and approaching the paid version if you use a good model. It lacks the image parts, but the project is on first steps.

You are right the project has some limitations on the network settings and would benefit by some changes to make it useful out of the LAN. For the moment it's restricted to one user, so it's not safe to expose a strangers. But I think that someday this changes will be introduced, and a private Perplexity clone is better for me than the real

[ishaan-jaff]

hi @spammenotinoz, thanks for using / mentioning LiteLLM. Can we hop on a quick call sometime this week? I’d love to learn how we can improve litellm for you

My cal for your convenience: https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version?month=2023-10
My linkedin if you prefer DMs: https://www.linkedin.com/in/reffajnaahsi/

[ItzCrazyKns]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

I have not YET reviewed the code, but I see this project uses langchain, so in theory if you used something like LiteLLM Proxy in the same docker network, just set the baseUrl to LiteLLM Proxy (don't expose LiteLLM), then simply reference lite LiteLLM by the internal container name, so you don't need to define an OpenAI key at all, just use a dummy key. Remove the settings \ section from the GUI. Add some authentication. You do not need to expose SearchNG, just reference it by the internal docker name. ie: in the config.toml [API_ENDPOINTS] SEARXNG = "http://searxng:8080" #use the internal port not the external port And in docker compose rem out the ports; In docker args for Perplexica revise to - SEARXNG_API_URL=http://searxng:8080

So that leave the issue of having to expose the perplexica-backend (which makes all the API calls exposed). Wouldn't be so much of a problem if a secret key is being used, but I didn't pickup on one.

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

While building the images we update the URL in the config and use the internal port automatically (you can see more in the Docker Images). It is left there for the non Docker users so they do not need to update anything.

[ItzCrazyKns]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

The keys were exposed in the browser to allow different users use different models (with different API keys) are the same time. The project is still meant to be hosted locally. If we had to change how keys are stored, we can store them in the config file and update them from the frontend but this would limit each user to use the same API key for each provider. Still we need another way (besides this) to use custom OpenAI because users might want to have a different URL for them (I believe that is not big of an issue, we can expose that because it generally doesn't involves an API key or even if it does it might be local).

[Wadera]

After few tries and errors I've managed to make it working and wanted to share my setup ;)

Ports:

I strongly suggest to leave defaults ports of 3000 for frontend and 3001 for backend. I've tried to adjust it in docker-compose.yml and config.toml for backend but then app doesn't want to work. Maybe this ports are hardcoded somewhere? Maybe my initial build cache it? I'm not sure, but if you struggle - leave them as defaults.

docker-compose.yml:

If you want to make it working over domain - you need to fallow documentation and adjust it to IP or domain name. That's my working config:

  perplexica-frontend:
    build:
      context: .
      dockerfile: app.dockerfile
      args:
        #- NEXT_PUBLIC_API_URL=http://192.168.40.10:3001/api                 # working over IP
        #- NEXT_PUBLIC_WS_URL=ws://192.168.40.10:3001                        # working over IP
        - NEXT_PUBLIC_API_URL=https://perpexica.mydomain.tdl/api              # working over domain
        - NEXT_PUBLIC_WS_URL=wss://perpexica.mydomain.tdl/wss                 # working over domain
    depends_on:
      - perplexica-backend
    ports:
      - 3000:3000
    networks:
      - perplexica-network
    restart: unless-stopped

Where perpexica.mydomain.tdl is my domain with working https protocol and 192.168.40.10 is my docker VM IPv4 (in my case in internal subnet, as I'm just running it on my home-lab environment).

If you decide to use IP - then you can visit your IP with port to display frontend, so example: http://192.168.40.10:3000

Reverse proxy:

If you decide that you want to run it over https and friendly domain - you need to deploy reverse proxy. I'm using Nginx Proxy Manager, but you can deploy or use any other proxy you want.

To avoid issues with CORS I strongly suggest to proxy everything under single domain - both frontend and backend. In my case it would be perpexica.mydomain.tdl

That's GUI setup:

Details:

Custom locations:

Be aware of /api being added next to IP!

SSL:

proxy code

And that's code generated by npm if you want to deploy it with different tools:

# ------------------------------------------------------------
# perpexica.mydomain.tdl
# ------------------------------------------------------------

map $scheme $hsts_header {
    https   "max-age=63072000; preload";
}

server {
  set $forward_scheme http;
  set $server         "192.168.40.10";
  set $port           3000;

  listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;

  server_name perpexica.mydomain.tdl;

  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-68/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-68/privkey.pem;

    # Force SSL
    include conf.d/include/force-ssl.conf;

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_http_version 1.1;

  access_log /data/logs/proxy-host-66_access.log proxy;
  error_log /data/logs/proxy-host-66_error.log warn;

  location /api {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto  $scheme;
    proxy_set_header X-Forwarded-For    $remote_addr;
    proxy_set_header X-Real-IP          $remote_addr;
    proxy_pass       http://192.168.40.10:3001/api;

    # Authorization
    auth_basic            "Authorization required";
    auth_basic_user_file  /data/access/1;

    proxy_set_header Authorization "";


    # Access Rules: 4 total
    allow 192.168.2.0/24;
    allow 192.168.40.0/24;
    allow 172.0.0.1/8;
    allow 192.168.1.0/24;
    deny all;

    # Access checks must...
    satisfy any;

    # Force SSL
    include conf.d/include/force-ssl.conf;

    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_http_version 1.1;

  }
  location /wss {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto  $scheme;
    proxy_set_header X-Forwarded-For    $remote_addr;
    proxy_set_header X-Real-IP          $remote_addr;
    proxy_pass       http://192.168.40.10:3001;

    # Authorization
    auth_basic            "Authorization required";
    auth_basic_user_file  /data/access/1;
    proxy_set_header Authorization "";

    # Access Rules: 4 total
    allow 192.168.2.0/24;
    allow 192.168.40.0/24;
    allow 172.0.0.1/8;
    allow 192.168.1.0/24;
    deny all;

    # Access checks must...
    satisfy any;

    # Force SSL
    include conf.d/include/force-ssl.conf;

    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_http_version 1.1;
  }

  location / {
    # Authorization
    auth_basic            "Authorization required";
    auth_basic_user_file  /data/access/1;

    proxy_set_header Authorization "";

    # Access Rules: 4 total
    allow 192.168.2.0/24;
    allow 192.168.40.0/24;
    allow 172.0.0.1/8;
    allow 192.168.1.0/24;
    deny all;

    # Access checks must...
    satisfy any;

    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_http_version 1.1;

    # Proxy!
    include conf.d/include/proxy.conf;
  }

  # Custom
  include /data/nginx/custom/server_proxy[.]conf;
}

Security

As you hopefully noticed - I'm using my Access List which would allow to visit my instance from local network by my family members or over VPN to my home.

Be aware that all your search history and API keys are exposed under cog setting page and potentially visible in requests (I didn't review code that deeply).

That's just one of the reason why ItzCrazyKns advising to not expose this service to public at this point of development.

Until proper login panel, user management and security would be improved - I strongly advise to use it locally.

Final words:

Thank you for bringing this project to live! You are all doing brilliant work with it's development. I hope that my notes would be useful to others ;)