-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
bb864df
commit a34cf80
Showing
21 changed files
with
468 additions
and
43 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
298 changes: 298 additions & 0 deletions
298
docs/posts/2024/lack-of-isolation-gpts-code-interpreter/index.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,298 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en-us"> | ||
| <head prefix="og: http://ogp.me/ns#"> | ||
| <meta charset="utf-8" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1" /> | ||
| <meta property="og:title" content=" ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs · Embrace The Red" /> | ||
|
|
||
| <meta property="og:site_name" content="Embrace The Red" /> | ||
| <meta property="og:url" content="https://embracethered.com/blog/posts/2024/lack-of-isolation-gpts-code-interpreter/" /> | ||
|
|
||
|
|
||
| <meta property="og:type" content="article" /> | ||
|
|
||
| <meta property="og:article:published_time" content="2024-02-14T03:30:17-08:00" /> | ||
|
|
||
| <meta property="og:article:tag" content="aiml" /> | ||
|
|
||
| <meta property="og:article:tag" content="machine learning" /> | ||
|
|
||
| <meta property="og:article:tag" content="threats" /> | ||
|
|
||
| <meta property="og:article:tag" content="ai injections" /> | ||
|
|
||
| <meta property="og:article:tag" content="llm" /> | ||
|
|
||
|
|
||
|
|
||
| <title> | ||
| ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs · Embrace The Red | ||
| </title> | ||
|
|
||
| <link rel="stylesheet" href="https://embracethered.com/blog/css/bootstrap.min.css" /> | ||
| <link rel="stylesheet" href="https://embracethered.com/blog/css/main.css" /> | ||
| <link rel="stylesheet" href="https://embracethered.com/blog/css/font-awesome.min.css" /> | ||
| <link rel="stylesheet" href="https://embracethered.com/blog/css/github.css" /> | ||
|
|
||
| <link rel="stylesheet" href="https://embracethered.com/blog/fonts/cachedfonts.css" type="text/css"> | ||
| <link rel="shortcut icon" type="image/png" href="https://embracethered.com/blog/images/favicon.ico?" /> | ||
| <link rel="apple-touch-icon" type="image/png" href="https://embracethered.com/blog/images/favicon.png" /> | ||
|
|
||
|
|
||
|
|
||
| <meta name="twitter:card" content="summary_large_image"> | ||
| <meta name="twitter:site" content="@wunderwuzzi23"> | ||
| <meta name="twitter:title" content="ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs"> | ||
| <meta name="twitter:description" content="Public GPTs can access and overwrite your private GPT files and vice versa."> | ||
| <meta name="twitter:image" content="https://embracethered.com/blog/images/2024/lack-of-isolation-gpts.png"> | ||
|
|
||
| <meta name="twitter:creator" content="@wunderwuzzi23"> | ||
|
|
||
|
|
||
|
|
||
|
|
||
| </head> | ||
| <body> | ||
| <header class="global-header" style="background-image:url( /blog/images/bg.png )"> | ||
| <section class="header-text"> | ||
| <h1><a href="https://embracethered.com/blog/">Embrace The Red</a></h1> | ||
|
|
||
| <div class="tag-line" style="min-width:fit-content; font-weight: 400;"> | ||
| wunderwuzzi's blog | ||
| <br><a style="color: greenyellow; font-weight:300;text-decoration: underline; " href="https://www.amazon.com/gp/product/1838828869/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1838828869&linkCode=as2&tag=wunderwuzzi-20&linkId=b6523e937607be47499c6010ff489537">OUT NOW: Cybersecurity Attacks - Red Team Strategies</a> | ||
| </div> | ||
|
|
||
| <div class="sns-links hidden-print"> | ||
|
|
||
| <a href="mailto:security@wunderwuzzi.net"> | ||
| <i class="fa fa-envelope"></i> | ||
| </a> | ||
|
|
||
|
|
||
| <a href="https://twitter.com/wunderwuzzi23" target="_blank"> | ||
| <i class="fa fa-twitter"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| <a href="https://github.com/wunderwuzzi23" target="_blank"> | ||
| <i class="fa fa-github"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| <a href="https://youtube.com/@embracethered" target="_blank"> | ||
| <i class="fa fa-youtube"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
| <a href="/blog/index.xml" target="_blank"> | ||
| <i class="fa fa-rss"></i> | ||
| </a> | ||
| </div> | ||
|
|
||
|
|
||
| <a href="https://embracethered.com/blog/" class="btn-header btn-back hidden-xs"> | ||
| <i class="fa fa-angle-left" aria-hidden="true"></i> | ||
| Home | ||
| </a> | ||
|
|
||
|
|
||
| <a href="/blog/index.xml" class="btn-header btn-subscribe hidden-xs"> | ||
| <i class="fa fa-rss" aria-hidden="true"></i> | ||
| Subscribe | ||
| </a> | ||
| </section> | ||
| </header> | ||
| <main class="container"> | ||
|
|
||
|
|
||
| <article> | ||
| <header> | ||
| <h1 class="text-primary">ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs</h1> | ||
| <div class="post-meta clearfix"> | ||
|
|
||
| <div class="post-date pull-left"> | ||
| Posted on | ||
| <time datetime="2024-02-14T03:30:17-08:00"> | ||
| Feb 14, 2024 | ||
| </time> | ||
| </div> | ||
|
|
||
| <div class="pull-right"> | ||
|
|
||
| <span class="post-tag small"><a href="https://embracethered.com/blog//tags/aiml">#aiml</a></span> | ||
|
|
||
| <span class="post-tag small"><a href="https://embracethered.com/blog//tags/machine-learning">#machine learning</a></span> | ||
|
|
||
| <span class="post-tag small"><a href="https://embracethered.com/blog//tags/threats">#threats</a></span> | ||
|
|
||
| <span class="post-tag small"><a href="https://embracethered.com/blog//tags/ai-injections">#ai injections</a></span> | ||
|
|
||
| <span class="post-tag small"><a href="https://embracethered.com/blog//tags/llm">#llm</a></span> | ||
|
|
||
| </div> | ||
| </div> | ||
| </header> | ||
| <section> | ||
| <p>This is an older bug, but it has been over 90 days that this vulnerability was reported to OpenAI. To adhere to <a href="https://about.google/intl/ALL_us/appsecurity/">industry norms of responsible disclosure</a> this post describes a security vulnerability in ChatGPT regarding the lack of isolation of GPTs when it comes to Code Interpreter sessions. At the bottom are also mitigation steps users and builders of GPTs can take.</p> | ||
| <h2 id="lack-of-isolation-with-gpts-and-code-interpreter">Lack of Isolation with GPTs and Code Interpreter</h2> | ||
| <p>Your Code Interpreter sandbox, also known as Advanced Data Analysis sessions, are shared between private and public GPTs. Yes, your actual compute container and its storage is shared. Each user gets their own isolated container, but if a user uses multiple GPTs and stores files in Code Interpreter <strong>all GPTs can access (and also overwrite) each others files</strong>.</p> | ||
| <p>This is true also for files uploaded/created with private GPTs and ChatGPT itself.</p> | ||
| <p><a href="/blog/images/2024/lack-of-isolation-gpts.png"><img src="/blog/images/2024/lack-of-isolation-gpts.png" alt="Lack of Isolation"></a></p> | ||
| <h2 id="simple-repro">Simple Repro</h2> | ||
| <p>The easiest way to repro this is to start a new ChatGPT chat, then upload a file or ask it to create a new file. You can ask for its <code>hostname</code> as a check, and <code>list files in /mnt/data</code> to view them.</p> | ||
| <p>Afterwards do the same in a new chat session of a Custom GPT (with Code Interpreter enabled) and observe that hostname and private files created from your ChatGPT chat are visible and can also be overwriten.</p> | ||
| <p>Here is a short demo video that highlights the problem:</p> | ||
|
|
||
| <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> | ||
| <iframe src="https://www.youtube.com/embed/b0pQ9DvdyZc" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe> | ||
| </div> | ||
|
|
||
| <p><!-- raw HTML omitted --><!-- raw HTML omitted --></p> | ||
| <p>The video shows how a regular ChatGPT session and a custom GPT (public) have access to the same files of the user as they share the same Code Interpreter session.</p> | ||
| <p><strong>Here are the repro steps in more detail:</strong></p> | ||
| <ol> | ||
| <li>Use ChatGPT and upload a test file (This file is stored inside your Code Interpreter environment)</li> | ||
| <li>Validate that you can see the file by asking ChatGPT something like <code>list the files in /mnt/data</code>. Observe that the file is indeed there (you can see how it writes the code and runs it when seeing the “Analyzing…” icon)</li> | ||
| <li>Now pick a random third party GPT, or create your own and start a new Chat session (it needs CI enabled)</li> | ||
| <li>Ask the third party GPT the same question <code>list the files in /mnt/data</code> and observe how the third party GPT has access to the same files.</li> | ||
| </ol> | ||
| <p>This still worked as of today, February, 14th 2024. It was first disclosed to the vendor early last November and confirmed to be a security problem. However, followup queries about status remain unanswered.</p> | ||
| <h2 id="what-does-this-mean-implications">What does this mean? Implications.</h2> | ||
| <p><strong>It allows the creator of a malicious GPT to steal and also overwrite files from your conversations with other GPTs, including ChatGPT</strong>.</p> | ||
| <h3 id="are-custom-knowledge-files-of-private-gpts-safe">Are custom knowledge files of private GPTs safe?</h3> | ||
| <p>As soon as Code Interpreter is invoked (visible via the “Analyzing…” UI indicator), any custom knowledge files of that GPT are copied into the shared Code Interpreter space and are available to all GPTs.</p> | ||
| <p>So, the answer is no. OpenAI cannot guarantee that private knowledge files in a private GPT are secured. There is the chance of knowledge of a private GPT becoming accessible to other GPTs.</p> | ||
| <p>If you build a private GPT with custom knowledge, then disabling Code Interpreter should mitigate this.</p> | ||
| <h3 id="resetting-the-environment">Resetting the environment</h3> | ||
| <p>After some idle time Code Interpreter sessions reset, which means all files are gone and the user starts with a new instance. It’s unclear exactly when that happens. And the user cannot decide to reset it themselves.</p> | ||
| <p>For a while I always used <code>%reset -f</code>, which seemed to force a reset, but that does not work anymore.</p> | ||
| <p><strong>There is only one CI container per user, across multiple GPT sessions.</strong></p> | ||
| <p>If you upload your own personal finance CSV to do some data analysis with ChatGPT, and then you use another custom third party GPT, the <strong>third party custom GPT has full access to your personal CSV file</strong>!</p> | ||
| <p>Similarly, custom knowledge of a private GPT might leak to other GPTs once the knowledge is loaded into CI.</p> | ||
| <h2 id="recommendations">Recommendations</h2> | ||
| <p>Code Interpreter sessions get discarded after a while, but it’s not clear when that happens. You can always prompt for <code>list files in /mnt/data</code> to check if there are any files.</p> | ||
| <ul> | ||
| <li>Users should not upload or create sensitive files that they do not want to have leaked.</li> | ||
| <li>Use third party GPTs (or prompts) with care, especially if they have Code Interpreter enabled.</li> | ||
| <li>Disable Code Interpreter when building private GPTs with private knowledge files (as they will be accessible to other GPTs, and <strong>other GPTs might influence your GPT by writing files</strong> to Code Interpreter)</li> | ||
| <li>Consider using <a href="https://chat.openai.com/g/g-YyyyMT9XH-chatgpt-classic">OpenAI’s Classic GPT</a> (without Code Interpreter) if the functionality is not needed.</li> | ||
| <li>Users could also delete uploaded files manually via Code Interpreter when tasks are completed.</li> | ||
| </ul> | ||
| <p>To OpenAI the following recommendations were provided in November:</p> | ||
| <ul> | ||
| <li>Isolate GPTs and their compute and storage (first party, private, public) - every GPT session needs to have its own container</li> | ||
| <li>Implement static analysis to catch GPTs with hidden/malicious instructions</li> | ||
| </ul> | ||
| <p>In comparison Google Bard (now Gemini) <a href="/blog/posts/2024/exploring-google-bard-vm/">seems to not share any state in the compute container between chat sessions</a>.</p> | ||
| <h2 id="references">References</h2> | ||
| <ul> | ||
| <li><a href="https://www.youtube.com/watch?v=b0pQ9DvdyZc">Demo Video of Lack of Isolation GPT</a></li> | ||
| <li><a href="https://chat.openai.com/g/g-YyyyMT9XH-chatgpt-classic">OpenAI’s Classic GPT</a></li> | ||
| <li><a href="/blog/posts/2024/exploring-google-bard-vm/">Exploring Google Bard’s Data Visualization Feature (Code Interpreter)</a></li> | ||
| <li><a href="https://about.google/intl/ALL_us/appsecurity/">How Google handles security vulnerabilities</a></li> | ||
| </ul> | ||
|
|
||
| </section> | ||
| <footer> | ||
|
|
||
|
|
||
|
|
||
| <footer></footer><hr/> | ||
|
|
||
| <ul class="pager"> | ||
|
|
||
| <li class="next disabled"><a href="#">Newer <span aria-hidden="true">→</span></a></li> | ||
|
|
||
|
|
||
| <li class="author-contact"> | ||
| <a href="mailto:security@wunderwuzzi.net"> | ||
| <i class="fa fa-envelope-o" aria-hidden="true"></i> | ||
| Contact me | ||
| </a> | ||
| </li> | ||
|
|
||
|
|
||
|
|
||
| <li class="previous"><a href="https://embracethered.com/blog/posts/2024/ascii-smuggling-and-hidden-prompt-instructions/"><span aria-hidden="true">←</span> Older</a></li> | ||
|
|
||
| </ul> | ||
| </footer> | ||
| </article> | ||
| </main> | ||
| <footer class="container global-footer"> | ||
| <div class="copyright-note pull-left"> | ||
|
|
||
| (c) WUNDERWUZZI 2018-2023 | ||
| <div class="sns-links hidden-print"> | ||
|
|
||
| <a href="mailto:security@wunderwuzzi.net"> | ||
| <i class="fa fa-envelope"></i> | ||
| </a> | ||
|
|
||
|
|
||
| <a href="https://twitter.com/wunderwuzzi23" target="_blank"> | ||
| <i class="fa fa-twitter"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| <a href="https://github.com/wunderwuzzi23" target="_blank"> | ||
| <i class="fa fa-github"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| <a href="https://youtube.com/@embracethered" target="_blank"> | ||
| <i class="fa fa-youtube"></i> | ||
| </a> | ||
|
|
||
|
|
||
|
|
||
| <a href="/blog/index.xml" target="_blank"> | ||
| <i class="fa fa-rss"></i> | ||
| </a> | ||
| </div> | ||
|
|
||
| <div style="font-size:small;font-style: italic;color:crimson"> | ||
| Disclaimer: Penetration testing requires authorization from proper stakeholders. Information on this blog is provided for research and educational purposes to advance understanding of attacks and countermeasures to help secure the Internet. | ||
| </div> | ||
| </div> | ||
| </footer> | ||
| <script src="https://embracethered.com/blog/js/highlight.pack.js"></script> | ||
| <script> | ||
| hljs.initHighlightingOnLoad(); | ||
| </script> | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| <script type="text/javascript"> | ||
| var _paq = window._paq || []; | ||
| _paq.push(["disableCookies"]); | ||
| _paq.push(['trackPageView']); | ||
| _paq.push(['enableLinkTracking']); | ||
| (function() { | ||
| var u="//wuzzi.net/anamato/inc/"; | ||
| _paq.push(['setTrackerUrl', u+'rts.php']); | ||
| _paq.push(['setSiteId', '1']); | ||
| var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; | ||
| g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'rts.js'; s.parentNode.insertBefore(g,s); | ||
| })(); | ||
| </script> | ||
| <img src="https://wuzzi.net/anamato/inc/rts.php?idsite=1&rec=1" style="border:0;" alt="" /> | ||
|
|
||
| </body> | ||
| </html> | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.