Slowdown explore

[jchate6]

Note from @jrfarah: this is now ready to be reviewed. See my comment below for a description of the new changes and improvements.

This is a draft PR.
This lets me make comments and relate our conversation more directly to the code in a self contained way.
The draft part of it is super useful because we all know the code isn't ready for review yet, but it could be converted into a real PR ready for review at any time.

[jrfarah]

Thanks Joey!

[jchate6]

We don't have anything like this in TOM Toolkit for a few reasons, the biggest one being we try to avoid javascript.
That does not mean that you shouldn't implement this if it works for you.
I'll also link this in TOMToolkit/tom_base#668 and maybe we can implement something similar for other slow pages.

[jrfarah]

Thanks Joey, this is super helpful! I'll pass the info along and we'll make sure to experiment carefully. @moira-andrews @dahowell

[moira-andrews]

Some preliminary testing from me was consistently giving this error, which is very similar to the Error loading plot error we get normally. Either way with everything loading asynchronously I was getting it like 50/50 every page load. Definitely want this looked into before merging and identify the best way to resolve it.

2025-11-13T17:18:33.893580834Z 2025-11-13 17:18:33.876    ERROR:             log: Internal Server Error: /django_plotly_dash/app/Spectra/initial/dpd-initial-args-345c2a89cdb047adb69cd79cf7f7b179/_dash-layout
2025-11-13T17:18:33.893603532Z Traceback (most recent call last):
2025-11-13T17:18:33.893607932Z   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
2025-11-13T17:18:33.893611641Z     response = get_response(request)
2025-11-13T17:18:33.893614773Z                ^^^^^^^^^^^^^^^^^^^^^
2025-11-13T17:18:33.893618063Z   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 199, in _get_response
2025-11-13T17:18:33.893621345Z     response = self.process_exception_by_middleware(e, request)
2025-11-13T17:18:33.893624656Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-13T17:18:33.893627844Z   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 362, in process_exception_by_middleware
2025-11-13T17:18:33.893630982Z     response = middleware_method(request, exception)
2025-11-13T17:18:33.893634047Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-13T17:18:33.893637036Z   File "/usr/local/lib/python3.11/site-packages/tom_common/middleware.py", line 29, in process_exception
2025-11-13T17:18:33.893640438Z     raise exception
2025-11-13T17:18:33.893643429Z   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
2025-11-13T17:18:33.893646789Z     response = wrapped_callback(request, *callback_args, **callback_kwargs)
2025-11-13T17:18:33.893649912Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-13T17:18:33.893653115Z   File "/usr/local/lib/python3.11/site-packages/django_plotly_dash/views.py", line 67, in layout
2025-11-13T17:18:33.893656273Z     initial_arguments = get_initial_arguments(request, cache_id)
2025-11-13T17:18:33.893659466Z                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-11-13T17:18:33.893662744Z   File "/usr/local/lib/python3.11/site-packages/django_plotly_dash/util.py", line 109, in get_initial_arguments
2025-11-13T17:18:33.893666083Z     return request.session[cache_id]
2025-11-13T17:18:33.893669113Z            ~~~~~~~~~~~~~~~^^^^^^^^^^
2025-11-13T17:18:33.893680192Z   File "/usr/local/lib/python3.11/site-packages/django/contrib/sessions/backends/base.py", line 53, in __getitem__
2025-11-13T17:18:33.893683637Z     return self._session[key]
2025-11-13T17:18:33.893686664Z            ~~~~~~~~~~~~~^^^^^
2025-11-13T17:18:33.893689639Z KeyError: 'dpd-initial-args-345c2a89cdb047adb69cd79cf7f7b179'

[jrfarah]

@moira-andrews thanks for raising this; I investigated a bit before I pushed the most recent changes but wasn't able to resolve it right away. I'll keep investigating and see if we can find a solution.

[jrfarah]

This branch fixes the main performance problems on the target detail page: slow first load, UI freezes during data fetches, and the recurring Dash “Error loading layout” failures. The solution is a shift to progressive lazy loading. Only the Overview tab renders on the initial request, while all other tabs (and the heavy components inside Overview) load asynchronously. Tabs stay disabled until their data arrives, unclickable and grayed out. Portions of the code for this PR were written using AI assistance (Cursor/Claude Sonnet).

The core behavior is driven through target_detail.html, which now handles:

• A lightweight JS loader
• Staggered background fetches
• Placeholder blocks for each heavy component

On the backend, I added dedicated endpoints for each tab and each plot in views.py and connected them in urls.py, so components like details, observations, spectra, photometry, thumbnails, airmass, and Dash plots are all delivered on demand.

The Dash “Error loading layout” bug turned out to be caused by expired Dash session keys. I added a client-side monitor that:

• Watches all Dash iframes shortly after load and at a few short intervals
• Identifies the error text and reloads only the failing component
• Tracks newly inserted iframes using a MutationObserver

On the backend, a new log filter compresses the long Dash traceback into a single line containing the app name and dead session key.

Testing was done in the new development environment created by @moira-andrews. She reproduced the speedups and flagged an issue I fixed by integrating the MutationObserver logic. The main code changes are in:

• snex2/templates/tom_targets/target_detail.html
• snex2/custom_code/views.py
• snex2/snex2/urls.py
• snex2/custom_code/log_filters.py (plus settings integration)
• custom_code/spectra_progressive_container.html for individual spectra loading

See a demo of the wildly improved performance below. I test opening 3 pages quickly in the production snex2 and the branch with the new changes. In the old version all 3 bug out and require reload. Even on reload performance is very slow and multiple "Error loading layout" errors occur. Both issues are solved in the new version; pages open instantaneously, and all layout load failures are quietly handled. Old snex2 is on left, new snex2 is on right.

snex2_performance_demo3.mp4

[moira-andrews]

I tested this on mock prod and found on pages like 23ixf that have lots of spectra, the spectroscopy tab has trouble loading, with there being some duplication in the dash, either with the labels, or duplicating the spectrum plot itself.

[moira-andrews]

The line plotting interface is not updating to show lines

[moira-andrews]

The info card on the left is not matching the spectrum, see here:

Vs. production