5.1.2

* Fix createColorScaleByType when offsets are available

* lint-fix

---------

Co-authored-by: David Sehnal <dsehnal@users.noreply.github.com>

.github/workflows/node.yml

@@ -13,7 +13,7 @@ jobs:
     - uses: actions/checkout@v2
     - uses: actions/setup-node@v2
       with:
-        node-version: 18
+        node-version: 20
     - run: npm ci
     - run: sudo apt-get install xvfb
     - name: Lint

CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file, following t
 Note that since we don't clearly distinguish between a public and private interfaces there will be changes in non-major versions that are potentially breaking. If we make breaking changes to less used interfaces we will highlight it in here.
 
 ## [Unreleased]
+
+## [v5.1.0] - 2025-10-25
+- Fix createColorScaleByType when offsets are available
+- Get bond orders from non-standard CONECT records in PDB files
+- Remove outdated `gl_FrontFacing` workaround for buggy drivers
+- Fix clip objects for direct-volume rendering
+- Support "magic window" style AR (via WebXR)
+- Fix `PluginState.getStateTransitionFrameIndex`
+- Update `GlycamSaccharideNames` and `Monosaccharides` in `carbohydrates/constants.ts`
+- Support custom ref resolvers in `State` 
+- Add full-screen mode support to layout manager
+  - Add `show-toggle-fullscreen` URL param option to Viewer app
+- MolViewSpec
+  - Support accessing Mol* State nodes by MVS-provided ref
+  - Add support for DX map format
+  - Better support for coarse structures in MVS:
+    - Support for MVS annotations on coarse structures (color_from_*, tooltip_from_*)
+    - Support for MVS labels on coarse structures (label, label_from_*)
+    - (Other things already worked on coarse structures before: tooltip, color,component, primitives, component_from_*, primitives_from_*)
+  - Tidy up MVS builder:
+    - Add `sphere` and `angle` methods
+    - [Breaking] Rename builder method primitives_from_uri -> primitivesFromUri
+
+## [v5.0.0] - 2025-09-28
 - [Breaking] Renamed some color schemes ('inferno' -> 'inferno-no-black', 'magma' -> 'magma-no-black', 'turbo' -> 'turbo-no-black', 'rainbow' -> 'simple-rainbow')
 - [Breaking] `Box3D.nearestIntersectionWithRay` -> `Ray3D.intersectBox3D`
 - [Breaking] `Plane3D.distanceToSpher3D` -> `distanceToSphere3D` (fix spelling)
@@ -12,6 +36,8 @@ Note that since we don't clearly distinguish between a public and private interf
 - [Breaking] Add `Volume.instances` support and a `VolumeInstances` transform to dynamically assign it
   - This change is breaking because all volume objects require the `instances` field now.
 - [Breaking] `Canvas3D.identify` now expects `Vec2` or `Ray3D`
+- [Breaking] `TrackballControlsParams.animate.spin.speed` now means "Number of rotations per second" instead of "radians per second"
+- [Breaking] `PluginStateSnapshotManager.play` now accepts an options object instead of a single boolean value
 - Update production build to use `esbuild`
 - Emit explicit paths in `import`s in `lib/`
 - Fix outlines on opaque elements using illumination mode
@@ -19,9 +45,10 @@ Note that since we don't clearly distinguish between a public and private interf
 - MolViewSpec extension:
   - Generic color schemes (`palette` parameter for color_from_* nodes)
   - Annotation field remapping (`field_remapping` parameter for color_from_* nodes)
-  - Representation node: support custom property `molstar_reprepresentation_params`
-  - Primitives node: support custom property `molstar_mesh/label/line_params`
-  - Canvas node: support custom property `molstar_postprocessing` with the ability to customize outline, depth of field, bloom, shadow, occlusion (SSAO), and fog
+  - `representation` node: support custom property `molstar_representation_params`
+  - Add `backbone` and `line` representation types
+  - `primitives` node: support custom property `molstar_mesh/label/line_params`
+  - `canvas` node: support custom property `molstar_postprocessing` with the ability to customize outline, depth of field, bloom, shadow, occlusion (SSAO), fog, and background
   - `clip` node support for structure and volume representations
   - `grid_slice` representation support for volumes
   - Support tethers and background for primitive labels
@@ -34,14 +61,24 @@ Note that since we don't clearly distinguish between a public and private interf
   - Support transforming and instancing of structures, components, and volumes
   - Use params hash for node version for more performant tree diffs
   - Add `Snapshot.animation` support that enables animating almost every property in a given tree
+  - Add `createMVSX` helper function
+  - Support Mol* trackball animation via `animation.custom.molstar_trackball`
+  - MVSX - use Murmur hash instead of FNV in archive URI
+  - Support additional file formats (pdbqt, gro, xyz, mol, sdf, mol2, xtc, lammpstrj)
+  - Support loading trajectory coordinates from separate nodes
+  - Trigger markdown commands from primitives using `molstar_markdown_commands` custom extensions
+  - Support `molstar_on_load_markdown_commands` custom state on the `root` node
+  - Print tree validation errors to plugin log
 - Added new color schemes, synchronized with D3.js ('inferno', 'magma', 'turbo', 'rainbow', 'sinebow', 'warm', 'cool', 'cubehelix-default', 'category-10', 'observable-10', 'tableau-10')
 - Snapshot Markdown improvements
   - Add `MarkdownExtensionManager` (`PluginContext.managers.markdownExtensions`)
   - Support custom markdown commands to control the plugin via the `[link](!command)` pattern
   - Support rendering custom elements via the `![alt](!parameters)` pattern
   - Support tables
-  - Support loading images from MVSX files
+  - Support loading images and audio from MVSX files
   - Indicate external links with ⤴
+  - Audio support
+  - Add `PluginState.Snapshot.onLoadMarkdownCommands`
 - Avoid calculating rings for coarse-grained structures
 - Fix isosurface compute shader normals when transformation matrix is applied to volume
 - Symmetry operator naming for spacegroup symmetry - parenthesize multi-character indices (1_111-1 -> 1_(11)1(-1))
@@ -57,7 +94,10 @@ Note that since we don't clearly distinguish between a public and private interf
 - Add `StructureInstances` transform
 - `mvs-stories` app
   - Add `story-id` URL arg support
+  - Add `story-session-url` URL arg support
   - Add "Download MVS State" link
+  - Add "Open in Mol*" link
+  - Add "Edit in MolViewStories" link for story states
 - Add ray-based picking
     - Render narrow view of scene scene from ray origin & direction to a few pixel sized viewport
     - Cast ray on every input as opposed to the standard "whole screen" picking
@@ -81,6 +121,34 @@ Note that since we don't clearly distinguish between a public and private interf
 - Add `Hsl` and (normalized) `Rgb` color spaces
 - Add `Color.interpolateHsl`
 - Add `rotationCenter` property to `TransformParam`
+- Add Monolayer transparency (exploiting dpoit).
+- Add plugin config item ShowReset (shows/hides "Reset Zoom" button)
+- Fix transform params not being normalized when used together with param hash version
+- Replace `immer` with `mutative`
+- Fix renderer transparency check
+- Add outlines improvements
+- VolumeServer & "VolumeCIF": default to P 1 spacegroup
+- Fix `ColorScale` for continuous case without offsets (broke in v4.13.0)
+- Experimental: support for custom color themes in Sequence Panel
+- Switch files.rcsb.org validation report URL to new endpoint /validation/view
+- Improve picking of objects with too many groups, pick whole instance/object
+- Add WebXR support
+    - Requires immersive AR/VR headset
+    - Supplements non-XR: enter/exit XR anytime and see (mostly) the same scene
+    - Add `Canvas3D.xr` for managing XR sessions
+    - Add `PointerHelper` for rendering XR input devices
+    - Add XR button to Viewer and Mesoscale Explorer
+    - Add XR button to render-structure in tests/browser
+- Fix illumination denoising with transparency on transparent background
+- Change the `to_mmCIF` function parameter from `structure` to `structures` to support either a single structure or an array of structures
+- ModelServer and VolumeServer: add configurable robots.txt
+- Adaptive parallel shader compilation
+    - Split shader compilation into linking and finalizing
+    - Start linking as early as possible and wait with finalizing to avoid blocking main thread
+        - Use of `KHR_parallel_shader_compile` extension when available to check status
+    - Add `ShaderManager` to compile shaders based on `Canvas3D` params and `Scene` content
+    - Draw `Scene` only when shaders are ready
+- Fix incorrect animation loop handling in the screenshot code
 
 ## [v4.18.0] - 2025-06-08
 - MolViewSpec extension:

docs/docs/plugin/instance.md

@@ -247,7 +247,7 @@ async function init() {
     const canvas = <HTMLCanvasElement> document.getElementById('molstar-canvas');
     const parent = <HTMLDivElement> document.getElementById('molstar-parent');
 
-    if (!(await plugin.initViewer(canvas, parent))) {
+    if (!(await plugin.initViewerAsync(canvas, parent))) {
         console.error('Failed to init Mol*');
         return;
     }

docs/docs/plugin/managers/markdown-extensions.md

@@ -20,14 +20,19 @@ Generally, the command should be URL encoded, e.g., `a b` => `a%20b` (in JS, `en
 
 - `center-camera` - Centers the camera
 - `apply-snapshot=key` - Loads snapshots with the provided key
+- `next-snapshot[=-1|1]` - Loads next/previous snapshot, the direction is optional and default to `1`
+- `play-snapshots` - Starts playback of state snapshots
+- `play-transition` - Plays an animation associated with the given snapshot
+- `stop-animation` - Stops currently playing animation
 - `focus-refs=ref1,ref2,...` - On click, focuses nodes with the provided refs
 - `highlight-refs=ref1,ref2,...` - On mouse over, highlights the provided refs
 - `query=...&lang=...&action=highlight,focus&focus-radius=...`
-  - `query` is an expression (e.g., `resn HEM` when using PyMol syntax)
-  - (optional) `lang` is one of `mol-script` (default), `pymol`, `vmd`, `jmol`
-  - (optional) `action` is an array of `highlight` (default), `focus` (multiple actions can be specified)
-  - (optional) `focus-radius` is extra distance applied when focusing the selection (default is `3`)
-  - Example: `[HEM](!query%3Dresn%20HEM%26lang%3Dpymol%26action%3Dhighlight%2Cfocus)` highlights or focuses the HEM residue (the command must be URL encoded because it contains spaces and possibly other special characters)
+    - `query` is an expression (e.g., `resn HEM` when using PyMol syntax)
+    - (optional) `lang` is one of `mol-script` (default), `pymol`, `vmd`, `jmol`
+    - (optional) `action` is an array of `highlight` (default), `focus` (multiple actions can be specified)
+    - (optional) `focus-radius` is extra distance applied when focusing the selection (default is `3`)
+    - Example: `[HEM](!query=resn%20HEM%26lang=pymol&action=highlight,focus)` highlights or focuses the HEM residue (the query must be URL encoded because it contains spaces and possibly other special characters)
+- `play-audio=src`, `toggle-audio[=src]`, `stop-audio`, `pause-audio`, `dispose-audio` - Audio playback support
 
 ## Custom Content
 
@@ -36,11 +41,11 @@ Extends Markdown Image syntax to support expressions of the form `![alt](!c1=v1&
 ### Built-in Custom Content
 - `color-swatch=color` - Renders a box with the provided color
 -  Color palettes:
-  - `color-palette-name=name` - Renders a gradient with the provided named color palette (see `mol-util/color/lists.ts` for supported color schemes)
-  - `color-palette-colors=color1,color2` - Renders a gradient with the provided colors
-  - `color-palette-width=CCS-value` - Specifies the width of the element, defaults to `150px`
-  - `color-palette-height=CCS-value` - Specified the height of the element, defaults to `0.5em`
-  - `color-palette-discrete` - Renders discrete color list instead of interpolating
+    - `color-palette-name=name` - Renders a gradient with the provided named color palette (see `mol-util/color/lists.ts` for supported color schemes)
+    - `color-palette-colors=color1,color2` - Renders a gradient with the provided colors
+    - `color-palette-width=CCS-value` - Specifies the width of the element, defaults to `150px`
+    - `color-palette-height=CCS-value` - Specified the height of the element, defaults to `0.5em`
+    - `color-palette-discrete` - Renders discrete color list instead of interpolating
 
 
 ## Example

eslint.config.mjs

@@ -39,6 +39,7 @@ export default defineConfig([{
         "comma-spacing": "off",
         "space-infix-ops": "off",
         "comma-dangle": "off",
+        quotes: ["warn", "single", { "allowTemplateLiterals": true, "avoidEscape": true }],
         eqeqeq: ["error", "smart"],
         "import/order": "off",
         "no-eval": "warn",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "molstar",
-  "version": "5.0.0-dev.6",
+  "version": "5.1.2",
   "description": "A comprehensive macromolecular library.",
   "homepage": "https://github.com/molstar/molstar#readme",
   "repository": {
@@ -11,7 +11,7 @@
     "url": "https://github.com/molstar/molstar/issues"
   },
   "engines": {
-    "node": ">=18.0.0"
+    "node": ">=20.0.0"
   },
   "scripts": {
     "lint": "eslint .",
@@ -74,7 +74,7 @@
       "js"
     ],
     "transform": {
-      "\\.ts$": "ts-jest"
+      "\\.ts$": "esbuild-jest-transform"
     },
     "moduleDirectories": [
       "node_modules",
@@ -122,58 +122,60 @@
     "Lukáš Polák <admin@lukaspolak.cz>",
     "Chetan Mishra <chetan.s115@gmail.com>",
     "Zach Charlop-Powers <zach.charlop.powers@gmail.com>",
-    "Kim Juho <juho_kim@outlook.com>"
+    "Kim Juho <juho_kim@outlook.com>",
+    "Victoria Doshchenko <doshchenko.victoria@gmail.com>"
   ],
   "license": "MIT",
   "devDependencies": {
     "@types/cors": "^2.8.19",
     "@types/gl": "^6.0.5",
-    "@types/jest": "^29.5.14",
+    "@types/jest": "^30.0.0",
     "@types/pngjs": "^6.0.5",
-    "@types/react": "^18.3.23",
+    "@types/react": "^18.3.24",
     "@types/react-dom": "^18.3.7",
-    "@typescript-eslint/eslint-plugin": "^8.34.0",
-    "@typescript-eslint/parser": "^8.34.0",
+    "@types/webxr": "^0.5.23",
+    "@typescript-eslint/eslint-plugin": "^8.44.1",
+    "@typescript-eslint/parser": "^8.44.1",
     "benchmark": "^2.1.4",
-    "concurrently": "^9.1.2",
+    "concurrently": "^9.2.1",
     "cpx2": "^8.0.0",
     "css-loader": "^7.1.2",
-    "esbuild": "^0.25.5",
+    "esbuild": "^0.25.10",
+    "esbuild-jest-transform": "^2.0.1",
     "esbuild-sass-plugin": "^3.3.1",
-    "eslint": "^9.29.0",
-    "fs-extra": "^11.3.0",
+    "eslint": "^9.36.0",
+    "fs-extra": "^11.3.2",
     "http-server": "^14.1.1",
-    "jest": "^29.7.0",
+    "jest": "^30.2.0",
     "jpeg-js": "^0.4.4",
     "react": "^18.3.1",
     "react-dom": "^18.3.1",
-    "sass": "^1.89.1",
+    "sass": "^1.93.2",
     "simple-git": "^3.28.0",
-    "ts-jest": "^29.3.4",
     "tsc-alias": "^1.8.16",
-    "typescript": "^5.8.3"
+    "typescript": "^5.9.2"
   },
   "dependencies": {
     "@types/argparse": "^2.0.17",
     "@types/benchmark": "^2.1.5",
     "@types/compression": "1.8.1",
     "@types/express": "^5.0.3",
-    "@types/node": "^18.19.111",
-    "@types/node-fetch": "^2.6.12",
-    "@types/swagger-ui-dist": "3.30.5",
+    "@types/node": "^20.19.17",
+    "@types/node-fetch": "^2.6.13",
+    "@types/swagger-ui-dist": "3.30.6",
     "argparse": "^2.0.1",
-    "compression": "^1.8.0",
+    "compression": "^1.8.1",
     "cors": "^2.8.5",
     "express": "^5.1.0",
     "h264-mp4-encoder": "^1.0.12",
-    "immer": "^10.1.1",
-    "immutable": "^5.1.2",
+    "immutable": "^5.1.3",
     "io-ts": "^2.2.22",
+    "mutative": "^1.3.0",
     "node-fetch": "^2.7.0",
     "react-markdown": "^10.1.0",
     "remark-gfm": "^4.0.1",
     "rxjs": "^7.8.2",
-    "swagger-ui-dist": "^5.24.0",
+    "swagger-ui-dist": "^5.29.0",
     "tslib": "^2.8.1",
     "util.promisify": "^1.1.3"
   },

scripts/build.mjs

@@ -131,8 +131,8 @@ function getPaths(app) {
 
 async function createBundle(app) {
     const { name, kind } = app;
-
     const { prefix, entry, outfile } = getPaths(app);
+    const NODE_ENV_PRD = isProduction || process.env.NODE_ENV === 'production';
 
     const ctx = await esbuild.context({
         entryPoints: [entry],
@@ -161,6 +161,7 @@ async function createBundle(app) {
         color: true,
         logLevel: 'info',
         define: {
+            'process.env.NODE_ENV': JSON.stringify(NODE_ENV_PRD ? 'production' : 'development'),
             'process.env.DEBUG': JSON.stringify(process.env.DEBUG || false),
             __MOLSTAR_PLUGIN_VERSION__: JSON.stringify(VERSION),
             __MOLSTAR_BUILD_TIMESTAMP__: `${TIMESTAMP}`,

scripts/deploy.js

@@ -7,14 +7,14 @@
 
 const git = require('simple-git');
 const path = require('path');
-const fs = require("fs");
-const fse = require("fs-extra");
+const fs = require('fs');
+const fse = require('fs-extra');
 const argparse = require('argparse');
 
 const VERSION = require(path.resolve(__dirname, '../package.json')).version;
 const MVS_STORIES_VERSION = require(path.resolve(__dirname, '../src/apps/mvs-stories/version.ts')).VERSION;
 
-const remoteUrl = "https://github.com/molstar/molstar.github.io.git";
+const remoteUrl = 'https://github.com/molstar/molstar.github.io.git';
 const dataDir = path.resolve(__dirname, '../data/');
 const buildDir = path.resolve(__dirname, '../build/');
 const deployDir = path.resolve(__dirname, '../deploy/');

src/apps/mesoscale-explorer/app.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2022-2024 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2022-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
@@ -147,6 +147,7 @@ export class MesoscaleExplorer {
             behaviors: [
                 PluginSpec.Behavior(PluginBehaviors.Camera.CameraAxisHelper),
                 PluginSpec.Behavior(PluginBehaviors.Camera.CameraControls),
+                PluginSpec.Behavior(PluginBehaviors.State.SnapshotControls),
 
                 PluginSpec.Behavior(MesoFocusLoci),
                 PluginSpec.Behavior(MesoSelectLoci),
@@ -252,6 +253,10 @@ export class MesoscaleExplorer {
             },
             cameraFog: { name: 'off', params: {} },
             hiZ: { enabled: true },
+            xr: {
+                disablePostprocessing: false,
+                sceneRadiusInMeters: 0.75,
+            },
         });
 
         plugin.representation.structure.registry.clear();
@@ -261,7 +266,6 @@ export class MesoscaleExplorer {
             image: true,
             componentManager: false,
             structureSelection: true,
-            behavior: true,
         });
 
         plugin.managers.lociLabels.clearProviders();

src/apps/mesoscale-explorer/behavior/camera.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2023-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
@@ -21,6 +21,7 @@ const Key = Binding.TriggerKey;
 const DefaultMesoFocusLociBindings = {
     clickCenter: Binding([
         Trigger(B.Flag.Primary, M.create()),
+        Trigger(B.Flag.Trigger),
     ], 'Camera center', 'Click element using ${triggers}'),
     clickCenterFocus: Binding([
         Trigger(B.Flag.Secondary, M.create()),

src/apps/mesoscale-explorer/behavior/select.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2023-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
@@ -24,7 +24,8 @@ const Trigger = Binding.Trigger;
 
 const DefaultMesoSelectLociBindings = {
     click: Binding([
-        Trigger(B.Flag.Primary, M.create())
+        Trigger(B.Flag.Primary, M.create()),
+        Trigger(B.Flag.Trigger),
     ], 'Click', 'Click element using ${triggers}'),
     clickToggleSelect: Binding([
         Trigger(B.Flag.Primary, M.create({ shift: true })),

src/apps/mesoscale-explorer/ui/panels.tsx

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2022-2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2022-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
@@ -18,12 +18,14 @@ import { ParamDefinition as PD } from '../../../mol-util/param-definition';
 import { TuneSvg } from '../../../mol-plugin-ui/controls/icons';
 import { RendererParams } from '../../../mol-gl/renderer';
 import { TrackballControlsParams } from '../../../mol-canvas3d/controls/trackball';
+import { XRManagerParams } from '../../../mol-canvas3d/helper/xr-manager';
 
 const Spacer = () => <div style={{ height: '2em' }} />;
 
 const ViewportParams = {
     renderer: PD.Group(RendererParams),
     trackball: PD.Group(TrackballControlsParams),
+    xr: PD.Group(XRManagerParams, { label: 'XR' }),
 };
 
 class ViewportSettingsUI extends CollapsableControls<{}, {}> {

src/apps/mvs-stories/context.ts

@@ -9,14 +9,14 @@ import { MVSData } from '../../extensions/mvs/mvs-data';
 import type { MVSStoriesViewerModel } from './elements/viewer';
 
 export type MVSStoriesCommand =
-    | { kind: 'load-mvs', format?: 'mvsj' | 'mvsx', url?: string, data?: MVSData | string | Uint8Array }
+    | { kind: 'load-mvs', format?: 'mvsj' | 'mvsx', url?: string, data?: MVSData | string | Uint8Array<ArrayBuffer> }
 
 
 export class MVSStoriesContext {
     commands = new BehaviorSubject<MVSStoriesCommand | undefined>(undefined);
     state = {
         viewers: new BehaviorSubject<{ name?: string, model: MVSStoriesViewerModel }[]>([]),
-        currentStoryData: new BehaviorSubject<string | Uint8Array | undefined>(undefined),
+        currentStoryData: new BehaviorSubject<string | Uint8Array<ArrayBuffer> | undefined>(undefined),
         isLoading: new BehaviorSubject(false),
     };
 

src/apps/mvs-stories/elements/viewer.tsx

@@ -66,7 +66,7 @@ export class MVSStoriesViewerModel extends PluginComponent {
                         loadedData = await loadMVSData(this.plugin, cmd.data, cmd.format ?? 'mvsj');
                     }
                     if (StringLike.is(loadedData) || loadedData instanceof Uint8Array) {
-                        this.context.state.currentStoryData.next(loadedData as string | Uint8Array);
+                        this.context.state.currentStoryData.next(loadedData as string | Uint8Array<ArrayBuffer>);
                     } else if (loadedData) {
                         this.context.state.currentStoryData.next(JSON.stringify(loadedData));
                     }

src/apps/mvs-stories/index.html

@@ -53,6 +53,10 @@
             text-decoration: none;
         }
 
+        #links .sep {
+            color: #aaa;
+        }
+
         @media (orientation:portrait) {
             #viewer {
                 position: absolute;
@@ -90,13 +94,16 @@
     </div>
 
     <div id="links">
-        <a href="#" id="mvs-data">Download MVS State</a> | <a href="https://github.com/molstar/molstar/tree/master/src/apps/mvs-stories" id="mvs-data" target="_blank" rel="noopener noreferrer">Source Code</a>
+        <span id="open-in-stories"><a href="#" id="open-in-stories-link" target="_blank" rel="noopener noreferrer" title="Open and edit the story in the MolViewStories app">Edit in MolViewStories</a>&nbsp;<span class="sep">•</span></span>
+        <span id="open-in-molstar"><a href="#" id="open-in-molstar-link" target="_blank" rel="noopener noreferrer" title="Open the story in the Mol* Viewer app. Enables exporting an animation.">Open in Mol* Viewer</a>&nbsp;<span class="sep">•</span></span>
+        <a href="#" id="mvs-data" title="MolViewSpec State for this story. Can be opened in the Mol* app.">Download MVS</a> <span class="sep">•</span> <a href="https://github.com/molstar/molstar/tree/master/src/apps/mvs-stories" id="mvs-data" target="_blank" rel="noopener noreferrer">Source Code</a>
     </div>
 
     <script>
         var urlParams = new URLSearchParams(window.location.search);
         var storyId = urlParams.get('story-id');
         var storyUrl = urlParams.get('story-url');
+        var storySessionUrl = urlParams.get('story-session-url');
         var format = urlParams.get('data-format');
 
         // For testing purposes:
@@ -104,12 +111,32 @@
         //     storyUrl = 'https://raw.githubusercontent.com/molstar/molstar/master/examples/mvs/kinase-story.mvsj';
         // }
 
+        var molstarDataLink = storyUrl;
+        var editInStoriesUrl = undefined;
+
         if (storyId) {
             mvsStories.loadFromID(storyId, { format: format || 'mvsj', contextName: 'story1' });
+            editInStoriesUrl = 'https://molstar.org/mol-view-stories/builder?published-session-id=' + storyId;
+            molstarDataLink = 'https://stories.molstar.org/api/story/' + storyId + '/data';
         } else if (storyUrl) {
             mvsStories.loadFromURL(storyUrl, { format: format || 'mvsj', contextName: 'story1' });
         }
 
+        if (!editInStoriesUrl && storySessionUrl) {
+            editInStoriesUrl = 'https://molstar.org/mol-view-stories/builder?session-url=' + encodeURIComponent(storySessionUrl);
+        }
+        
+        if (molstarDataLink) {
+            var molstarLink = 'https://molstar.org/viewer?mvs-url=' + encodeURIComponent(molstarDataLink) + '&mvs-format=' + encodeURIComponent(format || 'mvsj'); 
+            document.getElementById('open-in-molstar-link').setAttribute('href', molstarLink);
+            document.getElementById('open-in-molstar').style.display = 'inline';
+        }
+        
+        if (editInStoriesUrl) {
+            document.getElementById('open-in-stories-link').setAttribute('href', editInStoriesUrl);
+            document.getElementById('open-in-stories').style.display = 'inline';
+        }
+
         document.getElementById('mvs-data').addEventListener('click', (e) => {
             e.preventDefault();
             mvsStories.downloadCurrentStory({ contextName: 'story1' });

src/apps/mvs-stories/index.tsx

@@ -28,7 +28,7 @@ export function loadFromURL(url: string, options?: { format: 'mvsx' | 'mvsj', co
     }, 0);
 }
 
-export function loadFromData(data: MVSData | string | Uint8Array, options?: { format: 'mvsx' | 'mvsj', contextName?: string }) {
+export function loadFromData(data: MVSData | string | Uint8Array<ArrayBuffer>, options?: { format: 'mvsx' | 'mvsj', contextName?: string }) {
     setTimeout(() => {
         getContext(options?.contextName).dispatch({
             kind: 'load-mvs',

src/apps/mvs-stories/styles.scss

@@ -182,6 +182,15 @@
         max-width: 100%;
         height: auto;
     }
+
+    a {
+        text-decoration: none;
+        color: #1d4ed7;
+
+        &:hover {
+            text-decoration: underline;
+        }
+    }
 }
 
 @media (orientation:portrait) {

src/apps/viewer/app.ts

@@ -109,8 +109,11 @@ const DefaultViewerOptions = {
     resolutionMode: PluginConfig.General.ResolutionMode.defaultValue,
     illumination: false,
 
-    viewportShowExpand: PluginConfig.Viewport.ShowExpand.defaultValue,
+    viewportShowReset: PluginConfig.Viewport.ShowReset.defaultValue,
+    viewportShowScreenshotControls: PluginConfig.Viewport.ShowScreenshotControls.defaultValue,
     viewportShowControls: PluginConfig.Viewport.ShowControls.defaultValue,
+    viewportShowExpand: PluginConfig.Viewport.ShowExpand.defaultValue,
+    viewportShowToggleFullscreen: PluginConfig.Viewport.ShowToggleFullscreen.defaultValue,
     viewportShowSettings: PluginConfig.Viewport.ShowSettings.defaultValue,
     viewportShowSelectionMode: PluginConfig.Viewport.ShowSelectionMode.defaultValue,
     viewportShowAnimation: PluginConfig.Viewport.ShowAnimation.defaultValue,
@@ -187,8 +190,11 @@ export class Viewer {
                 [PluginConfig.General.AllowMajorPerformanceCaveat, o.allowMajorPerformanceCaveat],
                 [PluginConfig.General.PowerPreference, o.powerPreference],
                 [PluginConfig.General.ResolutionMode, o.resolutionMode],
-                [PluginConfig.Viewport.ShowExpand, o.viewportShowExpand],
+                [PluginConfig.Viewport.ShowReset, o.viewportShowReset],
+                [PluginConfig.Viewport.ShowScreenshotControls, o.viewportShowScreenshotControls],
                 [PluginConfig.Viewport.ShowControls, o.viewportShowControls],
+                [PluginConfig.Viewport.ShowExpand, o.viewportShowExpand],
+                [PluginConfig.Viewport.ShowToggleFullscreen, o.viewportShowToggleFullscreen],
                 [PluginConfig.Viewport.ShowSettings, o.viewportShowSettings],
                 [PluginConfig.Viewport.ShowSelectionMode, o.viewportShowSelectionMode],
                 [PluginConfig.Viewport.ShowAnimation, o.viewportShowAnimation],
@@ -536,7 +542,7 @@ export class Viewer {
     /** Load MolViewSpec from `data`.
      * If `format` is 'mvsj', `data` must be a string or a Uint8Array containing a UTF8-encoded string.
      * If `format` is 'mvsx', `data` must be a Uint8Array or a string containing base64-encoded binary data prefixed with 'base64,'. */
-    loadMvsData(data: string | Uint8Array, format: 'mvsj' | 'mvsx', options?: { appendSnapshots?: boolean, keepCamera?: boolean, extensions?: MolstarLoadingExtension<any>[] }) {
+    loadMvsData(data: string | Uint8Array<ArrayBuffer>, format: 'mvsj' | 'mvsx', options?: { appendSnapshots?: boolean, keepCamera?: boolean, extensions?: MolstarLoadingExtension<any>[] }) {
         return loadMVSData(this.plugin, data, format, options);
     }
 
@@ -580,12 +586,12 @@ export interface VolumeIsovalueInfo {
 
 export interface LoadTrajectoryParams {
     model: { kind: 'model-url', url: string, format?: BuiltInTrajectoryFormat /* mmcif */, isBinary?: boolean }
-    | { kind: 'model-data', data: string | number[] | ArrayBuffer | Uint8Array, format?: BuiltInTrajectoryFormat /* mmcif */ }
+    | { kind: 'model-data', data: string | number[] | ArrayBuffer | Uint8Array<ArrayBuffer>, format?: BuiltInTrajectoryFormat /* mmcif */ }
     | { kind: 'topology-url', url: string, format: BuiltInTopologyFormat, isBinary?: boolean }
-    | { kind: 'topology-data', data: string | number[] | ArrayBuffer | Uint8Array, format: BuiltInTopologyFormat },
+    | { kind: 'topology-data', data: string | number[] | ArrayBuffer | Uint8Array<ArrayBuffer>, format: BuiltInTopologyFormat },
     modelLabel?: string,
     coordinates: { kind: 'coordinates-url', url: string, format: BuiltInCoordinatesFormat, isBinary?: boolean }
-    | { kind: 'coordinates-data', data: string | number[] | ArrayBuffer | Uint8Array, format: BuiltInCoordinatesFormat },
+    | { kind: 'coordinates-data', data: string | number[] | ArrayBuffer | Uint8Array<ArrayBuffer>, format: BuiltInCoordinatesFormat },
     coordinatesLabel?: string,
     preset?: keyof PresetTrajectoryHierarchy
 }

src/apps/viewer/index.html

@@ -28,10 +28,10 @@
             }
             #app {
                 position: absolute;
-                left: 100px;
-                top: 100px;
-                width: 800px;
-                height: 600px;
+                left: 0;
+                top: 0;
+                right: 0;
+                bottom: 0;
             }
         </style>
         <link rel="stylesheet" type="text/css" href="molstar.css" />
@@ -66,6 +66,7 @@
             var powerPreference = getParam('power-preference', '[^&]+').trim().toLowerCase();
             var illumination = getParam('illumination', '[^&]+').trim() === '1';
             var resolutionMode = getParam('resolution-mode', '[^&]+').trim().toLowerCase();
+            var viewportShowToggleFullscreen = getParam('show-toggle-fullscreen', '[^&]+').trim() === '1';
 
             // console.log('Available extensions: ', Object.keys(molstar.ExtensionMap));
 
@@ -73,6 +74,7 @@
                 disabledExtensions: [], // anything from Object.keys(molstar.ExtensionMap)
                 layoutShowControls: !hideControls,
                 viewportShowExpand: false,
+                viewportShowToggleFullscreen: viewportShowToggleFullscreen,
                 collapseLeftPanel: collapseLeftPanel,
                 pdbProvider: pdbProvider || 'pdbe',
                 emdbProvider: emdbProvider || 'pdbe',
@@ -87,7 +89,7 @@
                 allowMajorPerformanceCaveat: allowMajorPerformanceCaveat,
                 powerPreference: powerPreference || 'high-performance',
                 illumination: illumination,
-                resolutionMode: resolutionMode || 'auto'
+                resolutionMode: resolutionMode || 'auto',
             }).then(viewer => {
                 var snapshotId = getParam('snapshot-id', '[^&]+').trim();
                 if (snapshotId) viewer.setRemoteSnapshot(snapshotId);

src/cli/cif2bcif/converter.ts

@@ -29,7 +29,7 @@ async function readFile(ctx: RuntimeContext, filename: string): Promise<ReaderRe
     const isGz = /\.gz$/i.test(filename);
     if (filename.match(/\.bcif/)) {
         let input = await readFileAsync(filename);
-        if (isGz) input = await unzipAsync(input);
+        if (isGz) input = await unzipAsync(input) as NonSharedBuffer;
         return await CIF.parseBinary(new Uint8Array(input)).runInContext(ctx);
     } else {
         const data = isGz ? await unzipAsync(await readFileAsync(filename)) : await readFileAsync(filename);

src/cli/mvs/mvs-print-schema.ts

@@ -11,7 +11,7 @@
  */
 
 import { ArgumentParser } from 'argparse';
-import { treeSchemaToMarkdown, treeSchemaToString } from '../../extensions/mvs/tree/generic/tree-schema';
+import { treeSchemaToMarkdown, treeSchemaToString } from '../../extensions/mvs/tree/generic/tree-validation';
 import { MVSTreeSchema } from '../../extensions/mvs/tree/mvs/mvs-tree';
 
 

src/examples/mvs-stories/stories/animation.ts

@@ -22,7 +22,15 @@ const Steps = [
         header: 'Animation Demo',
         key: 'intro',
         description: `### Molecular Animation
-A story showcasing MolViewSpec animation capabilities.`,
+A story showcasing MolViewSpec animation capabilities.
+
+[\[**🔄 Replay Intro**\]](!play-transition)
+[\[**⏵ Play Snapshots**\]](!play-snapshots)
+[\[**⏹ Stop Animation**\]](!stop-animation)
+
+[\[**➡️ Next Snapshot**\]](!next-snapshot)
+
+`,
         linger_duration_ms: 2000,
         transition_duration_ms: 500,
         state: (): Root => {
@@ -37,16 +45,35 @@ A story showcasing MolViewSpec animation capabilities.`,
             });
             prims.label({ text: 'Animation Demo', position: { label_asym_id: 'A' }, label_size: 10 });
 
-            const anim = builder.animation();
+            const anim = builder.animation({
+                custom: {
+                    molstar_trackball: {
+                        name: 'rock',
+                        params: { speed: 0.5 },
+                    }
+                }
+            });
             anim.interpolate({
                 kind: 'scalar',
                 ref: 'prims-opacity',
                 target_ref: 'prims',
-                duration_ms: 1000,
+                start_ms: 500,
+                duration_ms: 500,
                 property: 'label_opacity',
                 end: 1,
             });
 
+            anim.interpolate({
+                kind: 'scalar',
+                ref: 'prims-opacity',
+                target_ref: 'prims',
+                start_ms: 1500,
+                duration_ms: 500,
+                property: 'label_opacity',
+                start: 1,
+                end: 0.66,
+            });
+
 
             // Uncomment this to make 2nd frame render much faster
             // It will cause shader compilation to happen during the 1st snapshot
@@ -84,12 +111,28 @@ A story showcasing MolViewSpec animation capabilities.`,
             const builder = createMVSBuilder();
 
             const _1cbs = structure(builder, '1cbs');
-            const [poly,] = polymer(_1cbs, { color: Colors['1cbs'] });
+            const [poly, repr] = polymer(_1cbs, { color: Colors['1cbs'] });
+
+            repr.colorFromSource({
+                ref: 'residue_colors',
+                schema: 'residue',
+                category_name: 'atom_site',
+                field_name: 'label_comp_id',
+                palette: {
+                    kind: 'categorical',
+                    missing_color: 'white',
+                    colors: {
+                        ALA: 'red',
+                        ILE: 'white',
+                        LYS: 'white',
+                    }
+                }
+            });
 
             const surface = poly.representation({
                 type: 'surface',
                 surface_type: 'gaussian',
-            });
+            }).opacity({ opacity: 0.33 });
 
             _1cbs.component({ selector: 'ligand' })
                 .transform({
@@ -112,14 +155,32 @@ A story showcasing MolViewSpec animation capabilities.`,
 
             anim.interpolate({
                 kind: 'scalar',
-                ref: 'clip-transition',
                 target_ref: 'clip',
-                duration_ms: 2000,
+                duration_ms: 500,
                 property: ['point', 2],
                 end: 55,
                 easing: 'sin-in',
             });
 
+            anim.interpolate({
+                kind: 'scalar',
+                target_ref: 'clip',
+                start_ms: 600,
+                duration_ms: 800,
+                property: ['point', 2],
+                end: 0,
+                easing: 'sin-out',
+            });
+
+            anim.interpolate({
+                kind: 'scalar',
+                target_ref: 'clip',
+                start_ms: 1500,
+                duration_ms: 500,
+                property: ['point', 2],
+                end: 55,
+            });
+
             anim.interpolate({
                 kind: 'vec3',
                 target_ref: 'xform',
@@ -145,6 +206,20 @@ A story showcasing MolViewSpec animation capabilities.`,
                 end: Colors['ligand-docked'],
             });
 
+            anim.interpolate({
+                kind: 'color',
+                target_ref: 'residue_colors',
+                duration_ms: 2000,
+                property: ['palette', 'colors'],
+                start: {
+                    ALA: 'yellow',
+                },
+                end: {
+                    ILE: 'blue',
+                    LYS: 'purple',
+                },
+            });
+
             return builder;
         },
         camera: {
@@ -175,7 +250,7 @@ A story showcasing MolViewSpec animation capabilities.`,
                     ref: 'repr',
                     type: 'ball_and_stick',
                     custom: {
-                        molstar_reprepresentation_params: {
+                        molstar_representation_params: {
                             emissive: 0,
                         }
                     }
@@ -202,7 +277,7 @@ A story showcasing MolViewSpec animation capabilities.`,
                 kind: 'scalar',
                 target_ref: 'repr',
                 duration_ms: 1000,
-                property: ['custom', 'molstar_reprepresentation_params', 'emissive'],
+                property: ['custom', 'molstar_representation_params', 'emissive'],
                 end: 0.2,
             });
 
@@ -266,10 +341,12 @@ function structure(builder: Root, id: string): MVSStructure {
         .modelStructure();
 }
 
-function polymer(structure: MVSStructure, options: { color: ColorT }) {
+function polymer(structure: MVSStructure, options?: { color?: ColorT }) {
     const component = structure.component({ selector: { label_asym_id: 'A' } });
     const reprensentation = component.representation({ type: 'cartoon' });
-    reprensentation.color({ color: options.color });
+    if (options?.color) {
+        reprensentation.color({ color: options.color });
+    }
     return [component, reprensentation] as const;
 }
 
@@ -287,6 +364,21 @@ export function buildStory(): MVSData_States {
                 molstar_postprocessing: {
                     enable_outline: true,
                     enable_ssao: true,
+                    background: {
+                        name: 'horizontalGradient',
+                        params: {
+                            topColor: 0x777777,
+                            bottomColor: 0xffffff,
+                        }
+                    },
+                    // Example with background image:
+                    // background: {
+                    //     name: 'image',
+                    //     params: {
+                    //         // URL can also be filename in MVSX archive
+                    //         source: { name: 'url', params: 'URL' }
+                    //     }
+                    // }
                 }
             }
         });

src/examples/mvs-stories/stories/index.ts

@@ -7,9 +7,13 @@
 import { buildStory as kinase } from './kinase';
 import { buildStory as tbp } from './tbp';
 import { buildStory as animation } from './animation';
+import { buildStory as audio } from './audio';
+import { buildStory as motm1 } from './motm1';
 
 export const Stories = [
     { id: 'kinase', name: 'BCR-ABL: A Kinase Out of Control', buildStory: kinase },
     { id: 'tata', name: 'TATA-Binding Protein and its Role in Transcription Initiation ', buildStory: tbp },
-    { id: 'animation', name: 'Molecular Animation', buildStory: animation },
+    { id: 'motm1', name: 'RCSB PDB Molecule of the Month #1', buildStory: motm1 },
+    { id: 'animation-example', name: 'Molecular Animation Example', buildStory: animation },
+    { id: 'audio-example', name: 'Audio Playback Example', buildStory: audio },
 ] as const;

src/extensions/geo-export/glb-exporter.ts

@@ -56,7 +56,7 @@ export class GlbExporter extends MeshExporter<GlbData> {
     private materialMap = new Map<string, number>();
     private accessors: Record<string, any>[] = [];
     private bufferViews: Record<string, any>[] = [];
-    private binaryBuffer: ArrayBuffer[] = [];
+    private binaryBuffer: ArrayBufferLike[] = [];
     private byteOffset = 0;
     private centerTransform: Mat4;
 
@@ -72,7 +72,7 @@ export class GlbExporter extends MeshExporter<GlbData> {
         return [min, max];
     }
 
-    private addBuffer(buffer: ArrayBuffer, componentType: number, type: string, count: number, target: number, min?: any, max?: any, normalized?: boolean) {
+    private addBuffer(buffer: ArrayBufferLike, componentType: number, type: string, count: number, target: number, min?: any, max?: any, normalized?: boolean) {
         this.binaryBuffer.push(buffer);
 
         const bufferViewOffset = this.bufferViews.length;
@@ -304,7 +304,7 @@ export class GlbExporter extends MeshExporter<GlbData> {
             materials: this.materials
         };
 
-        const createChunk = (chunkType: number, data: ArrayBuffer[], byteLength: number, padChar: number): [ArrayBuffer[], number] => {
+        const createChunk = (chunkType: number, data: ArrayBufferLike[], byteLength: number, padChar: number): [ArrayBufferLike[], number] => {
             let padding = null;
             if (byteLength % 4 !== 0) {
                 const pad = 4 - (byteLength % 4);

src/extensions/model-archive/quality-assessment/pairwise/ui.tsx

@@ -305,7 +305,7 @@ function PlotInteractivity({ drawing, interactity }: { drawing: MAPairwiseMetric
     let labelNode: ReactNode | undefined;
     if (label) {
         const labelStyle: CSSProperties | undefined = label ? { fontSize: '45px', fill: 'black', fontWeight: 'bold', pointerEvents: 'none', userSelect: 'none' } : undefined;
-        let x: number, y: number, anchor: string;
+        let x: number, y: number, anchor: 'start' | 'end';
         if (crosshairOffset![0] < PlotSize / 2) {
             x = PlotOffset + crosshairOffset![0] + 20;
             anchor = 'start';

src/extensions/model-export/export.ts

@@ -39,7 +39,7 @@ function _exportHierarchy(plugin: PluginContext, options?: { format?: 'cif' | 'b
         const format = options?.format ?? 'cif';
         const { structures } = plugin.managers.structure.hierarchy.current;
 
-        const files: [name: string, data: string | Uint8Array][] = [];
+        const files: [name: string, data: string | Uint8Array<ArrayBuffer>][] = [];
         const entryMap = new Map<string, number>();
 
         for (const _s of structures) {
@@ -80,7 +80,7 @@ function _exportHierarchy(plugin: PluginContext, options?: { format?: 'cif' | 'b
         if (files.length === 1) {
             download(new Blob([files[0][1]]), files[0][0]);
         } else if (files.length > 1) {
-            const zipData: Record<string, Uint8Array> = {};
+            const zipData: Record<string, Uint8Array<ArrayBuffer>> = {};
             for (const [fn, data] of files) {
                 if (data instanceof Uint8Array) {
                     zipData[fn] = data;

src/extensions/mp4-export/encoder.ts

@@ -25,7 +25,7 @@ export interface Mp4EncoderParams<A extends PluginStateAnimation = PluginStateAn
     quantizationParameter?: number
 }
 
-export async function encodeMp4Animation<A extends PluginStateAnimation>(plugin: PluginContext, ctx: RuntimeContext, params: Mp4EncoderParams<A>) {
+export async function encodeMp4Animation<A extends PluginStateAnimation>(plugin: PluginContext, ctx: RuntimeContext, params: Mp4EncoderParams<A>): Promise<Uint8Array<ArrayBuffer>> {
     await ctx.update({ message: 'Initializing...', isIndeterminate: true });
 
     validateViewport(params);
@@ -88,7 +88,7 @@ export async function encodeMp4Animation<A extends PluginStateAnimation>(plugin:
         stoppedAnimation = true;
         encoder.finalize();
         finalized = true;
-        return encoder.FS.readFile(encoder.outputFilename);
+        return encoder.FS.readFile(encoder.outputFilename) as Uint8Array<ArrayBuffer>;
     } finally {
         if (finalized) encoder.delete();
         if (params.customBackground !== void 0) {

src/extensions/mp4-export/ui.tsx

@@ -15,7 +15,7 @@ import { Mp4AnimationParams, Mp4Controls } from './controls';
 
 interface State {
     busy?: boolean,
-    data?: { movie: Uint8Array, filename: string };
+    data?: { movie: Uint8Array<ArrayBuffer>, filename: string };
 }
 
 export class Mp4EncoderUI extends CollapsableControls<{}, State> {

src/extensions/mvs/README.md

@@ -1 +1 @@
-Find the MVS extension documentation [here](../../../docs/extensions/mvs/README.md).
+Please refer to the standalone documentation [here](https://molstar.org/mol-view-spec-docs/).

src/extensions/mvs/behavior.ts

@@ -12,7 +12,7 @@ import { LociLabelProvider } from '../../mol-plugin-state/manager/loci-label';
 import { PluginBehavior } from '../../mol-plugin/behavior/behavior';
 import { PluginContext } from '../../mol-plugin/context';
 import { StructureRepresentationProvider } from '../../mol-repr/structure/representation';
-import { StateAction, StateObjectCell, StateTree } from '../../mol-state';
+import { StateAction, StateObject, StateObjectCell, StateTree } from '../../mol-state';
 import { Task } from '../../mol-task';
 import { ColorTheme } from '../../mol-theme/color';
 import { fileToDataUri } from '../../mol-util/file';
@@ -111,6 +111,19 @@ export const MolViewSpec = PluginBehavior.create<{ autoAttach: boolean }>({
                 this.ctx.state.data.actions.add(action);
             }
 
+            this.ctx.state.data.registerRefResolver('mvs', (state, ref) => {
+                const tagSearch = StateTree.doPreOrder(state.tree, state.tree.root, { ref, ret: undefined as StateObject | undefined }, (n, _, s) => {
+                    if (!n.tags) return;
+                    for (const t of n.tags) {
+                        if (t.startsWith('mvs-ref:') && t.substring(8) === ref) {
+                            s.ret = state.cells.get(n.ref)?.obj?.data;
+                            return false;
+                        }
+                    }
+                });
+                return tagSearch.ret;
+            });
+
             this.ctx.managers.markdownExtensions.registerRefResolver('mvs', (plugin, refs) => {
                 const mvsRefs = new Set(refs.map(ref => `mvs-ref:${ref}`));
                 return StateTree.doPreOrder(
@@ -180,6 +193,7 @@ export const MolViewSpec = PluginBehavior.create<{ autoAttach: boolean }>({
             for (const action of this.registrables.actions ?? []) {
                 this.ctx.state.data.actions.remove(action);
             }
+            this.ctx.state.data.removeRefResolver('mvs');
             this.ctx.managers.markdownExtensions.removeRefResolver('mvs');
         }
     },

src/extensions/mvs/camera.ts

@@ -7,6 +7,8 @@
 
 import { Camera } from '../../mol-canvas3d/camera';
 import { CameraFogParams, Canvas3DParams, Canvas3DProps, DefaultCanvas3DParams } from '../../mol-canvas3d/canvas3d';
+import { TrackballControlsParams } from '../../mol-canvas3d/controls/trackball';
+import { BackgroundParams } from '../../mol-canvas3d/passes/background';
 import { BloomParams } from '../../mol-canvas3d/passes/bloom';
 import { DofParams } from '../../mol-canvas3d/passes/dof';
 import { OutlineParams } from '../../mol-canvas3d/passes/outline';
@@ -20,10 +22,12 @@ import { PluginState } from '../../mol-plugin/state';
 import { StateObjectSelector } from '../../mol-state';
 import { fovAdjustedPosition } from '../../mol-util/camera';
 import { ColorNames } from '../../mol-util/color/names';
+import { deepClone } from '../../mol-util/object';
 import { ParamDefinition } from '../../mol-util/param-definition';
 import { decodeColor } from './helpers/utils';
 import { MolstarLoadingContext } from './load';
 import { SnapshotMetadata } from './mvs-data';
+import { MVSAnimationNode } from './tree/animation/animation-tree';
 import { MolstarNode, MolstarNodeParams } from './tree/molstar/molstar-tree';
 import { MVSTreeSchema } from './tree/mvs/mvs-tree';
 
@@ -121,22 +125,22 @@ export function createPluginStateSnapshotCamera(plugin: PluginContext, context:
     return camera;
 }
 
-/** Set canvas properties based on a canvas node. */
-export function setCanvas(plugin: PluginContext, node: MolstarNode<'canvas'> | undefined) {
-    plugin.canvas3d?.setProps(old => modifyCanvasProps(old, node));
-}
-
 function optionalParams(enable: boolean | undefined, values: any, params: ParamDefinition.Params, fallback: any) {
     if (typeof enable === 'boolean') {
         return enable
             ? { name: 'on', params: { ...ParamDefinition.getDefaultValues(params), ...values } }
-            : { name: 'off', params: { } };
+            : { name: 'off', params: {} };
     }
     return fallback;
 }
 
+function normalizeBackground(variant: any, prev: any): any {
+    if (!variant) return prev;
+    return ParamDefinition.normalizeParams(BackgroundParams, { variant }, 'children');
+}
+
 /** Create a deep copy of `oldCanvasProps` with values modified according to a canvas node params. */
-export function modifyCanvasProps(oldCanvasProps: Canvas3DProps, canvasNode: MolstarNode<'canvas'> | undefined, custom?: Record<string, any>): Canvas3DProps {
+export function modifyCanvasProps(oldCanvasProps: Canvas3DProps, canvasNode: MolstarNode<'canvas'> | undefined, animationNode: MVSAnimationNode<'animation'> | undefined): Canvas3DProps {
     const params = canvasNode?.params;
     const backgroundColor = decodeColor(params?.background_color) ?? DefaultCanvasBackgroundColor;
 
@@ -160,6 +164,12 @@ export function modifyCanvasProps(oldCanvasProps: Canvas3DProps, canvasNode: Mol
     const bloom = molstar_postprocessing?.enable_bloom;
     const bloomParams = molstar_postprocessing?.bloom_params;
 
+    const background = molstar_postprocessing?.background;
+
+    const trackballAnimation = animationNode?.custom?.molstar_trackball;
+    const trackballAnimationName = trackballAnimation?.name;
+    const trackballAnimationParams = trackballAnimation?.params ?? {};
+
     return {
         ...oldCanvasProps,
         postprocessing: {
@@ -169,12 +179,28 @@ export function modifyCanvasProps(oldCanvasProps: Canvas3DProps, canvasNode: Mol
             occlusion: optionalParams(occlusion, occlusionParams, SsaoParams, oldCanvasProps.postprocessing.occlusion),
             dof: optionalParams(dof, dofParams, DofParams, oldCanvasProps.postprocessing.dof),
             bloom: optionalParams(bloom, bloomParams, BloomParams, oldCanvasProps.postprocessing.bloom),
+            background: normalizeBackground(background, oldCanvasProps.postprocessing.background),
         },
         cameraFog: optionalParams(fog, fogParams, CameraFogParams, oldCanvasProps.cameraFog),
         renderer: {
             ...oldCanvasProps.renderer,
             backgroundColor: backgroundColor,
         },
+        trackball: {
+            ...oldCanvasProps?.trackball,
+            ...(trackballAnimationName
+                ? {
+                    animate: {
+                        name: trackballAnimationName,
+                        params: {
+                            ...TrackballControlsParams.animate.map(trackballAnimationName)?.defaultValue,
+                            ...trackballAnimationParams
+                        }
+                    }
+                }
+                : {}
+            ),
+        }
     };
 }
 
@@ -184,12 +210,17 @@ export function resetCanvasProps(plugin: PluginContext) {
         ...old,
         postprocessing: {
             ...old,
-            outline: DefaultCanvas3DParams.postprocessing.outline,
-            shadow: DefaultCanvas3DParams.postprocessing.shadow,
-            occlusion: DefaultCanvas3DParams.postprocessing.occlusion,
-            dof: DefaultCanvas3DParams.postprocessing.dof,
-            bloom: DefaultCanvas3DParams.postprocessing.bloom,
+            outline: deepClone(DefaultCanvas3DParams.postprocessing.outline),
+            shadow: deepClone(DefaultCanvas3DParams.postprocessing.shadow),
+            occlusion: deepClone(DefaultCanvas3DParams.postprocessing.occlusion),
+            dof: deepClone(DefaultCanvas3DParams.postprocessing.dof),
+            bloom: deepClone(DefaultCanvas3DParams.postprocessing.bloom),
+            background: deepClone(DefaultCanvas3DParams.postprocessing.background),
         },
-        cameraFog: DefaultCanvas3DParams.cameraFog,
+        cameraFog: deepClone(DefaultCanvas3DParams.cameraFog),
+        trackball: {
+            ...old?.trackball,
+            animate: { name: 'off', params: {} },
+        }
     });
 }

src/extensions/mvs/components/annotation-label/visual.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2023-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Adam Midlik <midlik@gmail.com>
  */
@@ -57,7 +57,7 @@ function createLabelText(ctx: VisualContext, structure: Structure, theme: Theme,
         const labelText = annotation!.getValueForRow(iFirstRowInGroup, props.fieldName);
         if (!labelText) continue;
         const rowsInGroup = grouped.slice(offsets[iGroup], offsets[iGroup + 1]).map(j => rows[j]);
-        const p = textPropsForSelection(structure, theme.size.size, rowsInGroup, model);
+        const p = textPropsForSelection(structure, rowsInGroup, model);
         if (!p) continue;
         builder.add(labelText, p.center[0], p.center[1], p.center[2], p.depth, p.scale, p.group);
     }

src/extensions/mvs/components/annotation-prop.ts

@@ -12,17 +12,17 @@ import { CustomModelProperty } from '../../../mol-model-props/common/custom-mode
 import { CustomProperty } from '../../../mol-model-props/common/custom-property';
 import { CustomPropertyDescriptor } from '../../../mol-model/custom-property';
 import { Model } from '../../../mol-model/structure';
-import { Structure, StructureElement } from '../../../mol-model/structure/structure';
+import { Structure, StructureElement, Unit } from '../../../mol-model/structure/structure';
 import { Asset } from '../../../mol-util/assets';
 import { Jsonable, canonicalJsonString } from '../../../mol-util/json';
 import { objectOfArraysToArrayOfObjects, pickObjectKeysWithRemapping, promiseAllObj } from '../../../mol-util/object';
 import { Choice } from '../../../mol-util/param-choice';
 import { ParamDefinition as PD } from '../../../mol-util/param-definition';
-import { AtomRanges } from '../helpers/atom-ranges';
+import { ElementRanges } from '../helpers/element-ranges';
 import { IndicesAndSortings } from '../helpers/indexing';
 import { MaybeStringParamDefinition } from '../helpers/param-definition';
 import { MVSAnnotationRow, MVSAnnotationSchema, getCifAnnotationSchema } from '../helpers/schemas';
-import { getAtomRangesForRow } from '../helpers/selections';
+import { getAtomRangesForRow, getGaussianRangesForRow, getSphereRangesForRow } from '../helpers/selections';
 import { Maybe, isDefined, safePromise } from '../helpers/utils';
 
 
@@ -141,10 +141,25 @@ export function getMVSAnnotationForStructure(structure: Structure, annotationId:
 
 type FieldRemapping = Record<string, string | null>;
 
-/** Mapping `ElementIndex` -> annotation row index for all elements in a `Model`.
+/** Mapping `ElementIndex` -> annotation row index for all elements of one kind (atoms, spheres, gaussians) in a `Model`.
  * `-1` means no row applies to the element.
  * `null` means no row applies to any element. */
-type IndexedModel = number[] | null;
+type IndexedElements = number[] | null;
+
+/** Mapping `ElementIndex` -> annotation row index for atoms, spheres, and gaussians in a `Model`. */
+type IndexedModel = {
+    atoms: IndexedElements,
+    spheres: IndexedElements,
+    gaussians: IndexedElements,
+};
+
+function getIndexedElementsForUnitKind(indexedModel: IndexedModel, unitKind: Unit.Kind): IndexedElements {
+    if (unitKind === Unit.Kind.Atomic) return indexedModel.atoms;
+    if (unitKind === Unit.Kind.Spheres) return indexedModel.spheres;
+    if (unitKind === Unit.Kind.Gaussians) return indexedModel.gaussians;
+    console.warn(`Unknown Unit.Kind value: ${unitKind}`);
+    return null;
+}
 
 /** Main class for processing MVS annotation */
 export class MVSAnnotation {
@@ -202,7 +217,8 @@ export class MVSAnnotation {
     /** Return value of field `fieldName` assigned to location `loc`, if any */
     getValueForLocation(loc: StructureElement.Location, fieldName: string): string | undefined {
         const indexedModel = this.getIndexedModel(loc.unit.model, loc.unit.conformation.operator.instanceId);
-        const iRow = (indexedModel !== null) ? indexedModel[loc.element] : -1;
+        const indexedElements = getIndexedElementsForUnitKind(indexedModel, loc.unit.kind);
+        const iRow = indexedElements ? indexedElements[loc.element] : -1;
         return this.getValueForRow(iRow, fieldName);
     }
     /** Return value of field `fieldName` assigned to `i`-th annotation row, if any */
@@ -235,16 +251,22 @@ export class MVSAnnotation {
     private getRowForEachAtom(model: Model, instanceId: string): IndexedModel {
         const indices = IndicesAndSortings.get(model);
         const nAtoms = model.atomicHierarchy.atoms._rowCount;
-        let result: IndexedModel = null;
+        const nSpheres = model.coarseHierarchy.spheres.count;
+        const nGaussians = model.coarseHierarchy.gaussians.count;
+        let indexedAtoms: IndexedElements = null;
+        let indexedSpheres: IndexedElements = null;
+        let indexedGaussians: IndexedElements = null;
         const rows = this.getRows();
-        for (let i = 0, nRows = rows.length; i < nRows; i++) {
-            const row = rows[i];
+        for (let iRow = 0, nRows = rows.length; iRow < nRows; iRow++) {
+            const row = rows[iRow];
             const atomRanges = getAtomRangesForRow(row, model, instanceId, indices);
-            if (AtomRanges.count(atomRanges) === 0) continue;
-            result ??= Array(nAtoms).fill(-1);
-            AtomRanges.foreach(atomRanges, (from, to) => result!.fill(i, from, to));
+            indexedAtoms = fillValueOnRanges(indexedAtoms, nAtoms, atomRanges, iRow);
+            const sphereRanges = getSphereRangesForRow(row, model, instanceId, indices);
+            indexedSpheres = fillValueOnRanges(indexedSpheres, nSpheres, sphereRanges, iRow);
+            const gaussianRanges = getGaussianRangesForRow(row, model, instanceId, indices);
+            indexedGaussians = fillValueOnRanges(indexedGaussians, nGaussians, gaussianRanges, iRow);
         }
-        return result;
+        return { atoms: indexedAtoms, spheres: indexedSpheres, gaussians: indexedGaussians };
     }
 
     /** Parse and return all annotation rows in this annotation, or return cached result if available */
@@ -437,3 +459,11 @@ function annotationSourceFromSpec(s: MVSAnnotationSpec): MVSAnnotationSource {
             return { kind: 'source-cif' };
     }
 }
+
+/** In `array`, set value `fillValue` to all positions described by `fillRanges`. In case `array` is `null`, initialize it with length `n` prefilled with -1. */
+function fillValueOnRanges(array: IndexedElements, n: number, fillRanges: ElementRanges | undefined, fillValue: number): IndexedElements {
+    if (!fillRanges || ElementRanges.count(fillRanges) === 0) return array;
+    const out = array ?? Array(n).fill(-1);
+    ElementRanges.foreach(fillRanges, (from, to) => out.fill(fillValue, from, to));
+    return out;
+}

src/extensions/mvs/components/custom-label/visual.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2023-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Adam Midlik <midlik@gmail.com>
  */
@@ -75,7 +75,7 @@ function createLabelText(ctx: VisualContext, structure: Structure, theme: Theme,
                 break;
             case 'selection':
                 const substructure = substructureFromSelector(structure, item.position.params.selector);
-                const p = textPropsForSelection(substructure, theme.size.size, [{}]);
+                const p = textPropsForSelection(substructure, [{}]);
                 const group = serialIndexOfSubstructure(structure, substructure) ?? 0;
                 if (p) builder.add(item.text, p.center[0], p.center[1], p.center[2], p.depth, p.scale, group);
                 break;

src/extensions/mvs/components/formats.ts

@@ -5,7 +5,7 @@
  * @author David Sehnal <david.sehnal@gmail.com>
  */
 
-import { hashFnv32a } from '../../../mol-data/util';
+import { murmurHash3_128_fromBytes } from '../../../mol-data/util';
 import { StringLike } from '../../../mol-io/common/string-like';
 import { DataFormatProvider } from '../../../mol-plugin-state/formats/provider';
 import { PluginStateObject as SO } from '../../../mol-plugin-state/objects';
@@ -112,16 +112,16 @@ export const MVSXFormatProvider: DataFormatProvider<{}, StateObjectRef<Mvs>, any
  * add all contained files to `plugin`'s asset manager,
  * and parse the main file in the archive as MVSJ.
  * Return parsed MVS data and `sourceUrl` for resolution of relative URIs.  */
-export async function loadMVSX(plugin: PluginContext, runtimeCtx: RuntimeContext, data: Uint8Array, mainFilePath: string = 'index.mvsj'): Promise<{ mvsData: MVSData, sourceUrl: string }> {
+export async function loadMVSX(plugin: PluginContext, runtimeCtx: RuntimeContext, data: Uint8Array<ArrayBuffer>, mainFilePath: string = 'index.mvsj'): Promise<{ mvsData: MVSData, sourceUrl: string }> {
     // Ensure at most one generation of MVSX file assets exists in the asset manager.
     // Hopefully, this is a reasonable compromise to ensure MVSX files work in multi-snapshot
     // states.
     clearMVSXFileAssets(plugin);
 
-    const archiveId = `ni,fnv1a;${hashFnv32a(data)}`;
-    let files: { [path: string]: Uint8Array };
+    const archiveId = `ni,MurmurHash3_128;${murmurHash3_128_fromBytes(data, 42)}`;
+    let files: { [path: string]: Uint8Array<ArrayBuffer> };
     try {
-        files = await unzip(runtimeCtx, data) as typeof files;
+        files = await unzip(runtimeCtx, data.buffer) as typeof files;
     } catch (err) {
         plugin.log.error('Invalid MVSX file');
         throw err;
@@ -138,7 +138,7 @@ export async function loadMVSX(plugin: PluginContext, runtimeCtx: RuntimeContext
     return { mvsData, sourceUrl };
 }
 
-export async function loadMVSData(plugin: PluginContext, data: MVSData | StringLike | Uint8Array, format: 'mvsj' | 'mvsx', options?: MVSLoadOptions) {
+export async function loadMVSData(plugin: PluginContext, data: MVSData | StringLike | Uint8Array<ArrayBuffer>, format: 'mvsj' | 'mvsx', options?: MVSLoadOptions) {
     if (typeof data === 'string' && data.startsWith('base64')) {
         data = Uint8Array.from(atob(data.substring(7)), c => c.charCodeAt(0)); // Decode base64 string to Uint8Array
     }
@@ -160,7 +160,7 @@ export async function loadMVSData(plugin: PluginContext, data: MVSData | StringL
             throw new Error("loadMvsData: if `format` is 'mvsx', then `data` must be a Uint8Array or a base64-encoded string prefixed with 'base64,'.");
         }
         await plugin.runTask(Task.create('Load MVSX file', async ctx => {
-            const parsed = await loadMVSX(plugin, ctx, data as Uint8Array);
+            const parsed = await loadMVSX(plugin, ctx, data as Uint8Array<ArrayBuffer>);
             await loadMVS(plugin, parsed.mvsData, { sanityChecks: true, ...options, sourceUrl: parsed.sourceUrl });
         }));
     } else {
@@ -182,7 +182,7 @@ function tryGetDownloadUrl(pso: SO.Data.String, plugin: PluginContext): string |
 }
 
 /** Return a URI referencing a file within an archive, using ARCP scheme (https://arxiv.org/pdf/1809.06935.pdf).
- * `archiveId` corresponds to the `authority` part of URI (e.g. 'uuid,EYVwjDiZhM20PWbF1OWWvQ' or 'ni,fnv1a;938511930')
+ * `archiveId` corresponds to the `authority` part of URI (e.g. 'uuid,EYVwjDiZhM20PWbF1OWWvQ' or 'ni,MurmurHash3_128;e6494f6be71f34c556f3de73d306780c')
  * `path` corresponds to the path to a file within the archive */
 function arcpUri(archiveId: string, path: string): string {
     return new URL(path, `arcp://${archiveId}/`).href;
@@ -190,7 +190,7 @@ function arcpUri(archiveId: string, path: string): string {
 
 /** Add a URL asset to asset manager.
  * Skip if an asset with the same URL already exists. */
-function ensureUrlAsset(manager: AssetManager, url: string, data: Uint8Array, options?: { isFile?: boolean }) {
+function ensureUrlAsset(manager: AssetManager, url: string, data: Uint8Array<ArrayBuffer>, options?: { isFile?: boolean }) {
     const asset = Asset.getUrlAsset(manager, url);
     if (!manager.has(asset)) {
         const filename = url.split('/').pop() ?? 'file';

src/extensions/mvs/components/primitives.ts

@@ -149,6 +149,7 @@ export const MVSBuildPrimitiveShape = MVSTransform({
         const context: PrimitiveBuilderContext = { ...a.data, structureRefs };
 
         const snapshotKey = { snapshotKey: { ...SnapshotKey, defaultValue: a.data.options?.snapshot_key ?? '' } };
+        const markdownCommands = { markdownCommands: { ...MarkdownCommands, defaultValue: a.data.node?.custom?.molstar_markdown_commands } };
 
         const label = capitalize(params.kind);
         if (params.kind === 'mesh') {
@@ -161,6 +162,7 @@ export const MVSBuildPrimitiveShape = MVSTransform({
                 params: {
                     ...PD.withDefaults(Mesh.Params, { alpha: a.data.options?.opacity ?? 1, ...customMeshParams }),
                     ...snapshotKey,
+                    ...markdownCommands,
                 },
                 getShape: (_, data, __, prev: any) => buildPrimitiveMesh(data, prev?.geometry),
                 geometryUtils: Mesh.Utils,
@@ -185,6 +187,7 @@ export const MVSBuildPrimitiveShape = MVSTransform({
                         ...customLabelParams,
                     }),
                     ...snapshotKey,
+                    ...markdownCommands,
                 },
                 getShape: (_, data, props, prev: any) => buildPrimitiveLabels(data, prev?.geometry, props),
                 geometryUtils: Text.Utils,
@@ -199,6 +202,7 @@ export const MVSBuildPrimitiveShape = MVSTransform({
                 params: {
                     ...PD.withDefaults(Lines.Params, { alpha: a.data.options?.opacity ?? 1, ...customLineParams }),
                     ...snapshotKey,
+                    ...markdownCommands,
                 },
                 getShape: (_, data, __, prev: any) => buildPrimitiveLines(data, prev?.geometry),
                 geometryUtils: Lines.Utils,
@@ -227,7 +231,7 @@ export const MVSShapeRepresentation3D = MVSTransform({
             const repr = ShapeRepresentation(a.data.getShape, a.data.geometryUtils);
             await repr.createOrUpdate(props, a.data.data).runInContext(ctx);
 
-            const pickable = !!(params as any).snapshotKey?.trim();
+            const pickable = !!(params as any).snapshotKey?.trim() || !!(params as any).markdownCommands;
             if (pickable) {
                 repr.setState({ pickable, markerActions: MarkerActions.Highlighting });
             }
@@ -241,7 +245,7 @@ export const MVSShapeRepresentation3D = MVSTransform({
             await b.data.repr.createOrUpdate(props, a.data.data).runInContext(ctx);
             b.data.sourceData = a.data;
 
-            const pickable = !!(newParams as any).snapshotKey?.trim();
+            const pickable = !!(newParams as any).snapshotKey?.trim() || !!(newParams as any).markdownCommands;
             if (pickable) {
                 b.data.repr.setState({ pickable, markerActions: MarkerActions.Highlighting });
             }
@@ -252,6 +256,7 @@ export const MVSShapeRepresentation3D = MVSTransform({
 });
 
 const SnapshotKey = PD.Text('', { isEssential: true, disableInteractiveUpdates: true, description: 'Activate the snapshot with the provided key when clicking on the label' });
+const MarkdownCommands = PD.Value<any>(undefined, { isHidden: true });
 
 /* **************************************************** */
 

src/extensions/mvs/components/trajectory.ts

@@ -0,0 +1,36 @@
+/**
+ * Copyright (c) 2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ *
+ * @author David Sehnal <david.sehnal@gmail.com>
+ */
+
+import { PluginStateObject } from '../../../mol-plugin-state/objects';
+import { getTrajectory } from '../../../mol-plugin-state/transforms/model';
+import { Task } from '../../../mol-task';
+import { ParamDefinition } from '../../../mol-util/param-definition';
+import { getMVSReferenceObject } from '../helpers/utils';
+import { MVSTransform } from './annotation-structure-component';
+
+export const MVSTrajectoryWithCoordinates = MVSTransform({
+    name: 'trajectory-with-coordinates',
+    display: { name: 'Trajectory with Coordinates', description: 'Create a trajectory from existing model and the provided coordinates.' },
+    from: PluginStateObject.Molecule.Model,
+    to: PluginStateObject.Molecule.Trajectory,
+    params: {
+        coordinatesRef: ParamDefinition.Text('', { isHidden: true }),
+    }
+})({
+    apply({ a, params, dependencies }) {
+        return Task.create('Create trajectory from model/topology and coordinates', async ctx => {
+            const coordinates = getMVSReferenceObject([PluginStateObject.Molecule.Coordinates], dependencies, params.coordinatesRef);
+
+            if (!coordinates) {
+                throw new Error('Coordinates not found.');
+            }
+
+            const trajectory = await getTrajectory(ctx, a, coordinates.data);
+            const props = { label: 'Trajectory', description: `${trajectory.frameCount} model${trajectory.frameCount === 1 ? '' : 's'}` };
+            return new PluginStateObject.Molecule.Trajectory(trajectory, props);
+        });
+    }
+});

src/extensions/mvs/export.ts

@@ -0,0 +1,26 @@
+/**
+ * Copyright (c) 2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ *
+ * @author David Sehnal <david.sehnal@gmail.com>
+ */
+
+import { Zip } from '../../mol-util/zip/zip';
+import { MVSData } from './mvs-data';
+
+/**
+ * Creates an MVSX zip file with from the provided data and assets
+ */
+export async function createMVSX(data: MVSData, assets: { name: string, content: string | Uint8Array<ArrayBuffer> }[]) {
+    const encoder = new TextEncoder();
+    const files: Record<string, Uint8Array<ArrayBuffer>> = {
+        'index.mvsj': encoder.encode(JSON.stringify(data)),
+    };
+    for (const asset of assets) {
+        files[asset.name] = typeof asset.content === 'string'
+            ? encoder.encode(asset.content)
+            : asset.content;
+    }
+
+    const zip = await Zip(files).run();
+    return new Uint8Array(zip);
+}

src/extensions/mvs/helpers/_spec/atom-ranges.spec.ts

@@ -4,7 +4,7 @@
  * @author Adam Midlik <midlik@gmail.com>
  */
 
-import { AtomRanges } from '../atom-ranges';
+import { ElementRanges } from '../element-ranges';
 
 
 describe('union', () => {
@@ -12,39 +12,39 @@ describe('union', () => {
         const a = {
             from: [0, 20, 40, 60, 80],
             to: [10, 30, 50, 70, 90],
-        } as AtomRanges;
+        } as ElementRanges;
         const b = {
             from: [11, 37, 51, 205],
             to: [15, 39, 55, 210],
-        } as AtomRanges;
+        } as ElementRanges;
         const c = {
             from: [-10, 200, 300],
             to: [-5, 202, 305],
-        } as AtomRanges;
+        } as ElementRanges;
         const result = {
             from: [-10, 0, 11, 20, 37, 40, 51, 60, 80, 200, 205, 300],
             to: [-5, 10, 15, 30, 39, 50, 55, 70, 90, 202, 210, 305],
-        } as AtomRanges;
-        expect(AtomRanges.union([a, b, c])).toEqual(result);
+        } as ElementRanges;
+        expect(ElementRanges.union([a, b, c])).toEqual(result);
     });
     it('union overlapping', async () => {
         const a = {
             from: [0, 20, 40, 60, 80],
             to: [10, 30, 50, 70, 90],
-        } as AtomRanges;
+        } as ElementRanges;
         const b = {
             from: [10, 37, 51, 84, 205],
             to: [15, 40, 55, 88, 220],
-        } as AtomRanges;
+        } as ElementRanges;
         const c = {
             from: [-10, 67, 200, 300],
             to: [5, 80, 210, 305],
-        } as AtomRanges;
+        } as ElementRanges;
         const result = {
             from: [-10, 20, 37, 51, 60, 200, 300],
             to: [15, 30, 50, 55, 90, 220, 305],
-        } as AtomRanges;
-        expect(AtomRanges.union([a, b, c])).toEqual(result);
+        } as ElementRanges;
+        expect(ElementRanges.union([a, b, c])).toEqual(result);
     });
 });
 

src/extensions/mvs/helpers/animation.ts

@@ -5,24 +5,25 @@
  * @author Ludovic Autin <ludovic.autin@gmail.com>
  */
 
-import { produce } from 'immer';
-import { Snapshot } from '../mvs-data';
-import { Tree } from '../tree/generic/tree-schema';
-import { clamp, lerp } from '../../../mol-math/interpolate';
-import { MVSAnimationEasing, MVSAnimationNode, MVSAnimationSchema } from '../tree/animation/animation-tree';
-import { MVSTree } from '../tree/mvs/mvs-tree';
+import { SortedArray } from '../../../mol-data/int';
 import * as EasingFns from '../../../mol-math/easing';
-import { addDefaults } from '../tree/generic/tree-utils';
-import { RuntimeContext } from '../../../mol-task';
+import { clamp, lerp } from '../../../mol-math/interpolate';
 import { EPSILON, Mat3, Mat4, Quat, Vec3 } from '../../../mol-math/linear-algebra';
+import { RuntimeContext } from '../../../mol-task';
+import { deepEqual } from '../../../mol-util';
 import { Color } from '../../../mol-util/color';
+import { decodeColor } from '../../../mol-util/color/utils';
+import { produce } from '../../../mol-util/produce';
 import { makeContinuousPaletteCheckpoints, MVSContinuousPaletteProps, MVSDiscretePaletteProps } from '../components/annotation-color-theme';
 import { palettePropsFromMVSPalette } from '../load-helpers';
-import { SortedArray } from '../../../mol-data/int';
+import { Snapshot } from '../mvs-data';
+import { MVSAnimationEasing, MVSAnimationNode, MVSAnimationSchema } from '../tree/animation/animation-tree';
+import { Tree } from '../tree/generic/tree-schema';
+import { addDefaults } from '../tree/generic/tree-utils';
+import { MVSTree } from '../tree/mvs/mvs-tree';
 import { ColorT } from '../tree/mvs/param-types';
 
-
-export async function generateStateTransition(ctx: RuntimeContext, snapshot: Snapshot) {
+export async function generateStateTransition(ctx: RuntimeContext, snapshot: Snapshot, snapshotIndex: number, snapshotCount: number) {
     if (!snapshot.animation) return undefined;
 
     const tree = addDefaults(snapshot.animation, MVSAnimationSchema);
@@ -34,19 +35,30 @@ export async function generateStateTransition(ctx: RuntimeContext, snapshot: Sna
         ...transitions.map(t => (t.params.start_ms ?? 0) + t.params.duration_ms)
     );
 
-    const frames: MVSTree[] = [];
+    const frames: [tree: MVSTree, time: number][] = [];
     const dt = tree.params?.frame_time_ms ?? (1000 / 60);
     const N = Math.ceil(duration / dt);
 
+    const nodeMap = makeNodeMap(snapshot.root, new Map(), []);
     const cache = new Map<any, InterpolationCacheEntry>();
 
+    const transitionGroups = groupTranstions(transitions);
+
+    let prevRoot: MVSTree | undefined;
     for (let i = 0; i <= N; i++) {
         const t = i * dt;
-        const root = createSnapshot(snapshot.root, transitions, t, cache);
-        frames.push(root);
+        const root = createSnapshot(snapshot.root, transitionGroups, t, cache, nodeMap);
+
+        if (root === prevRoot || (prevRoot && deepEqual(root, prevRoot))) {
+            frames[frames.length - 1][1] += dt;
+        } else {
+            frames.push([root, dt]);
+        }
+
+        prevRoot = root;
 
         if (ctx.shouldUpdate) {
-            await ctx.update({ message: 'Generating transition...', current: i + 1, max: N });
+            await ctx.update({ message: `Generating transition for snapshot ${snapshotIndex + 1}/${snapshotCount}`, current: i + 1, max: N });
         }
     }
 
@@ -77,73 +89,96 @@ const EasingFnMap: Record<MVSAnimationEasing, (t: number) => number> = {
 
 interface InterpolationCacheEntry {
     paletteFn?: (value: number) => Color,
+    startColor?: Color | Record<number | string, Color>,
+    endColor?: Color | Record<number | string, Color>,
     rotation?: { axis: Vec3, angle: number, start: Quat, end: Quat },
 }
 
-function createSnapshot(tree: MVSTree, transitions: MVSAnimationNode<'interpolate'>[], time: number, cache: Map<any, InterpolationCacheEntry>) {
-    return produce(tree, (draft) => {
-        for (const transition of transitions) {
-            const node = findNode(draft, transition.params.target_ref);
-            if (!node) continue;
+function getTransitionKey(transition: MVSAnimationNode<'interpolate'>) {
+    const prop = transition.params.property;
+    if (Array.isArray(prop)) {
+        return `${transition.params.target_ref}:${prop.join('.')}`;
+    }
+    return `${transition.params.target_ref}:${prop}`;
+}
 
-            const target = transition.params.property[0] === 'custom' ? node?.custom : node?.params;
+function groupTranstions(transitions: MVSAnimationNode<'interpolate'>[]) {
+    const map = new Map<string, MVSAnimationNode<'interpolate'>[]>();
+    const groups: MVSAnimationNode<'interpolate'>[][] = [];
+    for (const t of transitions) {
+        const key = getTransitionKey(t);
+        if (!map.has(key)) {
+            const group: MVSAnimationNode<'interpolate'>[] = [];
+            map.set(key, group);
+            groups.push(group);
+        }
+        map.get(key)!.push(t);
+    }
+    for (const group of groups) {
+        group.sort((a, b) => {
+            const s = (a.params.start_ms ?? 0) - (b.params.start_ms ?? 0);
+            if (s !== 0) return s;
+            return a.params.duration_ms - b.params.duration_ms;
+        });
+    }
+    return groups;
+}
+
+function createSnapshot(tree: MVSTree, transitionGroups: MVSAnimationNode<'interpolate'>[][], time: number, cache: Map<any, InterpolationCacheEntry>, nodeMap: Map<string, (string | number)[]>) {
+    let modified = false;
+    const ret = produce(tree, (draft) => {
+        for (const transitionGroup of transitionGroups) {
+
+            const pivot = transitionGroup[0];
+            const nodePath = nodeMap.get(pivot.params.target_ref);
+            if (!nodePath) continue;
+
+            const node = select(draft, nodePath, 0);
+            const target = pivot.params.property[0] === 'custom' ? node?.custom : node?.params;
             if (!target) continue;
 
+
+            const offset = pivot.params.property[0] === 'custom' ? 1 : 0;
+
+            let transition: MVSAnimationNode<'interpolate'> = pivot;
+            let previous: MVSAnimationNode<'interpolate'> | undefined;
+
+            for (let i = transitionGroup.length - 1; i > 0; i--) {
+                const current = transitionGroup[i];
+                const currentStart = current.params.start_ms ?? 0;
+                if (time >= currentStart) {
+                    transition = current;
+                    previous = i > 0 ? transitionGroup[i - 1] : undefined;
+                    break;
+                }
+            }
+
             if (!cache.has(transition)) {
                 cache.set(transition, {});
             }
 
             const cacheEntry: InterpolationCacheEntry = cache.get(transition)!;
 
-            const startTime = transition.params.start_ms ?? 0;
-            let t = clamp((time - startTime) / transition.params.duration_ms, 0, 1);
-
-            if (transition.params.kind === 'transform_matrix') {
-                processTransformMatrix(transition, target, t, cacheEntry);
-                continue;
-            }
-
-            t = applyFrequency(t, transition.params.frequency ?? 1, !!transition.params.alternate_direction);
-
-            const offset = transition.params.property[0] === 'custom' ? 1 : 0;
-            const startBase = transition.params.start ?? select(target, transition.params.property, offset);
-
-            if (transition.params.kind === 'color' && !cacheEntry.paletteFn) {
-                cacheEntry.paletteFn = makePaletteFunction(transition, startBase, transition.params.end as ColorT | undefined);
-            }
-
-            const paletteFn = cacheEntry.paletteFn!;
-
-            const startValue: any = transition.params.kind === 'color'
-                ? Color.toHexStyle(paletteFn(0))
-                : startBase;
-            const endValue: any = transition.params.kind === 'color'
-                ? Color.toHexStyle(paletteFn(1))
-                : transition.params.end;
-
-            if (time <= startTime) {
-                assign(target, transition.params.property, startValue, offset);
-                continue;
-            }
-
-            const easing = EasingFnMap[transition.params.easing ?? 'linear'] ?? EasingFnMap['linear'];
-            t = easing(t);
+            const startTime: number = transition.params.start_ms ?? 0;
+            const durationMs: number = transition.params.duration_ms ?? 0;
+            const t = (time - startTime) / durationMs;
 
             let next: any;
-            if (transition.params.kind === 'scalar') {
-                next = interpolateScalars(startValue, endValue, t, transition.params.noise_magnitude ?? 0);
-            } else if (transition.params.kind === 'vec3') {
-                next = interpolateVectors(startValue, endValue, t, transition.params.noise_magnitude ?? 0, !!transition.params.spherical);
-            } else if (transition.params.kind === 'rotation_matrix') {
-                next = interpolateRotation(startValue, endValue, t, transition.params.noise_magnitude ?? 0, cacheEntry);
-            } else if (transition.params.kind === 'color') {
-                const color = paletteFn(t);
-                next = Color.toHexStyle(color);
+            if (transition.params.kind === 'transform_matrix') {
+                next = processTransformMatrix(transition, target, clamp(t, 0, 1), cacheEntry, offset, previous);
+            } else {
+                next = processScalarLike(transition, target, t, cacheEntry, offset, previous);
             }
 
+            if (next === undefined) {
+                continue;
+            }
+
+            modified = true;
             assign(target, transition.params.property, next, offset);
         }
     });
+    return modified ? ret : tree;
 }
 
 function applyFrequency(t: number, frequency: number, alternate: boolean) {
@@ -162,6 +197,53 @@ function applyFrequency(t: number, frequency: number, alternate: boolean) {
     return v;
 }
 
+function getPreviousScalarEnd(previous: MVSAnimationNode<'interpolate'> | undefined) {
+    if (!previous || previous.params.kind === 'transform_matrix') return undefined;
+    return previous.params.end;
+}
+
+function processScalarLike(transition: MVSAnimationNode<'interpolate'>, target: any, time: number, cacheEntry: InterpolationCacheEntry, offset: number, previous: MVSAnimationNode<'interpolate'> | undefined) {
+    if (transition.params.kind === 'transform_matrix') return;
+    if (previous && previous.params.kind === 'transform_matrix') return;
+
+    const startValue = transition.params.start ?? getPreviousScalarEnd(previous) ?? select(target, transition.params.property, offset);
+    if (transition.params.kind === 'color' && !cacheEntry.paletteFn) {
+        cacheEntry.paletteFn = makePaletteFunction(transition);
+    }
+
+    const endValue: any = transition.params.end;
+
+    if (time <= 0) return startValue;
+    else if (time >= 1 - EPSILON && !transition.params.alternate_direction && transition.params.kind !== 'color') return endValue;
+
+    let t = clamp(time, 0, 1);
+    t = applyFrequency(t, transition.params.frequency ?? 1, !!transition.params.alternate_direction);
+
+    const easing = EasingFnMap[transition.params.easing ?? 'linear'] ?? EasingFnMap['linear'];
+    t = easing(t);
+
+    if (transition.params.kind === 'scalar') {
+        return interpolateScalars(startValue, endValue, t, transition.params.noise_magnitude ?? 0, !!transition.params.discrete);
+    } else if (transition.params.kind === 'vec3') {
+        return interpolateVectors(startValue, endValue, t, transition.params.noise_magnitude ?? 0, !!transition.params.spherical);
+    } else if (transition.params.kind === 'rotation_matrix') {
+        return interpolateRotation(startValue, endValue, t, transition.params.noise_magnitude ?? 0, cacheEntry);
+    } else if (transition.params.kind === 'color') {
+        if (cacheEntry.paletteFn) {
+            const color = cacheEntry.paletteFn(t);
+            return Color.toHexStyle(color);
+        }
+
+        const baseColors = typeof startValue === 'object' ? select(target, transition.params.property, offset) : undefined;
+        return interpolateColors(startValue, endValue, t, cacheEntry, baseColors);
+    }
+}
+
+function getPreviousMatrixEnd(previous: MVSAnimationNode<'interpolate'> | undefined, prop: 'rotation_start' | 'translation_start' | 'scale_start') {
+    if (!previous || previous.params.kind !== 'transform_matrix') return undefined;
+    return previous.params[prop];
+}
+
 const TransformState = {
     pivotTranslation: Mat4(),
     pivotTranslationInv: Mat4(),
@@ -171,31 +253,41 @@ const TransformState = {
     pivotNeg: Vec3(),
     temp: Mat4(),
 };
-function processTransformMatrix(transition: MVSAnimationNode<'interpolate'>, target: any, time: number, cache: InterpolationCacheEntry) {
+function processTransformMatrix(transition: MVSAnimationNode<'interpolate'>, target: any, time: number, cache: InterpolationCacheEntry, offset: number, previous: MVSAnimationNode<'interpolate'> | undefined) {
     if (transition.params.kind !== 'transform_matrix') return;
+    if (previous && previous.params.kind !== 'transform_matrix') return;
 
-    const offset = transition.params.property[0] === 'custom' ? 1 : 0;
     const transform = select(target, transition.params.property, offset) ?? Mat4.identity();
 
-    const startRotation = transition.params.rotation_start ?? Mat3.fromMat4(Mat3(), transform);
-    const startTranslation = transition.params.translation_start ?? Mat4.getTranslation(Vec3(), transform);
-    const startScale = transition.params.scale_start ?? Mat4.getScaling(Vec3(), transform);
+    const startRotation = transition.params.rotation_start ?? getPreviousMatrixEnd(previous, 'rotation_start') ?? Mat3.fromMat4(Mat3(), transform);
+    const startTranslation = transition.params.translation_start ?? getPreviousMatrixEnd(previous, 'translation_start') ?? Mat4.getTranslation(Vec3(), transform);
+    const startScale = transition.params.scale_start ?? getPreviousMatrixEnd(previous, 'scale_start') ?? Mat4.getScaling(Vec3(), transform);
 
     const endRotation = transition.params.rotation_end;
     const endTranslation = transition.params.translation_end;
     const endScale = transition.params.scale_end;
 
-    let t = applyFrequency(time, transition.params.rotation_frequency ?? 1, !!transition.params.rotation_alternate_direction);
-    let easing = EasingFnMap[transition.params.rotation_easing ?? 'linear'] ?? EasingFnMap['linear'];
-    const rotation = interpolateRotation(startRotation as Mat3, endRotation as Mat3, easing(t), transition.params.rotation_noise_magnitude ?? 0, cache);
+    let rotation, translation, scale;
 
-    t = applyFrequency(time, transition.params.translation_frequency ?? 1, !!transition.params.translation_alternate_direction);
-    easing = EasingFnMap[transition.params.translation_easing ?? 'linear'] ?? EasingFnMap['linear'];
-    const translation = interpolateVec3(startTranslation as Vec3, endTranslation as Vec3 | undefined, easing(t), transition.params.translation_noise_magnitude ?? 0, false);
+    if (time <= 0) {
+        rotation = startRotation as Mat3;
+        translation = startTranslation as Vec3;
+        scale = startScale as Vec3;
+    } else {
+        const clampedTime = clamp(time, 0, 1);
 
-    t = applyFrequency(time, transition.params.scale_frequency ?? 1, !!transition.params.scale_alternate_direction);
-    easing = EasingFnMap[transition.params.scale_easing ?? 'linear'] ?? EasingFnMap['linear'];
-    const scale = interpolateVec3(startScale as Vec3, endScale as Vec3 | undefined, easing(t), transition.params.scale_noise_magnitude ?? 0, false);
+        let t = applyFrequency(clampedTime, transition.params.rotation_frequency ?? 1, !!transition.params.rotation_alternate_direction);
+        let easing = EasingFnMap[transition.params.rotation_easing ?? 'linear'] ?? EasingFnMap['linear'];
+        rotation = interpolateRotation(startRotation as Mat3, endRotation as Mat3, easing(t), transition.params.rotation_noise_magnitude ?? 0, cache);
+
+        t = applyFrequency(clampedTime, transition.params.translation_frequency ?? 1, !!transition.params.translation_alternate_direction);
+        easing = EasingFnMap[transition.params.translation_easing ?? 'linear'] ?? EasingFnMap['linear'];
+        translation = interpolateVec3(startTranslation as Vec3, endTranslation as Vec3 | undefined, easing(t), transition.params.translation_noise_magnitude ?? 0, false);
+
+        t = applyFrequency(clampedTime, transition.params.scale_frequency ?? 1, !!transition.params.scale_alternate_direction);
+        easing = EasingFnMap[transition.params.scale_easing ?? 'linear'] ?? EasingFnMap['linear'];
+        scale = interpolateVec3(startScale as Vec3, endScale as Vec3 | undefined, easing(t), transition.params.scale_noise_magnitude ?? 0, false);
+    }
 
     const pivot = transition.params.pivot ?? Vec3.zero();
 
@@ -212,21 +304,21 @@ function processTransformMatrix(transition: MVSAnimationNode<'interpolate'>, tar
     Mat4.mul(result, TransformState.rotation, result);
     Mat4.mul(result, TransformState.translation, result);
 
-    assign(target, transition.params.property, result, offset);
+    return result;
 }
 
-function interpolateScalars(start: number | number[], end: number | number[] | undefined, t: number, noise: number) {
+function interpolateScalars(start: number | number[], end: number | number[] | undefined, t: number, noise: number, discrete: boolean) {
     if (Array.isArray(start)) {
         const ret = Array.from<number>({ length: start.length }).fill(0.1);
         if (!end || !Array.isArray(end)) {
             for (let i = 0; i < start.length; i++) {
-                ret[i] = interpolateScalar(start[i], end, t, noise);
+                ret[i] = interpolateScalar(start[i], end, t, noise, discrete);
             }
             return ret;
         }
 
         for (let i = 0; i < start.length; i++) {
-            ret[i] = interpolateScalar(start[i], end[i], t, noise);
+            ret[i] = interpolateScalar(start[i], end[i], t, noise, discrete);
         }
         return ret;
     }
@@ -234,19 +326,22 @@ function interpolateScalars(start: number | number[], end: number | number[] | u
     if (Array.isArray(end)) {
         const ret = Array.from<number>({ length: end.length }).fill(0.1);
         for (let i = 0; i < end.length; i++) {
-            ret[i] = interpolateScalar(start, end[i], t, noise);
+            ret[i] = interpolateScalar(start, end[i], t, noise, discrete);
         }
         return ret;
     }
 
-    return interpolateScalar(start, end, t, noise);
+    return interpolateScalar(start, end, t, noise, discrete);
 }
 
-function interpolateScalar(start: number, end: number | undefined, t: number, noise: number) {
+function interpolateScalar(start: number, end: number | undefined, t: number, noise: number, discrete: boolean) {
     let v = typeof end === 'number' ? lerp(start, end, t) : start;
     if (noise) {
         v += (Math.random() - 0.5) * noise;
     }
+    if (discrete) {
+        v = Math.round(v);
+    }
     return v;
 }
 
@@ -347,6 +442,76 @@ function interpolateRotation(start: Mat3, end: Mat3 | undefined, t: number, nois
     return Mat3.fromMat4(Mat3(), RotationState.temp);
 }
 
+function decodeColors(color: ColorT | Record<number | string, ColorT> | undefined, baseColors: Record<number | string, ColorT> | undefined) {
+    if (color === undefined || color === null) return undefined;
+
+    if (typeof color === 'object') {
+        const ret: Record<number | string, Color> = {};
+        if (baseColors) {
+            for (const key of Object.keys(baseColors)) {
+                const decoded = decodeColor(baseColors[key]);
+                if (decoded !== undefined) {
+                    ret[key] = decoded;
+                }
+            }
+        }
+        for (const key of Object.keys(color)) {
+            const decoded = decodeColor(color[key]);
+            if (decoded !== undefined) {
+                ret[key] = decoded;
+            }
+        }
+        return ret;
+    }
+
+    return decodeColor(color);
+}
+
+function interpolateColors(start: ColorT | Record<number, ColorT>, end: ColorT | Record<number, ColorT> | undefined, time: number, cacheEntry: InterpolationCacheEntry, baseColors: Record<number, ColorT> | undefined) {
+    const t = clamp(time, 0, 1);
+
+    if (cacheEntry.paletteFn) {
+        const c = cacheEntry.paletteFn(t);
+        return Color.toHexStyle(c);
+    }
+
+    if (cacheEntry.startColor === undefined) {
+        cacheEntry.startColor = decodeColors(start, baseColors);
+    }
+    if (cacheEntry.endColor === undefined) {
+        cacheEntry.endColor = decodeColors(end, undefined);
+    }
+
+    const { startColor, endColor } = cacheEntry;
+
+    if (typeof startColor === 'object') {
+        if (typeof baseColors !== 'object') {
+            throw new Error('Cannot interpolate from scalar color to color mapping');
+        }
+
+        const ret = { ...baseColors as any, ...startColor as any };
+        if (typeof endColor === 'object') {
+            for (const key of Object.keys(endColor)) {
+                ret[key] = Color.toHexStyle(Color.interpolate(startColor[key], endColor[key], t));
+            }
+        } else if (typeof endColor === 'number') {
+            for (const key of Object.keys(startColor)) {
+                ret[key] = Color.toHexStyle(Color.interpolate(startColor[key], endColor, t));
+            }
+        }
+        return ret;
+    }
+    if (typeof endColor === 'object') {
+        throw new Error('Cannot interpolate from scalar color to color mapping');
+    }
+
+    if (typeof endColor === 'number' && typeof startColor === 'number') {
+        return Color.toHexStyle(Color.interpolate(startColor, endColor, t));
+    }
+
+    return start;
+}
+
 function select(params: any, path: string | (string | number)[], offset: number) {
     if (typeof path === 'string') {
         return params?.[path];
@@ -380,22 +545,29 @@ function assign(params: any, path: string | (string | number)[], value: any, off
     }
 }
 
-function findNode(tree: Tree, ref: string): Tree | undefined {
-    if (tree.ref === ref) return tree;
-    if (!tree.children) return undefined;
-    for (const child of tree.children) {
-        const result = findNode(child, ref);
-        if (result) return result;
+function makeNodeMap(tree: Tree, map: Map<string, (string | number)[]>, currentPath: (string | number)[]) {
+    if (tree.ref) {
+        map.set(tree.ref, [...currentPath]);
     }
-    return undefined;
+
+    if (!tree.children) return map;
+
+    currentPath.push('children');
+    for (let i = 0; i < tree.children.length; i++) {
+        const child = tree.children[i];
+        currentPath.push(i);
+        makeNodeMap(child, map, currentPath);
+        currentPath.pop();
+    }
+    currentPath.pop();
+
+    return map;
 }
 
-function makePaletteFunction(props: MVSAnimationNode<'interpolate'>, start: ColorT | undefined | null, end: ColorT | undefined | null): ((value: number) => Color) | undefined {
-    if (props.params.kind !== 'color') return undefined;
+function makePaletteFunction(props: MVSAnimationNode<'interpolate'>): ((value: number) => Color) | undefined {
+    if (props.params.kind !== 'color' || !props.params.palette) return undefined;
 
-    const params = props.params.palette
-        ? palettePropsFromMVSPalette(props.params.palette)
-        : palettePropsFromMVSPalette({ kind: 'continuous', colors: [start ?? 'black', end ?? start ?? 'black'] });
+    const params = palettePropsFromMVSPalette(props.params.palette);
     if (params.name === 'discrete') return makePaletteFunctionDiscrete(params.params);
     if (params.name === 'continuous') return makePaletteFunctionContinuous(params.params);
     throw new Error(`NotImplementedError: makePaletteFunction for ${(props as any).name}`);

src/extensions/mvs/helpers/element-ranges.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2023-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Adam Midlik <midlik@gmail.com>
  */
@@ -9,35 +9,35 @@ import { ElementIndex } from '../../../mol-model/structure';
 import { arrayExtend, range } from '../../../mol-util/array';
 
 
-/** Represents a collection of disjoint atom ranges in a model.
- * The number of ranges is `AtomRanges.count(ranges)`,
- * the i-th range covers atoms `[ranges.from[i], ranges.to[i])`. */
-export interface AtomRanges {
+/** Represents a collection of disjoint elements ranges in a model (atoms, spheres, or gaussians).
+ * The number of ranges is `ElementRanges.count(ranges)`,
+ * the i-th range covers elements `[ranges.from[i], ranges.to[i])`. */
+export interface ElementRanges {
     from: ElementIndex[],
     to: ElementIndex[],
 }
 
-export const AtomRanges = {
-    /** Return the number of disjoined ranges in a `AtomRanges` object */
-    count(ranges: AtomRanges): number {
+export const ElementRanges = {
+    /** Return the number of disjoined ranges in a `ElementRanges` object */
+    count(ranges: ElementRanges): number {
         return ranges.from.length;
     },
 
-    /** Create new `AtomRanges` without any atoms */
-    empty(): AtomRanges {
+    /** Create new `ElementRanges` without any elements */
+    empty(): ElementRanges {
         return { from: [], to: [] };
     },
 
-    /** Create new `AtomRanges` containing a single range of atoms `[from, to)` */
-    single(from: ElementIndex, to: ElementIndex): AtomRanges {
+    /** Create new `ElementRanges` containing a single range of elements `[from, to)` */
+    single(from: ElementIndex, to: ElementIndex): ElementRanges {
         return { from: [from], to: [to] };
     },
 
-    /** Add a range of atoms `[from, to)` to existing `AtomRanges` and return the modified original.
+    /** Add a range of elements `[from, to)` to existing `ElementRanges` and return the modified original.
      * The added range must start after the end of the last existing range
-     * (if it starts just on the next atom, these two ranges will get merged). */
-    add(ranges: AtomRanges, from: ElementIndex, to: ElementIndex): AtomRanges {
-        const n = AtomRanges.count(ranges);
+     * (if it starts just on the next element, these two ranges will get merged). */
+    add(ranges: ElementRanges, from: ElementIndex, to: ElementIndex): ElementRanges {
+        const n = ElementRanges.count(ranges);
         if (n > 0) {
             const lastTo = ranges.to[n - 1];
             if (from < lastTo) throw new Error('Overlapping ranges not allowed');
@@ -55,28 +55,30 @@ export const AtomRanges = {
     },
 
     /** Apply function `func` to each range in `ranges` */
-    foreach(ranges: AtomRanges, func: (from: ElementIndex, to: ElementIndex) => any) {
-        const n = AtomRanges.count(ranges);
+    foreach(ranges: ElementRanges, func: (from: ElementIndex, to: ElementIndex) => any) {
+        const n = ElementRanges.count(ranges);
         for (let i = 0; i < n; i++) func(ranges.from[i], ranges.to[i]);
     },
 
     /** Apply function `func` to each range in `ranges` and return an array with results */
-    map<T>(ranges: AtomRanges, func: (from: ElementIndex, to: ElementIndex) => T): T[] {
-        const n = AtomRanges.count(ranges);
+    map<T>(ranges: ElementRanges, func: (from: ElementIndex, to: ElementIndex) => T): T[] {
+        const n = ElementRanges.count(ranges);
         const result: T[] = new Array(n);
         for (let i = 0; i < n; i++) result[i] = func(ranges.from[i], ranges.to[i]);
         return result;
     },
 
-    /** Compute the set union of multiple `AtomRanges` objects (as sets of atoms) */
-    union(ranges: AtomRanges[]): AtomRanges {
-        const concat = AtomRanges.empty();
+    /** Compute the set union of multiple `ElementRanges` objects (as sets of elements) */
+    union(ranges: (ElementRanges | undefined)[]): ElementRanges {
+        const concat = ElementRanges.empty();
         for (const r of ranges) {
-            arrayExtend(concat.from, r.from);
-            arrayExtend(concat.to, r.to);
+            if (r) {
+                arrayExtend(concat.from, r.from);
+                arrayExtend(concat.to, r.to);
+            }
         }
         const indices = range(concat.from.length).sort((i, j) => concat.from[i] - concat.from[j]); // sort by start of range
-        const result = AtomRanges.empty();
+        const result = ElementRanges.empty();
         let last = -1;
         for (const i of indices) {
             const from = concat.from[i];
@@ -94,26 +96,26 @@ export const AtomRanges = {
         return result;
     },
 
-    /** Return a sorted subset of `atoms` which lie in any of `ranges` (i.e. set intersection of `atoms` and `ranges`).
+    /** Return a sorted subset of `elements` which lie in any of `ranges` (i.e. set intersection of `elements` and `ranges`).
      * If `out` is provided, use it to store the result (clear any old contents).
-     * If `outFirstAtomIndex` is provided, fill `outFirstAtomIndex.value` with the index of the first selected atom (if any). */
-    selectAtomsInRanges(atoms: SortedArray<ElementIndex>, ranges: AtomRanges, out?: ElementIndex[], outFirstAtomIndex: { value?: number } = {}): ElementIndex[] {
+     * If `outFirstElementIndex` is provided, fill `outFirstElementIndex.value` with the index of the first selected element (if any). */
+    selectElementsInRanges(elements: SortedArray<ElementIndex>, ranges: ElementRanges, out?: ElementIndex[], outFirstElementIndex: { value?: number } = {}): ElementIndex[] {
         out ??= [];
         out.length = 0;
-        outFirstAtomIndex.value = undefined;
+        outFirstElementIndex.value = undefined;
 
-        const nAtoms = atoms.length;
-        const nRanges = AtomRanges.count(ranges);
-        if (nAtoms <= nRanges) {
-            // Implementation 1 (more efficient when there are fewer atoms)
-            let iRange = SortedArray.findPredecessorIndex(SortedArray.ofSortedArray(ranges.to), atoms[0] + 1);
-            for (let iAtom = 0; iAtom < nAtoms; iAtom++) {
-                const a = atoms[iAtom];
+        const nElements = elements.length;
+        const nRanges = ElementRanges.count(ranges);
+        if (nElements <= nRanges) {
+            // Implementation 1 (more efficient when there are fewer elements)
+            let iRange = SortedArray.findPredecessorIndex(SortedArray.ofSortedArray(ranges.to), elements[0] + 1);
+            for (let iElem = 0; iElem < nElements; iElem++) {
+                const a = elements[iElem];
                 while (iRange < nRanges && ranges.to[iRange] <= a) iRange++;
                 const qualifies = iRange < nRanges && ranges.from[iRange] <= a;
                 if (qualifies) {
                     out.push(a);
-                    outFirstAtomIndex.value ??= iAtom;
+                    outFirstElementIndex.value ??= iElem;
                 }
             }
         } else {
@@ -121,11 +123,11 @@ export const AtomRanges = {
             for (let iRange = 0; iRange < nRanges; iRange++) {
                 const from = ranges.from[iRange];
                 const to = ranges.to[iRange];
-                for (let iAtom = SortedArray.findPredecessorIndex(atoms, from); iAtom < nAtoms; iAtom++) {
-                    const a = atoms[iAtom];
+                for (let iElem = SortedArray.findPredecessorIndex(elements, from); iElem < nElements; iElem++) {
+                    const a = elements[iElem];
                     if (a < to) {
                         out.push(a);
-                        outFirstAtomIndex.value ??= iAtom;
+                        outFirstElementIndex.value ??= iElem;
                     } else {
                         break;
                     }

src/extensions/mvs/helpers/indexing.ts

@@ -7,17 +7,42 @@
 import { Column } from '../../../mol-data/db';
 import { SortedArray } from '../../../mol-data/int';
 import { ChainIndex, ElementIndex, Model, ResidueIndex } from '../../../mol-model/structure';
+import { CoarseElements } from '../../../mol-model/structure/model/properties/coarse';
 import { filterInPlace, range, sortIfNeeded } from '../../../mol-util/array';
 import { Mapping, MultiMap, NumberMap } from './utils';
 
 
-/** Auxiliary data structure for efficiently finding chains/residues/atoms in a model by their properties */
 export interface IndicesAndSortings {
+    atomic?: AtomicIndicesAndSortings,
+    spheres?: CoarseIndicesAndSortings,
+    gaussians?: CoarseIndicesAndSortings,
+}
+
+export const IndicesAndSortings = {
+    /** Get `IndicesAndSortings` for a model (use a cached value or create if not available yet) */
+    get(model: Model): IndicesAndSortings {
+        return model._dynamicPropertyData['indices-and-sortings'] ??= this.create(model);
+    },
+
+    /** Create `IndicesAndSortings` for a model */
+    create(model: Model): IndicesAndSortings {
+        return {
+            atomic: createAtomicIndicesAndSortings(model),
+            spheres: createCoarseIndicesAndSortings(model.coarseHierarchy.spheres),
+            gaussians: createCoarseIndicesAndSortings(model.coarseHierarchy.gaussians),
+        };
+    },
+};
+
+
+/** Auxiliary data structure for efficiently finding chains/residues/atoms in an atomic model by their properties */
+export interface AtomicIndicesAndSortings {
     chainsByLabelEntityId: Mapping<string, readonly ChainIndex[]>,
     chainsByLabelAsymId: Mapping<string, readonly ChainIndex[]>,
     chainsByAuthAsymId: Mapping<string, readonly ChainIndex[]>,
     residuesSortedByLabelSeqId: Mapping<ChainIndex, Sorting<ResidueIndex, number>>,
     residuesSortedByAuthSeqId: Mapping<ChainIndex, Sorting<ResidueIndex, number>>,
+    residuesSortedBySourceIndex: Mapping<ChainIndex, Sorting<ResidueIndex, number>>,
     residuesByInsCode: Mapping<ChainIndex, Mapping<string, readonly ResidueIndex[]>>,
     residuesByLabelCompId: Mapping<ChainIndex, Mapping<string, readonly ResidueIndex[]>>,
     /** Indicates if each residue is listed only once in `residuesByLabelCompId` (i.e. if each residue has only one label_comp_id) */
@@ -26,95 +51,162 @@ export interface IndicesAndSortings {
     /** Indicates if each residue is listed only once in `residuesByAuthCompId` (i.e. if each residue has only one auth_comp_id) */
     residuesByAuthCompIdIsPure: boolean,
     atomsById: Mapping<number, ElementIndex>,
-    atomsByIndex: Mapping<number, ElementIndex>,
+    atomsBySourceIndex: Mapping<number, ElementIndex>,
 }
 
-export const IndicesAndSortings = {
-    /** Get `IndicesAndSortings` for a model (use a cached value or create if not available yet) */
-    get(model: Model): IndicesAndSortings {
-        return model._dynamicPropertyData['indices-and-sortings'] ??= IndicesAndSortings.create(model);
-    },
+/** Create `AtomicIndicesAndSortings` for a model */
+function createAtomicIndicesAndSortings(model: Model): AtomicIndicesAndSortings | undefined {
+    const h = model.atomicHierarchy;
+    const nAtoms = h.atoms._rowCount;
+    if (nAtoms === 0) return undefined;
 
-    /** Create `IndicesAndSortings` for a model */
-    create(model: Model): IndicesAndSortings {
-        const h = model.atomicHierarchy;
-        const nAtoms = h.atoms._rowCount;
-        const nChains = h.chains._rowCount;
-        const { label_entity_id, label_asym_id, auth_asym_id } = h.chains;
-        const { label_seq_id, auth_seq_id, pdbx_PDB_ins_code } = h.residues;
-        const { label_comp_id, auth_comp_id } = h.atoms;
-        const { Present } = Column.ValueKind;
+    const nChains = h.chains._rowCount;
+    const { label_entity_id, label_asym_id, auth_asym_id } = h.chains;
+    const { label_seq_id, auth_seq_id, pdbx_PDB_ins_code } = h.residues;
+    const { label_comp_id, auth_comp_id } = h.atoms;
+    const { Present } = Column.ValueKind;
 
-        const chainsByLabelEntityId = new MultiMap<string, ChainIndex>();
-        const chainsByLabelAsymId = new MultiMap<string, ChainIndex>();
-        const chainsByAuthAsymId = new MultiMap<string, ChainIndex>();
-        const residuesSortedByLabelSeqId = new Map<ChainIndex, Sorting<ResidueIndex, number>>();
-        const residuesSortedByAuthSeqId = new Map<ChainIndex, Sorting<ResidueIndex, number>>();
-        const residuesByInsCode = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
-        const residuesByLabelCompId = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
-        let residuesByLabelCompIdIsPure = true;
-        const residuesByAuthCompId = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
-        let residuesByAuthCompIdIsPure = true;
-        const atomsById = new NumberMap<number, ElementIndex>(nAtoms + 1);
-        const atomsByIndex = new NumberMap<number, ElementIndex>(nAtoms);
+    const chainsByLabelEntityId = new MultiMap<string, ChainIndex>();
+    const chainsByLabelAsymId = new MultiMap<string, ChainIndex>();
+    const chainsByAuthAsymId = new MultiMap<string, ChainIndex>();
+    const residuesSortedByLabelSeqId = new Map<ChainIndex, Sorting<ResidueIndex, number>>();
+    const residuesSortedByAuthSeqId = new Map<ChainIndex, Sorting<ResidueIndex, number>>();
+    const residuesSortedBySourceIndex = new Map<ChainIndex, Sorting<ResidueIndex, number>>();
+    const residuesByInsCode = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
+    const residuesByLabelCompId = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
+    let residuesByLabelCompIdIsPure = true;
+    const residuesByAuthCompId = new Map<ChainIndex, MultiMap<string, ResidueIndex>>();
+    let residuesByAuthCompIdIsPure = true;
+    const atomsById = new NumberMap<number, ElementIndex>(nAtoms + 1);
+    const atomsBySourceIndex = new NumberMap<number, ElementIndex>(nAtoms);
 
-        const _labelCompIdSet = new Set<string>();
-        const _authCompIdSet = new Set<string>();
+    const _labelCompIdSet = new Set<string>();
+    const _authCompIdSet = new Set<string>();
 
-        for (let iChain = 0 as ChainIndex; iChain < nChains; iChain++) {
-            chainsByLabelEntityId.add(label_entity_id.value(iChain), iChain);
-            chainsByLabelAsymId.add(label_asym_id.value(iChain), iChain);
-            chainsByAuthAsymId.add(auth_asym_id.value(iChain), iChain);
+    for (let iChain = 0 as ChainIndex; iChain < nChains; iChain++) {
+        chainsByLabelEntityId.add(label_entity_id.value(iChain), iChain);
+        chainsByLabelAsymId.add(label_asym_id.value(iChain), iChain);
+        chainsByAuthAsymId.add(auth_asym_id.value(iChain), iChain);
 
-            const iResFrom = h.residueAtomSegments.index[h.chainAtomSegments.offsets[iChain]];
-            const iResTo = h.residueAtomSegments.index[h.chainAtomSegments.offsets[iChain + 1] - 1] + 1;
+        const iResFrom = h.residueAtomSegments.index[h.chainAtomSegments.offsets[iChain]];
+        const iResTo = h.residueAtomSegments.index[h.chainAtomSegments.offsets[iChain + 1] - 1] + 1;
 
-            const residuesWithLabelSeqId = filterInPlace(range(iResFrom, iResTo) as ResidueIndex[], iRes => label_seq_id.valueKind(iRes) === Present);
-            residuesSortedByLabelSeqId.set(iChain, Sorting.create(residuesWithLabelSeqId, label_seq_id.value));
+        const residuesWithLabelSeqId = filterInPlace(range(iResFrom, iResTo) as ResidueIndex[], iRes => label_seq_id.valueKind(iRes) === Present);
+        residuesSortedByLabelSeqId.set(iChain, Sorting.create(residuesWithLabelSeqId, label_seq_id.value));
 
-            const residuesWithAuthSeqId = filterInPlace(range(iResFrom, iResTo) as ResidueIndex[], iRes => auth_seq_id.valueKind(iRes) === Present);
-            residuesSortedByAuthSeqId.set(iChain, Sorting.create(residuesWithAuthSeqId, auth_seq_id.value));
+        const residuesWithAuthSeqId = filterInPlace(range(iResFrom, iResTo) as ResidueIndex[], iRes => auth_seq_id.valueKind(iRes) === Present);
+        residuesSortedByAuthSeqId.set(iChain, Sorting.create(residuesWithAuthSeqId, auth_seq_id.value));
 
-            const residuesHereByInsCode = new MultiMap<string, ResidueIndex>();
-            const residuesHereByLabelCompId = new MultiMap<string, ResidueIndex>();
-            const residuesHereByAuthCompId = new MultiMap<string, ResidueIndex>();
-            for (let iRes = iResFrom; iRes < iResTo; iRes++) {
-                if (pdbx_PDB_ins_code.valueKind(iRes) === Present) {
-                    residuesHereByInsCode.add(pdbx_PDB_ins_code.value(iRes), iRes);
-                }
-                const iAtomFrom = h.residueAtomSegments.offsets[iRes];
-                const iAtomTo = h.residueAtomSegments.offsets[iRes + 1];
-                for (let iAtom = iAtomFrom; iAtom < iAtomTo; iAtom++) {
-                    _labelCompIdSet.add(label_comp_id.value(iAtom));
-                    _authCompIdSet.add(auth_comp_id.value(iAtom));
-                }
-                if (_labelCompIdSet.size > 1) residuesByLabelCompIdIsPure = false;
-                if (_authCompIdSet.size > 1) residuesByAuthCompIdIsPure = false;
-                for (const labelCompId of _labelCompIdSet) residuesHereByLabelCompId.add(labelCompId, iRes);
-                for (const authCompId of _authCompIdSet) residuesHereByAuthCompId.add(authCompId, iRes);
-                _labelCompIdSet.clear();
-                _authCompIdSet.clear();
+        const residuesWithSourceIndex = range(iResFrom, iResTo) as ResidueIndex[];
+        residuesSortedBySourceIndex.set(iChain, Sorting.create(residuesWithSourceIndex, h.residueSourceIndex.value));
+
+        const residuesHereByInsCode = new MultiMap<string, ResidueIndex>();
+        const residuesHereByLabelCompId = new MultiMap<string, ResidueIndex>();
+        const residuesHereByAuthCompId = new MultiMap<string, ResidueIndex>();
+        for (let iRes = iResFrom; iRes < iResTo; iRes++) {
+            if (pdbx_PDB_ins_code.valueKind(iRes) === Present) {
+                residuesHereByInsCode.add(pdbx_PDB_ins_code.value(iRes), iRes);
             }
-            residuesByInsCode.set(iChain, residuesHereByInsCode);
-            residuesByLabelCompId.set(iChain, residuesHereByLabelCompId);
-            residuesByAuthCompId.set(iChain, residuesHereByAuthCompId);
+            const iAtomFrom = h.residueAtomSegments.offsets[iRes];
+            const iAtomTo = h.residueAtomSegments.offsets[iRes + 1];
+            for (let iAtom = iAtomFrom; iAtom < iAtomTo; iAtom++) {
+                _labelCompIdSet.add(label_comp_id.value(iAtom));
+                _authCompIdSet.add(auth_comp_id.value(iAtom));
+            }
+            if (_labelCompIdSet.size > 1) residuesByLabelCompIdIsPure = false;
+            if (_authCompIdSet.size > 1) residuesByAuthCompIdIsPure = false;
+            for (const labelCompId of _labelCompIdSet) residuesHereByLabelCompId.add(labelCompId, iRes);
+            for (const authCompId of _authCompIdSet) residuesHereByAuthCompId.add(authCompId, iRes);
+            _labelCompIdSet.clear();
+            _authCompIdSet.clear();
         }
+        residuesByInsCode.set(iChain, residuesHereByInsCode);
+        residuesByLabelCompId.set(iChain, residuesHereByLabelCompId);
+        residuesByAuthCompId.set(iChain, residuesHereByAuthCompId);
+    }
 
-        const atomId = model.atomicConformation.atomId.value;
-        const atomIndex = h.atomSourceIndex.value;
-        for (let iAtom = 0 as ElementIndex; iAtom < nAtoms; iAtom++) {
-            atomsById.set(atomId(iAtom), iAtom);
-            atomsByIndex.set(atomIndex(iAtom), iAtom);
-        }
+    const atomId = model.atomicConformation.atomId.value;
+    const atomIndex = h.atomSourceIndex.value;
+    for (let iAtom = 0 as ElementIndex; iAtom < nAtoms; iAtom++) {
+        atomsById.set(atomId(iAtom), iAtom);
+        atomsBySourceIndex.set(atomIndex(iAtom), iAtom);
+    }
 
-        return {
-            chainsByLabelEntityId, chainsByLabelAsymId, chainsByAuthAsymId,
-            residuesSortedByLabelSeqId, residuesSortedByAuthSeqId, residuesByInsCode,
-            residuesByLabelCompId, residuesByLabelCompIdIsPure, residuesByAuthCompId, residuesByAuthCompIdIsPure,
-            atomsById, atomsByIndex,
-        };
+    return {
+        chainsByLabelEntityId, chainsByLabelAsymId, chainsByAuthAsymId,
+        residuesSortedByLabelSeqId, residuesSortedByAuthSeqId, residuesSortedBySourceIndex, residuesByInsCode,
+        residuesByLabelCompId, residuesByLabelCompIdIsPure, residuesByAuthCompId, residuesByAuthCompIdIsPure,
+        atomsById, atomsBySourceIndex,
+    };
+}
+
+
+/** Auxiliary data structure for efficiently finding chains/elements in a coarse model by their properties */
+export interface CoarseIndicesAndSortings {
+    /** Coarse equivalent to `model.atomicHierarchy.chains` */
+    chains: {
+        /** Number of chains */
+        count: number,
+        /** Maps chain index to `label_entity_id` value */
+        label_entity_id: string[],
+        /** Maps chain index to `label_asym_id` value */
+        label_asym_id: string[],
     },
-};
+    chainsByEntityId: Mapping<string, readonly ChainIndex[]>,
+    chainsByAsymId: Mapping<string, readonly ChainIndex[]>,
+    /** Coarse elements (per chain) sorted by `seq_id_begin`.
+     * This is used to get the range of elements which may overlap with a certain seq_id interval.
+     *
+     * (Filtering coarse elements by seq_id range is an interval search problem, so the worst-case-efficient solution would be to use a data structure optimized for that.
+     * But that would be overkill if we expect that in most cases the coarse elements cover non-overlapping seq_id ranges.
+     * So the current solution should be sufficient (fast for non-overlapping elements, while still correct if there are overlaps).) */
+    elementsSortedBySeqIdBegin: Mapping<ChainIndex,
+        Sorting<ElementIndex, number> & {
+            /** Non-decreasing upper bound for `seq_id_end` values of elements as listed in `keys` (`seq_id_end.value(keys[i]) <= endUpperBounds[i]`) */
+            endUpperBounds: SortedArray
+        }>,
+}
+
+/** Create `CoarseIndicesAndSortings` for a coarse elements hierarchy */
+function createCoarseIndicesAndSortings(coarseElements: CoarseElements): CoarseIndicesAndSortings | undefined {
+    if (coarseElements.count === 0) return undefined;
+    const { entity_id, asym_id, seq_id_begin, seq_id_end, chainElementSegments } = coarseElements;
+    const { Present } = Column.ValueKind;
+    const nChains = Math.max(chainElementSegments.count, 0); // chainElementSegments.count is -1 when there are no coarse elements
+
+    const chainsByEntityId = new MultiMap<string, ChainIndex>();
+    const chainsByAsymId = new MultiMap<string, ChainIndex>();
+    const elementsSortedBySeqIdBegin = new Map<ChainIndex, Sorting<ElementIndex, number> & { endUpperBounds: SortedArray }>();
+    const chains = {
+        count: nChains,
+        label_entity_id: new Array<string>(nChains),
+        label_asym_id: new Array<string>(nChains),
+    };
+
+    for (let iChain = 0 as ChainIndex; iChain < nChains; iChain++) {
+        const iElemFrom = chainElementSegments.offsets[iChain];
+        const iElemTo = chainElementSegments.offsets[iChain + 1];
+        const entityId = entity_id.value(iElemFrom);
+        const asymId = asym_id.value(iElemFrom);
+        chains.label_entity_id[iChain] = entityId;
+        chains.label_asym_id[iChain] = asymId;
+        chainsByEntityId.add(entityId, iChain);
+        chainsByAsymId.add(asymId, iChain);
+
+        const elementsWithSeqIds = filterInPlace(range(iElemFrom, iElemTo) as ElementIndex[], iElem => seq_id_begin.valueKind(iElem) === Present && seq_id_end.valueKind(iElem) === Present);
+        const sorting = Sorting.create(elementsWithSeqIds, seq_id_begin.value);
+        const endBounds = sorting.keys.map(seq_id_end.value);
+        // Ensure non-decreasing endBounds:
+        for (let i = 1; i < endBounds.length; i++) {
+            if (endBounds[i - 1] > endBounds[i]) {
+                endBounds[i] = endBounds[i - 1];
+            }
+        }
+        elementsSortedBySeqIdBegin.set(iChain, { ...sorting, endUpperBounds: SortedArray.ofSortedArray(endBounds) });
+    }
+
+    return { chains, chainsByEntityId, chainsByAsymId, elementsSortedBySeqIdBegin };
+}
 
 
 /** Represents a set of things (keys) of type `K`, sorted by some property (value) of type `V` */

src/extensions/mvs/helpers/label-text.ts

@@ -8,11 +8,12 @@ import { Sphere3D } from '../../../mol-math/geometry';
 import { BoundaryHelper } from '../../../mol-math/geometry/boundary-helper';
 import { Vec3 } from '../../../mol-math/linear-algebra';
 import { ElementIndex, Model, Structure, StructureElement, StructureProperties, Unit } from '../../../mol-model/structure';
+import { getPhysicalRadius } from '../../../mol-theme/size/physical';
 import { arrayExtend } from '../../../mol-util/array';
-import { AtomRanges } from './atom-ranges';
+import { ElementRanges } from './element-ranges';
 import { IndicesAndSortings } from './indexing';
 import { MVSAnnotationRow } from './schemas';
-import { getAtomRangesForRows } from './selections';
+import { getAtomRangesForRows, getGaussianRangesForRows, getSphereRangesForRows } from './selections';
 import { isDefined } from './utils';
 
 
@@ -24,63 +25,86 @@ export interface TextProps {
     depth: number,
     /** Relative text size */
     scale: number,
-    /** Index of the first atom within structure, to which this text is bound (for coloring and similar purposes) */
+    /** Index of the first element within structure, to which this text is bound (for coloring and similar purposes) */
     group: number,
 }
 
 const tmpVec = Vec3();
 const tmpArray: number[] = [];
 const boundaryHelper = new BoundaryHelper('98');
-const outAtoms: ElementIndex[] = [];
-const outFirstAtomIndex: { value?: number } = {};
+const outElements: ElementIndex[] = [];
+const outFirstElementIndex: { value?: number } = {};
 
-/** Helper for caching atom ranges qualifying to a group of annotation rows, per `Unit`. */
-class AtomRangesCache {
-    private readonly cache: { [key: string]: AtomRanges } = {};
+/** Helper for caching element ranges qualifying to a group of annotation rows, per `Unit`. */
+class ElementRangesCache {
+    private readonly cache: { [key: string]: ElementRanges } = {};
     private readonly hasOperators: boolean;
 
     constructor(private readonly rows: MVSAnnotationRow[]) {
         this.hasOperators = rows.some(row => isDefined(row.instance_id));
     }
 
-    get(unit: Unit): AtomRanges {
+    get(unit: Unit): ElementRanges {
         const instanceId = unit.conformation.operator.instanceId;
-        const key = this.hasOperators ? `${unit.model.id}:${instanceId}` : unit.model.id;
-        return this.cache[key] ??= getAtomRangesForRows(this.rows, unit.model, instanceId, IndicesAndSortings.get(unit.model));
+        const key = `${unit.model.id}:${unit.kind}:${this.hasOperators ? instanceId : '*'}`;
+        return this.cache[key] ??= this.compute(unit);
+    }
+    private compute(unit: Unit): ElementRanges {
+        const instanceId = unit.conformation.operator.instanceId;
+        const indices = IndicesAndSortings.get(unit.model);
+        switch (unit.kind) {
+            case Unit.Kind.Atomic:
+                return getAtomRangesForRows(this.rows, unit.model, instanceId, indices);
+            case Unit.Kind.Spheres:
+                return getSphereRangesForRows(this.rows, unit.model, instanceId, indices);
+            case Unit.Kind.Gaussians:
+                return getGaussianRangesForRows(this.rows, unit.model, instanceId, indices);
+        }
     }
 }
 
+/** Approximate number of heavy atoms per protein residue (I got 7.55 from 2e2n) */
+const AVG_ATOMS_PER_RESIDUE = 8;
+
 /** Return `TextProps` (position, size, etc.) for a text that is to be bound to a substructure of `structure` defined by union of `rows`.
  * Derives `center` and `depth` from the boundary sphere of the substructure, `scale` from the number of heavy atoms in the substructure. */
-export function textPropsForSelection(structure: Structure, sizeFunction: (location: StructureElement.Location) => number, rows: MVSAnnotationRow[], onlyInModel?: Model): TextProps | undefined {
+export function textPropsForSelection(structure: Structure, rows: MVSAnnotationRow[], onlyInModel?: Model): TextProps | undefined {
     const loc = StructureElement.Location.create(structure);
     const { units } = structure;
     const { type_symbol } = StructureProperties.atom;
     tmpArray.length = 0;
-    let includedAtoms = 0;
+    let includedElements = 0;
     let includedHeavyAtoms = 0;
     let group: number | undefined = undefined;
-    let atomSize: number | undefined = undefined;
-    const atomRangesCache = new AtomRangesCache(rows);
+    /** Used for `depth` in case the selection has only 1 element (hence bounding sphere radius is 0) */
+    let singularRadius: number | undefined = undefined;
+    const elementRangesCache = new ElementRangesCache(rows);
     for (let iUnit = 0, nUnits = units.length; iUnit < nUnits; iUnit++) {
         const unit = units[iUnit];
         if (onlyInModel && unit.model.id !== onlyInModel.id) continue;
-        const ranges = atomRangesCache.get(unit);
+        const coarseElements = unit.kind === Unit.Kind.Spheres ? unit.model.coarseHierarchy.spheres : unit.kind === Unit.Kind.Gaussians ? unit.model.coarseHierarchy.gaussians : undefined;
+        const ranges = elementRangesCache.get(unit);
+        ElementRanges.selectElementsInRanges(unit.elements, ranges, outElements, outFirstElementIndex);
         loc.unit = unit;
-        AtomRanges.selectAtomsInRanges(unit.elements, ranges, outAtoms, outFirstAtomIndex);
-        for (const atom of outAtoms) {
-            loc.element = atom;
-            unit.conformation.position(atom, tmpVec);
-            arrayExtend(tmpArray, tmpVec);
-            group ??= structure.serialMapping.cumulativeUnitElementCount[iUnit] + outFirstAtomIndex.value!;
-            atomSize ??= sizeFunction(loc);
-            includedAtoms++;
-            if (type_symbol(loc) !== 'H') includedHeavyAtoms++;
+        for (const iElem of outElements) {
+            loc.element = iElem;
+            arrayExtend(tmpArray, unit.conformation.position(iElem, tmpVec));
+            group ??= structure.serialMapping.cumulativeUnitElementCount[iUnit] + outFirstElementIndex.value!;
+            singularRadius ??= getPhysicalRadius(unit, iElem) * 1.2;
+            if (coarseElements) {
+                // coarse
+                const nResidues = coarseElements.seq_id_end.value(iElem) - coarseElements.seq_id_begin.value(iElem) + 1;
+                includedHeavyAtoms += nResidues * AVG_ATOMS_PER_RESIDUE;
+            } else {
+                // atomic
+                if (type_symbol(loc) !== 'H') includedHeavyAtoms++;
+            }
+            includedElements++;
         }
     }
-    if (includedAtoms > 0) {
-        const { center, radius } = (includedAtoms > 1) ? boundarySphere(tmpArray) : { center: Vec3.fromArray(Vec3(), tmpArray, 0), radius: 1.1 * atomSize! };
-        const scale = (includedHeavyAtoms || includedAtoms) ** (1 / 3);
+    if (includedElements > 0) {
+        const { center, radius } = (includedElements > 1) ? boundarySphere(tmpArray) : { center: Vec3.fromArray(Vec3(), tmpArray, 0), radius: singularRadius! };
+        const scale = (includedHeavyAtoms || includedElements) ** (1 / 3);
         return { center, depth: radius, scale, group: group! };
     }
 }

src/extensions/mvs/helpers/schemas.ts

@@ -65,7 +65,8 @@ const AllAtomicCifAnnotationSchema = {
     end_auth_seq_id: int,
     label_comp_id: str,
     auth_comp_id: str,
-    // residue_index: int, // 0-based residue index in the source file // TODO this is defined in Python builder but not supported by Molstar yet
+    /** 0-based residue index in the source file */
+    residue_index: int,
 
     /** Atom name like 'CA', 'N', 'O'... */
     label_atom_id: str,
@@ -89,11 +90,11 @@ const FieldsForSchemas = {
     entity: ['group_id', 'label_entity_id'],
     chain: ['group_id', 'label_entity_id', 'label_asym_id'],
     auth_chain: ['group_id', 'auth_asym_id'],
-    residue: ['group_id', 'label_entity_id', 'label_asym_id', 'label_seq_id'],
-    auth_residue: ['group_id', 'auth_asym_id', 'auth_seq_id', 'pdbx_PDB_ins_code'],
+    residue: ['group_id', 'label_entity_id', 'label_asym_id', 'label_seq_id', 'residue_index'],
+    auth_residue: ['group_id', 'auth_asym_id', 'auth_seq_id', 'pdbx_PDB_ins_code', 'residue_index'],
     residue_range: ['group_id', 'label_entity_id', 'label_asym_id', 'beg_label_seq_id', 'end_label_seq_id'],
     auth_residue_range: ['group_id', 'auth_asym_id', 'beg_auth_seq_id', 'end_auth_seq_id'],
-    atom: ['group_id', 'label_entity_id', 'label_asym_id', 'label_seq_id', 'label_atom_id', 'type_symbol', 'atom_id', 'atom_index'],
-    auth_atom: ['group_id', 'auth_asym_id', 'auth_seq_id', 'pdbx_PDB_ins_code', 'auth_atom_id', 'type_symbol', 'atom_id', 'atom_index'],
+    atom: ['group_id', 'label_entity_id', 'label_asym_id', 'label_seq_id', 'residue_index', 'label_atom_id', 'type_symbol', 'atom_id', 'atom_index'],
+    auth_atom: ['group_id', 'auth_asym_id', 'auth_seq_id', 'pdbx_PDB_ins_code', 'residue_index', 'auth_atom_id', 'type_symbol', 'atom_id', 'atom_index'],
     all_atomic: Object.keys(AllAtomicCifAnnotationSchema) as (keyof typeof AllAtomicCifAnnotationSchema)[],
 } satisfies { [schema in MVSAnnotationSchema]: (keyof typeof AllAtomicCifAnnotationSchema)[] };

src/extensions/mvs/helpers/selections.ts

@@ -6,11 +6,13 @@
  */
 
 import { Column } from '../../../mol-data/db';
+import { SortedArray } from '../../../mol-data/int';
 import { ChainIndex, ElementIndex, Model, ResidueIndex, StructureElement } from '../../../mol-model/structure';
+import { CoarseElements } from '../../../mol-model/structure/model/properties/coarse';
 import { Expression } from '../../../mol-script/language/expression';
-import { arrayExtend, filterInPlace, range } from '../../../mol-util/array';
-import { AtomRanges } from './atom-ranges';
-import { IndicesAndSortings, Sorting } from './indexing';
+import { arrayExtend, filterInPlace, range, sortIfNeeded } from '../../../mol-util/array';
+import { ElementRanges } from './element-ranges';
+import { AtomicIndicesAndSortings, CoarseIndicesAndSortings, IndicesAndSortings, Sorting } from './indexing';
 import { MVSAnnotationRow } from './schemas';
 import { isAnyDefined, isDefined } from './utils';
 
@@ -18,67 +20,72 @@ import { isAnyDefined, isDefined } from './utils';
 const EmptyArray: readonly any[] = [];
 
 
-/** Return atom ranges in `model` which satisfy criteria given by `row` */
-export function getAtomRangesForRow(row: MVSAnnotationRow, model: Model, instanceId: string, indices: IndicesAndSortings): AtomRanges {
-    if (isDefined(row.instance_id) && row.instance_id !== instanceId) return AtomRanges.empty();
+// ATOMIC SELECTIONS
 
+/** Return atom ranges in `model` which satisfy criteria given by `row` */
+export function getAtomRangesForRow(row: MVSAnnotationRow, model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges | undefined {
+    if (!indices.atomic) return undefined;
+    if (isDefined(row.instance_id) && row.instance_id !== instanceId) return undefined;
+
+    const atomicIndices = indices.atomic;
     const h = model.atomicHierarchy;
     const nAtoms = h.atoms._rowCount;
+    if (nAtoms === 0) return undefined;
 
     const hasAtomIds = isAnyDefined(row.atom_id, row.atom_index);
     const hasAtomFilter = isAnyDefined(row.label_atom_id, row.auth_atom_id, row.type_symbol)
-        || isDefined(row.label_comp_id) && !indices.residuesByLabelCompIdIsPure
-        || isDefined(row.auth_comp_id) && !indices.residuesByAuthCompIdIsPure;
+        || isDefined(row.label_comp_id) && !atomicIndices.residuesByLabelCompIdIsPure
+        || isDefined(row.auth_comp_id) && !atomicIndices.residuesByAuthCompIdIsPure;
     const hasResidueFilter = isAnyDefined(row.label_seq_id, row.auth_seq_id, row.pdbx_PDB_ins_code,
         row.beg_label_seq_id, row.end_label_seq_id, row.beg_auth_seq_id, row.end_auth_seq_id,
-        row.label_comp_id, row.auth_comp_id);
+        row.label_comp_id, row.auth_comp_id, row.residue_index);
     const hasChainFilter = isAnyDefined(row.label_asym_id, row.auth_asym_id, row.label_entity_id);
 
     if (hasAtomIds) {
-        const theAtom = getTheAtomForRow(model, row, indices);
-        return theAtom !== undefined ? AtomRanges.single(theAtom, theAtom + 1 as ElementIndex) : AtomRanges.empty();
+        const theAtom = getTheAtomForRow(model, row, atomicIndices);
+        return theAtom !== undefined ? ElementRanges.single(theAtom, theAtom + 1 as ElementIndex) : undefined;
     }
 
     if (!hasChainFilter && !hasResidueFilter && !hasAtomFilter) {
-        return AtomRanges.single(0 as ElementIndex, nAtoms as ElementIndex);
+        return ElementRanges.single(0 as ElementIndex, nAtoms as ElementIndex);
     }
 
-    const qualifyingChains = getQualifyingChains(model, row, indices);
+    const qualifyingChains = getQualifyingChains(model, row, atomicIndices);
     if (!hasResidueFilter && !hasAtomFilter) {
         const chainOffsets = h.chainAtomSegments.offsets;
-        const ranges = AtomRanges.empty();
+        const ranges = ElementRanges.empty();
         for (const iChain of qualifyingChains) {
-            AtomRanges.add(ranges, chainOffsets[iChain], chainOffsets[iChain + 1]);
+            ElementRanges.add(ranges, chainOffsets[iChain], chainOffsets[iChain + 1]);
         }
         return ranges;
     }
 
-    const qualifyingResidues = getQualifyingResidues(model, row, indices, qualifyingChains);
+    const qualifyingResidues = getQualifyingResidues(model, row, atomicIndices, qualifyingChains);
     if (!hasAtomFilter) {
         const residueOffsets = h.residueAtomSegments.offsets;
-        const ranges = AtomRanges.empty();
+        const ranges = ElementRanges.empty();
         for (const iRes of qualifyingResidues) {
-            AtomRanges.add(ranges, residueOffsets[iRes], residueOffsets[iRes + 1]);
+            ElementRanges.add(ranges, residueOffsets[iRes], residueOffsets[iRes + 1]);
         }
         return ranges;
     }
 
-    const qualifyingAtoms = getQualifyingAtoms(model, row, indices, qualifyingResidues);
-    const ranges = AtomRanges.empty();
+    const qualifyingAtoms = getQualifyingAtoms(model, row, atomicIndices, qualifyingResidues);
+    const ranges = ElementRanges.empty();
     for (const iAtom of qualifyingAtoms) {
-        AtomRanges.add(ranges, iAtom, iAtom + 1 as ElementIndex);
+        ElementRanges.add(ranges, iAtom, iAtom + 1 as ElementIndex);
     }
     return ranges;
 }
 
 /** Return atom ranges in `model` which satisfy criteria given by any of `rows` (atoms that satisfy more rows are still included only once) */
-export function getAtomRangesForRows(rows: MVSAnnotationRow[], model: Model, instanceId: string, indices: IndicesAndSortings): AtomRanges {
-    return AtomRanges.union(rows.map(row => getAtomRangesForRow(row, model, instanceId, indices)));
+export function getAtomRangesForRows(rows: MVSAnnotationRow[], model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges {
+    return ElementRanges.union(rows.map(row => getAtomRangesForRow(row, model, instanceId, indices)));
 }
 
 
 /** Return an array of chain indexes which satisfy criteria given by `row` */
-function getQualifyingChains(model: Model, row: MVSAnnotationRow, indices: IndicesAndSortings): readonly ChainIndex[] {
+function getQualifyingChains(model: Model, row: MVSAnnotationRow, indices: AtomicIndicesAndSortings): readonly ChainIndex[] {
     const { auth_asym_id, label_entity_id, _rowCount: nChains } = model.atomicHierarchy.chains;
     let result: readonly ChainIndex[] | undefined = undefined;
     if (isDefined(row.label_asym_id)) {
@@ -103,10 +110,10 @@ function getQualifyingChains(model: Model, row: MVSAnnotationRow, indices: Indic
 }
 
 /** Return an array of residue indexes which satisfy criteria given by `row` */
-function getQualifyingResidues(model: Model, row: MVSAnnotationRow, indices: IndicesAndSortings, fromChains: readonly ChainIndex[]): ResidueIndex[] {
+function getQualifyingResidues(model: Model, row: MVSAnnotationRow, indices: AtomicIndicesAndSortings, fromChains: readonly ChainIndex[]): ResidueIndex[] {
     const { label_seq_id, auth_seq_id, pdbx_PDB_ins_code } = model.atomicHierarchy.residues;
     const { label_comp_id, auth_comp_id } = model.atomicHierarchy.atoms;
-    const { residueAtomSegments, chainAtomSegments } = model.atomicHierarchy;
+    const { residueAtomSegments, chainAtomSegments, residueSourceIndex } = model.atomicHierarchy;
     const { Present } = Column.ValueKind;
     const result: ResidueIndex[] = [];
     for (const iChain of fromChains) {
@@ -123,6 +130,14 @@ function getQualifyingResidues(model: Model, row: MVSAnnotationRow, indices: Ind
                 residuesHere = Sorting.getKeysWithValue(sorting, row.auth_seq_id);
             }
         }
+        if (isDefined(row.residue_index)) {
+            if (residuesHere) {
+                residuesHere = residuesHere.filter(i => residueSourceIndex.value(i) === row.residue_index);
+            } else {
+                const sorting = indices.residuesSortedBySourceIndex.get(iChain)!;
+                residuesHere = Sorting.getKeysWithValue(sorting, row.residue_index);
+            }
+        }
         if (isDefined(row.pdbx_PDB_ins_code)) {
             if (residuesHere) {
                 residuesHere = residuesHere.filter(i => pdbx_PDB_ins_code.value(i) === row.pdbx_PDB_ins_code);
@@ -197,7 +212,7 @@ function getQualifyingResidues(model: Model, row: MVSAnnotationRow, indices: Ind
 }
 
 /** Return an array of atom indexes which satisfy criteria given by `row` */
-function getQualifyingAtoms(model: Model, row: MVSAnnotationRow, indices: IndicesAndSortings, fromResidues: readonly ResidueIndex[]): ElementIndex[] {
+function getQualifyingAtoms(model: Model, row: MVSAnnotationRow, indices: AtomicIndicesAndSortings, fromResidues: readonly ResidueIndex[]): ElementIndex[] {
     const { label_atom_id, auth_atom_id, type_symbol, label_comp_id, auth_comp_id } = model.atomicHierarchy.atoms;
     const residueAtomSegments_offsets = model.atomicHierarchy.residueAtomSegments.offsets;
     const result: ElementIndex[] = [];
@@ -225,12 +240,12 @@ function getQualifyingAtoms(model: Model, row: MVSAnnotationRow, indices: Indice
 
 /** Return index of atom in `model` which satistfies criteria given by `row`, if any.
  * Only works when `row.atom_id` and/or `row.atom_index` is defined (otherwise use `getAtomRangesForRow`). */
-function getTheAtomForRow(model: Model, row: MVSAnnotationRow, indices: IndicesAndSortings): ElementIndex | undefined {
+function getTheAtomForRow(model: Model, row: MVSAnnotationRow, indices: AtomicIndicesAndSortings): ElementIndex | undefined {
     let iAtom: ElementIndex | undefined = undefined;
     if (!isDefined(row.atom_id) && !isDefined(row.atom_index)) throw new Error('ArgumentError: at least one of row.atom_id, row.atom_index must be defined.');
     if (isDefined(row.atom_id) && isDefined(row.atom_index)) {
         const a1 = indices.atomsById.get(row.atom_id);
-        const a2 = indices.atomsByIndex.get(row.atom_index);
+        const a2 = indices.atomsBySourceIndex.get(row.atom_index);
         if (a1 !== a2) return undefined;
         iAtom = a1;
     }
@@ -238,7 +253,7 @@ function getTheAtomForRow(model: Model, row: MVSAnnotationRow, indices: IndicesA
         iAtom = indices.atomsById.get(row.atom_id);
     }
     if (isDefined(row.atom_index)) {
-        iAtom = indices.atomsByIndex.get(row.atom_index);
+        iAtom = indices.atomsBySourceIndex.get(row.atom_index);
     }
     if (iAtom === undefined) return undefined;
     if (!atomQualifies(model, iAtom, row)) return undefined;
@@ -261,11 +276,13 @@ export function atomQualifies(model: Model, iAtom: ElementIndex, row: MVSAnnotat
     const label_seq_id = (h.residues.label_seq_id.valueKind(iRes) === Column.ValueKind.Present) ? h.residues.label_seq_id.value(iRes) : undefined;
     const auth_seq_id = (h.residues.auth_seq_id.valueKind(iRes) === Column.ValueKind.Present) ? h.residues.auth_seq_id.value(iRes) : undefined;
     const pdbx_PDB_ins_code = h.residues.pdbx_PDB_ins_code.value(iRes);
+    const residue_index = h.residueSourceIndex.value(iRes);
     if (!matches(row.label_seq_id, label_seq_id)) return false;
     if (!matches(row.auth_seq_id, auth_seq_id)) return false;
     if (!matches(row.pdbx_PDB_ins_code, pdbx_PDB_ins_code)) return false;
     if (!matchesRange(row.beg_label_seq_id, row.end_label_seq_id, label_seq_id)) return false;
     if (!matchesRange(row.beg_auth_seq_id, row.end_auth_seq_id, auth_seq_id)) return false;
+    if (!matches(row.residue_index, residue_index)) return false;
 
     const label_comp_id = h.atoms.label_comp_id.value(iAtom);
     const auth_comp_id = h.atoms.auth_comp_id.value(iAtom);
@@ -299,6 +316,124 @@ function matchesRange<T>(requiredMin: T | undefined | null, requiredMax: T | und
     return true;
 }
 
+
+// COARSE SELECTIONS
+
+/** Return sphere ranges in `model` which satisfy criteria given by `row` */
+export function getSphereRangesForRow(row: MVSAnnotationRow, model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges | undefined {
+    if (!indices.spheres) return undefined;
+    if (isDefined(row.instance_id) && row.instance_id !== instanceId) return undefined;
+    return getCoarseElementRangesForRow(row, model.coarseHierarchy.spheres, indices.spheres);
+}
+
+/** Return sphere ranges in `model` which satisfy criteria given by any of `rows` (spheres that satisfy more rows are still included only once) */
+export function getSphereRangesForRows(rows: MVSAnnotationRow[], model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges {
+    return ElementRanges.union(rows.map(row => getSphereRangesForRow(row, model, instanceId, indices)));
+}
+
+/** Return gaussian ranges in `model` which satisfy criteria given by `row` */
+export function getGaussianRangesForRow(row: MVSAnnotationRow, model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges | undefined {
+    if (!indices.gaussians) return undefined;
+    if (isDefined(row.instance_id) && row.instance_id !== instanceId) return undefined;
+    return getCoarseElementRangesForRow(row, model.coarseHierarchy.gaussians, indices.gaussians);
+}
+
+/** Return gaussian ranges in `model` which satisfy criteria given by any of `rows` (gaussians that satisfy more rows are still included only once) */
+export function getGaussianRangesForRows(rows: MVSAnnotationRow[], model: Model, instanceId: string, indices: IndicesAndSortings): ElementRanges {
+    return ElementRanges.union(rows.map(row => getGaussianRangesForRow(row, model, instanceId, indices)));
+}
+
+/** Return ranges of coarse elements (spheres or gaussians) which satisfy criteria given by `row` */
+export function getCoarseElementRangesForRow(row: MVSAnnotationRow, coarseElements: CoarseElements, indices: CoarseIndicesAndSortings): ElementRanges | undefined {
+    const nElements = coarseElements.count;
+    if (nElements === 0) return undefined;
+
+    const hasResidueFilter = isAnyDefined(row.label_seq_id, row.beg_label_seq_id, row.end_label_seq_id);
+    const hasChainFilter = isAnyDefined(row.label_asym_id, row.label_entity_id);
+    const hasInvalidFilter = isAnyDefined(
+        row.auth_asym_id,
+        row.auth_seq_id, row.pdbx_PDB_ins_code, row.beg_auth_seq_id, row.end_auth_seq_id, row.label_comp_id, row.auth_comp_id, row.residue_index,
+        row.label_atom_id, row.auth_atom_id, row.type_symbol, row.atom_id, row.atom_index);
+
+    if (hasInvalidFilter) {
+        printCoarseSelectorWarning();
+        return undefined;
+    }
+
+    if (!hasChainFilter && !hasResidueFilter) {
+        return ElementRanges.single(0 as ElementIndex, nElements as ElementIndex);
+    }
+
+    const qualifyingChains = getQualifyingCoarseChains(coarseElements, row, indices);
+    if (!hasResidueFilter) {
+        const chainOffsets = coarseElements.chainElementSegments.offsets;
+        const ranges = ElementRanges.empty();
+        for (const iChain of qualifyingChains) {
+            ElementRanges.add(ranges, chainOffsets[iChain], chainOffsets[iChain + 1]);
+        }
+        return ranges;
+    }
+
+    const qualifyingElements = getQualifyingCoarseElements(coarseElements, row, indices, qualifyingChains);
+    const ranges = ElementRanges.empty();
+    for (const iElem of qualifyingElements) {
+        ElementRanges.add(ranges, iElem, iElem + 1 as ElementIndex);
+    }
+    return ranges;
+}
+
+
+/** Return an array of chain indexes which satisfy criteria given by `row` */
+function getQualifyingCoarseChains(coarseElements: CoarseElements, row: MVSAnnotationRow, indices: CoarseIndicesAndSortings): readonly ChainIndex[] {
+    let result: readonly ChainIndex[] | undefined = undefined;
+    if (isDefined(row.label_asym_id)) {
+        result = indices.chainsByAsymId.get(row.label_asym_id) ?? EmptyArray;
+    }
+    if (isDefined(row.label_entity_id)) {
+        if (result) {
+            result = result.filter(iChain => coarseElements.entity_id.value(coarseElements.chainElementSegments.offsets[iChain]) === row.label_entity_id);
+        } else {
+            result = indices.chainsByEntityId.get(row.label_entity_id) ?? EmptyArray;
+        }
+    }
+    result ??= range(coarseElements.chainElementSegments.count) as ChainIndex[];
+    return result;
+}
+
+/** Return an array of residue indexes which satisfy criteria given by `row` */
+function getQualifyingCoarseElements(coarseElements: CoarseElements, row: MVSAnnotationRow, indices: CoarseIndicesAndSortings, fromChains: readonly ChainIndex[]): ElementIndex[] {
+    const result: ElementIndex[] = [];
+    for (const iChain of fromChains) {
+        const sorting = indices.elementsSortedBySeqIdBegin.get(iChain)!;
+        const queryStart = Math.max(row.label_seq_id ?? -Infinity, row.beg_label_seq_id ?? -Infinity);
+        const queryEnd = Math.min(row.label_seq_id ?? Infinity, row.end_label_seq_id ?? Infinity); // inclusive
+        const iStart = SortedArray.findPredecessorIndex(sorting.endUpperBounds, queryStart); // select elements potentially ending >=queryStart (necessary condition)
+        const iStop = SortedArray.findPredecessorIndex(sorting.values, queryEnd + 1); // select elements starting <=queryEnd (necessary and suffient condition) // exclusive
+
+        for (let i = iStart; i < iStop; i++) {
+            const iElem = sorting.keys[i];
+            if (coarseElements.seq_id_end.value(iElem) >= queryStart) { // rechecking seq_id_end, as the condition was not sufficient
+                result.push(iElem);
+            }
+        }
+        // This implementation can yield some elements even when queryStart>queryEnd (e.g. { beg_label_seq_id: 70, end_label_seq_id: 58, label_seq_id: 60 } -> sphere 51-100 qualifies ).
+        // This is on purpose, to have the same behavior as MolScript.
+    }
+    sortIfNeeded(result, iElem => iElem);
+    return result;
+}
+
+let coarseSelectorWarningPrinted = false;
+function printCoarseSelectorWarning() {
+    if (!coarseSelectorWarningPrinted) {
+        console.warn('Using unsupported selector fields (auth_asym_id, auth_seq_id, pdbx_PDB_ins_code, beg_auth_seq_id, end_auth_seq_id, label_comp_id, auth_comp_id, residue_index, label_atom_id, auth_atom_id, type_symbol, atom_id, atom_index) on a coarse structure. The resulting selection will be empty.');
+        coarseSelectorWarningPrinted = true;
+    }
+}
+
+
+// GENERAL
+
 /** Convert an annotation row into a MolScript expression */
 export function rowToExpression(row: MVSAnnotationRow): Expression {
     return StructureElement.Schema.toExpression(row);

src/extensions/mvs/helpers/utils.ts

@@ -152,4 +152,25 @@ export function collectMVSReferences<T extends StateObject.Ctor>(type: T[], depe
     }
 
     return ret;
+}
+
+export function getMVSReferenceObject<T extends StateObject.Ctor>(type: T[], dependencies: Record<string, StateObject> | undefined, ref: string): StateObject | undefined {
+    if (!dependencies) return undefined;
+
+    for (const key of Object.keys(dependencies)) {
+        const o = dependencies[key];
+        let okType = false;
+        for (const t of type) {
+            if (t.is(o)) {
+                okType = true;
+                break;
+            }
+        }
+        if (!okType || !o.tags) continue;
+        for (const tag of o.tags) {
+            if (tag.startsWith('mvs-ref:')) {
+                if (tag.substring(8) === ref) return o;
+            }
+        }
+    }
 }

src/extensions/mvs/load-helpers.ts

@@ -219,7 +219,7 @@ export function collectInlineTooltips(tree: MolstarSubtree<'structure'>, context
                         text: node.params.text,
                         selector: {
                             name: 'annotation',
-                            params: { annotationId: p.annotationId, fieldName: p.fieldName, fieldValues: p.fieldValues },
+                            params: { annotationId: p.annotationId, fieldName: p.fieldName, fieldValues: p.fieldValues, label: p.label || 'Annotation' },
                         },
                     });
                 }
@@ -253,7 +253,7 @@ export function collectInlineLabels(tree: MolstarSubtree<'structure'>, context:
                             params: {
                                 selector: {
                                     name: 'annotation',
-                                    params: { annotationId: p.annotationId, fieldName: p.fieldName, fieldValues: p.fieldValues },
+                                    params: { annotationId: p.annotationId, fieldName: p.fieldName, fieldValues: p.fieldValues, label: p.label || 'Annotation' },
                                 },
                             },
                         },
@@ -369,10 +369,20 @@ function representationPropsBase(node: MolstarSubtree<'representation'>): Partia
                 type: { name: 'cartoon', params: { alpha, tubularHelices: params.tubular_helices } },
                 sizeTheme: { name: 'uniform', params: { value: params.size_factor } },
             };
+        case 'backbone':
+            return {
+                type: { name: 'backbone', params: { alpha } },
+                sizeTheme: { name: 'uniform', params: { value: params.size_factor } },
+            };
         case 'ball_and_stick':
             return {
                 type: { name: 'ball-and-stick', params: { sizeFactor: (params.size_factor ?? 1) * 0.5, sizeAspectRatio: 0.5, alpha, ignoreHydrogens: params.ignore_hydrogens } },
             };
+        case 'line':
+            return {
+                type: { name: 'line', params: { alpha, ignoreHydrogens: params.ignore_hydrogens } },
+                sizeTheme: { name: 'uniform', params: { value: params.size_factor } },
+            };
         case 'spacefill':
             return {
                 type: { name: 'spacefill', params: { alpha, ignoreHydrogens: params.ignore_hydrogens } },
@@ -402,8 +412,8 @@ export function representationProps(node: MolstarSubtree<'representation'>): Par
     if (clip) {
         base.type!.params = { ...base.type?.params, clip };
     }
-    if (node.custom?.molstar_reprepresentation_params) {
-        base.type!.params = { ...base.type!.params, ...node.custom.molstar_reprepresentation_params };
+    if (node.custom?.molstar_representation_params) {
+        base.type!.params = { ...base.type!.params, ...node.custom.molstar_representation_params };
     }
     return base;
 }
@@ -484,8 +494,16 @@ function hasMolStarUseDefaultColoring(node: MolstarNode): boolean {
     return 'molstar_use_default_coloring' in node.custom || 'molstar_color_theme_name' in node.custom;
 }
 
+function customColoring(custom: any) {
+    if (custom?.molstar_use_default_coloring) return undefined;
+    return {
+        name: custom?.molstar_color_theme_name ?? undefined,
+        params: custom?.molstar_color_theme_params ?? {},
+    };
+}
+
 /** Create value for `colorTheme` prop for `StructureRepresentation3D` transformer from a representation node based on color* nodes in its subtree. */
-export function colorThemeForNode(node: MolstarSubtree<'color' | 'color_from_uri' | 'color_from_source' | 'representation'> | undefined, context: MolstarLoadingContext): StateTransformer.Params<StructureRepresentation3D>['colorTheme'] | undefined {
+export function colorThemeForNode(node: MolstarSubtree<'color' | 'color_from_uri' | 'color_from_source' | 'representation' | 'volume'> | undefined, context: MolstarLoadingContext): StateTransformer.Params<StructureRepresentation3D>['colorTheme'] | undefined {
     if (node?.kind === 'representation') {
         const children = getChildren(node).filter(c => c.kind === 'color' || c.kind === 'color_from_uri' || c.kind === 'color_from_source') as MolstarNode<'color' | 'color_from_uri' | 'color_from_source'>[];
         if (children.length === 0) {
@@ -494,12 +512,7 @@ export function colorThemeForNode(node: MolstarSubtree<'color' | 'color_from_uri
                 params: { value: decodeColor(DefaultColor) },
             };
         } else if (children.length === 1 && hasMolStarUseDefaultColoring(children[0])) {
-            if (children[0].custom?.molstar_use_default_coloring) return undefined;
-            const custom = children[0].custom;
-            return {
-                name: custom?.molstar_color_theme_name ?? undefined,
-                params: custom?.molstar_color_theme_params ?? {},
-            };
+            return customColoring(children[0].custom);
         } else if (children.length === 1 && appliesColorToWholeRepr(children[0])) {
             return colorThemeForNode(children[0], context);
         } else {
@@ -517,6 +530,10 @@ export function colorThemeForNode(node: MolstarSubtree<'color' | 'color_from_uri
         }
     }
     if (node?.kind === 'color') {
+        if (hasMolStarUseDefaultColoring(node)) {
+            return customColoring(node.custom);
+        }
+
         return {
             name: 'uniform',
             params: { value: decodeColor(node.params.color) },

src/extensions/mvs/load.ts

@@ -8,10 +8,10 @@
 
 import { PluginStateSnapshotManager } from '../../mol-plugin-state/manager/snapshots';
 import { PluginStateObject } from '../../mol-plugin-state/objects';
-import { Download, ParseCcp4, ParseCif } from '../../mol-plugin-state/transforms/data';
-import { CustomModelProperties, CustomStructureProperties, ModelFromTrajectory, StructureComponent, StructureFromModel, TrajectoryFromMmCif, TrajectoryFromPDB } from '../../mol-plugin-state/transforms/model';
+import { Download, ParseCif, ParseCcp4, ParseDx } from '../../mol-plugin-state/transforms/data';
+import { CoordinatesFromLammpstraj, CoordinatesFromXtc, CustomModelProperties, CustomStructureProperties, ModelFromTrajectory, StructureComponent, StructureFromModel, TrajectoryFromGRO, TrajectoryFromLammpsTrajData, TrajectoryFromMmCif, TrajectoryFromMOL, TrajectoryFromMOL2, TrajectoryFromPDB, TrajectoryFromSDF, TrajectoryFromXYZ } from '../../mol-plugin-state/transforms/model';
 import { StructureRepresentation3D, VolumeRepresentation3D } from '../../mol-plugin-state/transforms/representation';
-import { VolumeFromCcp4, VolumeFromDensityServerCif } from '../../mol-plugin-state/transforms/volume';
+import { VolumeFromCcp4, VolumeFromDensityServerCif, VolumeFromDx } from '../../mol-plugin-state/transforms/volume';
 import { PluginCommands } from '../../mol-plugin/commands';
 import { PluginContext } from '../../mol-plugin/context';
 import { PluginState } from '../../mol-plugin/state';
@@ -26,16 +26,18 @@ import { CustomLabelProps, CustomLabelRepresentationProvider } from './component
 import { CustomTooltipsProvider } from './components/custom-tooltips-prop';
 import { IsMVSModelProps, IsMVSModelProvider } from './components/is-mvs-model-prop';
 import { getPrimitiveStructureRefs, MVSBuildPrimitiveShape, MVSDownloadPrimitiveData, MVSInlinePrimitiveData, MVSShapeRepresentation3D } from './components/primitives';
+import { MVSTrajectoryWithCoordinates } from './components/trajectory';
 import { generateStateTransition } from './helpers/animation';
 import { IsHiddenCustomStateExtension } from './load-extensions/is-hidden-custom-state';
 import { NonCovalentInteractionsExtension } from './load-extensions/non-covalent-interactions';
 import { LoadingActions, LoadingExtension, loadTreeVirtual, UpdateTarget } from './load-generic';
 import { AnnotationFromSourceKind, AnnotationFromUriKind, collectAnnotationReferences, collectAnnotationTooltips, collectInlineLabels, collectInlineTooltips, colorThemeForNode, componentFromXProps, componentPropsFromSelector, isPhantomComponent, labelFromXProps, makeNearestReprMap, prettyNameFromSelector, representationProps, structureProps, transformAndInstantiateStructure, transformAndInstantiateVolume, volumeColorThemeForNode, volumeRepresentationProps } from './load-helpers';
 import { MVSData, MVSData_States, Snapshot, SnapshotMetadata } from './mvs-data';
-import { validateTree } from './tree/generic/tree-schema';
+import { MVSAnimationNode, MVSAnimationSchema } from './tree/animation/animation-tree';
+import { validateTree } from './tree/generic/tree-validation';
 import { convertMvsToMolstar, mvsSanityCheck } from './tree/molstar/conversion';
 import { MolstarNode, MolstarNodeParams, MolstarSubtree, MolstarTree, MolstarTreeSchema } from './tree/molstar/molstar-tree';
-import { type MVSTree, MVSTreeSchema } from './tree/mvs/mvs-tree';
+import { MVSTreeSchema } from './tree/mvs/mvs-tree';
 
 
 export interface MVSLoadOptions {
@@ -49,6 +51,7 @@ export interface MVSLoadOptions {
     sanityChecks?: boolean,
     /** Base for resolving relative URLs/URIs. May itself be a relative URL (relative to the window URL). */
     sourceUrl?: string,
+
     doNotReportErrors?: boolean
 }
 
@@ -64,6 +67,9 @@ async function _loadMVS(ctx: RuntimeContext, plugin: PluginContext, data: MVSDat
         const mvsExtensionLoaded = plugin.state.hasBehavior(MolViewSpec);
         if (!mvsExtensionLoaded) throw new Error('MolViewSpec extension is not loaded.');
 
+        // Stop any currently running audio
+        plugin.managers.markdownExtensions.audio.dispose();
+
         // Reset canvas props to default so that modifyCanvasProps works as expected
         resetCanvasProps(plugin);
 
@@ -73,18 +79,21 @@ async function _loadMVS(ctx: RuntimeContext, plugin: PluginContext, data: MVSDat
         for (let i = 0; i < multiData.snapshots.length; i++) {
             const snapshot = multiData.snapshots[i];
             const previousSnapshot = i > 0 ? multiData.snapshots[i - 1] : multiData.snapshots[multiData.snapshots.length - 1];
-            validateTree(MVSTreeSchema, snapshot.root, 'MVS');
+            validateTree(MVSTreeSchema, snapshot.root, 'MVS', plugin);
+            if (snapshot.animation) {
+                validateTree(MVSAnimationSchema, snapshot.animation, 'Animation', plugin);
+            }
             if (options.sanityChecks) mvsSanityCheck(snapshot.root);
             const molstarTree = convertMvsToMolstar(snapshot.root, options.sourceUrl);
-            validateTree(MolstarTreeSchema, molstarTree, 'Converted Molstar');
+            validateTree(MolstarTreeSchema, molstarTree, 'Converted Molstar', plugin);
             const entry = molstarTreeToEntry(
                 plugin,
                 molstarTree,
-                snapshot.root,
+                snapshot.animation,
                 { ...snapshot.metadata, previousTransitionDurationMs: previousSnapshot.metadata.transition_duration_ms },
                 options
             );
-            await assignStateTransition(ctx, plugin, entry, snapshot, options);
+            await assignStateTransition(ctx, plugin, entry, snapshot, options, i, multiData.snapshots.length);
             entries.push(entry);
 
             if (ctx.shouldUpdate) {
@@ -119,8 +128,8 @@ async function _loadMVS(ctx: RuntimeContext, plugin: PluginContext, data: MVSDat
     }
 }
 
-async function assignStateTransition(ctx: RuntimeContext, plugin: PluginContext, parentEntry: PluginStateSnapshotManager.Entry, parent: Snapshot, options: MVSLoadOptions = {}) {
-    const transitions = await generateStateTransition(ctx, parent);
+async function assignStateTransition(ctx: RuntimeContext, plugin: PluginContext, parentEntry: PluginStateSnapshotManager.Entry, parent: Snapshot, options: MVSLoadOptions, snapshotIndex: number, snapshotCount: number) {
+    const transitions = await generateStateTransition(ctx, parent, snapshotIndex, snapshotCount);
     if (!transitions?.frames.length) return;
 
     const animation: PluginState.StateTransition = {
@@ -131,11 +140,11 @@ async function assignStateTransition(ctx: RuntimeContext, plugin: PluginContext,
 
     for (let i = 0; i < transitions.frames.length; i++) {
         const frame = transitions.frames[i];
-        const molstarTree = convertMvsToMolstar(frame, options.sourceUrl);
+        const molstarTree = convertMvsToMolstar(frame[0], options.sourceUrl);
         const entry = molstarTreeToEntry(
             plugin,
             molstarTree,
-            frame,
+            parent.animation,
             { ...parent.metadata, previousTransitionDurationMs: transitions.frametimeMs },
             options
         );
@@ -143,14 +152,14 @@ async function assignStateTransition(ctx: RuntimeContext, plugin: PluginContext,
         StateTree.reuseTransformParams(entry.snapshot.data!.tree, parentEntry.snapshot.data!.tree);
 
         animation.frames.push({
-            durationInMs: transitions.frametimeMs,
+            durationInMs: frame[1],
             data: entry.snapshot.data!,
             camera: transitions.tree.params?.include_camera ? entry.snapshot.camera : undefined,
             canvas3d: transitions.tree.params?.include_canvas ? entry.snapshot.canvas3d : undefined,
         });
 
         if (ctx.shouldUpdate) {
-            await ctx.update({ message: 'Loading animation...', current: i + 1, max: transitions.frames.length });
+            await ctx.update({ message: `Loading animation for snapshot ${snapshotIndex + 1}/${snapshotCount}...`, current: i + 1, max: transitions.frames.length });
         }
     }
 
@@ -160,20 +169,24 @@ async function assignStateTransition(ctx: RuntimeContext, plugin: PluginContext,
 function molstarTreeToEntry(
     plugin: PluginContext,
     tree: MolstarTree,
-    mvsTree: MVSTree,
+    animation: MVSAnimationNode<'animation'> | undefined,
     metadata: SnapshotMetadata & { previousTransitionDurationMs?: number },
     options: { keepCamera?: boolean, extensions?: MolstarLoadingExtension<any>[] }
 ) {
     const context = MolstarLoadingContext.create();
     const snapshot = loadTreeVirtual(plugin, tree, MolstarLoadingActions, context, { replaceExisting: true, extensions: options?.extensions ?? BuiltinLoadingExtensions });
     snapshot.canvas3d = {
-        props: plugin.canvas3d ? modifyCanvasProps(plugin.canvas3d.props, context.canvas, mvsTree.custom) : undefined,
+        props: plugin.canvas3d ? modifyCanvasProps(plugin.canvas3d.props, context.canvas, animation) : undefined,
     };
     if (!options?.keepCamera) {
         snapshot.camera = createPluginStateSnapshotCamera(plugin, context, metadata);
     }
     snapshot.durationInMs = metadata.linger_duration_ms + (metadata.previousTransitionDurationMs ?? 0);
 
+    if (tree.custom?.molstar_on_load_markdown_commands) {
+        snapshot.onLoadMarkdownCommands = tree.custom.molstar_on_load_markdown_commands;
+    }
+
     const entryParams: PluginStateSnapshotManager.EntryParams = {
         key: metadata.key,
         name: metadata.title,
@@ -220,31 +233,75 @@ const MolstarLoadingActions: LoadingActions<MolstarTree, MolstarLoadingContext>
     },
     parse(updateParent: UpdateTarget, node: MolstarNode<'parse'>): UpdateTarget | undefined {
         const format = node.params.format;
-        if (format === 'cif') {
-            return UpdateTarget.apply(updateParent, ParseCif, {});
-        } else if (format === 'pdb') {
-            return updateParent;
-        } else if (format === 'map') {
-            return UpdateTarget.apply(updateParent, ParseCcp4, {});
-        } else {
-            console.error(`Unknown format in "parse" node: "${format}"`);
-            return undefined;
+        switch (format) {
+            case 'cif':
+                return UpdateTarget.apply(updateParent, ParseCif, {});
+            case 'pdb':
+            case 'pdbqt':
+            case 'gro':
+            case 'xyz':
+            case 'mol':
+            case 'sdf':
+            case 'mol2':
+            case 'xtc':
+            case 'lammpstrj':
+                return updateParent;
+            case 'map':
+                return UpdateTarget.apply(updateParent, ParseCcp4, {});
+            case 'dx':
+            case 'dxbin':
+                return UpdateTarget.apply(updateParent, ParseDx, {});
+            default:
+                console.error(`Unknown format in "parse" node: "${format}"`);
+                return undefined;
+        }
+    },
+    coordinates(updateParent: UpdateTarget, node: MolstarNode<'coordinates'>): UpdateTarget | undefined {
+        const format = node.params.format;
+        switch (format) {
+            case 'xtc':
+                return UpdateTarget.apply(updateParent, CoordinatesFromXtc);
+            case 'lammpstrj':
+                return UpdateTarget.apply(updateParent, CoordinatesFromLammpstraj);
+            default:
+                console.error(`Unknown format in "coordinates" node: "${format}"`);
+                return undefined;
         }
     },
     trajectory(updateParent: UpdateTarget, node: MolstarNode<'trajectory'>): UpdateTarget | undefined {
         const format = node.params.format;
-        if (format === 'cif') {
-            return UpdateTarget.apply(updateParent, TrajectoryFromMmCif, {
-                blockHeader: node.params.block_header ?? '', // Must set to '' because just undefined would get overwritten by createDefaults
-                blockIndex: node.params.block_index ?? undefined,
-            });
-        } else if (format === 'pdb') {
-            return UpdateTarget.apply(updateParent, TrajectoryFromPDB, {});
-        } else {
-            console.error(`Unknown format in "trajectory" node: "${format}"`);
-            return undefined;
+        switch (format) {
+            case 'cif':
+                return UpdateTarget.apply(updateParent, TrajectoryFromMmCif, {
+                    blockHeader: node.params.block_header ?? '', // Must set to '' because just undefined would get overwritten by createDefaults
+                    blockIndex: node.params.block_index ?? undefined,
+                });
+            case 'pdb':
+            case 'pdbqt':
+                return UpdateTarget.apply(updateParent, TrajectoryFromPDB, { isPdbqt: format === 'pdbqt' });
+            case 'gro':
+                return UpdateTarget.apply(updateParent, TrajectoryFromGRO);
+            case 'xyz':
+                return UpdateTarget.apply(updateParent, TrajectoryFromXYZ);
+            case 'mol':
+                return UpdateTarget.apply(updateParent, TrajectoryFromMOL);
+            case 'sdf':
+                return UpdateTarget.apply(updateParent, TrajectoryFromSDF);
+            case 'mol2':
+                return UpdateTarget.apply(updateParent, TrajectoryFromMOL2);
+            case 'lammpstrj':
+                return UpdateTarget.apply(updateParent, TrajectoryFromLammpsTrajData);
+            default:
+                console.error(`Unknown format in "trajectory" node: "${format}"`);
+                return undefined;
         }
     },
+    trajectory_with_coordinates(updateParent: UpdateTarget, node: MolstarNode<'trajectory_with_coordinates'>): UpdateTarget | undefined {
+        const result = UpdateTarget.apply(updateParent, MVSTrajectoryWithCoordinates, {
+            coordinatesRef: node.params.coordinates_ref,
+        });
+        return UpdateTarget.setMvsDependencies(result, [node.params.coordinates_ref]);
+    },
     model(updateParent: UpdateTarget, node: MolstarSubtree<'model'>, context: MolstarLoadingContext): UpdateTarget {
         const annotations = collectAnnotationReferences(node, context);
         const model = UpdateTarget.apply(updateParent, ModelFromTrajectory, {
@@ -327,6 +384,8 @@ const MolstarLoadingActions: LoadingActions<MolstarTree, MolstarLoadingContext>
         let volume: UpdateTarget;
         if (updateParent.transformer?.definition.to.includes(PluginStateObject.Format.Ccp4)) {
             volume = UpdateTarget.apply(updateParent, VolumeFromCcp4, {});
+        } else if (updateParent.transformer?.definition.to.includes(PluginStateObject.Format.Dx)) {
+            volume = UpdateTarget.apply(updateParent, VolumeFromDx, {});
         } else if (updateParent.transformer?.definition.to.includes(PluginStateObject.Format.Cif)) {
             volume = UpdateTarget.apply(updateParent, VolumeFromDensityServerCif, { blockHeader: node.params.channel_id || undefined });
         } else {

src/extensions/mvs/mvs-data.ts

@@ -5,7 +5,7 @@
  * @author David Sehnal <david.sehnal@gmail.com>
  */
 
-import { treeValidationIssues } from './tree/generic/tree-schema';
+import { treeValidationIssues } from './tree/generic/tree-validation';
 import { treeToString } from './tree/generic/tree-utils';
 import { MVSAnimationSchema, MVSAnimationTree } from './tree/animation/animation-tree';
 import { Root, createMVSBuilder } from './tree/mvs/mvs-builder';

src/extensions/mvs/tree/animation/animation-tree.ts

@@ -4,7 +4,7 @@
  * @author David Sehnal <david.sehnal@gmail.com>
  */
 
-import { bool, float, int, list, OptionalField, RequiredField, str, union, nullable, literal, ValueFor } from '../generic/field-schema';
+import { bool, float, int, list, OptionalField, RequiredField, str, union, nullable, literal, ValueFor, dict } from '../generic/field-schema';
 import { SimpleParamsSchema, UnionParamsSchema } from '../generic/params-schema';
 import { NodeFor, ParamsOfKind, SubtreeOfKind, TreeFor, TreeSchema } from '../generic/tree-schema';
 import { ColorT, ContinuousPalette, DiscretePalette, Matrix, Vector3 } from '../mvs/param-types';
@@ -48,6 +48,7 @@ const ScalarInterpolation = {
     ..._Easing,
     start: OptionalField(nullable(union(float, list(float))), null, 'Start value. If a list of values is provided, each element will be interpolated separately. If unset, parent state value is used.'),
     end: OptionalField(nullable(union(float, list(float))), null, 'End value. If a list of values is provided, each element will be interpolated separately. If unset, only noise is applied.'),
+    discrete: OptionalField(bool, false, 'Whether to round the values to the closest integer. Useful for example for trajectory animation.'),
     ..._Noise,
 };
 
@@ -74,8 +75,8 @@ const ColorInterpolation = {
     ..._Common,
     ..._Frequency,
     ..._Easing,
-    start: OptionalField(nullable(ColorT), null, 'Start value. If unset, parent state value is used.'),
-    end: OptionalField(nullable(ColorT), null, 'End value.'),
+    start: OptionalField(union(nullable(ColorT), dict(union(int, str), ColorT)), null, 'Start value. If unset, parent state value is used.'),
+    end: OptionalField(union(nullable(ColorT), dict(union(int, str), ColorT)), null, 'End value.'),
     palette: OptionalField(nullable(union(DiscretePalette, ContinuousPalette)), null, 'Palette to sample colors from. Overrides start and end values.'),
 };
 

src/extensions/mvs/tree/generic/tree-schema.ts

@@ -4,12 +4,8 @@
  * @author Adam Midlik <midlik@gmail.com>
  */
 
-import { onelinerJsonString } from '../../../../mol-util/json';
-import { isPlainObject, mapObjectMap } from '../../../../mol-util/object';
-import { Field } from './field-schema';
-import { AllRequired, ParamsSchema, SimpleParamsSchema, ValuesFor, paramsValidationIssues } from './params-schema';
-import { treeToString } from './tree-utils';
-
+import { mapObjectMap } from '../../../../mol-util/object';
+import { AllRequired, ParamsSchema, ValuesFor } from './params-schema';
 
 /** Type of "custom" of a tree node (key-value storage with arbitrary JSONable values) */
 export type CustomProps = Partial<Record<string, any>>
@@ -114,120 +110,3 @@ export type NodeFor<TTreeSchema extends TreeSchema, TKind extends keyof ParamsSc
 
 /** Type of tree which conforms to tree schema `TTreeSchema` */
 export type TreeFor<TTreeSchema extends TreeSchema> = Tree<NodeFor<TTreeSchema>, RootFor<TTreeSchema> & NodeFor<TTreeSchema>>
-
-
-/** Return `undefined` if a tree conforms to the given schema,
- * return validation issues (as a list of lines) if it does not conform.
- * If `options.requireAll`, all parameters (including optional) must have a value provided.
- * If `options.noExtra` is true, presence of any extra parameters is treated as an issue.
- * If `options.anyRoot` is true, the kind of the root node is not enforced.
- */
-export function treeValidationIssues(schema: TreeSchema, tree: Tree, options: { requireAll?: boolean, noExtra?: boolean, anyRoot?: boolean, parent?: string } = {}): string[] | undefined {
-    if (!isPlainObject(tree)) return [`Node must be an object, not ${tree}`];
-    if (!options.anyRoot && tree.kind !== schema.rootKind) return [`Invalid root node kind "${tree.kind}", root must be of kind "${schema.rootKind}"`];
-    const nodeSchema = schema.nodes[tree.kind];
-    if (!nodeSchema) return [`Unknown node kind "${tree.kind}"`];
-    if (nodeSchema.parent && (options.parent !== undefined) && !nodeSchema.parent.includes(options.parent)) {
-        return [`Node of kind "${tree.kind}" cannot appear as a child of "${options.parent}". Allowed parents for "${tree.kind}" are: ${nodeSchema.parent.map(s => `"${s}"`).join(', ')}`];
-    }
-    const issues = paramsValidationIssues(nodeSchema.params, getParams(tree), options);
-    if (issues) return [`Invalid parameters for node of kind "${tree.kind}":`, ...issues.map(s => '  ' + s)];
-    if (tree.custom !== undefined && (typeof tree.custom !== 'object' || tree.custom === null)) {
-        return [`Invalid "custom" for node of kind "${tree.kind}": must be an object, not ${tree.custom}.`];
-    }
-    for (const child of getChildren(tree)) {
-        const issues = treeValidationIssues(schema, child, { ...options, anyRoot: true, parent: tree.kind });
-        if (issues) return issues;
-    }
-    return undefined;
-}
-
-/** Validate a tree against the given schema.
- * Do nothing if OK; print validation issues on console and throw an error is the tree does not conform.
- * Include `label` in the printed output. */
-export function validateTree(schema: TreeSchema, tree: Tree, label: string): void {
-    const issues = treeValidationIssues(schema, tree, { noExtra: true });
-    if (issues) {
-        console.warn(`Invalid ${label} tree:\n${treeToString(tree)}`);
-        console.error(`${label} tree validation issues:`);
-        for (const line of issues) {
-            console.error(' ', line);
-        }
-        throw new Error('FormatError');
-    }
-}
-
-/** Return documentation for a tree schema as plain text */
-export function treeSchemaToString<S extends TreeSchema>(schema: S): string {
-    return treeSchemaToString_(schema, false);
-}
-/** Return documentation for a tree schema as markdown text */
-export function treeSchemaToMarkdown<S extends TreeSchema>(schema: S): string {
-    return treeSchemaToString_(schema, true);
-}
-function treeSchemaToString_<S extends TreeSchema>(schema: S, markdown: boolean = false): string {
-    const out: string[] = [];
-    const bold = (str: string) => markdown ? `**${str}**` : str;
-    const code = (str: string) => markdown ? `\`${str}\`` : str;
-    const h1 = markdown ? '## ' : '  - ';
-    const p1 = markdown ? '' : '    ';
-    const h2 = markdown ? '- ' : '      - ';
-    const p2 = markdown ? '  ' : '        ';
-    const h3 = markdown ? '  - ' : '          - ';
-    const p3 = markdown ? '    ' : '            ';
-    const newline = markdown ? '\n\n' : '\n';
-    out.push(`Tree schema:`);
-    for (const kind in schema.nodes) {
-        const { description, params, parent } = schema.nodes[kind];
-        out.push(`${h1}${code(kind)}`);
-        if (kind === schema.rootKind) {
-            out.push(`${p1}[Root of the tree must be of this kind]`);
-        }
-        if (description) {
-            out.push(`${p1}${description}`);
-        }
-        out.push(`${p1}Parent: ${!parent ? 'any' : parent.length === 0 ? 'none' : parent.map(code).join(' or ')}`);
-        out.push(`${p1}Params:${Object.keys(params).length > 0 ? '' : ' none'}`);
-        if (params.type === 'simple') {
-            formatSimpleParams(out, params, { h: h2, p: p2, code, bold });
-        } else {
-            const key = params.discriminator;
-            const casesStr = Object.keys(params.cases).join(' | ');
-            out.push(`${h2}${bold(code(key + ': '))}${code(casesStr)}`);
-            if (params.discriminatorDescription) {
-                out.push(`${p2}${params.discriminatorDescription}`);
-            }
-            out.push(`${p2}[This parameter determines the rest of parameters]`);
-            for (const case_ in params.cases) {
-                const caseStr = `${params.discriminator}: "${case_}"`;
-                out.push(`${p2}${bold(`Case ${code(caseStr)}:`)}`);
-                formatSimpleParams(out, params.cases[case_], { h: h3, p: p3, code, bold });
-            }
-        }
-    }
-    return out.join(newline);
-}
-
-function formatSimpleParams(out: string[], params: SimpleParamsSchema, formatting: { h: string, p: string, code: (str: string) => string, bold: (str: string) => string }): void {
-    const { h, p, code, bold } = formatting;
-    for (const key in params.fields) {
-        const field = params.fields[key];
-        out.push(`${h}${bold(code(key + (field.required ? ': ' : '?: ')))}${code(formatFieldType(field))}`);
-        const defaultValue = field.required ? undefined : field.default;
-        if (field.description) {
-            out.push(`${p}${field.description}`);
-        }
-        if (defaultValue !== undefined) {
-            out.push(`${p}Default: ${code(onelinerJsonString(defaultValue))}`);
-        }
-    }
-}
-
-function formatFieldType(field: Field): string {
-    const typeString = field.type.name;
-    if (typeString.startsWith('(') && typeString.endsWith(')')) {
-        return typeString.slice(1, -1);
-    } else {
-        return typeString;
-    }
-}

src/extensions/mvs/tree/generic/tree-utils.ts

@@ -86,7 +86,7 @@ export function copyTree<T extends Tree>(root: T): T {
  * nodes of kind `C` will be converted to `Y` with a child `Z` (original children moved to `Z`),
  * nodes of other kinds will just be copied. */
 export type ConversionRules<A extends Tree, B extends Tree> = {
-    [kind in Kind<Subtree<A>>]?: (node: SubtreeOfKind<A, kind>, parent?: Subtree<A>) => Subtree<B>[]
+    [kind in Kind<Subtree<A>>]?: (node: SubtreeOfKind<A, kind>, parent?: Subtree<A>) => { subtree: Subtree<B>[] }
 };
 
 /** Apply a set of conversion rules to a tree to change to a different schema. */
@@ -94,12 +94,12 @@ export function convertTree<A extends Tree, B extends Tree>(root: A, conversions
     const mapping = new Map<Subtree<A>, Subtree<B>>();
     let convertedRoot: Subtree<B>;
     dfs<A>(root, (node, parent) => {
-        const conversion = conversions[node.kind as (typeof node)['kind']] as ((n: typeof node, p?: Subtree<A>) => Subtree<B>[]) | undefined;
+        const conversion = conversions[node.kind as (typeof node)['kind']] as ((n: typeof node, p?: Subtree<A>) => { subtree: Subtree<B>[] }) | undefined;
         if (conversion) {
-            const convertidos = conversion(node, parent);
-            if (!parent && convertidos.length === 0) throw new Error('Cannot convert root to empty path');
+            const converted = conversion(node, parent);
+            if (!parent && converted?.subtree.length === 0) throw new Error('Cannot convert root to empty path');
             let convParent = parent ? mapping.get(parent) : undefined;
-            for (const conv of convertidos) {
+            for (const conv of converted.subtree) {
                 if (convParent) {
                     (convParent.children ??= []).push(conv);
                 } else {
@@ -153,12 +153,14 @@ export function addDefaults<S extends TreeSchema>(tree: TreeFor<S>, treeSchema:
     type TTree = TreeFor<S>;
     const rules: ConversionRules<TTree, TTree> = {};
     for (const kind in treeSchema.nodes) {
-        rules[kind as Kind<Subtree<TTree>>] = node => [{
-            kind: node.kind,
-            params: addParamDefaults(treeSchema.nodes[kind].params, node.params as any),
-            custom: node.custom,
-            ref: node.ref,
-        } as Node as any];
+        rules[kind as Kind<Subtree<TTree>>] = node => ({
+            subtree: [{
+                kind: node.kind,
+                params: addParamDefaults(treeSchema.nodes[kind].params, node.params as any),
+                custom: node.custom,
+                ref: node.ref,
+            } as Node as any]
+        });
     }
     return convertTree(tree, rules) as any;
 }

src/extensions/mvs/tree/generic/tree-validation.ts

@@ -0,0 +1,125 @@
+import { PluginContext } from '../../../../mol-plugin/context';
+import { onelinerJsonString } from '../../../../mol-util/json';
+import { isPlainObject } from '../../../../mol-util/object';
+import { Field } from './field-schema';
+import { SimpleParamsSchema, paramsValidationIssues } from './params-schema';
+import { getChildren, getParams, Tree, TreeSchema } from './tree-schema';
+import { treeToString } from './tree-utils';
+
+/** Return `undefined` if a tree conforms to the given schema,
+ * return validation issues (as a list of lines) if it does not conform.
+ * If `options.requireAll`, all parameters (including optional) must have a value provided.
+ * If `options.noExtra` is true, presence of any extra parameters is treated as an issue.
+ * If `options.anyRoot` is true, the kind of the root node is not enforced.
+ */
+export function treeValidationIssues(schema: TreeSchema, tree: Tree, options: { requireAll?: boolean, noExtra?: boolean, anyRoot?: boolean, parent?: string } = {}): string[] | undefined {
+    if (!isPlainObject(tree)) return [`Node must be an object, not ${tree}`];
+    if (!options.anyRoot && tree.kind !== schema.rootKind) return [`Invalid root node kind "${tree.kind}", root must be of kind "${schema.rootKind}"`];
+    const nodeSchema = schema.nodes[tree.kind];
+    if (!nodeSchema) return [`Unknown node kind "${tree.kind}"`];
+    if (nodeSchema.parent && (options.parent !== undefined) && !nodeSchema.parent.includes(options.parent)) {
+        return [`Node of kind "${tree.kind}" cannot appear as a child of "${options.parent}". Allowed parents for "${tree.kind}" are: ${nodeSchema.parent.map(s => `"${s}"`).join(', ')}`];
+    }
+    const issues = paramsValidationIssues(nodeSchema.params, getParams(tree), options);
+    if (issues) return [`Invalid parameters for node of kind "${tree.kind}":`, ...issues.map(s => '  ' + s)];
+    if (tree.custom !== undefined && (typeof tree.custom !== 'object' || tree.custom === null)) {
+        return [`Invalid "custom" for node of kind "${tree.kind}": must be an object, not ${tree.custom}.`];
+    }
+    for (const child of getChildren(tree)) {
+        const issues = treeValidationIssues(schema, child, { ...options, anyRoot: true, parent: tree.kind });
+        if (issues) return issues;
+    }
+    return undefined;
+}
+
+/** Validate a tree against the given schema.
+ * Do nothing if OK; print validation issues on console and throw an error is the tree does not conform.
+ * Include `label` in the printed output. */
+export function validateTree(schema: TreeSchema, tree: Tree, label: string, plugin: PluginContext): void {
+    const issues = treeValidationIssues(schema, tree, { noExtra: true });
+    if (issues) {
+        console.warn(`Invalid ${label} tree:\n${treeToString(tree)}`);
+        console.error(`${label} tree validation issues:`);
+        plugin.log.error(`${label} tree validation issues:`);
+        for (const line of issues) {
+            console.error(' ', line);
+            plugin.log.error(line);
+        }
+        throw new Error('FormatError');
+    }
+}
+
+/** Return documentation for a tree schema as plain text */
+export function treeSchemaToString<S extends TreeSchema>(schema: S): string {
+    return treeSchemaToString_(schema, false);
+}
+/** Return documentation for a tree schema as markdown text */
+export function treeSchemaToMarkdown<S extends TreeSchema>(schema: S): string {
+    return treeSchemaToString_(schema, true);
+}
+function treeSchemaToString_<S extends TreeSchema>(schema: S, markdown: boolean = false): string {
+    const out: string[] = [];
+    const bold = (str: string) => markdown ? `**${str}**` : str;
+    const code = (str: string) => markdown ? `\`${str}\`` : str;
+    const h1 = markdown ? '## ' : '  - ';
+    const p1 = markdown ? '' : '    ';
+    const h2 = markdown ? '- ' : '      - ';
+    const p2 = markdown ? '  ' : '        ';
+    const h3 = markdown ? '  - ' : '          - ';
+    const p3 = markdown ? '    ' : '            ';
+    const newline = markdown ? '\n\n' : '\n';
+    out.push(`Tree schema:`);
+    for (const kind in schema.nodes) {
+        const { description, params, parent } = schema.nodes[kind];
+        out.push(`${h1}${code(kind)}`);
+        if (kind === schema.rootKind) {
+            out.push(`${p1}[Root of the tree must be of this kind]`);
+        }
+        if (description) {
+            out.push(`${p1}${description}`);
+        }
+        out.push(`${p1}Parent: ${!parent ? 'any' : parent.length === 0 ? 'none' : parent.map(code).join(' or ')}`);
+        out.push(`${p1}Params:${Object.keys(params).length > 0 ? '' : ' none'}`);
+        if (params.type === 'simple') {
+            formatSimpleParams(out, params, { h: h2, p: p2, code, bold });
+        } else {
+            const key = params.discriminator;
+            const casesStr = Object.keys(params.cases).join(' | ');
+            out.push(`${h2}${bold(code(key + ': '))}${code(casesStr)}`);
+            if (params.discriminatorDescription) {
+                out.push(`${p2}${params.discriminatorDescription}`);
+            }
+            out.push(`${p2}[This parameter determines the rest of parameters]`);
+            for (const case_ in params.cases) {
+                const caseStr = `${params.discriminator}: "${case_}"`;
+                out.push(`${p2}${bold(`Case ${code(caseStr)}:`)}`);
+                formatSimpleParams(out, params.cases[case_], { h: h3, p: p3, code, bold });
+            }
+        }
+    }
+    return out.join(newline);
+}
+
+function formatSimpleParams(out: string[], params: SimpleParamsSchema, formatting: { h: string, p: string, code: (str: string) => string, bold: (str: string) => string }): void {
+    const { h, p, code, bold } = formatting;
+    for (const key in params.fields) {
+        const field = params.fields[key];
+        out.push(`${h}${bold(code(key + (field.required ? ': ' : '?: ')))}${code(formatFieldType(field))}`);
+        const defaultValue = field.required ? undefined : field.default;
+        if (field.description) {
+            out.push(`${p}${field.description}`);
+        }
+        if (defaultValue !== undefined) {
+            out.push(`${p}Default: ${code(onelinerJsonString(defaultValue))}`);
+        }
+    }
+}
+
+function formatFieldType(field: Field): string {
+    const typeString = field.type.name;
+    if (typeString.startsWith('(') && typeString.endsWith(')')) {
+        return typeString.slice(1, -1);
+    } else {
+        return typeString;
+    }
+}

src/extensions/mvs/tree/molstar/conversion.ts

@@ -15,37 +15,78 @@ import { MolstarKind, MolstarNode, MolstarTree } from './molstar-tree';
 /** Convert `format` parameter of `parse` node in `MolstarTree`
  * into `format` and `is_binary` parameters in `MolstarTree` */
 export const ParseFormatMvsToMolstar = {
+    // trajectory
     mmcif: { format: 'cif', is_binary: false },
     bcif: { format: 'cif', is_binary: true },
     pdb: { format: 'pdb', is_binary: false },
+    pdbqt: { format: 'pdbqt', is_binary: false },
+    gro: { format: 'gro', is_binary: false },
+    xyz: { format: 'xyz', is_binary: false },
+    mol: { format: 'mol', is_binary: false },
+    sdf: { format: 'sdf', is_binary: false },
+    mol2: { format: 'mol2', is_binary: false },
+    lammpstrj: { format: 'lammpstrj', is_binary: false },
+    // coordinates
+    xtc: { format: 'xtc', is_binary: true },
+    // maps
     map: { format: 'map', is_binary: true },
+    dx: { format: 'dx', is_binary: false },
+    dxbin: { format: 'dxbin', is_binary: true },
 } satisfies { [p in ParseFormatT]: { format: MolstarParseFormatT, is_binary: boolean } };
 
-
 /** Conversion rules for conversion from `MVSTree` (with all parameter values) to `MolstarTree` */
 const mvsToMolstarConversionRules: ConversionRules<FullMVSTree, MolstarTree> = {
-    'download': node => [],
+    'download': node => ({ subtree: [] }),
     'parse': (node, parent) => {
         const { format, is_binary } = ParseFormatMvsToMolstar[node.params.format];
-        const convertedNode: MolstarNode<'parse'> = { kind: 'parse', params: { ...node.params, format }, custom: node.custom, ref: node.ref };
         if (parent?.kind === 'download') {
-            return [
-                { kind: 'download', params: { ...parent.params, is_binary }, custom: parent.custom, ref: parent.ref },
-                convertedNode,
-            ] satisfies MolstarNode[];
+            return {
+                subtree: [
+                    { kind: 'download', params: { ...parent.params, is_binary }, custom: parent.custom, ref: parent.ref },
+                    { kind: 'parse', params: { ...node.params, format }, custom: node.custom, ref: node.ref }
+                ] satisfies MolstarNode[]
+            };
         } else {
             console.warn('"parse" node is not being converted, this is suspicious');
-            return [convertedNode] satisfies MolstarNode[];
+            return {
+                subtree: [
+                    { kind: 'parse', params: { ...node.params, format }, custom: node.custom, ref: node.ref }
+                ] satisfies MolstarNode[]
+            };
         }
     },
+    'coordinates': (node, parent) => {
+        if (parent?.kind !== 'parse') throw new Error(`Parent of "coordinates" must be "parse", not "${parent?.kind}".`);
+        const { format } = ParseFormatMvsToMolstar[parent.params.format];
+        return {
+            subtree: [
+                { kind: 'coordinates', params: { format }, custom: node.custom, ref: node.ref }
+            ] satisfies MolstarNode[]
+        };
+    },
     'structure': (node, parent) => {
         if (parent?.kind !== 'parse') throw new Error(`Parent of "structure" must be "parse", not "${parent?.kind}".`);
         const { format } = ParseFormatMvsToMolstar[parent.params.format];
-        return [
-            { kind: 'trajectory', params: { format, ...pickObjectKeys(node.params, ['block_header', 'block_index']) } },
-            { kind: 'model', params: pickObjectKeys(node.params, ['model_index']) },
-            { kind: 'structure', params: omitObjectKeys(node.params, ['block_header', 'block_index', 'model_index']), custom: node.custom, ref: node.ref },
-        ] satisfies MolstarNode[];
+
+        if (node.params.coordinates_ref) {
+            return {
+                subtree: [
+                    { kind: 'trajectory', params: { format, ...pickObjectKeys(node.params, ['block_header', 'block_index']) } },
+                    { kind: 'model', params: { model_index: 0 } },
+                    { kind: 'trajectory_with_coordinates', params: { coordinates_ref: node.params.coordinates_ref } },
+                    { kind: 'model', params: pickObjectKeys(node.params, ['model_index']) },
+                    { kind: 'structure', params: omitObjectKeys(node.params, ['block_header', 'block_index', 'model_index', 'coordinates_ref']), custom: node.custom, ref: node.ref },
+                ] satisfies MolstarNode[]
+            };
+        } else {
+            return {
+                subtree: [
+                    { kind: 'trajectory', params: { format, ...pickObjectKeys(node.params, ['block_header', 'block_index']) } },
+                    { kind: 'model', params: pickObjectKeys(node.params, ['model_index']) },
+                    { kind: 'structure', params: omitObjectKeys(node.params, ['block_header', 'block_index', 'model_index', 'coordinates_ref']), custom: node.custom, ref: node.ref },
+                ] satisfies MolstarNode[]
+            };
+        }
     },
 };
 
@@ -70,10 +111,23 @@ function fileExtensionMatches(filename: string, extensions: (FileExtension | '*'
 }
 
 const StructureFormatExtensions: Record<ParseFormatT, (FileExtension | '*')[]> = {
+    // trajectory
     mmcif: ['.cif', '.mmif'],
     bcif: ['.bcif'],
     pdb: ['.pdb', '.ent'],
+    pdbqt: ['.pdbqt'],
+    gro: ['.gro'],
+    xyz: ['.xyz'],
+    mol: ['.mol'],
+    sdf: ['.sdf'],
+    mol2: ['.mol2'],
+    lammpstrj: ['.lammpstrj'],
+    // coordinates
+    xtc: ['.xtc'],
+    // volumes
     map: ['.map', '.ccp4', '.mrc', '.mrcs'],
+    dx: ['.dx'],
+    dxbin: ['.dxbin'],
 };
 
 /** Run some sanity check on a MVSTree. Return a list of potential problems (`undefined` if there are none) */

src/extensions/mvs/tree/molstar/molstar-tree.ts

@@ -5,7 +5,7 @@
  */
 
 import { omitObjectKeys, pickObjectKeys } from '../../../../mol-util/object';
-import { RequiredField, bool } from '../generic/field-schema';
+import { RequiredField, bool, str } from '../generic/field-schema';
 import { SimpleParamsSchema } from '../generic/params-schema';
 import { NodeFor, ParamsOfKind, SubtreeOfKind, TreeFor, TreeSchema } from '../generic/tree-schema';
 import { FullMVSTreeSchema } from '../mvs/mvs-tree';
@@ -30,6 +30,14 @@ export const MolstarTreeSchema = TreeSchema({
                 format: RequiredField(MolstarParseFormatT, 'File format'),
             }),
         },
+        /** Auxiliary node corresponding to Molstar's CoordinatesFrom*. */
+        coordinates: {
+            description: "Auxiliary node corresponding to Molstar's CoordinatesFrom*.",
+            parent: ['parse'],
+            params: SimpleParamsSchema({
+                format: RequiredField(MolstarParseFormatT, 'File format'),
+            }),
+        },
         /** Auxiliary node corresponding to Molstar's TrajectoryFrom*. */
         trajectory: {
             description: "Auxiliary node corresponding to Molstar's TrajectoryFrom*.",
@@ -39,10 +47,18 @@ export const MolstarTreeSchema = TreeSchema({
                 ...pickObjectKeys(FullMVSTreeSchema.nodes.structure.params.fields, ['block_header', 'block_index'] as const),
             }),
         },
+        /** Auxiliary node corresponding to Molstar's TrajectoryFrom*. */
+        trajectory_with_coordinates: {
+            description: 'Auxiliary node corresponding to assigning a separate coordinates to a trajectory.',
+            parent: ['model'],
+            params: SimpleParamsSchema({
+                coordinates_ref: RequiredField(str, 'Coordinates reference'),
+            }),
+        },
         /** Auxiliary node corresponding to Molstar's ModelFromTrajectory. */
         model: {
             description: "Auxiliary node corresponding to Molstar's ModelFromTrajectory.",
-            parent: ['trajectory'],
+            parent: ['trajectory', 'trajectory_with_coordinates'],
             params: SimpleParamsSchema(
                 pickObjectKeys(FullMVSTreeSchema.nodes.structure.params.fields, ['model_index'] as const)
             ),
@@ -52,7 +68,7 @@ export const MolstarTreeSchema = TreeSchema({
             ...FullMVSTreeSchema.nodes.structure,
             parent: ['model'],
             params: SimpleParamsSchema(
-                omitObjectKeys(FullMVSTreeSchema.nodes.structure.params.fields, ['block_header', 'block_index', 'model_index'] as const)
+                omitObjectKeys(FullMVSTreeSchema.nodes.structure.params.fields, ['block_header', 'block_index', 'model_index', 'coordinates_ref'] as const)
             ),
         },
     }

src/extensions/mvs/tree/mvs/mvs-builder.ts

@@ -7,9 +7,10 @@
 
 import { deepClone, pickObjectKeys } from '../../../../mol-util/object';
 import { GlobalMetadata, MVSData_State, Snapshot, SnapshotMetadata } from '../../mvs-data';
-import { CustomProps } from '../generic/tree-schema';
 import { MVSAnimationNodeParams, MVSAnimationSubtree } from '../animation/animation-tree';
+import { CustomProps } from '../generic/tree-schema';
 import { MVSKind, MVSNode, MVSNodeParams, MVSSubtree } from './mvs-tree';
+import { ColorT, PrimitivePositionT } from './param-types';
 
 
 /** Create a new MolViewSpec builder containing only a root node. Example of MVS builder usage:
@@ -92,12 +93,18 @@ export class Root extends _Base<'root'> implements FocusMixin, PrimitivesMixin {
     }
     focus = bindMethod(this, FocusMixinImpl, 'focus');
     primitives = bindMethod(this, PrimitivesMixinImpl, 'primitives');
-    primitives_from_uri = bindMethod(this, PrimitivesMixinImpl, 'primitives_from_uri');
+    primitivesFromUri = bindMethod(this, PrimitivesMixinImpl, 'primitivesFromUri');
 
     animation(params: MVSAnimationNodeParams<'animation'> & CustomAndRef = {}): Animation {
         this._animation ??= new Animation(params);
         return this._animation;
     }
+
+    /** Modifies custom state of the root */
+    extendRootCustomState(custom: Record<string, any>): this {
+        this._node.custom = { ...this._node.custom, ...custom };
+        return this;
+    }
 }
 
 export class Animation {
@@ -138,10 +145,10 @@ export class Download extends _Base<'download'> {
 
 /** Subsets of 'structure' node params which will be passed to individual builder functions. */
 const StructureParamsSubsets = {
-    model: ['block_header', 'block_index', 'model_index'],
-    assembly: ['block_header', 'block_index', 'model_index', 'assembly_id'],
-    symmetry: ['block_header', 'block_index', 'model_index', 'ijk_min', 'ijk_max'],
-    symmetry_mates: ['block_header', 'block_index', 'model_index', 'radius'],
+    model: ['block_header', 'block_index', 'model_index', 'coordinates_ref'],
+    assembly: ['block_header', 'block_index', 'model_index', 'assembly_id', 'coordinates_ref'],
+    symmetry: ['block_header', 'block_index', 'model_index', 'ijk_min', 'ijk_max', 'coordinates_ref'],
+    symmetry_mates: ['block_header', 'block_index', 'model_index', 'radius', 'coordinates_ref'],
 } satisfies { [kind in MVSNodeParams<'structure'>['type']]: (keyof MVSNodeParams<'structure'>)[] };
 
 
@@ -191,6 +198,11 @@ export class Parse extends _Base<'parse'> {
     volume(params: MVSNodeParams<'volume'> & CustomAndRef = {}): Volume {
         return new Volume(this._root, this.addChild('volume', params));
     }
+    /** Add a 'coordinates' node indicating the parsed data type */
+    coordinates(params: MVSNodeParams<'coordinates'> & CustomAndRef = {}): Parse {
+        this.addChild('coordinates', params);
+        return this;
+    }
 }
 
 
@@ -232,7 +244,7 @@ export class Structure extends _Base<'structure'> implements PrimitivesMixin, Tr
     transform = bindMethod(this, TransformMixinImpl, 'transform');
     instance = bindMethod(this, TransformMixinImpl, 'instance');
     primitives = bindMethod(this, PrimitivesMixinImpl, 'primitives');
-    primitives_from_uri = bindMethod(this, PrimitivesMixinImpl, 'primitives_from_uri');
+    primitivesFromUri = bindMethod(this, PrimitivesMixinImpl, 'primitivesFromUri');
 }
 
 
@@ -354,6 +366,11 @@ export class Primitives extends _Base<'primitives'> implements FocusMixin {
         this.addChild('primitive', { kind: 'distance_measurement', ...params });
         return this;
     }
+    /** Defines an angle between vectors (b - a) and (c - b). */
+    angle(params: MVSPrimitiveSubparams<'angle_measurement'> & CustomAndRef): Primitives {
+        this.addChild('primitive', { kind: 'angle_measurement', ...params });
+        return this;
+    }
     /** Defines a label. */
     label(params: MVSPrimitiveSubparams<'label'> & CustomAndRef): Primitives {
         this.addChild('primitive', { kind: 'label', ...params });
@@ -364,11 +381,22 @@ export class Primitives extends _Base<'primitives'> implements FocusMixin {
         this.addChild('primitive', { kind: 'ellipse', ...params });
         return this;
     }
-    /** Defines an ellipsoid */
+    /** Defines an ellipsoid. */
     ellipsoid(params: MVSPrimitiveSubparams<'ellipsoid'> & CustomAndRef): Primitives {
         this.addChild('primitive', { kind: 'ellipsoid', ...params });
         return this;
     }
+    /** Defines a sphere (a special case of ellipsoid). */
+    sphere(params: {
+        center: PrimitivePositionT,
+        radius?: number | null,
+        radius_extent?: number | null,
+        color?: ColorT | null,
+        tooltip?: string | null,
+    } & CustomAndRef): Primitives {
+        this.addChild('primitive', { kind: 'ellipsoid', ...params });
+        return this;
+    }
     /** Defines a box. */
     box(params: MVSPrimitiveSubparams<'box'> & CustomAndRef): Primitives {
         this.addChild('primitive', { kind: 'box', ...params });
@@ -414,13 +442,13 @@ interface PrimitivesMixin {
     /** Allows the definition of a (group of) geometric primitives. You can add any number of primitives and then assign shared options (color, opacity etc.). */
     primitives(params: MVSNodeParams<'primitives'> & CustomAndRef): Primitives,
     /** Allows the definition of a (group of) geometric primitives provided dynamically. */
-    primitives_from_uri(params: MVSNodeParams<'primitives_from_uri'> & CustomAndRef): PrimitivesFromUri,
+    primitivesFromUri(params: MVSNodeParams<'primitives_from_uri'> & CustomAndRef): PrimitivesFromUri,
 };
 class PrimitivesMixinImpl extends _Base<MVSKind> implements PrimitivesMixin {
     primitives(params: MVSNodeParams<'primitives'> & CustomAndRef = {}): Primitives {
         return new Primitives(this._root, this.addChild('primitives', params));
     }
-    primitives_from_uri(params: MVSNodeParams<'primitives_from_uri'> & CustomAndRef): PrimitivesFromUri {
+    primitivesFromUri(params: MVSNodeParams<'primitives_from_uri'> & CustomAndRef): PrimitivesFromUri {
         return new PrimitivesFromUri(this._root, this.addChild('primitives_from_uri', params));
     }
 };

src/extensions/mvs/tree/mvs/mvs-tree-primitives.ts

@@ -53,7 +53,7 @@ const LinesParams = {
     vertices: RequiredField(FloatList, '3*n_vertices length array of floats with vertex position (x1, y1, z1, ...).'),
     /** 2*n_lines length array of indices into vertices that form lines (l1_1, l1_2, ...). */
     indices: RequiredField(IntList, '2*n_lines length array of indices into vertices that form lines (l1_1, l1_2, ...).'),
-    /** Assign a number to each triangle to group them. If not specified, each line is considered a separate group (line i = group i). */
+    /** Assign a number to each line to group them. If not specified, each line is considered a separate group (line i = group i). */
     line_groups: OptionalField(nullable(IntList), null, 'Assign a number to each triangle to group them. If not specified, each line is considered a separate group (line i = group i).'),
     /** Assign a color to each group. Where not assigned, uses `color`. */
     group_colors: OptionalField(dict(int, ColorT), {}, 'Assign a color to each group. Where not assigned, uses `color`.'),

src/extensions/mvs/tree/mvs/mvs-tree-representations.ts

@@ -15,6 +15,11 @@ const Cartoon = {
     tubular_helices: OptionalField(bool, false, 'Simplify corkscrew helices to tubes.'),
 };
 
+const Backbone = {
+    /** Scales the corresponding visuals */
+    size_factor: OptionalField(float, 1, 'Scales the corresponding visuals.'),
+};
+
 const BallAndStick = {
     /** Scales the corresponding visuals */
     size_factor: OptionalField(float, 1, 'Scales the corresponding visuals.'),
@@ -22,6 +27,13 @@ const BallAndStick = {
     ignore_hydrogens: OptionalField(bool, false, 'Controls whether hydrogen atoms are drawn.'),
 };
 
+const Line = {
+    /** Scales the corresponding visuals */
+    size_factor: OptionalField(float, 1, 'Scales the corresponding visuals.'),
+    /** Controls whether hydrogen atoms are drawn. */
+    ignore_hydrogens: OptionalField(bool, false, 'Controls whether hydrogen atoms are drawn.'),
+};
+
 const Spacefill = {
     /** Scales the corresponding visuals */
     size_factor: OptionalField(float, 1, 'Scales the corresponding visuals.'),
@@ -48,7 +60,9 @@ export const MVSRepresentationParams = UnionParamsSchema(
     'Representation type',
     {
         cartoon: SimpleParamsSchema(Cartoon),
+        backbone: SimpleParamsSchema(Backbone),
         ball_and_stick: SimpleParamsSchema(BallAndStick),
+        line: SimpleParamsSchema(Line),
         spacefill: SimpleParamsSchema(Spacefill),
         carbohydrate: SimpleParamsSchema(Carbohydrate),
         surface: SimpleParamsSchema(Surface),

src/extensions/mvs/tree/mvs/mvs-tree.ts

@@ -92,6 +92,12 @@ export const MVSTreeSchema = TreeSchema({
                 format: RequiredField(ParseFormatT, 'Format of the input data resource.'),
             }),
         },
+        /** This node instructs to retrieve molecular coordinates from a parsed data resource. */
+        coordinates: {
+            description: 'This node instructs to retrieve molecular coordinates from a parsed data resource.',
+            parent: ['parse'],
+            params: SimpleParamsSchema({}),
+        },
         /** This node instructs to create a structure from a parsed data resource. "Structure" refers to an internal representation of molecular coordinates without any visual representation. */
         structure: {
             description: 'This node instructs to create a structure from a parsed data resource. "Structure" refers to an internal representation of molecular coordinates without any visual representation.',
@@ -113,6 +119,8 @@ export const MVSTreeSchema = TreeSchema({
                 ijk_min: OptionalField(tuple([int, int, int]), [-1, -1, -1], 'Miller indices of the bottom-left unit cell to be included (only applies when `kind` is `"symmetry"`).'),
                 /** Miller indices of the top-right unit cell to be included (only applies when `kind` is `"symmetry"`). */
                 ijk_max: OptionalField(tuple([int, int, int]), [1, 1, 1], 'Miller indices of the top-right unit cell to be included (only applies when `kind` is `"symmetry"`).'),
+                /** Reference to a specific set of coordinates. */
+                coordinates_ref: OptionalField(nullable(str), null, 'Reference to a specific set of coordinates.')
             }),
         },
         /** This node instructs to rotate and/or translate structure coordinates. */

src/extensions/mvs/tree/mvs/param-types.ts

@@ -11,11 +11,46 @@ import { ValueFor, bool, dict, float, int, list, literal, nullable, object, part
 
 
 /** `format` parameter values for `parse` node in MVS tree */
-export const ParseFormatT = literal('mmcif', 'bcif', 'pdb', 'map');
+export const ParseFormatT = literal(
+    // trajectory
+    'mmcif',
+    'bcif', // +volumes
+    'pdb',
+    'pdbqt',
+    'gro',
+    'xyz',
+    'mol',
+    'sdf',
+    'mol2',
+    'lammpstrj', // + coordinates
+    // coordinates
+    'xtc',
+    // volumes
+    'map',
+    'dx',
+    'dxbin',
+);
 export type ParseFormatT = ValueFor<typeof ParseFormatT>
 
 /** `format` parameter values for `parse` node in Molstar tree */
-export const MolstarParseFormatT = literal('cif', 'pdb', 'map');
+export const MolstarParseFormatT = literal(
+    // trajectory
+    'cif', // +volumes
+    'pdb',
+    'pdbqt',
+    'gro',
+    'xyz',
+    'mol',
+    'sdf',
+    'mol2',
+    'lammpstrj',
+    // coordinates
+    'xtc',
+    // volumes
+    'map',
+    'dx',
+    'dxbin',
+);
 export type MolstarParseFormatT = ValueFor<typeof MolstarParseFormatT>
 
 /** `kind` parameter values for `structure` node in MVS tree */
@@ -38,7 +73,8 @@ export const ComponentExpressionT = partial({
     end_auth_seq_id: int,
     label_comp_id: str,
     auth_comp_id: str,
-    // residue_index: int, // 0-based residue index in the source file // TODO this is defined in Python builder but not supported by Molstar yet
+    /** 0-based residue index in the source file */
+    residue_index: int, // TODO this is defined in Python builder but not supported by Molstar yet
     label_atom_id: str,
     auth_atom_id: str,
     type_symbol: str,

src/extensions/rcsb/validation-report/prop.ts

@@ -85,10 +85,10 @@ namespace ValidationReport {
         Clashes = 'rcsb-clashes',
     }
 
-    export const DefaultBaseUrl = 'https://files.rcsb.org/pub/pdb/validation_reports';
+    export const DefaultBaseUrl = 'https://files.rcsb.org/validation/view';
     export function getEntryUrl(pdbId: string, baseUrl: string) {
         const id = pdbId.toLowerCase();
-        return `${baseUrl}/${id.substr(1, 2)}/${id}/${id}_validation.xml.gz`;
+        return `${baseUrl}/${id}_validation.xml`;
     }
 
     export function isApplicable(model?: Model): boolean {

src/mol-canvas3d/camera.ts

@@ -28,6 +28,14 @@ interface ICamera {
     readonly fogFar: number,
     readonly fogNear: number,
     readonly headRotation: Mat4,
+    readonly viewEye: Mat4,
+    readonly isAsymmetricProjection: boolean,
+
+    readonly forceFull: boolean;
+    readonly scale: number;
+    readonly minTargetDistance: number;
+
+    readonly disabled: boolean;
 }
 
 const tmpClip = Vec4();
@@ -38,17 +46,25 @@ export class Camera implements ICamera {
     readonly projectionView: Mat4 = Mat4.identity();
     readonly inverseProjectionView: Mat4 = Mat4.identity();
     readonly headRotation: Mat4 = Mat4.zero();
+    readonly viewEye: Mat4 = Mat4.zero();
+    readonly isAsymmetricProjection = false;
 
     readonly viewport: Viewport;
     readonly state: Readonly<Camera.Snapshot> = Camera.createDefaultSnapshot();
     readonly viewOffset = Camera.ViewOffset();
 
+    readonly disabled = false as const;
+
     near = 1;
     far = 10000;
     fogNear = 5000;
     fogFar = 10000;
     zoom = 1;
 
+    forceFull = false;
+    scale = 1;
+    minTargetDistance = 0;
+
     readonly transition: CameraTransitionManager = new CameraTransitionManager(this);
     readonly stateChanged = new BehaviorSubject<Partial<Camera.Snapshot>>(this.state);
 
@@ -72,7 +88,15 @@ export class Camera implements ICamera {
             return false;
         }
 
-        const height = 2 * Math.tan(snapshot.fov / 2) * Vec3.distance(snapshot.position, snapshot.target) * this.state.scale;
+        const distance = Vec3.distance(snapshot.position, snapshot.target);
+        const minTargetDistance = this.minTargetDistance / this.scale;
+        if (distance < minTargetDistance) {
+            Vec3.sub(this.deltaDirection, snapshot.target, snapshot.position);
+            Vec3.setMagnitude(this.deltaDirection, this.deltaDirection, minTargetDistance);
+            Vec3.sub(snapshot.position, snapshot.target, this.deltaDirection);
+        }
+
+        const height = 2 * Math.tan(snapshot.fov / 2) * Vec3.distance(snapshot.position, snapshot.target) * this.scale;
         this.zoom = this.viewport.height / height;
 
         updateClip(this);
@@ -111,7 +135,7 @@ export class Camera implements ICamera {
     }
 
     getTargetDistance(radius: number) {
-        return Camera.targetDistance(radius, this.state.mode, this.state.fov, this.viewport.width, this.viewport.height);
+        return Math.max(this.minTargetDistance / this.scale, Camera.targetDistance(radius, this.state.mode, this.state.fov, this.viewport.width, this.viewport.height));
     }
 
     getFocus(target: Vec3, radius: number, up?: Vec3, dir?: Vec3, snapshot?: Partial<Camera.Snapshot>): Partial<Camera.Snapshot> {
@@ -202,7 +226,7 @@ export class Camera implements ICamera {
             Vec3.scaleAndAdd(out.origin, out.origin, out.direction, -this.near);
         } else {
             Vec3.copy(out.origin, this.state.position);
-            Vec3.scale(out.origin, out.origin, this.state.scale);
+            Vec3.scale(out.origin, out.origin, this.scale);
             Vec3.set(out.direction, x, y, 0.5);
             this.unproject(out.direction, out.direction);
             Vec3.normalize(out.direction, Vec3.sub(out.direction, out.direction, out.origin));
@@ -268,10 +292,11 @@ export namespace Camera {
         const r = Math.max(radius, 0.01);
         const aspect = width / height;
         const aspectFactor = (height < width ? 1 : aspect);
-        if (mode === 'orthographic')
+        if (mode === 'orthographic') {
             return Math.abs((r / aspectFactor) / Math.tan(fov / 2));
-        else
+        } else {
             return Math.abs((r / aspectFactor) / Math.sin(fov / 2));
+        }
     }
 
     export function createDefaultSnapshot(): Snapshot {
@@ -289,8 +314,6 @@ export namespace Camera {
             clipFar: true,
             minNear: 5,
             minFar: 0,
-
-            scale: 1,
         };
     }
 
@@ -308,8 +331,6 @@ export namespace Camera {
         clipFar: boolean
         minNear: number
         minFar: number
-
-        scale: number
     }
 
     export function copySnapshot(out: Snapshot, source?: Partial<Snapshot>) {
@@ -329,8 +350,6 @@ export namespace Camera {
         if (typeof source.minNear !== 'undefined') out.minNear = source.minNear;
         if (typeof source.minFar !== 'undefined') out.minFar = source.minFar;
 
-        if (typeof source.scale !== 'undefined') out.scale = source.scale;
-
         return out;
     }
 
@@ -343,7 +362,6 @@ export namespace Camera {
             && a.clipFar === b.clipFar
             && a.minNear === b.minNear
             && a.minFar === b.minFar
-            && a.scale === b.scale
             && Vec3.exactEquals(a.position, b.position)
             && Vec3.exactEquals(a.up, b.up)
             && Vec3.exactEquals(a.target, b.target);
@@ -354,11 +372,11 @@ const tmpPosition = Vec3();
 const tmpTarget = Vec3();
 
 function updateView(camera: Camera) {
-    if (camera.state.scale === 1) {
+    if (camera.scale === 1) {
         Mat4.lookAt(camera.view, camera.state.position, camera.state.target, camera.state.up);
     } else {
-        Vec3.scale(tmpPosition, camera.state.position, camera.state.scale);
-        Vec3.scale(tmpTarget, camera.state.target, camera.state.scale);
+        Vec3.scale(tmpPosition, camera.state.position, camera.scale);
+        Vec3.scale(tmpTarget, camera.state.target, camera.scale);
         Mat4.lookAt(camera.view, tmpPosition, tmpTarget, camera.state.up);
     }
 }
@@ -424,11 +442,13 @@ function updatePers(camera: Camera) {
 }
 
 function updateClip(camera: Camera) {
-    let { radius, radiusMax, mode, fog, clipFar, minNear, minFar, scale } = camera.state;
+    const { forceFull, scale } = camera;
+    let { radius, radiusMax, mode, fog, clipFar, minNear, minFar } = camera.state;
     radiusMax *= scale;
     minFar *= scale;
     minNear *= scale;
     radius *= scale;
+    if (forceFull) radius = radiusMax;
 
     const minRadius = 0.01 * scale;
     if (radius < minRadius) radius = minRadius;
@@ -437,8 +457,9 @@ function updateClip(camera: Camera) {
     Vec3.scale(tmpTarget, camera.state.target, scale);
     Vec3.scale(tmpPosition, camera.state.position, scale);
     const cameraDistance = Vec3.distance(tmpPosition, tmpTarget);
-    let near = cameraDistance - radius;
+    let near = forceFull ? 0.01 : cameraDistance - radius;
     let far = cameraDistance + normalizedFar;
+    if (forceFull) minNear = near;
 
     if (mode === 'perspective') {
         // set at least to 5 to avoid slow sphere impostor rendering

src/mol-canvas3d/camera/stereo.ts

@@ -7,10 +7,11 @@
  * Adapted from three.js, The MIT License, Copyright © 2010-2020 three.js authors
  */
 
-import { Mat4 } from '../../mol-math/linear-algebra';
+import { Ray3D } from '../../mol-math/geometry/primitives/ray3d';
+import { Mat4, Vec3 } from '../../mol-math/linear-algebra';
 import { ParamDefinition as PD } from '../../mol-util/param-definition';
 import { Camera, ICamera } from '../camera';
-import { Viewport } from './util';
+import { cameraUnproject, Viewport } from './util';
 
 export const StereoCameraParams = {
     eyeSeparation: PD.Numeric(0.062, { min: 0.02, max: 0.1, step: 0.001 }, { description: 'Distance between left and right camera.' }),
@@ -22,8 +23,8 @@ export type StereoCameraProps = PD.Values<typeof StereoCameraParams>
 export { StereoCamera };
 
 class StereoCamera {
-    readonly left: ICamera = new EyeCamera();
-    readonly right: ICamera = new EyeCamera();
+    readonly left = new EyeCamera();
+    readonly right = new EyeCamera();
 
     get viewport() {
         return this.parent.viewport;
@@ -43,9 +44,13 @@ class StereoCamera {
         Object.assign(this.props, props);
     }
 
-    update() {
+    update(xr?: { pose: XRViewerPose, layer: XRWebGLLayer }) {
         this.parent.update();
-        update(this.parent, this.props, this.left as EyeCamera, this.right as EyeCamera);
+        if (xr) {
+            xrUpdate(this.parent, this.left, this.right, xr);
+        } else {
+            update(this.parent, this.props, this.left, this.right);
+        }
     }
 }
 
@@ -62,12 +67,29 @@ class EyeCamera implements ICamera {
     projectionView = Mat4();
     inverseProjectionView = Mat4();
     headRotation = Mat4();
+    viewEye = Mat4();
+    isAsymmetricProjection = true;
+
     state: Readonly<Camera.Snapshot> = Camera.createDefaultSnapshot();
     viewOffset: Readonly<Camera.ViewOffset> = Camera.ViewOffset();
     far: number = 0;
     near: number = 0;
     fogFar: number = 0;
     fogNear: number = 0;
+
+    forceFull: boolean = false;
+    scale: number = 0;
+    minTargetDistance: number = 0;
+
+    disabled = false;
+
+    getRay(out: Ray3D, x: number, y: number) {
+        Mat4.getTranslation(out.origin, Mat4.invert(Mat4(), this.view));
+        Vec3.set(out.direction, x, y, 0.5);
+        cameraUnproject(out.direction, out.direction, this.viewport, this.inverseProjectionView);
+        Vec3.normalize(out.direction, Vec3.sub(out.direction, out.direction, out.origin));
+        return out;
+    }
 }
 
 const tmpEyeLeft = Mat4.identity();
@@ -84,6 +106,10 @@ function copyStates(parent: Camera, eye: EyeCamera) {
     eye.near = parent.near;
     eye.fogFar = parent.fogFar;
     eye.fogNear = parent.fogNear;
+
+    eye.forceFull = parent.forceFull;
+    eye.scale = parent.scale;
+    eye.minTargetDistance = parent.minTargetDistance;
 }
 
 //
@@ -138,4 +164,31 @@ function update(camera: Camera, props: StereoCameraProps, left: EyeCamera, right
     Mat4.mul(right.view, right.view, tmpEyeRight);
     Mat4.mul(right.projectionView, right.projection, right.view);
     Mat4.invert(right.inverseProjectionView, right.projectionView);
-}
+
+    // ensure enabled
+
+    left.disabled = false;
+    right.disabled = false;
+}
+
+//
+
+function xrUpdate(camera: Camera, left: EyeCamera, right: EyeCamera, xr: { pose: XRViewerPose, layer: XRWebGLLayer }) {
+    _xrUpdate(camera, left, xr.pose.views[0], xr.layer);
+    if (xr.pose.views.length === 1) {
+        right.disabled = true;
+    } else {
+        _xrUpdate(camera, right, xr.pose.views[1], xr.layer);
+    }
+}
+
+function _xrUpdate(camera: Camera, eye: EyeCamera, view: XRView, layer: XRWebGLLayer) {
+    copyStates(camera, eye);
+    const lvp = layer.getViewport(view)!;
+    Viewport.set(eye.viewport, lvp.x, lvp.y, lvp.width, lvp.height);
+    Mat4.fromArray(eye.projection, view.projectionMatrix, 0);
+    Mat4.fromArray(eye.view, view.transform.inverse.matrix, 0);
+    Mat4.mul(eye.projectionView, eye.projection, eye.view);
+    Mat4.invert(eye.inverseProjectionView, eye.projectionView);
+    eye.disabled = false;
+}

src/mol-canvas3d/canvas3d.ts

@@ -13,7 +13,7 @@ import { Vec3, Vec2 } from '../mol-math/linear-algebra';
 import { InputObserver, ModifiersKeys, ButtonsType } from '../mol-util/input/input-observer';
 import { Renderer, RendererStats, RendererParams } from '../mol-gl/renderer';
 import { GraphicsRenderObject } from '../mol-gl/render-object';
-import { TrackballControls, TrackballControlsParams } from './controls/trackball';
+import { DefaultTrackballControlsAttribs, TrackballControls, TrackballControlsParams } from './controls/trackball';
 import { Viewport } from './camera/util';
 import { createContext, WebGLContext, getGLContext } from '../mol-gl/webgl/context';
 import { Representation } from '../mol-repr/representation';
@@ -34,7 +34,6 @@ import { ImagePass, ImageProps } from './passes/image';
 import { Sphere3D } from '../mol-math/geometry';
 import { addConsoleStatsProvider, isDebugMode, isTimingMode, removeConsoleStatsProvider } from '../mol-util/debug';
 import { CameraHelperParams } from './helper/camera-helper';
-import { produce } from 'immer';
 import { HandleHelperParams } from './helper/handle-helper';
 import { StereoCamera, StereoCameraParams } from './camera/stereo';
 import { Helper } from './helper/helper';
@@ -47,8 +46,13 @@ import { deepClone } from '../mol-util/object';
 import { HiZParams, HiZPass } from './passes/hi-z';
 import { IlluminationParams } from './passes/illumination';
 import { isMobileBrowser } from '../mol-util/browser';
+import { PointerHelperParams } from './helper/pointer-helper';
+import { DefaultXRManagerAttribs, XRManager, XRManagerParams } from './helper/xr-manager';
 import { Ray3D } from '../mol-math/geometry/primitives/ray3d';
 import { RayHelper } from './helper/ray-helper';
+import { produce } from '../mol-util/produce';
+import { ShaderManager } from './helper/shader-manager';
+import { toFixed } from '../mol-util/number';
 
 export const CameraFogParams = {
     intensity: PD.Numeric(15, { min: 1, max: 100, step: 1 }),
@@ -62,7 +66,6 @@ export const Canvas3DParams = {
             off: PD.Group({})
         }, { cycle: true, hideIf: p => p?.mode !== 'perspective' }),
         fov: PD.Numeric(45, { min: 10, max: 130, step: 1 }, { label: 'Field of View' }),
-        scale: PD.Numeric(1, { min: 0.001, max: 1, step: 0.001 }, { label: 'Scene scale' }),
         manualReset: PD.Boolean(false, { isHidden: true }),
     }, { pivot: 'mode' }),
     cameraFog: PD.MappedStatic('on', {
@@ -107,6 +110,8 @@ export const Canvas3DParams = {
     interaction: PD.Group(Canvas3dInteractionHelperParams),
     debug: PD.Group(DebugHelperParams),
     handle: PD.Group(HandleHelperParams),
+    pointer: PD.Group(PointerHelperParams),
+    xr: PD.Group(XRManagerParams, { label: 'XR' }),
 };
 export const DefaultCanvas3DParams = PD.getDefaultValues(Canvas3DParams);
 export type Canvas3DProps = PD.Values<typeof Canvas3DParams>
@@ -114,6 +119,12 @@ export type PartialCanvas3DProps = {
     [K in keyof Canvas3DProps]?: Canvas3DProps[K] extends { name: string, params: any } ? Canvas3DProps[K] : Partial<Canvas3DProps[K]>
 }
 
+export const DefaultCanvas3DAttribs = {
+    trackball: DefaultTrackballControlsAttribs,
+    xr: DefaultXRManagerAttribs,
+};
+export type Canvas3DAttribs = typeof DefaultCanvas3DAttribs
+
 export { Canvas3DContext };
 
 /** Can be used to create multiple Canvas3D objects */
@@ -297,6 +308,9 @@ namespace Canvas3DContext {
                 canvas.removeEventListener('webglcontextlost', handleWebglContextLost, false);
                 canvas.removeEventListener('webglcontextrestored', handlewWebglContextRestored, false);
                 webgl.destroy(options);
+
+                contextLost.complete();
+                changed.complete();
             }
         };
     }
@@ -317,7 +331,7 @@ interface Canvas3D {
      * Function for external "animation" control
      * Calls commit.
      */
-    tick(t: now.Timestamp, options?: { isSynchronous?: boolean, manualDraw?: boolean, updateControls?: boolean }): void
+    tick(t: now.Timestamp, options?: { isSynchronous?: boolean, manualDraw?: boolean, updateControls?: boolean, xrFrame?: XRFrame }): void
     update(repr?: Representation.Any, keepBoundingSphere?: boolean): void
     clear(): void
     syncVisibility(): void
@@ -334,6 +348,10 @@ interface Canvas3D {
     pause(noDraw?: boolean): void
     /** Sets drawPaused = false without starting the built in animation loop */
     resume(): void
+
+    requestAnimationFrame(callback: FrameRequestCallback | XRFrameRequestCallback): number
+    cancelAnimationFrame(handle: number): void
+
     identify(target: Vec2 | Ray3D): PickData | undefined
     asyncIdentify(target: Vec2 | Ray3D): AsyncPickData | undefined
     mark(loci: Representation.Loci, action: MarkerAction): void
@@ -360,10 +378,19 @@ interface Canvas3D {
 
     /** Returns a copy of the current Canvas3D instance props */
     readonly props: Readonly<Canvas3DProps>
+    readonly attribs: Readonly<Canvas3DAttribs>
     readonly input: InputObserver
     readonly stats: RendererStats
     readonly interaction: Canvas3dInteractionHelper['events']
 
+    readonly xr: {
+        request(): Promise<void>
+        end(): Promise<void>
+        readonly isSupported: BehaviorSubject<boolean>
+        readonly isPresenting: BehaviorSubject<boolean>
+        readonly requestFailed: Subject<string>
+    }
+
     dispose(): void
 }
 
@@ -379,9 +406,10 @@ namespace Canvas3D {
     export interface DragEvent { current: Representation.Loci, buttons: ButtonsType, button: ButtonsType.Flag, modifiers: ModifiersKeys, pageStart: Vec2, pageEnd: Vec2 }
     export interface ClickEvent { current: Representation.Loci, buttons: ButtonsType, button: ButtonsType.Flag, modifiers: ModifiersKeys, page?: Vec2, position?: Vec3 }
 
-    export function create(ctx: Canvas3DContext, props: Partial<Canvas3DProps> = {}): Canvas3D {
+    export function create(ctx: Canvas3DContext, props: Partial<Canvas3DProps> = {}, attribs: Partial<Canvas3DAttribs> = {}): Canvas3D {
         const { webgl, input, passes, assetManager, canvas, contextLost } = ctx;
         const p: Canvas3DProps = { ...deepClone(DefaultCanvas3DParams), ...deepClone(props) };
+        const a = { ...deepClone(DefaultCanvas3DAttribs), ...deepClone(attribs) };
 
         const reprRenderObjects = new Map<Representation.Any, Set<GraphicsRenderObject>>();
         const reprUpdatedSubscriptions = new Map<Representation.Any, Subscription>();
@@ -404,7 +432,10 @@ namespace Canvas3D {
         let currentTime = 0;
 
         updateViewport();
-        const scene = Scene.create(webgl, passes.draw.transparency);
+        const scene = Scene.create(webgl, passes.draw.transparency, {
+            dColorMarker: p.renderer.colorMarker,
+            dLightCount: p.renderer.light?.length,
+        });
 
         function getSceneRadius() {
             return scene.boundingSphere.radius * p.sceneRadiusFactor;
@@ -417,17 +448,19 @@ namespace Canvas3D {
             clipFar: p.cameraClipping.far,
             minNear: p.cameraClipping.minNear,
             fov: degToRad(p.camera.fov),
-            scale: p.camera.scale,
         }, { x, y, width, height });
         const stereoCamera = new StereoCamera(camera, p.camera.stereo.params);
 
-        const controls = TrackballControls.create(input, camera, scene, p.trackball);
+        const controls = TrackballControls.create(input, camera, scene, p.trackball, a.trackball);
         const helper = new Helper(webgl, scene, p);
         const hiZ = new HiZPass(webgl, passes.draw, canvas, p.hiZ);
 
         const renderer = Renderer.create(webgl, p.renderer);
         renderer.setOcclusionTest(hiZ.isOccluded);
 
+        const shaderManager = new ShaderManager(webgl, scene);
+        shaderManager.updateRequired(p);
+
         const pickOptions = {
             pickPadding: p.pickPadding,
             maxAsyncReadLag: DefaultPickOptions.maxAsyncReadLag,
@@ -446,6 +479,85 @@ namespace Canvas3D {
         let nextCameraResetSnapshot: Camera.SnapshotProvider | undefined = void 0;
         let resizeRequested = false;
 
+        //
+
+        function getNonXRProps() {
+            return {
+                transparency: ctx.props.transparency,
+                transparentBackground: p.transparentBackground,
+                hiZ: hiZ.props.enabled,
+                postprocessing: p.postprocessing.enabled,
+                axes: deepClone(helper.camera.props.axes),
+            };
+        }
+        const nonXRProps = getNonXRProps();
+
+        function saveNonXRProps() {
+            Object.assign(nonXRProps, getNonXRProps());
+        }
+
+        function loadNonXRProps() {
+            p.postprocessing.enabled = nonXRProps.postprocessing;
+            p.transparentBackground = nonXRProps.transparentBackground;
+            ctx.setProps({ transparency: nonXRProps.transparency });
+            hiZ.setProps({ enabled: nonXRProps.hiZ });
+            helper.camera.setProps({ axes: nonXRProps.axes });
+        }
+
+        function setXRProps() {
+            p.postprocessing.enabled = !xrManager.props.disablePostprocessing;
+            ctx.setProps({ transparency: 'blended' });
+            hiZ.setProps({ enabled: false });
+            helper.camera.setProps({ axes: { name: 'off', params: {} } });
+
+            if (xrManager.session?.environmentBlendMode === 'alpha-blend') {
+                p.transparentBackground = xrPassthrough;
+            }
+        }
+
+        const xrManager = new XRManager(webgl, input, scene, camera, stereoCamera, helper.pointer, interactionHelper);
+
+        const xr = {
+            request: async () => {
+                try {
+                    await xrManager.request();
+                } catch (e) {
+                    console.error(e);
+                    xr.requestFailed.next(e);
+                }
+            },
+            end: () => xrManager.end(),
+            isSupported: new BehaviorSubject(false),
+            isPresenting: new BehaviorSubject(false),
+            requestFailed: new Subject<string>(),
+        };
+
+        let xrPassthrough = false;
+
+        const xrSubs = [
+            xrManager.isSupported.subscribe(e => xr.isSupported.next(e)),
+            xrManager.togglePassthrough.subscribe(() => {
+                if (xrManager.session?.environmentBlendMode === 'alpha-blend') {
+                    xrPassthrough = !p.transparentBackground;
+                }
+            }),
+            xrManager.sessionChanged.subscribe(() => {
+                fenceSync = null;
+                resizeRequested = true;
+                if (xrManager.session) {
+                    saveNonXRProps();
+                    xrPassthrough = xrManager.session?.environmentBlendMode === 'alpha-blend';
+                    setXRProps();
+                } else {
+                    loadNonXRProps();
+                }
+                resume();
+                xr.isPresenting.next(!!xrManager.session);
+            }),
+        ];
+
+        //
+
         let notifyDidDraw = true;
 
         function getLoci(pickingId: PickingId | undefined) {
@@ -491,6 +603,9 @@ namespace Canvas3D {
                 helper.handle.scene.update(void 0, true);
                 helper.camera.scene.update(void 0, true);
 
+                shaderManager.updateRequired(p);
+                shaderManager.finalizeRequired(true);
+
                 interactionEvent.next();
             }
             return changed;
@@ -511,8 +626,9 @@ namespace Canvas3D {
 
         let fenceSync: WebGLSync | null = null;
 
-        function render(force: boolean) {
+        function render(force: boolean, xrFrame?: XRFrame) {
             if (webgl.isContextLost) return false;
+            if (webgl.xr.session && !xrFrame) return false;
 
             let resized = false;
             if (resizeRequested) {
@@ -526,7 +642,7 @@ namespace Canvas3D {
                 y > drs.height || y + height < 0
             ) return false;
 
-            if (fenceSync !== null) {
+            if (fenceSync !== null && !xrFrame) {
                 if (webgl.checkSyncStatus(fenceSync)) {
                     fenceSync = null;
                 } else {
@@ -534,22 +650,29 @@ namespace Canvas3D {
                 }
             }
 
+            if (xrFrame) {
+                setXRProps();
+                p.transparentBackground = xrPassthrough;
+            }
+
             const markingUpdated = resolveMarking() && (renderer.props.colorMarker || p.marking.enabled);
 
             let didRender = false;
             controls.update(currentTime);
             const cameraChanged = camera.update();
+            const xrChanged = xrManager.update(xrFrame);
+            if (!xrChanged && xrFrame) return false;
 
-            const shouldRender = force || cameraChanged || resized || forceNextRender;
+            const shouldRender = force || cameraChanged || resized || forceNextRender || xrChanged;
             forceNextRender = false;
 
-            if (passes.illumination.supported && p.illumination.enabled) {
+            if (passes.illumination.supported && p.illumination.enabled && !xrFrame) {
                 if (shouldRender || markingUpdated) {
                     renderer.setOcclusionTest(null);
                     passes.illumination.restart();
                 }
 
-                if (passes.illumination.shouldRender(p)
+                if (passes.illumination.shouldRender(p.illumination)
                     && ((!isActivelyInteracting && scene.count > 0) || passes.illumination.iteration === 0 || p.userInteractionReleaseMs === 0)
                 ) {
                     if (isTimingMode) webgl.timer.mark('Canvas3D.render', { captureStats: true });
@@ -562,20 +685,20 @@ namespace Canvas3D {
                     didRender = true;
                 }
             } else {
-                const multiSampleChanged = multiSampleHelper.update(markingUpdated || shouldRender, p.multiSample);
+                const multiSampleChanged = multiSampleHelper.update(markingUpdated || shouldRender, p.multiSample) && !xrFrame;
 
                 if (shouldRender || multiSampleChanged || markingUpdated) {
                     renderer.setOcclusionTest(hiZ.isOccluded);
 
                     let cam: Camera | StereoCamera = camera;
-                    if (p.camera.stereo.name === 'on') {
-                        stereoCamera.update();
+                    if (p.camera.stereo.name === 'on' || xrChanged) {
+                        if (!xrChanged) stereoCamera.update();
                         cam = stereoCamera;
                     }
 
                     if (isTimingMode) webgl.timer.mark('Canvas3D.render', { captureStats: true });
                     const ctx = { renderer, camera: cam, scene, helper };
-                    if (MultiSamplePass.isEnabled(p.multiSample)) {
+                    if (MultiSamplePass.isEnabled(p.multiSample) && !xrFrame) {
                         const forceOn = p.multiSample.reduceFlicker && !cameraChanged && markingUpdated && !controls.isAnimating;
                         multiSampleHelper.render(ctx, p, true, forceOn);
                     } else {
@@ -590,7 +713,7 @@ namespace Canvas3D {
                 }
             }
 
-            if (didRender) {
+            if (didRender && !xrFrame) {
                 fenceSync = webgl.getFenceSync();
             }
 
@@ -601,9 +724,13 @@ namespace Canvas3D {
         let drawPaused = false;
         let isContextLost = false;
 
-        function draw(options?: { force?: boolean }) {
+        function draw(options?: { force?: boolean, isSynchronous?: boolean, xrFrame?: XRFrame }) {
             if (drawPaused || isContextLost) return;
-            if (render(!!options?.force) && notifyDidDraw) {
+            if (!shaderManager.finalizeRequired(options?.isSynchronous)) {
+                forceNextRender = true;
+                return;
+            }
+            if (render(!!options?.force, options?.xrFrame) && notifyDidDraw) {
                 didDraw.next(now() - startTime as now.Timestamp);
             }
         }
@@ -614,8 +741,9 @@ namespace Canvas3D {
 
         let animationFrameHandle = 0;
 
-        function tick(t: now.Timestamp, options?: { isSynchronous?: boolean, manualDraw?: boolean, updateControls?: boolean }) {
+        function tick(t: now.Timestamp, options?: { isSynchronous?: boolean, manualDraw?: boolean, updateControls?: boolean, xrFrame?: XRFrame }) {
             if (isContextLost) return;
+            if (webgl.xr.session && !options?.xrFrame) return;
 
             currentTime = t;
             commit(options?.isSynchronous);
@@ -632,15 +760,31 @@ namespace Canvas3D {
                 return;
             }
 
-            draw();
+            draw({ isSynchronous: options?.isSynchronous, xrFrame: options?.xrFrame });
             if (!camera.transition.inTransition && !webgl.isContextLost) {
                 interactionHelper.tick(currentTime);
             }
         }
 
-        function _animate() {
-            tick(now());
-            animationFrameHandle = requestAnimationFrame(_animate);
+        let animationFrameCB: FrameRequestCallback | XRFrameRequestCallback | undefined = undefined;
+
+        function _requestAnimationFrame(callback: FrameRequestCallback | XRFrameRequestCallback): number {
+            animationFrameCB = callback;
+            return webgl.xr.session
+                ? webgl.xr.session.requestAnimationFrame(callback)
+                : requestAnimationFrame(callback as FrameRequestCallback);
+        }
+
+        function _cancelAnimationFrame(handle: number): void {
+            animationFrameCB = undefined;
+            webgl.xr.session
+                ? webgl.xr.session.cancelAnimationFrame(handle)
+                : cancelAnimationFrame(handle);
+        }
+
+        function _animate(_timestamp: number, xrFrame?: XRFrame) {
+            tick(now(), { xrFrame });
+            animationFrameHandle = _requestAnimationFrame(_animate);
         }
 
         function resetTime(t: now.Timestamp) {
@@ -651,17 +795,25 @@ namespace Canvas3D {
         function animate() {
             drawPaused = false;
             controls.start(now());
-            if (animationFrameHandle === 0) _animate();
+            if (animationFrameHandle === 0) _animate(0);
         }
 
         function pause(noDraw = false) {
             drawPaused = noDraw;
-            cancelAnimationFrame(animationFrameHandle);
-            animationFrameHandle = 0;
+            if (animationFrameHandle !== 0) {
+                _cancelAnimationFrame(animationFrameHandle);
+                animationFrameHandle = 0;
+            }
+        }
+
+        function resume() {
+            drawPaused = false;
+            if (animationFrameCB) _requestAnimationFrame(animationFrameCB);
         }
 
         function identify(target: Vec2 | Ray3D): PickData | undefined {
             if (webgl.isContextLost) return undefined;
+            shaderManager.finalize(['pick'], true);
 
             if ('origin' in target) {
                 return rayHelper.identify(target, camera);
@@ -673,6 +825,7 @@ namespace Canvas3D {
 
         function asyncIdentify(target: Vec2 | Ray3D): AsyncPickData | undefined {
             if (webgl.isContextLost) return undefined;
+            shaderManager.finalize(['pick'], true);
 
             if ('origin' in target) {
                 return rayHelper.asyncIdentify(target, camera);
@@ -684,6 +837,7 @@ namespace Canvas3D {
 
         function commit(isSynchronous: boolean = false) {
             const allCommited = commitScene(isSynchronous);
+            shaderManager.updateRequired(p);
             // Only reset the camera after the full scene has been commited.
             if (allCommited) {
                 resolveCameraReset();
@@ -699,6 +853,10 @@ namespace Canvas3D {
         function resolveCameraReset() {
             if (!cameraResetRequested) return;
 
+            if (!xr.isPresenting.value) {
+                xrManager.resetScale();
+            }
+
             const boundingSphere = scene.boundingSphereVisible;
             const { center, radius } = boundingSphere;
 
@@ -785,25 +943,42 @@ namespace Canvas3D {
                 instanceCount: r.values.instanceCount.ref.value,
                 materialId: r.materialId,
                 renderItemId: r.id,
+                geometryType: r.values.dGeometryType.ref.value,
+                'byteCount [MiB]': toFixed(r.getByteCount() / 1024 / 1024, 3),
             }));
 
             console.groupCollapsed(`${items.length} RenderItems`);
 
-            if (items.length < 50) {
+            if (items.length <= 64) {
                 console.table(items);
             } else {
                 console.log(items);
             }
             console.log(JSON.stringify(webgl.stats, undefined, 4));
 
-            const { texture, attribute, elements } = webgl.resources.getByteCounts();
+            const { texture, cubeTexture, attribute, elements, pixelPack, renderbuffer } = webgl.resources.getByteCounts();
             console.log(JSON.stringify({
                 texture: `${(texture / 1024 / 1024).toFixed(3)} MiB`,
+                cubeTexture: `${(cubeTexture / 1024 / 1024).toFixed(3)} MiB`,
                 attribute: `${(attribute / 1024 / 1024).toFixed(3)} MiB`,
                 elements: `${(elements / 1024 / 1024).toFixed(3)} MiB`,
+                pixelPack: `${(pixelPack / 1024 / 1024).toFixed(3)} MiB`,
+                renderbuffer: `${(renderbuffer / 1024 / 1024).toFixed(3)} MiB`,
             }, undefined, 4));
 
-            console.log(JSON.stringify(webgl.timer.formatedStats(), undefined, 4));
+            console.log(JSON.stringify({
+                renderables: `${(scene.renderables.reduce((sum, r) => sum + r.getByteCount(), 0) / 1024 / 1024).toFixed(3)} MiB`,
+                passes: {
+                    draw: `${(passes.draw.getByteCount() / 1024 / 1024).toFixed(3)} MiB`,
+                    illumination: `${(passes.illumination.getByteCount() / 1024 / 1024).toFixed(3)} MiB`,
+                    pick: `${(passes.pick.getByteCount() / 1024 / 1024).toFixed(3)} MiB`,
+                    hiZ: `${(hiZ.getByteCount() / 1024 / 1024).toFixed(3)} MiB`,
+                }
+            }, undefined, 4));
+
+            if (isTimingMode) {
+                console.log(JSON.stringify(webgl.timer.formatedStats(), undefined, 4));
+            }
 
             console.groupEnd();
         }
@@ -869,13 +1044,16 @@ namespace Canvas3D {
                     helper: { ...helper.camera.props },
                     stereo: { ...p.camera.stereo },
                     fov: Math.round(radToDeg(camera.state.fov)),
-                    scale: camera.state.scale,
                     manualReset: !!p.camera.manualReset
                 },
                 cameraFog: camera.state.fog > 0
                     ? { name: 'on' as const, params: { intensity: camera.state.fog } }
                     : { name: 'off' as const, params: {} },
-                cameraClipping: { far: camera.state.clipFar, radius, minNear: camera.state.minNear },
+                cameraClipping: {
+                    far: camera.state.clipFar,
+                    radius,
+                    minNear: camera.state.minNear,
+                },
                 cameraResetDurationMs: p.cameraResetDurationMs,
                 sceneRadiusFactor: p.sceneRadiusFactor,
                 transparentBackground: p.transparentBackground,
@@ -894,6 +1072,8 @@ namespace Canvas3D {
                 interaction: { ...interactionHelper.props },
                 debug: { ...helper.debug.props },
                 handle: { ...helper.handle.props },
+                pointer: { ...helper.pointer.props },
+                xr: { ...xrManager.props },
             };
         }
 
@@ -944,7 +1124,7 @@ namespace Canvas3D {
         // Monitor user interactions
         let isDragging = false;
         let isActivelyInteracting = false;
-        let interactionSubs = [
+        const interactionSubs = [
             input.drag.subscribe(() => {
                 isDragging = true;
             }),
@@ -1051,7 +1231,11 @@ namespace Canvas3D {
             animate,
             resetTime,
             pause,
-            resume: () => { drawPaused = false; },
+            resume,
+
+            requestAnimationFrame: _requestAnimationFrame,
+            cancelAnimationFrame: _cancelAnimationFrame,
+
             identify,
             asyncIdentify,
             mark,
@@ -1094,9 +1278,6 @@ namespace Canvas3D {
                 if (props.camera && props.camera.fov !== undefined && props.camera.fov !== oldFov) {
                     cameraState.fov = degToRad(props.camera.fov);
                 }
-                if (props.camera && props.camera.scale !== undefined && props.camera.scale !== cameraState.scale) {
-                    cameraState.scale = props.camera.scale;
-                }
                 if (props.cameraFog !== undefined && props.cameraFog.params) {
                     const newFog = props.cameraFog.name === 'on' ? props.cameraFog.params.intensity : 0;
                     if (newFog !== camera.state.fog) cameraState.fog = newFog;
@@ -1154,16 +1335,25 @@ namespace Canvas3D {
                 if (props.illumination) Object.assign(p.illumination, props.illumination);
                 if (props.multiSample) Object.assign(p.multiSample, props.multiSample);
                 if (props.hiZ) hiZ.setProps(props.hiZ);
-                if (props.renderer) renderer.setProps(props.renderer);
+                if (props.renderer) {
+                    scene.setGlobals({
+                        dColorMarker: props.renderer.colorMarker ?? renderer.props.colorMarker,
+                        dLightCount: props.renderer.light?.length ?? renderer.props.light.length,
+                    });
+                    renderer.setProps(props.renderer);
+                }
                 if (props.trackball) controls.setProps(props.trackball);
                 if (props.interaction) interactionHelper.setProps(props.interaction);
                 if (props.debug) helper.debug.setProps(props.debug);
                 if (props.handle) helper.handle.setProps(props.handle);
+                if (props.pointer) helper.pointer.setProps(props.pointer);
+                if (props.xr) xrManager.setProps(props.xr);
 
                 if (cameraState.mode === 'orthographic') {
                     p.camera.stereo.name = 'off';
                 }
 
+                shaderManager.updateRequired(p);
                 if (!doNotRequestDraw) {
                     requestDraw();
                 }
@@ -1180,6 +1370,9 @@ namespace Canvas3D {
             get props() {
                 return getProps();
             },
+            get attribs() {
+                return a;
+            },
             get input() {
                 return input;
             },
@@ -1189,15 +1382,20 @@ namespace Canvas3D {
             get interaction() {
                 return interactionHelper.events;
             },
+            xr,
             dispose: () => {
                 contextLostSub?.unsubscribe();
                 contextRestoredSub.unsubscribe();
                 ctxChangedSub?.unsubscribe();
 
+                for (const s of xrSubs) s.unsubscribe();
+                xrSubs.length = 0;
+
                 for (const s of interactionSubs) s.unsubscribe();
-                interactionSubs = [];
+                interactionSubs.length = 0;
 
                 cancelAnimationFrame(animationFrameHandle);
+                animationFrameCB = undefined;
 
                 markBuffer = [];
 
@@ -1209,12 +1407,23 @@ namespace Canvas3D {
                 hiZ.dispose();
                 pickHelper.dispose();
                 rayHelper.dispose();
+                xrManager.dispose();
 
                 if (fenceSync !== null) {
                     webgl.deleteSync(fenceSync);
                     fenceSync = null;
                 }
 
+                reprCount.complete();
+                interactionEvent.complete();
+                didDraw.complete();
+                resized.complete();
+                commited.complete();
+                commitQueueSize.complete();
+                xr.isPresenting.complete();
+                xr.isSupported.complete();
+                xr.requestFailed.complete();
+
                 removeConsoleStatsProvider(consoleStats);
             }
         };

src/mol-canvas3d/controls/trackball.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2018-2023 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2018-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  * @author David Sehnal <david.sehnal@gmail.com>
@@ -24,7 +24,10 @@ const Trigger = Binding.Trigger;
 const Key = Binding.TriggerKey;
 
 export const DefaultTrackballBindings = {
-    dragRotate: Binding([Trigger(B.Flag.Primary, M.create())], 'Rotate', 'Drag using ${triggers}'),
+    dragRotate: Binding([
+        Trigger(B.Flag.Primary, M.create()),
+        Trigger(B.Flag.Trigger)
+    ], 'Rotate', 'Drag using ${triggers}'),
     dragRotateZ: Binding([Trigger(B.Flag.Primary, M.create({ shift: true, control: true }))], 'Rotate around z-axis (roll)', 'Drag using ${triggers}'),
     dragPan: Binding([
         Trigger(B.Flag.Secondary, M.create()),
@@ -38,8 +41,14 @@ export const DefaultTrackballBindings = {
     scrollFocus: Binding([Trigger(B.Flag.Auxilary, M.create({ shift: true }))], 'Clip', 'Scroll using ${triggers}'),
     scrollFocusZoom: Binding.Empty,
 
-    keyMoveForward: Binding([Key('KeyW')], 'Move forward', 'Press ${triggers}'),
-    keyMoveBack: Binding([Key('KeyS')], 'Move back', 'Press ${triggers}'),
+    keyMoveForward: Binding([
+        Key('KeyW'),
+        Key('GamepadUp'),
+    ], 'Move forward', 'Press ${triggers}'),
+    keyMoveBack: Binding([
+        Key('KeyS'),
+        Key('GamepadDown'),
+    ], 'Move back', 'Press ${triggers}'),
     keyMoveLeft: Binding([Key('KeyA')], 'Move left', 'Press ${triggers}'),
     keyMoveRight: Binding([Key('KeyD')], 'Move right', 'Press ${triggers}'),
     keyMoveUp: Binding([Key('KeyR')], 'Move up', 'Press ${triggers}'),
@@ -56,8 +65,6 @@ export const DefaultTrackballBindings = {
 };
 
 export const TrackballControlsParams = {
-    noScroll: PD.Boolean(true, { isHidden: true }),
-
     rotateSpeed: PD.Numeric(5.0, { min: 1, max: 10, step: 1 }),
     zoomSpeed: PD.Numeric(7.0, { min: 1, max: 15, step: 1 }),
     panSpeed: PD.Numeric(1.0, { min: 0.1, max: 5, step: 0.1 }),
@@ -68,10 +75,10 @@ export const TrackballControlsParams = {
     animate: PD.MappedStatic('off', {
         off: PD.EmptyGroup(),
         spin: PD.Group({
-            speed: PD.Numeric(1, { min: -20, max: 20, step: 1 }, { description: 'Rotation speed in radians per second' }),
+            speed: PD.Numeric(0.3, { min: -5, max: 5, step: 0.1 }, { description: 'Number of rotations per second' }),
         }, { description: 'Spin the 3D scene around the x-axis in view space' }),
         rock: PD.Group({
-            speed: PD.Numeric(0.3, { min: -5, max: 5, step: 0.1 }),
+            speed: PD.Numeric(0.3, { min: -5, max: 5, step: 0.1 }, { description: 'Number of oscilations per second' }),
             angle: PD.Numeric(10, { min: 0, max: 90, step: 1 }, { description: 'How many degrees to rotate in each direction.' }),
         }, { description: 'Rock the 3D scene around the x-axis in view space' })
     }),
@@ -85,8 +92,6 @@ export const TrackballControlsParams = {
     gestureScaleFactor: PD.Numeric(1, {}, { isHidden: true }),
     maxWheelDelta: PD.Numeric(0.02, {}, { isHidden: true }),
 
-    bindings: PD.Value(DefaultTrackballBindings, { isHidden: true }),
-
     /**
      * minDistance = minDistanceFactor * boundingSphere.radius + minDistancePadding
      * maxDistance = max(maxDistanceFactor * boundingSphere.radius, maxDistanceMin)
@@ -103,6 +108,11 @@ export const TrackballControlsParams = {
 };
 export type TrackballControlsProps = PD.Values<typeof TrackballControlsParams>
 
+export const DefaultTrackballControlsAttribs = {
+    bindings: DefaultTrackballBindings,
+};
+export type TrackballControlsAttribs = typeof DefaultTrackballControlsAttribs
+
 export { TrackballControls };
 interface TrackballControls {
     readonly viewport: Viewport
@@ -112,20 +122,25 @@ interface TrackballControls {
     readonly props: Readonly<TrackballControlsProps>
     setProps: (props: Partial<TrackballControlsProps>) => void
 
+    readonly attribs: Readonly<TrackballControlsAttribs>
+    setAttribs: (attribs: Partial<TrackballControlsAttribs>) => void
+
     start: (t: number) => void
     update: (t: number) => void
     reset: () => void
     dispose: () => void
 }
 namespace TrackballControls {
-    export function create(input: InputObserver, camera: Camera, scene: Scene, props: Partial<TrackballControlsProps> = {}): TrackballControls {
+    export function create(input: InputObserver, camera: Camera, scene: Scene, props: Partial<TrackballControlsProps> = {}, attribs: Partial<TrackballControlsAttribs> = {}): TrackballControls {
         const p: TrackballControlsProps = {
             ...PD.getDefaultValues(TrackballControlsParams),
             ...props,
-            // include default bindings for backwards state compatibility
-            bindings: { ...DefaultTrackballBindings, ...props.bindings }
         };
-        const b = p.bindings;
+        const a: TrackballControlsAttribs = {
+            ...DefaultTrackballControlsAttribs,
+            ...attribs
+        };
+        const b = a.bindings;
 
         const viewport = Viewport.clone(camera.viewport);
 
@@ -384,20 +399,35 @@ namespace TrackballControls {
             const minDistance = Math.max(camera.state.minNear, p.minDistance);
             Vec3.setMagnitude(moveEye, moveEye, minDistance);
 
+            const moveTarget = p.flyMode || input.pointerLock;
             const moveSpeed = deltaT * (60 / 1000) * p.moveSpeed * (keyState.boostMove === 1 ? p.boostMoveFactor : 1);
 
             if (keyState.moveForward === 1) {
-                Vec3.normalize(moveDir, moveEye);
-                Vec3.scaleAndSub(camera.position, camera.position, moveDir, moveSpeed);
-                if (p.flyMode || input.pointerLock) {
+                const cameraDistance = Vec3.distance(camera.position, scene.boundingSphereVisible.center);
+                if (cameraDistance < scene.boundingSphereVisible.radius && moveTarget) {
+                    Vec3.normalize(moveDir, moveEye);
+                    Vec3.scaleAndSub(camera.position, camera.position, moveDir, moveSpeed);
+                } else {
+                    Vec3.sub(moveDir, camera.position, camera.target);
+                    Vec3.scale(moveDir, moveDir, 1 - moveSpeed / 100);
+                    Vec3.add(camera.position, camera.target, moveDir);
+                }
+                if (moveTarget) {
                     Vec3.sub(camera.target, camera.position, moveEye);
                 }
             }
 
             if (keyState.moveBack === 1) {
-                Vec3.normalize(moveDir, moveEye);
-                Vec3.scaleAndAdd(camera.position, camera.position, moveDir, moveSpeed);
-                if (p.flyMode || input.pointerLock) {
+                const cameraDistance = Vec3.distance(camera.position, scene.boundingSphereVisible.center);
+                if (cameraDistance < scene.boundingSphereVisible.radius && moveTarget) {
+                    Vec3.normalize(moveDir, moveEye);
+                    Vec3.scaleAndAdd(camera.position, camera.position, moveDir, moveSpeed);
+                } else {
+                    Vec3.sub(moveDir, camera.position, camera.target);
+                    Vec3.scale(moveDir, moveDir, 1 + moveSpeed / 100);
+                    Vec3.add(camera.position, camera.target, moveDir);
+                }
+                if (moveTarget) {
                     Vec3.sub(camera.target, camera.position, moveEye);
                 }
             }
@@ -405,7 +435,7 @@ namespace TrackballControls {
             if (keyState.moveLeft === 1) {
                 Vec3.cross(moveDir, moveEye, camera.up);
                 Vec3.normalize(moveDir, moveDir);
-                if (p.flyMode || input.pointerLock) {
+                if (moveTarget) {
                     Vec3.scaleAndAdd(camera.position, camera.position, moveDir, moveSpeed);
                     Vec3.sub(camera.target, camera.position, moveEye);
                 } else {
@@ -417,7 +447,7 @@ namespace TrackballControls {
             if (keyState.moveRight === 1) {
                 Vec3.cross(moveDir, moveEye, camera.up);
                 Vec3.normalize(moveDir, moveDir);
-                if (p.flyMode || input.pointerLock) {
+                if (moveTarget) {
                     Vec3.scaleAndSub(camera.position, camera.position, moveDir, moveSpeed);
                     Vec3.sub(camera.target, camera.position, moveEye);
                 } else {
@@ -428,7 +458,7 @@ namespace TrackballControls {
 
             if (keyState.moveUp === 1) {
                 Vec3.normalize(moveDir, camera.up);
-                if (p.flyMode || input.pointerLock) {
+                if (moveTarget) {
                     Vec3.scaleAndAdd(camera.position, camera.position, moveDir, moveSpeed);
                     Vec3.sub(camera.target, camera.position, moveEye);
                 } else {
@@ -439,7 +469,7 @@ namespace TrackballControls {
 
             if (keyState.moveDown === 1) {
                 Vec3.normalize(moveDir, camera.up);
-                if (p.flyMode || input.pointerLock) {
+                if (moveTarget) {
                     Vec3.scaleAndSub(camera.position, camera.position, moveDir, moveSpeed);
                     Vec3.sub(camera.target, camera.position, moveEye);
                 } else {
@@ -448,7 +478,7 @@ namespace TrackballControls {
                 }
             }
 
-            if (p.flyMode || input.pointerLock) {
+            if (moveTarget) {
                 const cameraDistance = Vec3.distance(camera.position, scene.boundingSphereVisible.center);
                 camera.setState({ minFar: cameraDistance + scene.boundingSphereVisible.radius });
             }
@@ -538,8 +568,8 @@ namespace TrackballControls {
 
         // listeners
 
-        function onDrag({ x, y, pageX, pageY, buttons, modifiers, isStart }: DragInput) {
-            const isOutside = outsideViewport(x, y);
+        function onDrag({ x, y, dx, dy, pageX, pageY, buttons, modifiers, isStart, useDelta }: DragInput) {
+            const isOutside = !useDelta && outsideViewport(x, y);
 
             if (isStart && isOutside) return;
             if (!isStart && !_isInteracting) return;
@@ -554,6 +584,10 @@ namespace TrackballControls {
             const dragFocus = Binding.match(b.dragFocus, buttons, modifiers);
             const dragFocusZoom = Binding.match(b.dragFocusZoom, buttons, modifiers);
 
+            if (useDelta && dragRotate) {
+                Vec2.copy(_rotPrev, getMouseOnCircle(pageX - dx, pageY - dy));
+            }
+
             getMouseOnCircle(pageX, pageY);
             getMouseOnScreen(pageX, pageY);
 
@@ -825,7 +859,7 @@ namespace TrackballControls {
         function spin(deltaT: number) {
             if (p.animate.name !== 'spin' || p.animate.params.speed === 0 || _isInteracting) return;
 
-            const radPerMs = p.animate.params.speed / 1000;
+            const radPerMs = 2 * Math.PI * p.animate.params.speed / 1000;
             _spinSpeed[0] = deltaT * radPerMs / getRotateFactor();
             Vec2.add(_rotCurr, _rotPrev, _spinSpeed);
         }
@@ -885,7 +919,12 @@ namespace TrackballControls {
                     }
                 }
                 Object.assign(p, props);
-                Object.assign(b, props.bindings);
+            },
+
+            get attribs() { return a as Readonly<TrackballControlsAttribs>; },
+            setAttribs: (attribs: Partial<TrackballControlsAttribs>) => {
+                Object.assign(a, attribs);
+                Object.assign(b, a.bindings);
             },
 
             start,

src/mol-canvas3d/helper/camera-helper.ts

@@ -4,7 +4,7 @@
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
 
-import { produce } from 'immer';
+import { produce } from '../../mol-util/produce';
 import { Interval } from '../../mol-data/int/interval';
 import { addCylinder } from '../../mol-geo/geometry/mesh/builder/cylinder';
 import { addSphere } from '../../mol-geo/geometry/mesh/builder/sphere';

src/mol-canvas3d/helper/handle-helper.ts

@@ -16,7 +16,7 @@ import { addCylinder } from '../../mol-geo/geometry/mesh/builder/cylinder';
 import { ValueCell } from '../../mol-util';
 import { Sphere3D } from '../../mol-math/geometry';
 import { ParamDefinition as PD } from '../../mol-util/param-definition';
-import { produce } from 'immer';
+import { produce } from '../../mol-util/produce';
 import { Shape } from '../../mol-model/shape';
 import { PickingId } from '../../mol-geo/geometry/picking';
 import { Camera } from '../camera';

src/mol-canvas3d/helper/helper.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2020 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2020-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  */
@@ -10,6 +10,7 @@ import { ParamDefinition as PD } from '../../mol-util/param-definition';
 import { BoundingSphereHelper, DebugHelperParams } from './bounding-sphere-helper';
 import { CameraHelper, CameraHelperParams } from './camera-helper';
 import { HandleHelper, HandleHelperParams } from './handle-helper';
+import { PointerHelper, PointerHelperParams } from './pointer-helper';
 
 export const HelperParams = {
     debug: PD.Group(DebugHelperParams),
@@ -17,6 +18,7 @@ export const HelperParams = {
         helper: PD.Group(CameraHelperParams)
     }),
     handle: PD.Group(HandleHelperParams),
+    pointer: PD.Group(PointerHelperParams),
 };
 export const DefaultHelperProps = PD.getDefaultValues(HelperParams);
 export type HelperProps = PD.Values<typeof HelperParams>
@@ -26,6 +28,7 @@ export class Helper {
     readonly debug: BoundingSphereHelper;
     readonly camera: CameraHelper;
     readonly handle: HandleHelper;
+    readonly pointer: PointerHelper;
 
     constructor(webgl: WebGLContext, scene: Scene, props: Partial<HelperProps> = {}) {
         const p = { ...DefaultHelperProps, ...props };
@@ -33,5 +36,6 @@ export class Helper {
         this.debug = new BoundingSphereHelper(webgl, scene, p.debug);
         this.camera = new CameraHelper(webgl, p.camera.helper);
         this.handle = new HandleHelper(webgl, p.handle);
+        this.pointer = new PointerHelper(webgl, p.pointer);
     }
 }

src/mol-canvas3d/helper/pointer-helper.ts

@@ -0,0 +1,196 @@
+/**
+ * Copyright (c) 2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ *
+ * @author Alexander Rose <alexander.rose@weirdbyte.de>
+ */
+
+import { WebGLContext } from '../../mol-gl/webgl/context';
+import { Scene } from '../../mol-gl/scene';
+import { MeshBuilder } from '../../mol-geo/geometry/mesh/mesh-builder';
+import { Mat4, Vec3 } from '../../mol-math/linear-algebra';
+import { GraphicsRenderObject } from '../../mol-gl/render-object';
+import { Mesh } from '../../mol-geo/geometry/mesh/mesh';
+import { ColorNames } from '../../mol-util/color/names';
+import { ValueCell } from '../../mol-util';
+import { ParamDefinition as PD } from '../../mol-util/param-definition';
+import { Geometry } from '../../mol-geo/geometry/geometry';
+import { addCylinderFromRay3D } from '../../mol-geo/geometry/mesh/builder/cylinder';
+import { addSphere } from '../../mol-geo/geometry/mesh/builder/sphere';
+import { Camera, ICamera } from '../camera';
+import { Ray3D } from '../../mol-math/geometry/primitives/ray3d';
+import { Viewport } from '../camera/util';
+import { Shape } from '../../mol-model/shape/shape';
+
+export const PointerHelperParams = {
+    ...Mesh.Params,
+    enabled: PD.Select('off', PD.arrayToOptions(['on', 'off']), { isEssential: true }),
+    ignoreLight: { ...Mesh.Params.ignoreLight, defaultValue: true },
+    color: PD.Color(ColorNames.grey, { isEssential: true }),
+    hitColor: PD.Color(ColorNames.pink, { isEssential: true }),
+};
+export type PointerHelperParams = typeof PointerHelperParams
+export type PointerHelperProps = PD.Values<PointerHelperParams>
+
+export class PointerHelper {
+    readonly scene: Scene;
+    readonly camera: Camera;
+    readonly props: PointerHelperProps;
+
+    pixelScale = 1;
+
+    private renderObject: GraphicsRenderObject<'mesh'>;
+    private shape: Shape<Mesh>;
+
+    private modelScale = 1;
+    private pointers: Ray3D[] = [];
+    private points: Vec3[] = [];
+    private hit: Vec3 | undefined = undefined;
+
+    setProps(props: Partial<PointerHelperProps>) {
+        Object.assign(this.props, props);
+        if (this.isEnabled) this.update(this.pointers, this.points, this.hit);
+    }
+
+    ensureEnabled() {
+        if (this.props.enabled !== 'on') this.props.enabled = 'on';
+    }
+
+    get isEnabled() {
+        return this.props.enabled === 'on';
+    }
+
+    setCamera(camera: ICamera) {
+        Camera.copySnapshot(this.camera.state, camera.state);
+        Viewport.copy(this.camera.viewport, camera.viewport);
+        Mat4.copy(this.camera.view, camera.view);
+        Mat4.copy(this.camera.projection, camera.projection);
+        Mat4.copy(this.camera.projectionView, camera.projectionView);
+        Mat4.copy(this.camera.headRotation, camera.headRotation);
+        Camera.copyViewOffset(this.camera.viewOffset, camera.viewOffset);
+        this.camera.far = camera.far;
+        this.camera.near = camera.near;
+        this.camera.fogFar = camera.fogFar;
+        this.camera.fogNear = camera.fogNear;
+
+        this.camera.forceFull = camera.forceFull;
+        this.camera.scale = 1;
+
+        this.modelScale = camera.scale;
+    }
+
+    update(pointers: Ray3D[], points: Vec3[], hit: Vec3 | undefined) {
+        this.pointers = pointers;
+        this.points = points;
+        this.hit = hit;
+
+        const p = this.props;
+        if (p.enabled !== 'on') {
+            if (this.renderObject) this.renderObject.state.visible = false;
+            return;
+        }
+
+        this.shape = getPointerMeshShape(this.getData(), this.props, this.shape);
+
+        ValueCell.updateIfChanged(this.renderObject.values.drawCount, Geometry.getDrawCount(this.shape.geometry));
+        ValueCell.updateIfChanged(this.renderObject.values.uVertexCount, Geometry.getVertexCount(this.shape.geometry));
+        ValueCell.updateIfChanged(this.renderObject.values.uGroupCount, 2);
+        Mesh.Utils.updateBoundingSphere(this.renderObject.values, this.shape.geometry);
+        Mesh.Utils.updateValues(this.renderObject.values, this.props);
+        Mesh.Utils.updateRenderableState(this.renderObject.state, this.props);
+
+        this.renderObject.state.visible = true;
+
+        this.scene.update(void 0, false);
+        this.scene.commit();
+    }
+
+    private getData() {
+        return {
+            pointers: this.pointers,
+            points: this.points,
+            hit: this.hit,
+            modelScale: this.modelScale,
+            camera: this.camera,
+            pixels: 12,
+        };
+    }
+
+    constructor(webgl: WebGLContext, props: Partial<PointerHelperProps> = {}) {
+        this.scene = Scene.create(webgl, 'blended');
+        this.props = { ...PD.getDefaultValues(PointerHelperParams), ...props };
+
+        this.camera = new Camera();
+
+        this.shape = getPointerMeshShape(this.getData(), this.props, this.shape);
+        this.renderObject = createMeshRenderObject(this.shape, this.props);
+        this.scene.add(this.renderObject);
+    }
+}
+
+type PointerData = {
+    pointers: Ray3D[]
+    points: Vec3[]
+    hit?: Vec3
+    modelScale: number
+    camera: ICamera
+    pixels: number
+}
+
+export enum PointerHelperGroup {
+    None = 0,
+    Hit,
+}
+
+const tmpV = Vec3();
+function getSizeForPixels(position: Vec3, pixels: number, camera: ICamera, modelScale: number) {
+    const cameraPosition = Vec3.scale(tmpV, camera.state.position, modelScale);
+    const d = Vec3.distance(position, cameraPosition);
+    const height = 2 * Math.tan(camera.state.fov / 2) * d;
+    return (height / camera.viewport.height) * pixels;
+};
+
+function createPointerMesh(data: PointerData, mesh?: Mesh) {
+    const state = MeshBuilder.createState(512, 256, mesh);
+    const radius = 0.0005;
+    const cylinderProps = { radiusTop: radius, radiusBottom: radius, radialSegments: 32 };
+    const { modelScale, camera, pixels } = data;
+
+    state.currentGroup = PointerHelperGroup.None;
+    for (const pointer of data.pointers) {
+        addCylinderFromRay3D(state, pointer, 0.2, cylinderProps);
+        const size = getSizeForPixels(pointer.origin, pixels, camera, modelScale);
+        addSphere(state, pointer.origin, size, 1);
+    }
+    for (const point of data.points) {
+        const size = getSizeForPixels(point, pixels, camera, modelScale);
+        addSphere(state, point, size, 1);
+    }
+
+    if (data.hit) {
+        state.currentGroup = PointerHelperGroup.Hit;
+        const size = getSizeForPixels(data.hit, pixels, camera, modelScale);
+        addSphere(state, data.hit, size, 1);
+    }
+
+    return MeshBuilder.getMesh(state);
+}
+
+function getPointerMeshShape(data: PointerData, props: PointerHelperProps, shape?: Shape<Mesh>) {
+    const mesh = createPointerMesh(data, shape?.geometry);
+    const getColor = (groupId: number) => {
+        switch (groupId) {
+            case PointerHelperGroup.Hit: return props.hitColor;
+            default: return props.color;
+        }
+    };
+    return Shape.create('pointer-mesh', data, mesh, getColor, () => 1, () => '', undefined, 2);
+}
+
+function createMeshRenderObject(shape: Shape<Mesh>, props: PointerHelperProps) {
+    return Shape.createRenderObject(shape, {
+        ...PD.getDefaultValues(Mesh.Params),
+        ...props,
+        ignoreLight: props.ignoreLight,
+        cellSize: 0,
+    }) as GraphicsRenderObject<'mesh'>;
+}

src/mol-canvas3d/helper/ray-helper.ts

@@ -80,12 +80,16 @@ export class RayHelper {
         this.camera.near = cam.near;
         this.camera.fogFar = cam.fogFar;
         this.camera.fogNear = cam.fogNear;
+        this.camera.forceFull = cam.forceFull;
+        this.camera.scale = cam.scale;
         Viewport.copy(this.camera.viewport, this.viewport);
         Camera.copySnapshot(this.camera.state, { ...cam.state, mode: 'orthographic' });
 
-        updateOrthoRayCamera(this.camera, ray);
+        updateOrthoRayCamera(this.camera, ray, cam.up);
         Mat4.mul(this.camera.projectionView, this.camera.projection, this.camera.view);
         Mat4.tryInvert(this.camera.inverseProjectionView, this.camera.projectionView);
+
+        Mat4.copy(this.camera.viewEye, cam.view);
     }
 
     private getPickData(): PickData | undefined {
@@ -104,7 +108,7 @@ export class RayHelper {
     }
 
     identify(ray: Ray3D, cam: Camera): PickData | undefined {
-        if (!this.intersectsScene(ray, cam.state.scale)) return;
+        if (!this.intersectsScene(ray, cam.scale)) return;
 
         this.prepare(ray, cam);
 
@@ -117,7 +121,7 @@ export class RayHelper {
     }
 
     asyncIdentify(ray: Ray3D, cam: Camera): AsyncPickData | undefined {
-        if (!this.intersectsScene(ray, cam.state.scale)) return;
+        if (!this.intersectsScene(ray, cam.scale)) return;
 
         this.prepare(ray, cam);
 
@@ -168,10 +172,10 @@ export class RayHelper {
 
 //
 
-function updateOrthoRayCamera(camera: Camera, ray: Ray3D) {
+function updateOrthoRayCamera(camera: Camera, ray: Ray3D, up: Vec3) {
     const { near, far, viewport } = camera;
 
-    const height = 2 * Math.tan(degToRad(0.1) / 2) * Vec3.distance(camera.position, camera.target) * camera.state.scale;
+    const height = 2 * Math.tan(degToRad(0.1) / 2) * Vec3.distance(camera.position, camera.target) * camera.scale;
     const zoom = viewport.height / height;
 
     const fullLeft = -viewport.width / 2;
@@ -197,7 +201,6 @@ function updateOrthoRayCamera(camera: Camera, ray: Ray3D) {
     Quat.invert(r, r);
 
     const eye = Vec3.clone(ray.origin);
-    const up = Vec3.transformQuat(Vec3(), Vec3.unitY, r);
     const target = Vec3.add(Vec3(), eye, direction);
 
     // build view matrix

src/mol-canvas3d/helper/shader-manager.ts

@@ -0,0 +1,57 @@
+/**
+ * Copyright (c) 2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ *
+ * @author Alexander Rose <alexander.rose@weirdbyte.de>
+ */
+
+import { Scene } from '../../mol-gl/scene';
+import { WebGLContext } from '../../mol-gl/webgl/context';
+import { GraphicsRenderVariant } from '../../mol-gl/webgl/render-item';
+import { BloomPass } from '../passes/bloom';
+import { IlluminationPass, IlluminationProps } from '../passes/illumination';
+import { MarkingPass, MarkingProps } from '../passes/marking';
+import { PostprocessingPass, PostprocessingProps } from '../passes/postprocessing';
+
+export type ShaderManagerProps = {
+    marking: MarkingProps
+    postprocessing: PostprocessingProps
+    illumination: IlluminationProps
+}
+
+export class ShaderManager {
+    static ensureRequired(webgl: WebGLContext, scene: Scene, p: ShaderManagerProps) {
+        const sm = new ShaderManager(webgl, scene);
+        sm.updateRequired(p);
+        sm.finalizeRequired(true);
+    }
+
+    private readonly required: GraphicsRenderVariant[] = [];
+
+    constructor(private readonly webgl: WebGLContext, private readonly scene: Scene) { }
+
+    updateRequired(p: ShaderManagerProps) {
+        this.required.length = 0;
+        this.required.push('color');
+        if (IlluminationPass.isEnabled(this.webgl, p.illumination)) {
+            this.required.push('tracing');
+        }
+        if (MarkingPass.isEnabled(p.marking) && this.scene.markerAverage > 0) {
+            this.required.push('marking');
+        }
+        if (BloomPass.isEnabled(p.postprocessing) && this.scene.emissiveAverage > 0) {
+            this.required.push('emissive');
+        }
+        if (PostprocessingPass.isTransparentDepthRequired(this.scene, p.postprocessing) || !this.webgl.extensions.drawBuffers || !this.webgl.extensions.depthTexture || IlluminationPass.isEnabled(this.webgl, p.illumination)) {
+            this.required.push('depth');
+        }
+        this.webgl.resources.linkPrograms(this.required);
+    }
+
+    finalizeRequired(isSynchronous?: boolean) {
+        return this.finalize(this.required, isSynchronous);
+    }
+
+    finalize(variants?: GraphicsRenderVariant[], isSynchronous?: boolean) {
+        return this.webgl.resources.finalizePrograms(variants, isSynchronous);
+    }
+}

src/mol-canvas3d/helper/xr-manager.ts

@@ -0,0 +1,344 @@
+/**
+ * Copyright (c) 2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ *
+ * @author Alexander Rose <alexander.rose@weirdbyte.de>
+ */
+
+import { BehaviorSubject, Subject, Subscription } from 'rxjs';
+import { WebGLContext } from '../../mol-gl/webgl/context';
+import { Vec3 } from '../../mol-math/linear-algebra/3d/vec3';
+import { Quat } from '../../mol-math/linear-algebra/3d/quat';
+import { Mat4 } from '../../mol-math/linear-algebra/3d/mat4';
+import { Camera, ICamera } from '../camera';
+import { PointerHelper } from './pointer-helper';
+import { Vec2 } from '../../mol-math/linear-algebra/3d/vec2';
+import { ButtonsType, InputObserver, ScreenTouchInput, TrackedPointerInput } from '../../mol-util/input/input-observer';
+import { Plane3D } from '../../mol-math/geometry/primitives/plane3d';
+import { Vec4 } from '../../mol-math/linear-algebra/3d/vec4';
+import { StereoCamera } from '../camera/stereo';
+import { Ray3D } from '../../mol-math/geometry/primitives/ray3d';
+import { Scene } from '../../mol-gl/scene';
+import { Sphere3D } from '../../mol-math/geometry';
+import { Canvas3dInteractionHelper } from './interaction-events';
+import { ParamDefinition as PD } from '../../mol-util/param-definition';
+import { cameraProject } from '../camera/util';
+import { Binding } from '../../mol-util/binding';
+
+const B = ButtonsType;
+const Trigger = Binding.Trigger;
+const Key = Binding.TriggerKey;
+
+function getRigidTransformFromMat4(m: Mat4): XRRigidTransform {
+    const d = Mat4.getDecomposition(m);
+    return new XRRigidTransform(Vec3.toObj(d.position), Quat.toObj(d.quaternion));
+}
+
+function getRayFromPose(pose: XRPose, view?: Mat4): Ray3D {
+    const origin = Vec3.fromObj(pose.transform.position);
+    const t = Mat4.fromArray(Mat4(), pose.transform.matrix, 0);
+    const td = Mat4.getDecomposition(t);
+    const m = Mat4.fromQuat(Mat4(), td.quaternion);
+    const direction = Vec3.transformMat4(Vec3(), Vec3.negUnitZ, m);
+    const ray = Ray3D.create(origin, direction);
+    if (view) Ray3D.transform(ray, ray, Mat4.invert(Mat4(), view));
+    return ray;
+}
+
+type InputInfo = {
+    targetRayPose: XRPose,
+}
+
+export const DefaultXRManagerBindings = {
+    exit: Binding([Key('GamepadB')]),
+    togglePassthrough: Binding([Key('GamepadA')]),
+    gestureScale: Binding([Trigger(B.Flag.Trigger)]),
+};
+export const DefaultXRManagerAttribs = {
+    bindings: DefaultXRManagerBindings,
+};
+export type XRManagerAttribs = typeof DefaultXRManagerAttribs
+
+export const XRManagerParams = {
+    minTargetDistance: PD.Numeric(0.4, { min: 0.001, max: 1, step: 0.001 }),
+    disablePostprocessing: PD.Boolean(true),
+    resolutionScale: PD.Numeric(1, { min: 0.1, max: 2, step: 0.1 }),
+    sceneRadiusInMeters: PD.Numeric(0.25, { min: 0.01, max: 2, step: 0.01 }, { description: 'The radius of the scene bounding sphere in meters, used to set the initial camera scale.' }),
+};
+export type XRManagerParams = typeof XRManagerParams
+export type XRManagerProps = PD.Values<XRManagerParams>
+
+export class XRManager {
+    private hoverSub: Subscription;
+    private keyUpSub: Subscription;
+    private gestureSub: Subscription;
+    private sessionChangedSub: Subscription;
+
+    readonly togglePassthrough = new Subject<void>();
+    readonly sessionChanged = new Subject<void>();
+    readonly isSupported = new BehaviorSubject(false);
+
+    private xrSession: XRSession | undefined = undefined;
+    get session() {
+        return this.xrSession;
+    }
+
+    private xrRefSpace: XRReferenceSpace | undefined = undefined;
+
+    private scaleFactor = 1;
+    private prevScale = 0;
+    private prevInput: { left?: InputInfo, right?: InputInfo } = {};
+    private hit: Vec3 | undefined = undefined;
+
+    readonly props: XRManagerProps;
+
+    setProps(props: Partial<XRManagerProps>) {
+        Object.assign(this.props, props);
+    }
+
+    private intersect(camera: ICamera, view: Mat4, plane: Plane3D, targetRayPose: XRPose): { point: Vec3, screen: Vec2 } | undefined {
+        const point = Vec3();
+        const ray = getRayFromPose(targetRayPose, view);
+        if (Plane3D.intersectRay3D(point, plane, ray)) {
+            const { height } = camera.viewport;
+            const v = cameraProject(Vec4(), point, camera.viewport, camera.projectionView);
+            const screen = Vec2.create(Math.floor(v[0]), height - Math.floor(v[1]));
+            return { point, screen };
+        }
+    }
+
+    setScaleFactor(factor: number) {
+        this.scaleFactor = factor;
+    }
+
+    resetScale() {
+        this.scaleFactor = 1;
+        this.prevScale = 0;
+    }
+
+    update(xrFrame?: XRFrame): boolean {
+        const { xrSession, xrRefSpace, input, camera, stereoCamera, pointerHelper } = this;
+        if (!xrFrame || !xrSession || !xrRefSpace) return false;
+
+        camera.scale = camera.scale * this.scaleFactor;
+        this.prevScale = camera.scale;
+        const camDirUnscaled = Vec3.sub(Vec3(), camera.position, camera.target);
+        Vec3.scaleAndAdd(camera.position, camera.position, camDirUnscaled, 1 - this.scaleFactor);
+        this.scaleFactor = 1;
+
+        const xform = getRigidTransformFromMat4(camera.view);
+        const xrOffsetRefSpace = xrRefSpace.getOffsetReferenceSpace(xform);
+        const xrPose = xrFrame.getViewerPose(xrOffsetRefSpace);
+        if (!xrPose) return false;
+
+        const xrHeadPose = xrFrame.getViewerPose(xrRefSpace);
+        if (xrHeadPose) {
+            const hq = Quat.fromObj(xrHeadPose.transform.orientation);
+            Mat4.fromQuat(camera.headRotation, hq);
+        }
+
+        const { depthFar, depthNear, baseLayer } = xrSession.renderState;
+        if (!baseLayer) return false;
+
+        if (depthFar !== camera.far || depthNear !== camera.near) {
+            xrSession.updateRenderState({
+                depthNear: camera.near,
+                depthFar: camera.far,
+            });
+        }
+
+        stereoCamera.update({ pose: xrPose, layer: baseLayer });
+        const camLeft = stereoCamera.left;
+
+        const cameraTarget = Vec3.scale(Vec3(), camLeft.state.target, camLeft.scale);
+        const cameraPosition = Mat4.getTranslation(Vec3(), Mat4.invert(Mat4(), camLeft.view));
+        const cameraDirection = Vec3.sub(Vec3(), cameraPosition, cameraTarget);
+        const cameraPlane = Plane3D.fromNormalAndCoplanarPoint(Plane3D(), cameraDirection, cameraTarget);
+
+        //
+
+        const pointers: Ray3D[] = [];
+        const points: Vec3[] = [];
+
+        const trackedPointers: TrackedPointerInput[] = [];
+        const screenTouches: ScreenTouchInput[] = [];
+
+        if (xrSession.inputSources) {
+            for (const inputSource of xrSession.inputSources) {
+                if (inputSource.targetRayMode === 'screen') {
+                    if (inputSource.gamepad) {
+                        const { axes } = inputSource.gamepad;
+                        const { width, height } = camLeft.viewport;
+                        const x = ((axes[0] + 1) / 2) * width;
+                        const y = ((axes[1] + 1) / 2) * height;
+                        const ray = camLeft.getRay(Ray3D(), x, height - y);
+                        screenTouches.push({ x, y, ray });
+                    }
+                    continue;
+                }
+
+                if (inputSource.targetRayMode !== 'tracked-pointer') continue;
+
+                const { handedness, targetRaySpace, gamepad } = inputSource;
+                if (!handedness) continue;
+
+                const targetRayPose = xrFrame.getPose(targetRaySpace!, xrRefSpace);
+                if (!targetRayPose) continue;
+
+                const ray = getRayFromPose(targetRayPose, camera.view);
+                pointers.push(ray);
+
+                const sceneBoundingSphere = Sphere3D.scaleNX(Sphere3D(), this.scene.boundingSphereVisible, camLeft.scale);
+
+                const si = Vec3();
+                if (Ray3D.intersectSphere3D(si, ray, sceneBoundingSphere)) {
+                    points.push(si);
+                }
+
+                let buttons = ButtonsType.create(ButtonsType.Flag.None);
+                if (gamepad?.buttons[0]?.pressed) buttons |= ButtonsType.Flag.Primary;
+                if (gamepad?.buttons[1]?.pressed) buttons |= ButtonsType.Flag.Secondary;
+                if (gamepad?.buttons[3]?.pressed) buttons |= ButtonsType.Flag.Auxilary;
+                if (gamepad?.buttons[4]?.pressed) buttons |= ButtonsType.Flag.Forth;
+                if (gamepad?.buttons[5]?.pressed) buttons |= ButtonsType.Flag.Five;
+
+                const prevInput = handedness === 'left' ? this.prevInput.left : this.prevInput.right;
+
+                const intersection = this.intersect(camLeft, camera.view, cameraPlane, targetRayPose);
+                const prevIntersection = prevInput ? this.intersect(camLeft, camera.view, cameraPlane, prevInput.targetRayPose) : undefined;
+
+                const [x, y] = intersection?.screen ?? [0, 0];
+                const [prevX, prevY] = prevIntersection?.screen ?? [x, y];
+
+                const dd = Vec2.set(Vec2(), x - prevX, y - prevY);
+                Vec2.setMagnitude(dd, dd, Math.min(100, Vec2.magnitude(dd)));
+                const [dx, dy] = Vec2.round(dd, dd);
+
+                trackedPointers.push({
+                    handedness,
+                    buttons,
+                    x, y, dx, dy, ray,
+                    axes: gamepad?.axes
+                });
+
+                if (handedness === 'left') {
+                    this.prevInput.left = { targetRayPose };
+                } else {
+                    this.prevInput.right = { targetRayPose };
+                }
+            }
+        } else {
+            this.prevInput.left = undefined;
+            this.prevInput.right = undefined;
+        }
+
+        input.updateTrackedPointers(trackedPointers);
+        input.updateScreenTouches(screenTouches);
+
+        pointerHelper.ensureEnabled();
+        pointerHelper.update(pointers, points, this.hit);
+
+        return true;
+    }
+
+    private async setSession(xrSession: XRSession | undefined) {
+        if (this.xrSession === xrSession) return;
+
+        await this.webgl.xr.set(xrSession, { resolutionScale: this.props.resolutionScale });
+
+        this.xrSession = this.webgl.xr.session;
+        this.prevInput = {};
+        this.hit = undefined;
+
+        if (this.xrSession) {
+            this.xrRefSpace = await this.xrSession.requestReferenceSpace('local');
+            this.pointerHelper.setProps({ enabled: 'on' });
+            let scale = this.prevScale;
+            if (scale === 0) {
+                const { radius } = this.scene.boundingSphereVisible;
+                scale = radius ? (1 / radius) * this.props.sceneRadiusInMeters : 0.01;
+            }
+            this.camera.forceFull = true;
+            this.camera.scale = scale;
+            this.camera.minTargetDistance = this.props.minTargetDistance;
+            this.prevScale = scale;
+        } else {
+            this.xrRefSpace = undefined;
+            Mat4.setZero(this.camera.headRotation);
+            this.pointerHelper.setProps({ enabled: 'off' });
+            this.camera.forceFull = false;
+            this.camera.scale = 1;
+            this.camera.minTargetDistance = 0;
+        }
+    }
+
+    async end() {
+        await this.webgl.xr.end();
+    }
+
+    private checkSupported = async () => {
+        if (!navigator.xr) return false;
+
+        const [arSupported, vrSupported] = await Promise.all([
+            navigator.xr.isSessionSupported('immersive-ar'),
+            navigator.xr.isSessionSupported('immersive-vr'),
+        ]);
+        this.isSupported.next(arSupported || vrSupported);
+    };
+
+    async request() {
+        if (!navigator.xr) return;
+
+        const session = await navigator.xr.isSessionSupported('immersive-ar')
+            ? await navigator.xr.requestSession('immersive-ar')
+            : await navigator.xr.requestSession('immersive-vr');
+
+        await this.setSession(session);
+    }
+
+    dispose() {
+        this.hoverSub.unsubscribe();
+        this.keyUpSub.unsubscribe();
+        this.gestureSub.unsubscribe();
+        this.sessionChangedSub.unsubscribe();
+
+        this.togglePassthrough.complete();
+        this.sessionChanged.complete();
+        this.isSupported.complete();
+
+        navigator.xr?.removeEventListener('devicechange', this.checkSupported);
+    }
+
+    constructor(private webgl: WebGLContext, private input: InputObserver, private scene: Scene, private camera: Camera, private stereoCamera: StereoCamera, private pointerHelper: PointerHelper, private interactionHelper: Canvas3dInteractionHelper, props: Partial<XRManagerProps> = {}, attribs: Partial<XRManagerAttribs> = {}) {
+        this.props = { ...PD.getDefaultValues(XRManagerParams), ...props };
+
+        this.hoverSub = this.interactionHelper.events.hover.subscribe(({ position }) => {
+            this.hit = position;
+        });
+
+        this.sessionChangedSub = webgl.xr.changed.subscribe(async () => {
+            await this.setSession(webgl.xr.session);
+            this.sessionChanged.next();
+        });
+
+        this.checkSupported();
+        navigator.xr?.addEventListener('devicechange', this.checkSupported);
+
+        const b = { ...DefaultXRManagerBindings, ...attribs.bindings };
+
+        this.keyUpSub = input.keyUp.subscribe(({ code, modifiers, key }) => {
+            if (Binding.matchKey(b.exit, code, modifiers, key)) {
+                this.end();
+            }
+
+            if (Binding.matchKey(b.togglePassthrough, code, modifiers, key)) {
+                this.togglePassthrough.next();
+            }
+        });
+
+        this.gestureSub = input.gesture.subscribe(({ scale, button, modifiers }) => {
+            if (Binding.match(b.gestureScale, button, modifiers)) {
+                this.setScaleFactor(scale);
+            }
+        });
+    }
+}

src/mol-canvas3d/passes/background.ts

@@ -121,8 +121,6 @@ export class BackgroundPass {
     private readonly position = Vec3();
     private readonly dir = Vec3();
 
-    readonly texture: Texture;
-
     constructor(private readonly webgl: WebGLContext, private readonly assetManager: AssetManager, width: number, height: number) {
         this.renderable = getBackgroundRenderable(webgl, width, height);
     }
@@ -445,8 +443,9 @@ function areImageTexturePropsEqual(sourceA: ImageProps['source'], sourceB: Image
 
 function getImageTexture(ctx: WebGLContext, assetManager: AssetManager, source: ImageProps['source'], onload?: (errored?: boolean) => void): { texture: Texture, asset: Asset } {
     const asset = source.name === 'url'
-        ? Asset.getUrlAsset(assetManager, source.params)
+        ? assetManager.tryFindFilename(source.params) ?? Asset.getUrlAsset(assetManager, source.params)
         : source.params!;
+
     if (typeof HTMLImageElement === 'undefined') {
         console.error(`Missing "HTMLImageElement" required for background image`);
         onload?.(true);

src/mol-canvas3d/passes/bloom.ts

@@ -75,6 +75,16 @@ export class BloomPass {
         this.copyRenderable = createCopyRenderable(webgl, this.compositeTarget.texture);
     }
 
+    getByteCount() {
+        return (
+            this.emissiveTarget.getByteCount() +
+            this.luminosityTarget.getByteCount() +
+            this.compositeTarget.getByteCount() +
+            this.horizontalBlurTargets.reduce((sum, t) => sum + t.getByteCount(), 0) +
+            this.verticalBlurTargets.reduce((sum, t) => sum + t.getByteCount(), 0)
+        );
+    }
+
     setSize(width: number, height: number) {
         const w = this.luminosityTarget.getWidth();
         const h = this.luminosityTarget.getHeight();

src/mol-canvas3d/passes/dof.ts

@@ -50,6 +50,10 @@ export class DofPass {
         this.renderable = getDofRenderable(webgl, nullTexture, nullTexture, nullTexture);
     }
 
+    getByteCount() {
+        return this.target.getByteCount();
+    }
+
     private updateState(viewport: Viewport) {
         const { gl, state } = this.webgl;
 
@@ -122,7 +126,7 @@ export class DofPass {
         const worldCenter = (props.center === 'scene-center' ? sphere.center : camera.state.target);
         const distance = Vec3.distance(camera.state.position, worldCenter);
         const inFocus = distance + props.inFocus;
-        ValueCell.updateIfChanged(this.renderable.values.uInFocus, inFocus * camera.state.scale);
+        ValueCell.updateIfChanged(this.renderable.values.uInFocus, inFocus * camera.scale);
 
         // transform center in view space
         const center = this.renderable.values.uCenter.ref.value;
@@ -130,7 +134,7 @@ export class DofPass {
         ValueCell.update(this.renderable.values.uCenter, center);
 
         ValueCell.updateIfChanged(this.renderable.values.uBlurSpread, props.blurSpread);
-        ValueCell.updateIfChanged(this.renderable.values.uPPM, props.PPM * camera.state.scale);
+        ValueCell.updateIfChanged(this.renderable.values.uPPM, props.PPM * camera.scale);
 
         if (needsUpdate) {
             this.renderable.update();

src/mol-canvas3d/passes/dpoit.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2022 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2022-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Gianluca Tomasello <giagitom@gmail.com>
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
@@ -89,6 +89,18 @@ export class DpoitPass {
         return this._supported;
     }
 
+    getByteCount() {
+        if (!this._supported) return 0;
+        return (
+            this.depthTextures[0].getByteCount() +
+            this.depthTextures[1].getByteCount() +
+            this.colorFrontTextures[0].getByteCount() +
+            this.colorFrontTextures[1].getByteCount() +
+            this.colorBackTextures[0].getByteCount() +
+            this.colorBackTextures[1].getByteCount()
+        );
+    }
+
     bind() {
         const { state, gl, extensions: { blendMinMax } } = this.webgl;
 
@@ -167,6 +179,7 @@ export class DpoitPass {
         if (isTimingMode) this.webgl.timer.mark('DpoitPass.render');
         const { state, gl } = this.webgl;
 
+        state.blendEquation(gl.FUNC_ADD);
         state.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);
 
         ValueCell.update(this.renderable.values.tDpoitFrontColor, this.colorFrontTextures[this.writeId]);

src/mol-canvas3d/passes/draw.ts

@@ -120,6 +120,25 @@ export class DrawPass {
         this.setTransparency(transparency);
     }
 
+    getByteCount() {
+        return (
+            this.drawTarget.getByteCount() +
+            this.colorTarget.getByteCount() +
+            this.transparentColorTarget.getByteCount() +
+            this.depthTargetTransparent.getByteCount() +
+            (this.depthTargetOpaque
+                ? this.depthTargetOpaque.getByteCount()
+                : this.depthTextureOpaque.getByteCount()) +
+            this.wboit.getByteCount() +
+            this.dpoit.getByteCount() +
+            this.marking.getByteCount() +
+            this.postprocessing.getByteCount() +
+            this.antialiasing.getByteCount() +
+            this.bloom.getByteCount() +
+            this.dof.getByteCount()
+        );
+    }
+
     reset() {
         this.wboit.reset();
         this.dpoit.reset();
@@ -198,8 +217,10 @@ export class DrawPass {
                 const dpoitTextures = this.dpoit.bindDualDepthPeeling();
                 renderer.renderDpoitTransparent(scene.primitives, camera, this.depthTextureOpaque, dpoitTextures);
 
-                target.bind();
-                this.dpoit.renderBlendBack();
+                if (iterations > 1) {
+                    target.bind();
+                    this.dpoit.renderBlendBack();
+                }
                 if (isTimingMode) this.webgl.timer.markEnd('DpoitPass.layer');
             }
 
@@ -372,6 +393,8 @@ export class DrawPass {
     }
 
     private _render(renderer: Renderer, camera: ICamera, scene: Scene, helper: Helper, toDrawingBuffer: boolean, transparentBackground: boolean, props: Props) {
+        if (camera.disabled) return;
+
         const volumeRendering = scene.volumes.renderables.length > 0;
         const postprocessingEnabled = PostprocessingPass.isEnabled(props.postprocessing);
         const antialiasingEnabled = AntialiasingPass.isEnabled(props.postprocessing);
@@ -430,6 +453,11 @@ export class DrawPass {
         if (helper.handle.isEnabled) {
             renderer.renderBlended(helper.handle.scene, camera);
         }
+        if (helper.pointer.isEnabled) {
+            helper.pointer.setCamera(camera);
+            renderer.update(helper.pointer.camera, helper.pointer.scene);
+            renderer.renderBlended(helper.pointer.scene, helper.pointer.camera);
+        }
         if (helper.camera.isEnabled) {
             helper.camera.update(camera);
             renderer.update(helper.camera.camera, helper.camera.scene);

src/mol-canvas3d/passes/hi-z.ts

@@ -140,6 +140,15 @@ export class HiZPass {
 
     readonly props: HiZProps;
 
+    getByteCount() {
+        if (!this.supported) return 0;
+        return (
+            this.tex.getByteCount() +
+            this.buf.getByteCount() +
+            this.levelData.reduce((sum, l) => sum + l.texture.getByteCount(), 0)
+        );
+    }
+
     clear() {
         if (!this.supported) return;
 

src/mol-canvas3d/passes/illumination.ts

@@ -36,6 +36,26 @@ import { SsaoProps } from './ssao';
 import { OutlinePass } from './outline';
 import { BloomPass } from './bloom';
 
+let IlluminationWarningShown = false;
+
+function checkIlluminationSupport(webgl: WebGLContext) {
+    const { drawBuffers, textureFloat, colorBufferFloat, depthTexture } = webgl.extensions;
+    if (!textureFloat || !colorBufferFloat || !depthTexture || !drawBuffers) {
+        if (isDebugMode && !IlluminationWarningShown) {
+            const missing: string[] = [];
+            if (!textureFloat) missing.push('textureFloat');
+            if (!colorBufferFloat) missing.push('colorBufferFloat');
+            if (!depthTexture) missing.push('depthTexture');
+            if (!drawBuffers) missing.push('drawBuffers');
+            console.log(`Missing "${missing.join('", "')}" extensions required for "illumination"`);
+            IlluminationWarningShown = true;
+        }
+        return false;
+    } else {
+        return true;
+    }
+}
+
 type Props = {
     transparentBackground: boolean;
     dpoitIterations: number;
@@ -90,25 +110,28 @@ export class IlluminationPass {
         return this._supported;
     }
 
-    getMaxIterations(props: Props) {
-        return Math.pow(2, props.illumination.maxIterations);
+    getByteCount() {
+        if (!this._supported) return 0;
+        return (
+            this.tracing.getByteCount() +
+            this.transparentTarget.getByteCount() +
+            this.outputTarget.getByteCount() +
+            this.multiSampleComposeTarget.getByteCount() +
+            this.multiSampleHoldTarget.getByteCount() +
+            this.multiSampleAccumulateTarget.getByteCount()
+        );
+    }
+
+    getMaxIterations(props: IlluminationProps) {
+        return Math.pow(2, props.maxIterations);
     }
 
     static isSupported(webgl: WebGLContext) {
-        const { drawBuffers, textureFloat, colorBufferFloat, depthTexture } = webgl.extensions;
-        if (!textureFloat || !colorBufferFloat || !depthTexture || !drawBuffers) {
-            if (isDebugMode) {
-                const missing: string[] = [];
-                if (!textureFloat) missing.push('textureFloat');
-                if (!colorBufferFloat) missing.push('colorBufferFloat');
-                if (!depthTexture) missing.push('depthTexture');
-                if (!drawBuffers) missing.push('drawBuffers');
-                console.log(`Missing "${missing.join('", "')}" extensions required for "illumination"`);
-            }
-            return false;
-        } else {
-            return true;
-        }
+        return checkIlluminationSupport(webgl);
+    }
+
+    static isEnabled(webgl: WebGLContext, props: IlluminationProps) {
+        return props.enabled && checkIlluminationSupport(webgl);
     }
 
     constructor(private readonly webgl: WebGLContext, private readonly drawPass: DrawPass) {
@@ -171,13 +194,15 @@ export class IlluminationPass {
                 const dpoitTextures = this.drawPass.dpoit.bind();
                 renderer.renderDpoitTransparent(scene.primitives, camera, this.drawPass.depthTextureOpaque, dpoitTextures);
 
-                for (let i = 0, il = props.dpoitIterations; i < il; i++) {
+                for (let i = 0, iterations = props.dpoitIterations; i < iterations; i++) {
                     if (isTimingMode) this.webgl.timer.mark('DpoitPass.layer');
                     const dpoitTextures = this.drawPass.dpoit.bindDualDepthPeeling();
                     renderer.renderDpoitTransparent(scene.primitives, camera, this.drawPass.depthTextureOpaque, dpoitTextures);
 
-                    this.transparentTarget.bind();
-                    this.drawPass.dpoit.renderBlendBack();
+                    if (iterations > 1) {
+                        this.transparentTarget.bind();
+                        this.drawPass.dpoit.renderBlendBack();
+                    }
                     if (isTimingMode) this.webgl.timer.markEnd('DpoitPass.layer');
                 }
 
@@ -238,8 +263,8 @@ export class IlluminationPass {
         if (isTimingMode) this.webgl.timer.markEnd('IlluminationPass.renderInput');
     }
 
-    shouldRender(props: Props) {
-        return this._supported && props.illumination.enabled && this._iteration < this.getMaxIterations(props);
+    shouldRender(props: IlluminationProps) {
+        return this._supported && props.enabled && this._iteration < this.getMaxIterations(props);
     }
 
     setSize(width: number, height: number) {
@@ -283,11 +308,11 @@ export class IlluminationPass {
     }
 
     private renderInternal(ctx: RenderContext, props: Props, toDrawingBuffer: boolean, forceRenderInput: boolean) {
-        if (!this.shouldRender(props)) return;
+        if (!this.shouldRender(props.illumination)) return;
 
         if (isTimingMode) {
             this.webgl.timer.mark('IlluminationPass.render', {
-                note: `iteration ${this._iteration + 1} of ${this.getMaxIterations(props)}`
+                note: `iteration ${this._iteration + 1} of ${this.getMaxIterations(props.illumination)}`
             });
         }
         this.tracing.render(ctx, props.transparentBackground, props.illumination, this._iteration, forceRenderInput);
@@ -396,7 +421,7 @@ export class IlluminationPass {
         }
         const denoiseThreshold = props.multiSample.mode === 'on'
             ? props.illumination.denoiseThreshold[0]
-            : lerp(props.illumination.denoiseThreshold[1], props.illumination.denoiseThreshold[0], clamp(this.iteration / (this.getMaxIterations(props) / 2), 0, 1));
+            : lerp(props.illumination.denoiseThreshold[1], props.illumination.denoiseThreshold[0], clamp(this.iteration / (this.getMaxIterations(props.illumination) / 2), 0, 1));
         ValueCell.updateIfChanged(this.composeRenderable.values.uDenoiseThreshold, denoiseThreshold);
         if (needsUpdateCompose) this.composeRenderable.update();
         this.composeRenderable.render();
@@ -474,7 +499,7 @@ export class IlluminationPass {
         // each sample with camera jitter and accumulates the results.
         const offsetList = JitterVectors[Math.max(0, Math.min(props.multiSample.sampleLevel, 5))];
 
-        const maxIterations = this.getMaxIterations(props);
+        const maxIterations = this.getMaxIterations(props.illumination);
         const iteration = Math.min(this._iteration, maxIterations);
 
         const sampleIndex = Math.floor((iteration / maxIterations) * offsetList.length);

src/mol-canvas3d/passes/image.ts

@@ -21,8 +21,9 @@ import { MarkingParams } from './marking';
 import { AssetManager } from '../../mol-util/assets';
 import { IlluminationParams, IlluminationPass } from './illumination';
 import { RuntimeContext } from '../../mol-task';
-import { isTimingMode } from '../../mol-util/debug';
+import { isDebugMode, isTimingMode } from '../../mol-util/debug';
 import { printTimerResults } from '../../mol-gl/webgl/timer';
+import { ShaderManager } from '../helper/shader-manager';
 
 export const ImageParams = {
     transparentBackground: PD.Boolean(false),
@@ -68,11 +69,16 @@ export class ImagePass {
             camera: new CameraHelper(webgl, this.props.cameraHelper),
             debug: helper.debug,
             handle: helper.handle,
+            pointer: helper.pointer,
         };
 
         this.setSize(1024, 768);
     }
 
+    getByteCount() {
+        return this.drawPass.getByteCount() + this.illuminationPass.getByteCount() + this.multiSamplePass.getByteCount();
+    }
+
     updateBackground() {
         return new Promise<void>(resolve => {
             this.drawPass.postprocessing.background.update(this.camera, this.props.postprocessing.background, () => {
@@ -98,15 +104,16 @@ export class ImagePass {
     }
 
     async render(runtime: RuntimeContext) {
+        ShaderManager.ensureRequired(this.webgl, this.scene, this.props);
         Camera.copySnapshot(this._camera.state, this.camera.state);
         Viewport.set(this._camera.viewport, 0, 0, this._width, this._height);
         this._camera.update();
 
         const ctx = { renderer: this.renderer, camera: this._camera, scene: this.scene, helper: this.helper };
         if (this.illuminationPass.supported && this.props.illumination.enabled) {
-            await runtime.update({ message: 'Tracing...', current: 1, max: this.illuminationPass.getMaxIterations(this.props) });
+            await runtime.update({ message: 'Tracing...', current: 1, max: this.illuminationPass.getMaxIterations(this.props.illumination) });
             this.illuminationPass.restart(true);
-            while (this.illuminationPass.shouldRender(this.props)) {
+            while (this.illuminationPass.shouldRender(this.props.illumination)) {
                 if (isTimingMode) this.webgl.timer.mark('ImagePass.render', { captureStats: true });
                 this.illuminationPass.render(ctx, this.props, false);
                 if (isTimingMode) this.webgl.timer.markEnd('ImagePass.render');
@@ -137,13 +144,8 @@ export class ImagePass {
             }
         }
 
-        if (isTimingMode) {
-            const timerResults = this.webgl.timer.resolve();
-            if (timerResults) {
-                for (const result of timerResults) {
-                    printTimerResults([result]);
-                }
-            }
+        if (isDebugMode) {
+            console.log(`image pass byte count ${(this.getByteCount() / 1024 / 1024).toFixed(3)} MiB`);
         }
     }
 

src/mol-canvas3d/passes/marking.ts

@@ -55,6 +55,10 @@ export class MarkingPass {
         this.overlay = getOverlayRenderable(webgl, this.edgesTarget.texture);
     }
 
+    getByteCount() {
+        return this.depthTarget.getByteCount() + this.maskTarget.getByteCount() + this.edgesTarget.getByteCount();
+    }
+
     private setEdgeState(viewport: Viewport) {
         const { gl, state } = this.webgl;
 

src/mol-canvas3d/passes/multi-sample.ts

@@ -96,6 +96,10 @@ export class MultiSamplePass {
         this.compose = getComposeRenderable(webgl, drawPass.colorTarget.texture);
     }
 
+    getByteCount() {
+        return this.colorTarget.getByteCount() + this.composeTarget.getByteCount() + this.holdTarget.getByteCount();
+    }
+
     syncSize() {
         const width = this.drawPass.colorTarget.getWidth();
         const height = this.drawPass.colorTarget.getHeight();

src/mol-canvas3d/passes/outline.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2019-2024 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2019-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
  * @author Áron Samuel Kovács <aron.kovacs@mail.muni.cz>
@@ -47,6 +47,10 @@ export class OutlinePass {
         this.renderable = getOutlinesRenderable(webgl, depthTextureOpaque, depthTextureTransparent, true);
     }
 
+    getByteCount() {
+        return this.target.getByteCount();
+    }
+
     setSize(width: number, height: number) {
         const [w, h] = this.renderable.values.uTexSize.ref.value;
         if (width !== w || height !== h) {

src/mol-canvas3d/passes/passes.ts

@@ -25,6 +25,10 @@ export class Passes {
         this.illumination = new IlluminationPass(webgl, this.draw);
     }
 
+    getByteCount() {
+        return this.draw.getByteCount() + this.pick.getByteCount() + this.multiSample.getByteCount() + this.illumination.getByteCount();
+    }
+
     setPickScale(pickScale: number) {
         this.pick.setPickScale(pickScale);
     }

src/mol-canvas3d/passes/pick.ts

@@ -22,8 +22,6 @@ import { ICamera } from '../camera';
 import { Viewport } from '../camera/util';
 import { Helper } from '../helper/helper';
 
-const NullId = Math.pow(2, 24) - 2;
-
 export type PickData = { id: PickingId, position: Vec3 }
 
 export type AsyncPickData = {
@@ -119,6 +117,25 @@ export class PickPass {
         }
     }
 
+    getByteCount() {
+        if (this.webgl.extensions.drawBuffers) {
+            return (
+                this.objectPickTexture.getByteCount() +
+                this.instancePickTexture.getByteCount() +
+                this.groupPickTexture.getByteCount() +
+                this.depthPickTexture.getByteCount() +
+                this.depthRenderbuffer.getByteCount()
+            );
+        } else {
+            return (
+                this.objectPickTarget.getByteCount() +
+                this.instancePickTarget.getByteCount() +
+                this.groupPickTarget.getByteCount() +
+                this.depthPickTarget.getByteCount()
+            );
+        }
+    }
+
     dispose() {
         if (this.webgl.extensions.drawBuffers) {
             this.framebuffer.destroy();
@@ -261,7 +278,9 @@ export class PickPass {
         if (this.webgl.extensions.drawBuffers) {
             this.framebuffer.bind();
             this.renderVariant(renderer, camera, scene, helper, 'pick', PickType.None);
-            // printTextureImage(readTexture(this.webgl, this.groupPickTexture, new Uint8Array(this.pickWidth * this.pickHeight * 4)), { scale: 16, id: 'group', pixelated: true, useCanvas: true, flipY: true });
+            // if (this.pickWidth < 256) {
+            //     printTextureImage(readTexture(this.webgl, this.groupPickTexture, new Uint8Array(this.pickWidth * this.pickHeight * 4)), { scale: 16, id: 'group', pixelated: true, useCanvas: true, flipY: true });
+            // }
         } else {
             this.objectPickTarget.bind();
             this.renderVariant(renderer, camera, scene, helper, 'pick', PickType.Object);
@@ -450,15 +469,15 @@ export class PickBuffers {
     getPickingId(x: number, y: number): PickingId | undefined {
         const objectId = this.getObjectId(x, y);
         // console.log('objectId', objectId);
-        if (objectId === -1 || objectId === NullId) return;
+        if (objectId === -1 || objectId === PickingId.Null) return;
 
         const instanceId = this.getInstanceId(x, y);
         // console.log('instanceId', instanceId);
-        if (instanceId === -1 || instanceId === NullId) return;
+        if (instanceId === -1 || instanceId === PickingId.Null) return;
 
         const groupId = this.getGroupId(x, y);
         // console.log('groupId', groupId);
-        if (groupId === -1 || groupId === NullId) return;
+        if (groupId === -1) return;
 
         return { objectId, instanceId, groupId };
     }

src/mol-canvas3d/passes/postprocessing.ts

@@ -161,7 +161,7 @@ export class PostprocessingPass {
     }
 
     static isTransparentDepthRequired(scene: Scene, props: PostprocessingProps) {
-        return props.enabled && (DofPass.isEnabled(props) || OutlinePass.isEnabled(props) && PostprocessingPass.isTransparentOutlineEnabled(props) || SsaoPass.isEnabled(props) && PostprocessingPass.isTransparentSsaoEnabled(scene, props));
+        return props.enabled && (DofPass.isEnabled(props) || OutlinePass.isEnabled(props) && PostprocessingPass.isTransparentOutlineEnabled(props) || SsaoPass.isEnabled(props) && PostprocessingPass.isTransparentSsaoEnabled(scene, props)) && scene.opacityAverage < 1;
     }
 
     static isTransparentOutlineEnabled(props: PostprocessingProps) {
@@ -202,6 +202,15 @@ export class PostprocessingPass {
         this.background = new BackgroundPass(webgl, assetManager, width, height);
     }
 
+    getByteCount() {
+        return (
+            this.target.getByteCount() +
+            this.ssao.getByteCount() +
+            this.shadow.getByteCount() +
+            this.outline.getByteCount()
+        );
+    }
+
     setSize(width: number, height: number) {
         const [w, h] = this.renderable.values.uTexSize.ref.value;
 
@@ -374,6 +383,14 @@ export class AntialiasingPass {
         this.cas = new CasPass(webgl, this.target.texture);
     }
 
+    getByteCount() {
+        return (
+            this.target.getByteCount() +
+            this.internalTarget.getByteCount() +
+            this.smaa.getByteCount()
+        );
+    }
+
     setSize(width: number, height: number) {
         const w = this.target.texture.getWidth();
         const h = this.target.texture.getHeight();

src/mol-canvas3d/passes/shadow.ts

@@ -21,7 +21,7 @@ import { RenderTarget } from '../../mol-gl/webgl/render-target';
 import { ICamera } from '../../mol-canvas3d/camera';
 import { quad_vert } from '../../mol-gl/shader/quad.vert';
 import { isTimingMode } from '../../mol-util/debug';
-import { Light } from '../../mol-gl/renderer';
+import { getTransformedLightDirection, Light } from '../../mol-gl/renderer';
 import { shadows_frag } from '../../mol-gl/shader/shadows.frag';
 import { PostprocessingProps } from './postprocessing';
 
@@ -41,11 +41,18 @@ export class ShadowPass {
     readonly target: RenderTarget;
     private readonly renderable: ShadowsRenderable;
 
+    private invProjection = Mat4.identity();
+    private invHeadRotation = Mat4.identity();
+
     constructor(readonly webgl: WebGLContext, width: number, height: number, depthTextureOpaque: Texture) {
         this.target = webgl.createRenderTarget(width, height, false);
         this.renderable = getShadowsRenderable(webgl, depthTextureOpaque);
     }
 
+    getByteCount() {
+        return this.target.getByteCount();
+    }
+
     setSize(width: number, height: number) {
         const [w, h] = this.renderable.values.uTexSize.ref.value;
         if (width !== w || height !== h) {
@@ -59,14 +66,11 @@ export class ShadowPass {
 
         const orthographic = camera.state.mode === 'orthographic' ? 1 : 0;
 
-        const invProjection = Mat4.identity();
-        Mat4.invert(invProjection, camera.projection);
-
         const [w, h] = this.renderable.values.uTexSize.ref.value;
         const v = camera.viewport;
 
         ValueCell.update(this.renderable.values.uProjection, camera.projection);
-        ValueCell.update(this.renderable.values.uInvProjection, invProjection);
+        ValueCell.update(this.renderable.values.uInvProjection, Mat4.invert(this.invProjection, camera.projection));
 
         Vec4.set(this.renderable.values.uBounds.ref.value,
             v.x / w,
@@ -83,14 +87,19 @@ export class ShadowPass {
             needsUpdateShadows = true;
         }
 
-        ValueCell.updateIfChanged(this.renderable.values.uMaxDistance, props.maxDistance * camera.state.scale);
-        ValueCell.updateIfChanged(this.renderable.values.uTolerance, props.tolerance * camera.state.scale);
+        ValueCell.updateIfChanged(this.renderable.values.uMaxDistance, props.maxDistance * camera.scale);
+        ValueCell.updateIfChanged(this.renderable.values.uTolerance, props.tolerance * camera.scale);
         if (this.renderable.values.dSteps.ref.value !== props.steps) {
             ValueCell.update(this.renderable.values.dSteps, props.steps);
             needsUpdateShadows = true;
         }
 
-        ValueCell.update(this.renderable.values.uLightDirection, light.direction);
+        const hasHeadRotation = !Mat4.isZero(camera.headRotation);
+        if (hasHeadRotation) {
+            ValueCell.update(this.renderable.values.uLightDirection, getTransformedLightDirection(light, Mat4.invert(this.invHeadRotation, camera.headRotation)));
+        } else {
+            ValueCell.update(this.renderable.values.uLightDirection, light.direction);
+        }
         ValueCell.update(this.renderable.values.uLightColor, light.color);
         if (this.renderable.values.dLightCount.ref.value !== light.count) {
             ValueCell.update(this.renderable.values.dLightCount, light.count);

src/mol-canvas3d/passes/smaa.ts

@@ -62,6 +62,11 @@ export class SmaaPass {
         this._supported = true;
     }
 
+    getByteCount() {
+        if (!this.supported) return 0;
+        return this.edgesTarget.getByteCount() + this.weightsTarget.getByteCount();
+    }
+
     private updateState(viewport: Viewport) {
         const { gl, state } = this.webgl;
 

src/mol-canvas3d/passes/ssao.ts

@@ -80,15 +80,13 @@ function getLevels(props: { radius: number, bias: number }[], scale: number, lev
 
 export class SsaoPass {
     static isEnabled(props: PostprocessingProps) {
-        return props.occlusion.name !== 'off';
+        return props.enabled && props.occlusion.name !== 'off';
     }
 
     static isTransparentEnabled(scene: Scene, props: SsaoProps) {
         return scene.opacityAverage < 1 && scene.transparencyMin < props.transparentThreshold;
     }
 
-    readonly target: RenderTarget;
-
     private readonly framebuffer: Framebuffer;
     private readonly blurFirstPassFramebuffer: Framebuffer;
     private readonly blurSecondPassFramebuffer: Framebuffer;
@@ -134,6 +132,7 @@ export class SsaoPass {
         return Math.min(1, 1 / this.webgl.pixelRatio) * resolutionScale;
     }
 
+    private levelsCameraScale = -1;
     private levels: { radius: number, bias: number }[];
 
     private getDepthTexture() {
@@ -213,6 +212,20 @@ export class SsaoPass {
         this.blurSecondPassRenderable = getSsaoBlurRenderable(webgl, this.depthBlurProxyTexture, 'vertical');
     }
 
+    getByteCount() {
+        return (
+            this.downsampledDepthTargetOpaque.getByteCount() +
+            this.depthHalfTargetOpaque.getByteCount() +
+            this.depthQuarterTargetOpaque.getByteCount() +
+            this.downsampledDepthTargetTransparent.getByteCount() +
+            this.depthHalfTargetTransparent.getByteCount() +
+            this.depthQuarterTargetTransparent.getByteCount() +
+            this.ssaoDepthTexture.getByteCount() +
+            this.ssaoDepthTransparentTexture.getByteCount() +
+            this.depthBlurProxyTexture.getByteCount()
+        );
+    }
+
     setSize(width: number, height: number) {
         const [w, h] = this.texSize;
         const ssaoScale = this.calcSsaoScale(1);
@@ -305,8 +318,8 @@ export class SsaoPass {
         ValueCell.update(this.blurFirstPassRenderable.values.uInvProjection, invProjection);
         ValueCell.update(this.blurSecondPassRenderable.values.uInvProjection, invProjection);
 
-        ValueCell.update(this.blurFirstPassRenderable.values.uBlurDepthBias, props.blurDepthBias * camera.state.scale);
-        ValueCell.update(this.blurSecondPassRenderable.values.uBlurDepthBias, props.blurDepthBias * camera.state.scale);
+        ValueCell.update(this.blurFirstPassRenderable.values.uBlurDepthBias, props.blurDepthBias * camera.scale);
+        ValueCell.update(this.blurSecondPassRenderable.values.uBlurDepthBias, props.blurDepthBias * camera.scale);
 
         if (this.blurFirstPassRenderable.values.dOrthographic.ref.value !== orthographic) {
             needsUpdateSsaoBlur = true;
@@ -344,11 +357,12 @@ export class SsaoPass {
 
         if (props.multiScale.name === 'on') {
             const mp = props.multiScale.params;
-            if (!deepEqual(this.levels, mp.levels)) {
+            if (this.levelsCameraScale !== camera.scale || !deepEqual(this.levels, mp.levels)) {
                 needsUpdateSsao = true;
 
+                this.levelsCameraScale = camera.scale;
                 this.levels = mp.levels;
-                const levels = getLevels(mp.levels, camera.state.scale);
+                const levels = getLevels(mp.levels, camera.scale);
                 ValueCell.updateIfChanged(this.renderable.values.dLevels, levels.count);
 
                 ValueCell.update(this.renderable.values.uLevelRadius, levels.radius);
@@ -357,7 +371,7 @@ export class SsaoPass {
             ValueCell.updateIfChanged(this.renderable.values.uNearThreshold, mp.nearThreshold);
             ValueCell.updateIfChanged(this.renderable.values.uFarThreshold, mp.farThreshold);
         } else {
-            ValueCell.updateIfChanged(this.renderable.values.uRadius, Math.pow(2, props.radius) * camera.state.scale);
+            ValueCell.updateIfChanged(this.renderable.values.uRadius, Math.pow(2, props.radius) * camera.scale);
         }
         ValueCell.updateIfChanged(this.renderable.values.uBias, props.bias);
 

src/mol-canvas3d/passes/tracing.ts

@@ -126,6 +126,18 @@ export class TracingPass {
         this.accumulateRenderable = getAccumulateRenderable(webgl, this.holdTarget.texture);
     }
 
+    getByteCount() {
+        return (
+            this.thicknessTarget.getByteCount() +
+            this.holdTarget.getByteCount() +
+            this.accumulateTarget.getByteCount() +
+            this.composeTarget.getByteCount() +
+            this.colorTextureOpaque.getByteCount() +
+            this.normalTextureOpaque.getByteCount() +
+            this.shadedTextureOpaque.getByteCount()
+        );
+    }
+
     private renderInput(renderer: Renderer, camera: ICamera, scene: Scene, props: TracingProps) {
         if (isTimingMode) this.webgl.timer.mark('TracePass.renderInput');
         const { gl, state } = this.webgl;

src/mol-canvas3d/passes/wboit.ts

@@ -1,5 +1,5 @@
 /**
- * Copyright (c) 2020 mol* contributors, licensed under MIT, See LICENSE file for more info.
+ * Copyright (c) 2020-2025 mol* contributors, licensed under MIT, See LICENSE file for more info.
  *
  * @author Áron Samuel Kovács <aron.kovacs@mail.muni.cz>
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
@@ -59,6 +59,11 @@ export class WboitPass {
         return this._supported;
     }
 
+    getByteCount() {
+        if (!this._supported) return 0;
+        return this.textureA.getByteCount() + this.textureB.getByteCount() + this.depthRenderbuffer.getByteCount();
+    }
+
     bind() {
         const { state, gl } = this.webgl;
 

src/mol-data/util/hash-functions.ts

@@ -3,6 +3,7 @@
  *
  * @author David Sehnal <david.sehnal@gmail.com>
  * @author Alexander Rose <alexander.rose@weirdbyte.de>
+ * @author Adam Midlik <midlik@gmail.com>
  */
 
 // from http://burtleburtle.net/bob/hash/integer.html
@@ -365,6 +366,21 @@ export function murmurHash3_32(key: string, seed: number): number {
     return h >>> 0;
 }
 
+/**
+ * MurmurHash3 128-bit implementation
+ * @param key - The input data to hash
+ * @param seed - The seed value (default: 0)
+ * @returns The 128-bit hash as a hexadecimal string
+ */
+export function murmurHash3_128_fromBytes(key: Uint8Array, seed: number): string {
+    // This fakeString approach is much faster than `new TextDecoder('ascii').decode(key)`
+    const fakeString = {
+        length: key.length,
+        charCodeAt(i: number) { return key[i]; },
+    };
+    return murmurHash3_128(fakeString as string, seed);
+}
+
 /**
  * MurmurHash3 128-bit implementation
  * @param key - The input string to hash

src/mol-geo/geometry/base.ts

@@ -120,8 +120,6 @@ export namespace BaseGeometry {
             uBumpiness: ValueCell.create(props.material.bumpiness),
             uEmissive: ValueCell.create(props.emissive),
             uDensity: ValueCell.create(props.density),
-            dLightCount: ValueCell.create(1),
-            dColorMarker: ValueCell.create(true),
 
             dClipObjectCount: ValueCell.create(clip.objects.count),
             dClipVariant: ValueCell.create(clip.variant),