Multi-panel plot in Jupyter Lab

[HeavenlyBerserker]

I am trying to create a multi-panel interactive plot in Jupyter Lab in Ubuntu 20. The code visualizes multiple meshes and points in the space on each panel. The goal is to be able to visualize and interact with all the panels while the points update, i.e. rotate them at the same time.

I tried running it with the vtk default_backend, but I can't interact with it, and I cannot close it, which is problematic. That said, it works at displaying the changing points, and I can interact with it after all updates are done, but I can't interact during the updates.

import vedo

vedo.settings.default_backend= 'vtk'

path = "some/path/meshes/"

mesh_file_names = os.listdir(path)

# this is one instance of the class Plotter
vp1 = vedo.Plotter(N = len(mesh_file_names), title='Mesh Particle Visualizer')

meshes = []

for i in range(num_meshes):
    fname = new_meshes[i]

    mesh_i = vedo.Mesh(fname)
    mesh_i.alpha(0.8)
    meshes.append(mesh_i)

    vp1.show(os.path.basename(fname), at=i)
    vp1.show(meshes[i], at=i)

# Adding points
for i in range(num_meshes):
        points = vedo.Points(some_pts[i], c=colors, r=5)
        vp1.add(points, at=i)

Then I update the points multiple times using

# Updating points
for some iterations:
    for i in range(num_meshes):
        some_pts[I] = some_change(some_pts[I])
    sleep(100)
    vp1.render().reset_camera()

I thought the k3d default_backend would help me circumvent the window-not-closing issue by having the visualization in the notebook output, but I'm getting the following error when running vp1.add(points, at=i):

File ~/anaconda3/envs/some_conda_env/lib/python3.9/site-packages/vedo/plotter.py:815, in Plotter.at(self, nren, yren)
    812         vedo.logger.error(f"at({nren, yren}) is malformed!")
    813         raise RuntimeError
--> 815 self.renderer = self.renderers[nren]
    816 self.camera = self.renderer.GetActiveCamera()
    817 return self

IndexError: list index out of range

I believe this is because the k3d backend does not support multi-panel visualization. The Vedo documentation says that Vedo can run on other backends like ipyvtklink and trame. Does anybody know if this will work for me?

Update: I conda installed ipyvtklink and set the backend to ipyvtk and no visualization showed and also no error. Still can't figure it out.

All in all, I'm unsure how to get this to work. Again, my goal is to be able to visualize N meshes and M changing points in their panels while interacting with them during the changes. Would also be great if I could close the window by clicking the x of the vtkplotter. Any help or pointers would be appreciated!

[marcomusy]

Consider the following example:

import vedo
import numpy as np

vedo.settings.default_backend = "vtk"
vedo.settings.immediate_rendering = False

meshes = [vedo.Ellipsoid().color(i) for i in range(30)]
num_meshes = len(meshes)

plt = vedo.Plotter(N=num_meshes, title="Mesh Particle Visualizer", size=(1800, 1300))

for i in range(num_meshes):
    mesh_i = meshes[i].rotate_z(i)
    mesh_i.alpha(0.8)
    plt.at(i).show(f"mesh #{i}", mesh_i)

def loop_func(event):
    for i in range(num_meshes):
        plt.at(i).remove("Points")
        some_pts = np.random.rand(100, 3)
        points = vedo.Points(some_pts, c=i, r=5)
        plt.add(points)
    plt.render()

plt.add_callback("timer", loop_func)
plt.timer_callback("start")
plt.interactive()
plt.close()

Note that if you interact with the scene the program flow will hold.

[HeavenlyBerserker]

Thank you for the prompt answer. This is very helpful! The exact code you send me works, but it's not exactly what I want. I'm not running a timer exactly, but just running code that computes the updated points in an optimization loop (this time taken is symbolized by sleep(100) in my code above). Is there any way to keep running the code and update the results through a function (say update_points() ) while maintaining interactivity?

My code keeps running if I don't do plt.interactive() and plt.close(), but naturally, this removes interactivity. Is there any callback or event setup to maintain interactivity while still running the rest of my code? Please let me know if this doesn't make sense.

Also, I'm getting this error on interrupt when running your code:

Kernel Restarting
The kernel for notebook.ipynb appears to have died. It will restart automatically.

I'm forced to restart the kernel after that. It's not a big deal, but I wonder if it's a replicable issue.

[marcomusy]

Uhm.. does it also crash in the latest dev?

pip install -U git+https://github.com/marcomusy/vedo.git

[HeavenlyBerserker]

It works with the latest dev. It's also significantly faster! Thanks.

I still have the problem of keeping my optimization code running while maintaining interactivity though, something like this

import vedo
import numpy as np

vedo.settings.default_backend = "vtk"
vedo.settings.immediate_rendering = False

meshes = [vedo.Ellipsoid().color(i) for i in range(30)]
num_meshes = len(meshes)

plt = vedo.Plotter(N=num_meshes, title="Mesh Particle Visualizer", size=(1800, 1300))

for i in range(num_meshes):
    mesh_i = meshes[i].rotate_z(i)
    mesh_i.alpha(0.8)
    plt.at(i).show(f"mesh #{i}", mesh_i)

# Adds initial points and shows them
current_points = set_initial_points()

# Keep interactivity while running an optimization that updates the points
for i in num_optimization_iterations:
    new_points = optimization_iteration(current_points) # Some optimization routine function
    update_points() # This can be the same loop_func you provided
    current_points = new_points

plt.close()

Is there a way to do what I'm describing with Vedo, or is this impossible?

[marcomusy]

That is not possible i'm afraid (but I think it's a general limitation of the upstream vtk).
You can try with embedding your window in a Qt application. Check out this example:
/examples/other/qt_window1.py

[HeavenlyBerserker]

(1) There might be a bug with /examples/other/qt_window1.py. I get the following error when I click on the button labeled "My Button makes the cone red"

..calling onClick
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [1], line 43, in MainWindow.onClick(self)
     41 printc("..calling onClick")
     42 self.plt.objects[0].color('red').rotate_z(40)
---> 43 self.plt.interactor.Render()

AttributeError: 'NoneType' object has no attribute 'Render'

(2) I'm not having any luck with modifying /examples/other/qt_window1.py to continue running code while running in the backgound. I tried adding a print loop to test if the code will continue running, but no success. Any pointers on what you had in mind? This is what I modified in the main call.

if __name__ == "__main__":
    app = Qt.QApplication(sys.argv)
    window = MainWindow()
    app.aboutToQuit.connect(window.onClose) # <-- connect the onClose event
    app.exec_()
    x = 1
    while(True):
        time.sleep(1000)
        print(x)
        x += 1

(3) Another problem I'm finding might be related to #908. I remember why I deliberately used an old version of vedo. I used to be able to use 'vedo.Points(pts, c=colors, r=5)' with colors being a Nx3 or Nx4 matrix, but now I get an error when running this

import vedo

pts = [[0,0,0], [0,0,0]]
colors = [(255,255,255), (255,255,255)]

vedo.Points(pts, c=colors, r=5)

This is the error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [2], line 6
      3 pts = [[0,0,0], [0,0,0]]
      4 colors = [(255,255,255), (255,255,255)]
----> 6 vedo.Points(pts, c=colors, r=5)

File ~/anaconda3/envs/novassm/lib/python3.9/site-packages/vedo/pointcloud.py:585, in Points.__init__(self, inputobj, r, c, alpha)
    582 self.actor.SetMapper(self.mapper)
    583 self.mapper.SetInputData(self.dataset)
--> 585 self.properties.SetColor(colors.get_color(c))
    586 self.properties.SetOpacity(alpha)
    587 self.properties.SetRepresentationToPoints()

TypeError: SetColor argument 1: expected a sequence of 3 values, got 2 values

I don't know if removing this feature was intentional. The documentation still says "For very large point clouds a list of colors and alpha can be assigned to each point in the form c=[(R,G,B,A), ... ] where 0<=R<256, ... 0<=A<256." so I assume it's a bug. It's a very useful feature so I hope it can be kept!

[marcomusy]

Thanks for reporting all this and for your patience, I will have a look at all this over the weekend!

[HeavenlyBerserker]

Hi @marcomusy, have you been able to look into (3) above? I have confirmed that vedo-2023.4.4 has the correct behavior and vedo-2024.5.1+dev04 does not.

[HeavenlyBerserker]

I don't know if removing this feature was intentional. The documentation still says "For very large point clouds a list of colors and alpha can be assigned to each point in the form c=[(R,G,B,A), ... ] where 0<=R<256, ... 0<=A<256." so I assume it's a bug. It's a very useful feature so I hope it can be kept!

Oh, I see what I had wrong. I can directly do object.c = [(R,G,B,A), ... ], and not object.c([(R,G,B,A), ... ]). This aspect is resolved. This should be resolved now. Thank you!