vispy flickering on Windows with real time data

[ParthShah91]

I am trying to create a real time application using vispy. Basically I need to maintain a window of 1000 data at a time. So once the data that I receive crosses 1000 I will try to shift the window as well as the x-axis using 'set_range' method. This is when the flickering issues start appearing. Below is the code for the same. I am executing the code on Windows 10 and my python version is 3.12.1.

Have also attached the execution logs for your reference
realtime_execution_logs.txt

import sys
import numpy as np
from vispy import app, scene, sys_info
import random
from vispy.color.color_array import Color

print('='*80)
print(sys_info())
print('='*80)

WINDOW_SIZE = 1000
MAX_DATA_TO_PLOT = 3000
MAX_DATA_STREAMS = 8

# vertex positions of data to draw
N = WINDOW_SIZE

canvas = scene.SceneCanvas(keys='interactive', show=True)
grid = canvas.central_widget.add_grid(spacing=0)

viewbox = grid.add_view(row=0, col=1, camera='panzoom')

# add some axes
x_axis = scene.AxisWidget(orientation='bottom')
x_axis.stretch = (1, 0.1)
grid.add_widget(x_axis, row=1, col=1)
x_axis.link_view(viewbox)
y_axis = scene.AxisWidget(orientation='left')
y_axis.stretch = (0.1, 1)
grid.add_widget(y_axis, row=0, col=0)
y_axis.link_view(viewbox)

# add a line plot inside the viewbox
lines_and_data = []
for i in range(MAX_DATA_STREAMS):
    data = np.empty((0, 2), dtype=np.float32)
    print(data.shape)
    line = scene.Line(data, color=None, parent=viewbox.scene, method='gl')
    color = np.empty((0, 4), dtype=np.float32)
    lines_and_data.append([line, data, color])

value_ranges = [
    (10, 20, [0, 0, 1, 1]),
    (25,35, [0, 1, 0, 1]),
    (40, 50, [0, 1, 1, 1]),
    (55,65, [1, 0, 0, 1]),
    (70,80, [1, 0, 1, 1]),
    (85,95, [1, 1, 0, 1]),
    (100,110, [1, 1, 1, 1]),
    (115,125, [0, 1, 0, 1])
]

def update(ev):
    global color, line, update_call_cnt, lines_and_data

    for stream_idx, el in enumerate(lines_and_data):
        line, data, color = el

        new_element_x = update_call_cnt
        new_element_y = random.randint(value_ranges[stream_idx][0], value_ranges[stream_idx][1])

        new_row = np.array([[new_element_x, new_element_y]])
        data = np.vstack((data, new_row))        
        lines_and_data[stream_idx][1] = data.copy()

        new_row = np.array(value_ranges[stream_idx][2])
        color = np.vstack((color, new_row))
        lines_and_data[stream_idx][2] = color.copy()
        
        line.set_data(pos=data, color=color)

        stream_idx = stream_idx + 1

    update_call_cnt = update_call_cnt + 1

    if update_call_cnt > WINDOW_SIZE:
        viewbox.camera.set_range((update_call_cnt - WINDOW_SIZE, update_call_cnt), (0, 140))
    else:
        viewbox.camera.set_range((0,WINDOW_SIZE), (0,140))

    return

timer = app.Timer(interval='auto', iterations=MAX_DATA_TO_PLOT)
timer.connect(update)
timer.start()
update_call_cnt = 0

if __name__ == '__main__' and sys.flags.interactive == 0:
    app.run()

[ParthShah91]

@djhoese Sorry to tag you directly but can you help us here?

[aganders3]

I can take a look tomorrow most likely. I tried the code and didn't notice issues on my Mac but I have access to a Windows machine now.

[ParthShah91]

Sure thank you

[aganders3]

Okay I played around with this more. If we're seeing the same thing, this is aliasing. I can see it on my Mac if using my external (4k) monitor. I'm not sure there's much you can do to fix it easily, but maybe others have ideas.

You could start with changing the blending mode with line.set_gl_state("additive") which for me seems to make it slightly better.

There is a vispy demo of using shaders to plot high frequency signals that might be helpful.

There are also some previous issues discussing MSAA, but unfortunately it's still not supported in vispy.

Sorry I can't be more helpful. I will come back if anything else comes to me.

[ParthShah91]

Thanks for the help. I tried searching for things on my own as well.

I integrated my code into PySide6. We had data coming in at 100 ms and faced flickering issues once we started windowing. As I debugged I realised if we plot just 1 in 10 times i.e every 1 second the flickering goes away.

As I checked the examples.. ex03a_data_sources_timer, ex03b_data_sources_threaded_loop and ex03c_data_sources_threaded_events I realized that they too are being toggled at every 1 second either using time.sleep or passing "1.0" as argument to Timer method.
(https://github.com/vispy/vispy/tree/main/examples/scene/realtime_data)

Is this really a known constraint? Then we can convey the same to our customer and be happy with plotting data every 1 second. Maybe later you guys can fix this

[aganders3]

I don't think it's an issue with the timer interval. It's probably just that the aliasing artifact is not as apparent with the slower update.

It's kind of hard to capture in a recording (the jump is because this is only a 3s GIF), but I see a big improvement when setting samples=16 on the Qt backend. I am using PyQt6 here.

[ParthShah91]

Cool. We have a limitation that we can only use PySide6 due to licensing limitations but that's ok. Could you help us where do you set the 'samples' method/parameter in Qt backend that you mentioned? Maybe we can try something similar in PySide6 if that exists.

[aganders3]

I don't think it's exposed in the API, but I could be wrong. I hard-coded the value here which should apply to both PySide and PyQt backends:

vispy/vispy/app/backends/_qt.py

Line 300 in 512a1ce

glformat.setSamples(c['samples'] if c['samples'] else 0)

[ParthShah91]

Ah ok will try it for my code as well. Who knows it might be a break through we need.

[ParthShah91]

Hi! Sadly this still remains a problem for us. I tried changing it to 16 and of course saw some improvement but sadly it still does not match our requirements.

The configuration for our system is:

Windows version
Edition Windows 10 Home Single Language
Version 22H2
Installed on ‎04-‎03-‎2024
OS build 19045.4046
Experience Windows Feature Experience Pack 1000.19053.1000.0

CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz 2.90 GHz
GPU: Intel(R) HD Graphics 620 and NVIDIA GeForce 940 MX
Nvidia driver version: 545.84

Below is the output of "vispy.sys_info()": One key thing that I observe is egl is none. Is that the issue?

Platform: Windows-10-10.0.19045-SP0
Python:   3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)]
NumPy:    1.26.4
Backend:  PySide6
pyqt4:    None
pyqt5:    None
pyqt6:    None
pyside:   None
pyside2:  None
pyside6:  ('PySide6', '6.6.2', '6.6.2')
pyglet:   None
glfw:     None
sdl2:     None
wx:       None
egl:      None
osmesa:   None
tkinter:  None
jupyter_rfb: None
_test:    None

GL version:  '4.6.0 NVIDIA 545.84'
MAX_TEXTURE_SIZE: 16384
Extensions: 'GL_AMD_multi_draw_indirect GL_AMD_seamless_cubemap_per_texture GL_ARB_arrays_of_arrays GL_ARB_base_instance GL_ARB_bindless_texture GL_ARB_blend_func_extended GL_ARB_buffer_storage GL_ARB_clear_buffer_object GL_ARB_clear_texture GL_ARB_clip_control GL_ARB_color_buffer_float GL_ARB_compatibility GL_ARB_compressed_texture_pixel_storage GL_ARB_conservative_depth GL_ARB_compute_shader GL_ARB_compute_variable_group_size GL_ARB_conditional_render_inverted GL_ARB_copy_buffer GL_ARB_copy_image GL_ARB_cull_distance GL_ARB_debug_output GL_ARB_depth_buffer_float GL_ARB_depth_clamp GL_ARB_depth_texture GL_ARB_derivative_control GL_ARB_direct_state_access GL_ARB_draw_buffers GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_draw_elements_base_vertex GL_ARB_draw_instanced GL_ARB_enhanced_layouts 
GL_ARB_ES2_compatibility GL_ARB_ES3_compatibility GL_ARB_ES3_1_compatibility GL_ARB_ES3_2_compatibility GL_ARB_explicit_attrib_location GL_ARB_explicit_uniform_location GL_ARB_fragment_coord_conventions GL_ARB_fragment_layer_viewport GL_ARB_fragment_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_framebuffer_no_attachments GL_ARB_framebuffer_object GL_ARB_framebuffer_sRGB GL_ARB_geometry_shader4 GL_ARB_get_program_binary GL_ARB_get_texture_sub_image GL_ARB_gl_spirv GL_ARB_gpu_shader5 GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader_int64 GL_ARB_half_float_pixel GL_ARB_half_float_vertex GL_ARB_imaging GL_ARB_indirect_parameters GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_internalformat_query2 GL_ARB_invalidate_subdata GL_ARB_map_buffer_alignment GL_ARB_map_buffer_range GL_ARB_multi_bind GL_ARB_multi_draw_indirect GL_ARB_multisample GL_ARB_multitexture GL_ARB_occlusion_query GL_ARB_occlusion_query2 GL_ARB_parallel_shader_compile GL_ARB_pipeline_statistics_query GL_ARB_pixel_buffer_object GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_polygon_offset_clamp GL_ARB_program_interface_query GL_ARB_provoking_vertex GL_ARB_query_buffer_object GL_ARB_robust_buffer_access_behavior GL_ARB_robustness GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_seamless_cube_map GL_ARB_seamless_cubemap_per_texture GL_ARB_separate_shader_objects GL_ARB_shader_atomic_counter_ops GL_ARB_shader_atomic_counters GL_ARB_shader_ballot GL_ARB_shader_bit_encoding GL_ARB_shader_clock GL_ARB_shader_draw_parameters GL_ARB_shader_group_vote GL_ARB_shader_image_load_store GL_ARB_shader_image_size GL_ARB_shader_objects GL_ARB_shader_precision GL_ARB_shader_storage_buffer_object GL_ARB_shader_subroutine GL_ARB_shader_texture_image_samples GL_ARB_shader_texture_lod GL_ARB_shading_language_100 GL_ARB_shading_language_420pack GL_ARB_shading_language_include GL_ARB_shading_language_packing GL_ARB_shadow GL_ARB_sparse_buffer GL_ARB_sparse_texture GL_ARB_spirv_extensions GL_ARB_stencil_texturing GL_ARB_sync GL_ARB_tessellation_shader GL_ARB_texture_barrier GL_ARB_texture_border_clamp GL_ARB_texture_buffer_object GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_buffer_range GL_ARB_texture_compression GL_ARB_texture_compression_bptc GL_ARB_texture_compression_rgtc GL_ARB_texture_cube_map GL_ARB_texture_cube_map_array GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_filter_anisotropic GL_ARB_texture_float GL_ARB_texture_gather GL_ARB_texture_mirror_clamp_to_edge GL_ARB_texture_mirrored_repeat GL_ARB_texture_multisample GL_ARB_texture_non_power_of_two GL_ARB_texture_query_levels GL_ARB_texture_query_lod GL_ARB_texture_rectangle GL_ARB_texture_rg GL_ARB_texture_rgb10_a2ui GL_ARB_texture_stencil8 GL_ARB_texture_storage GL_ARB_texture_storage_multisample GL_ARB_texture_swizzle GL_ARB_texture_view GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_transform_feedback_instanced GL_ARB_transform_feedback_overflow_query GL_ARB_transpose_matrix GL_ARB_uniform_buffer_object GL_ARB_vertex_array_bgra GL_ARB_vertex_array_object GL_ARB_vertex_attrib_64bit GL_ARB_vertex_attrib_binding GL_ARB_vertex_buffer_object GL_ARB_vertex_program GL_ARB_vertex_shader GL_ARB_vertex_type_10f_11f_11f_rev GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_ARB_window_pos GL_ATI_draw_buffers GL_ATI_texture_float GL_ATI_texture_mirror_once GL_S3_s3tc GL_EXT_texture_env_add GL_EXT_abgr GL_EXT_bgra GL_EXT_bindable_uniform GL_EXT_blend_color GL_EXT_blend_equation_separate GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_compiled_vertex_array GL_EXT_Cg_shader GL_EXT_depth_bounds_test GL_EXT_direct_state_access GL_EXT_draw_buffers2 GL_EXT_draw_instanced GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_blit GL_EXT_framebuffer_multisample GL_EXTX_framebuffer_mixed_formats GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_framebuffer_object GL_EXT_framebuffer_sRGB GL_EXT_geometry_shader4 GL_EXT_gpu_program_parameters GL_EXT_gpu_shader4 GL_EXT_multi_draw_arrays GL_EXT_multiview_texture_multisample GL_EXT_multiview_timer_query GL_EXT_packed_depth_stencil GL_EXT_packed_float GL_EXT_packed_pixels GL_EXT_pixel_buffer_object GL_EXT_point_parameters GL_EXT_polygon_offset_clamp GL_EXT_provoking_vertex GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_shader_objects GL_EXT_separate_specular_color GL_EXT_shader_image_load_formatted GL_EXT_shader_image_load_store GL_EXT_shader_integer_mix GL_EXT_shadow_funcs GL_EXT_stencil_two_side GL_EXT_stencil_wrap GL_EXT_texture3D GL_EXT_texture_array GL_EXT_texture_buffer_object GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_latc GL_EXT_texture_compression_rgtc GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map GL_EXT_texture_edge_clamp GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_EXT_texture_filter_anisotropic GL_EXT_texture_integer GL_EXT_texture_lod GL_EXT_texture_lod_bias GL_EXT_texture_mirror_clamp GL_EXT_texture_object GL_EXT_texture_shadow_lod GL_EXT_texture_shared_exponent GL_EXT_texture_sRGB GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_decode GL_EXT_texture_storage GL_EXT_texture_swizzle GL_EXT_timer_query GL_EXT_transform_feedback2 GL_EXT_vertex_array GL_EXT_vertex_array_bgra GL_EXT_vertex_attrib_64bit GL_EXT_window_rectangles GL_EXT_import_sync_object GL_IBM_rasterpos_clip GL_IBM_texture_mirrored_repeat GL_KHR_context_flush_control GL_KHR_debug GL_EXT_memory_object GL_EXT_memory_object_win32 GL_EXT_win32_keyed_mutex GL_KHR_parallel_shader_compile GL_KHR_no_error GL_KHR_robust_buffer_access_behavior GL_KHR_robustness GL_EXT_semaphore GL_EXT_semaphore_win32 GL_NV_timeline_semaphore GL_KHR_shader_subgroup GL_KTX_buffer_region GL_NV_alpha_to_coverage_dither_control GL_NV_bindless_multi_draw_indirect GL_NV_bindless_multi_draw_indirect_count GL_NV_bindless_texture GL_NV_blend_equation_advanced GL_NV_blend_equation_advanced_coherent GL_NV_blend_minmax_factor GL_NV_blend_square GL_NV_command_list GL_NV_compute_program5 GL_NV_conditional_render GL_NV_copy_depth_to_color GL_NV_copy_image GL_NV_depth_buffer_float GL_NV_depth_clamp GL_NV_draw_texture GL_NV_draw_vulkan_image GL_NV_ES1_1_compatibility GL_NV_ES3_1_compatibility GL_NV_explicit_multisample GL_NV_feature_query GL_NV_fence GL_NV_float_buffer GL_NV_fog_distance GL_NV_fragment_program GL_NV_fragment_program_option GL_NV_fragment_program2 GL_NV_framebuffer_multisample_coverage GL_NV_geometry_shader4 GL_NV_gpu_program4 GL_NV_internalformat_sample_query GL_NV_gpu_program4_1 GL_NV_gpu_program5 GL_NV_gpu_program5_mem_extended GL_NV_gpu_program_fp64 GL_NV_gpu_shader5 GL_NV_half_float GL_NV_light_max_exponent GL_NV_multisample_coverage GL_NV_multisample_filter_hint GL_NV_occlusion_query GL_NV_packed_depth_stencil GL_NV_parameter_buffer_object GL_NV_parameter_buffer_object2 GL_NV_path_rendering GL_NV_pixel_data_range GL_NV_point_sprite GL_NV_primitive_restart GL_NV_query_resource GL_NV_query_resource_tag GL_NV_register_combiners GL_NV_register_combiners2 GL_NV_shader_atomic_counters GL_NV_shader_atomic_float GL_NV_shader_atomic_int64 GL_NV_shader_buffer_load GL_NV_shader_storage_buffer_object GL_NV_shader_subgroup_partitioned GL_NV_texgen_reflection GL_NV_texture_barrier GL_NV_texture_compression_vtc GL_NV_texture_env_combine4 GL_NV_texture_multisample GL_NV_texture_rectangle GL_NV_texture_rectangle_compressed GL_NV_texture_shader GL_NV_texture_shader2 GL_NV_texture_shader3 GL_NV_transform_feedback GL_NV_transform_feedback2 GL_NV_uniform_buffer_unified_memory GL_NV_uniform_buffer_std430_layout GL_NV_vertex_array_range GL_NV_vertex_array_range2 GL_NV_vertex_attrib_integer_64bit GL_NV_vertex_buffer_unified_memory GL_NV_vertex_program GL_NV_vertex_program1_1 GL_NV_vertex_program2 GL_NV_vertex_program2_option GL_NV_vertex_program3 GL_NVX_conditional_render GL_NVX_linked_gpu_multicast GL_NV_gpu_multicast GL_NVX_gpu_multicast2 GL_NVX_progress_fence GL_NVX_gpu_memory_info GL_NVX_multigpu_info GL_NVX_nvenc_interop GL_NV_shader_thread_group GL_NV_shader_thread_shuffle GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OVR_multiview GL_OVR_multiview2 GL_SGIS_generate_mipmap GL_SGIS_texture_lod GL_SGIX_depth_texture GL_SGIX_shadow GL_SUN_slice_accum GL_WIN_swap_hint WGL_EXT_swap_control '

[ParthShah91]

@djhoese Hi! Can you please look into this? We would really like to see the flickering getting stopped. We evaluated wide range of libraries for our project and based on client's requirements vispy seems to be the frontrunner. A bit of help from you or someone else would help us a great great deal.

[djhoese]

Sorry, I do not have time and do not have a Windows machine to test on...nor would I know how to fix this.

My priorities: #2570