Contouring

[ianthomas23]

This is a place for us to discuss how to proceed with contouring in bokeh. There is a demo PR at #12020. There is lots to discuss eventually, such as

1. How to deal with selecting levels to contour at.
2. How to deal with use of colormaps/palettes.
3. Other style issues such as dashed lines.
4. Do we just have the Python side creating standard MultiPolygons and MultiLine glyphs, or do we need a bespoke ContourRenderer that is a bit like the GraphRenderer.

My first real question is about the use of MultiLine for contour lines. Using a MultiPolygons object for filled contours works really well, the demo contours multiple levels and put them all together in one MultiPolygons object with a vector fill_color. But there isn't an equivalent for contour lines because we don't have a MultiLines (note the s at the end) object. So (1) I can create a single MultiLine glyph for each contour line level, with a scalar line_color, but then I am dealing with multiple of these for the complete set of contour lines compared to the single MultiPolygons object for the complete set of filled polygons. Alternatively (2) I could use a single MultiLine glyph for the complete set, but this becomes a little trickier to manage as the vector line_color is the same length as the total number of lines across all contour levels not just one per contour level. Or (3) create a new MultiLines glyph analogous to MultiPolygons.

[ianthomas23]

I will spend some time looking at the GraphRenderer to see how such an approach might or might not work with contouring.

[bryevdv]

copy from #12020

Reiterating the comment in the weekly meeting, I think a main decision is whether to do something along the lines of GraphRenderer, i.e. have a ContourRenderer. I do think the new serialization work might afford some opportunities here, as well as learning some lessons from GraphRenderer to do things better.

As far as rationale: the reason I think a ContourRenderer would be better for users is that the data for the contour lines and polys is not independent. We really should afford an API to allow updating all of this "atomically" in one go. If we just return two glyph renderers and call it done, then updating contours will always necessitate users updating two things, sequentially, and I just foresee issues and questions arising from that. I think it may also make render updates more prone to stuttering artifacts.

So what might we do better with a ContourRenderer?

• JS side manages internal (non-synced) MultiPolygon and MultiLine glyphs that it delegates to, when it is called to render.
• Data for both contour lines and regions can be set in one go. e.g. we could have a struct type with two ColumnData sub-fields:

  new_contour_data = ContourData(level_data=line_data_dict, region_data=poly_data_dict)
  
  contour_renderer.data = new_contour_data

  This would not preclude users from trying to update contour_renderer.data.level_data independently but we don't have to demonstrate that anywhere.

With something like this, the only things that are synced are standard and familiar visual properties as well as the new contour data struct.

Issues/unknowns:

• Interactions with hit-testing, hover, and selection would require some thought
• How should legend-ing work? (We actually have more control here with a new renderer)

Thoughts?

[bryevdv]

@ianthomas23 I believe each line in a MulitLine can include nan values in the data, indicating disjoint portions of the same "line". If so, that would afford one line per level, even if the level has disconnected parts. It's a bit more tedious to construct, I suppose, but we would hide all that from users.

Edit: yes, see: https://docs.bokeh.org/en/latest/docs/user_guide/plotting.html#missing-points

Alternatively, extending the above discussion, a ContourRenderer could also adopt a hybrid approach: manage an internal MultiPolygons for the filled areas, and render the iso-level lines itself. This might be nice because using MultiLine we would not actually be able to do any line joins to close iso-levels that are fully closed inside the contour region.

[bryevdv]

I should add, I am not opposed to adding a MultiLines glyph that has a uniform "list of lists of lists" data structure that mirrors MultiPolygons. The issues are that MultiLines is the correct name, to mirror MultiPolygons and that would be some confusion for users I think, with existing MultiLine. Also the data format is somewhat more tedious to deal with for common cases I think (an extra level of nesting). I'd be happy to hear any ideas about how to make things simpler for end users of a new MultiLines glyph.

[ianthomas23]

1. I like the use of nan in MultiLine, so I will proceed with that. I should have remembered this, my first Bokeh contribution was fixing nans in webgl lines!
2. I agree that it would be too confusing to have both a MultiLine and MultiLines, so if we can we should just stick with what we have available now.
3. I like the idea of a ContourRenderer, I just need to think it through.
4. We do need to calculate and store both the filled contours and the contour lines (unless the user doesn't want one of them). Deriving the lines from the polygons works badly when contouring masked arrays, which is very common in geoscience (e.g. islands in the ocean).

[ianthomas23]

The nan in MultiLine works well. If this is how we end up doing it, I can move the nan insertion down into contourpy as it will be much faster in C++.

[mattpap]

I'm wondering if we could eventually go a step further an not encode xs and ys as lists of lists/arrays, but as continuous memory buffers implemented as ragged/offset arrays. bokehjs internally uses RaggedArray (from core/util/ragged_array) in MultiLine glyph (and other "multi" glyphs) to efficiently store coordinates. With the new serializer, we could fairly easily allow passing such specialized arrays from/to bokeh (fully binary encoded if permitted).

[ianthomas23]

Yes, that would be a really good approach. My benchmarking of complicated scenarios in Matplotlib shows that using offset arrays can speed up calculation and rendering time by a factor of 3 by avoiding the creating and manipulation of a large number of numpy arrays.

[bryevdv]

Agree offset arrays would be ideal from the protocol perspective. I think they are less friendly for end users, but given that we will be affording API to prepare the data, this might be a good situation to start exploring their use.

[gmerritt123]

Hi there, first off I really appreciate the effort.

I have a few questions/thoughts about this. I've dabbled in implementing JS-side contouring in a few scenarios (see my "bokriging" example below)... but a full on framework would be just amazing.

1. Is the intent to for the contour geometries to be calculated JS side? I guess specifically I'm wondering if once implemented, the idea is that I could pass an updated 2D array/ragged array as described by @mattpap to the ContourRenderer model in a CustomJS callback, and the contour geometries (i.e. the MultiLine or MultiPolygon glyphs displaying the contours) will be recalculated based on that array via some JS side interpolation algo? I've dabbled with this extension of d3 to do this before: https://github.com/Fil/d3-tricontour ....

This kinda which leads me to my next question:

2. There are a boatload of gridding-->interpolation algos/techniques out there...kriging, radial basis functions, inverse distance weighted etc.... is the intent for this ContourRenderer to calculate contours consistent with other tricontour routines (like the d3 one above and/or matplotlib's tricontour) or to facilitate custom extension using arbitrary/user defined interpolation routines?

My example harnessing krige.js (https://oeo4b.github.io/) to make a little sandbox visualization of the kriging parameters:

Either way I really appreciate there's chatter about this kind of feature :-D Thanks!

[bryevdv]

@gmerritt123 The current plan is to build around @ianthomas23 recently released contourpy package, which provides python bindings for the C++ contouring algorithm used by Matplotlib (but standalone, split out from MPL). While there would be obvious benefits to having a JS-side contouring facility, I would personally say it is out of scope for this current work. However, if we adopt something along the lines I describe above where a ContourRenderer is configured with a single ContourData value then I don't see why that value couldn't also be supplied by JS code (e.g. from a JS callback or a thin custom extension that adapts the output of a JS contour library to a ContourData).

[bryevdv]

As an aside: BTW great demo I'd love to RT from the Bokeh twitter account if you are inclined to tweet out that GIF

[ianthomas23]

@gmerritt123 We are talking about quad grid contouring, not tri grid. The gridded data and calculations all occur on the Python side, and only the line and polygon geometries are passed to JS. So unfortunately that doesn't include anything that you would like to see!

Use cases we are considering:

1. Single shot calculation passed to JS where it remains as static geometries that can be interacted with in the usual manner.
2. Animating through e.g. different timesteps of Earth surface temperature. Each timestep is a 2D slice of 3D (latitude, longitude, time) data. Animation could be controlled from Python or JS.

I am happy to receive suggestions for other use cases of course.

Calculating quad grid contours on the JS side could be done, but I wouldn't do it in JS, I would use WASM or WebGPU instead. This is a whole new research project.

Tri grid contouring is also possible, but not planned. As it happens I also wrote the tri grid contouring code in Matplotlib so it would be possible to pull this out into a separate Python/C++ project. But again this is a whole new project.

Probably progress on either of these possibilities depends on obtaining funding. But there are other improvements that I would prioritise over these for general funding. So we would probably need to find someone who wanted JS-side quad contouring or tri contouring enough to at least part-fund that specific work.

[gmerritt123]

Awesome thanks for the clarification/down low.

@ianthomas23 There are approximately a gajillion applications for this in my field (hydrogeology) alone. The whole field is basically an interpolation/extrapolation haha:

• Hydraulic head (pressure)/water level gets measured at a few wells --> we need to infer the regional gradient --> make a contour map of equipotential lines ---> water flows from high pressure to low pressure (i.e. downhill) --> we can visualize/interpret groundwater flow direction

• Contaminant plumes --> we measure contaminant concentration at a few wells --> use an interpolation routine + contours to estimate contaminant extent, contaminant mass, volume of contaminated soil, etc. etc.

• Geologic surfaces --> we drill a bunch of boreholes at different locations and use those datapoints to interpolate the elevation of a particular identified geologic unit.

All of these interpolations/contours go into big numerical models, a lot of which require a crap tonne of expert software training/experience to really utilize/understand, not to mention insane licensing costs. Bokeh comes in for me to pull only the relevant stuff out of those models and to convey that info to the client in a way that's engaging and meaningful for them.

A basic example of this would be to run a model where they pump a well really hard for a period of time. The system around it depressurizes/dewaters and the model calculates a simulated water level decline (i.e. "Drawdown" or DD) at thousands of model nodes for several time steps. It's a finite element model usually for me meaning these nodes are irregularly spaced. Anyway, I dump the X-Y--DD--Time data from the model, then generate DD contours for each time step with this kind function, using mpl's tricontour, some finagling of accessing the contour geoms, and using geopandas/shapely to return a geodataframe of line geoms:

def contour_lines(df,ints):
    '''function to take a dataframe with x,y and Var column, and ints (ascending list only)
    and return a geodataframe with contour line geometries'''
    hdf = df
    fig,ax = plt.subplots()
    print('contouring...')
    cntr = ax.tricontour(hdf['X'],hdf['Y'],hdf['Var'],ints)
    mdf = pd.DataFrame(columns=['lvl','geometry'])
    for i,collection in enumerate(cntr.collections):
        lvl = cntr.levels[i]
        for path in collection.get_paths():
            ls = LineString(path.vertices)
            app_df = pd.DataFrame(data={'geometry':[ls]})
            app_df['lvl'] = lvl
            mdf = pd.concat([mdf,app_df],sort=True)
    mgdf = geopd.GeoDataFrame(mdf,geometry=mdf['geometry'])
    return mgdf

Then I transform that geodataframe into a bokeh CDS or a dictionary containing the xs/ys components of the line geoms. Then a CustomJS callback on a slider lets the user "move through time" and see the drawdown propagate in space etc. It's super efficient data wise because basically I'm only pulling the relevant contour geoms and not the actual data itself into the html, and that's all I need to convey what's happening to the client/decision maker etc.

That kinda leads me to my next point... this ContourRenderer model would obviously streamline the above described kind of workflow (like... big time :-D ), but will it dump the "raw" data to the html as well as the contour line/patch geom? That'll come at signficant size expense (like thousands of nodes for thousands for potentially hundreds of time steps in my above example)...

But... if I ever want to be able to contour JS side.... obviously I need access to the raw data (and that's what I was kinda alluding to before and demonstrating with my gif --> I'd like the ability to change the raw data on JS side and get the contours re-rendered there via my own custom stuff...

So is the idea to have two separate CDS's, one containing the raw data, and one containing the contour geoms, both held within the ContourRenderer model? This was the approach to GraphRenderer correct? My very, (very), humble suggestion would be to NOT write the raw data CDS component to the html (or at the very least give the user to option not to), in the interest of not writing redundant data that is not being rendered nor being used js-side, unless of course the user WANTS it to.... this would just require them passing it as an arg in a CustomJS model, or using it to drive a different renderer (e.g. the circles in my gif). Then kinda like @bryevdv mentioned, if I can access and manipulate the contour geom CDS via CustomJS as well.. then I'm free to implement my own crazy js-side interpolation routines if I want. But it also gives me the option of passing only the contour CDS into the html as well if i don't need the "raw" data.

Hope that kinda makes sense, happy to try and elaborate if it doesn't at all...

Regarding tweeting that gif @bryevdv a lot of the colors got all messed up when i made the gif --> the circle points aren't actually supposed to change in color! So let me try and make a higher quality one and i'll pass that to you for twitterland :-)

[bryevdv]

but will it dump the "raw" data to the html as well as the contour line/patch geom?

The current proposal does not do this, it would only send the contour level and polygon coordinates (and any extra columns of per-level or per-polygon metadata, e.g. for colormapping or hover) in a combined ContourData model. [1] If you wanted to send the raw data that would be something you'd have to do separately and manually, in a CDS or possibly a DataModel you define.

[1] The reason for a new model instead of two CDS's like the graph renderer is so that everything can be updated "atomically" in one go. Having two separate CDS's is one of the biggest pain points of the graph renderer IMO.

[bryevdv]

Regarding twitter, if you would not be tweeting out yourself, perhaps you could make a short post with the GIF in the Showcase categorig on the Discourse? https://discourse.bokeh.org/c/showcase/13

Also, all those use cases sound really fantastic and interesting. As @ianthomas23 mentions, the best chance for serious work in that direction would be to secure real funding. So if you ever know of any grant opportunities or potential sponsors in this sector, please don't hesitate to let us know or make an introduction!

[gmerritt123]

but will it dump the "raw" data to the html as well as the contour line/patch geom?

The current proposal does not do this, it would only send the contour level and polygon coordinates (and any extra columns of per-level or per-polygon metadata, e.g. for colormapping or hover) in a combined ContourData model. [1] If you wanted to send the raw data that would be something you'd have to do separately and manually, in a CDS or possibly a DataModel you define.

[1] The reason for a new model instead of two CDS's like the graph renderer is so that everything can be updated "atomically" in one go. Having two separate CDS's is one of the biggest pain points of the graph renderer IMO.

Sweet, that's exactly how I was hoping the approach would go, that'll be awesome :-D as usual you guys think of everything.

Regarding twitter, if you would not be tweeting out yourself, perhaps you could make a short post with the GIF in the Showcase categorig on the Discourse? https://discourse.bokeh.org/c/showcase/13

Alright i'll prep a gif and post on showcase with a basic abstract when I get the chance.

[ianthomas23]

I have made some progress with the contouring PR. It is still very much experimental, these are my latest thoughts. Example output

using this code

from bokeh.io import show
from bokeh.palettes import cividis
from bokeh.plotting import figure, from_contour
import numpy as np

x, y = np.meshgrid(np.linspace(0, 10, 20), np.linspace(0, 6, 10))
z = np.sin(x)*np.cos(y)
levels = np.linspace(-1.0, 1.0, 11)

fig = figure()
contour_renderer = from_contour(x, y, z, levels, fill_color=cividis, line_color="black")
fig.renderers.append(contour_renderer)
colorbar = contour_renderer.color_bar()
fig.add_layout(colorbar, 'right')
show(fig)

from_contour takes the x, y, z 2D arrays (x and y will eventually be optional) then the levels to contour. This is currently a sequence but it will also support an integer for the number of levels required that will be sensibly calculated based on the z limits, probably with linear and logarithmic options initially. This is similar to the Ticker classes, in fact maybe it could reuse the tickers for this purpose? If so we'd have to copy across the ticker maths from JS to Python side.

Currently only supporting the visual properties fill_color and line_color. These can be a single color or a sequence/palette. The plan is to support the full set of line, fill and hatch properties, the line properties will go to the contour MultiLine and the fill and hatch to the MultiPolygons. You will be able to have just line or just fill, or both. It would be nice to have a Callable option for visual properties so that you could have e.g. red lines for contour levels < 0 and black for >= 0, if the contour levels are being calculated for you.

The questions start with the ColorBar. I am assuming most contour users want a colorbar, if they want a legend instead that could be created from the MultiLine and MultiPolygons. All we have currently are the fill colors in the colorbar. I'd like the full set of fill and hatch visual properties to be in the colorbar, and indeed also the line visual properties at the appropriate positions. So this will need another ContourColorBar class, probably derived from ColorBar with the extra functionality. These will only ever need to be instantiated from a ContourRenderer I think?

Also, ideally, the visual properties in the colorbar wouldn't be separate copies of the same colors, etc, but would be some sort of proxy into the MultiLine and MultiPolygons properties (or the other way round?) so that it would be possible on the JS side to change for example the fill colors and the colorbar would also be updated in sync.

There is a use case of a set or grid of contour plots with a single shared colorbar. But I would not go out of my way to help too much with this, I think it would be fine for the user to create the colorbar from one of the contour plots, reuse the same levels, colors, etc on the other contour plots, and if they want to change some visual properties on the JS side they will have to iterate over the plots rather than us having a more complicated many-to-one proxy for the visual properties of the contour plots and colorbar.

[ianthomas23]

Matplotlib example that includes fill, lines, alpha and hatching:

This wins no awards for presentation 😵‍💫, but there is some demand for contour hatching.

[ianthomas23]

It's possible a CustomJSTransform could already be used here.

Maybe, I'll look into it. I was thinking of a callback on the Python side but if there is JS solution that can be reused for setting/changing visual properties then that would suffice.

[ianthomas23]

For the ContourColorBar and ContourRenderer sharing the visual properties, the ContourColorBar could just hold references to the underlying MultiLine and MultiPolygons and use their line/fill/hatch properties when rendering.

[mattpap]

This wins no awards for presentation face_with_spiral_eyes, but there is some demand for contour hatching.

Bokeh should be unopinionated (us much as possible) with regards to what we support. If it makes sense to have and we have resources to implement this, then it should be done, regardless of our view of aesthetics, etc.

[bryevdv]

I have definitely put time an effort into things I thought people would want, only to see them completely unused years later. Anyway, only for the record, my opinion is that this is not worth expending resources on until/unless there is an explicit ask.