Extended Python interface in the neuron module

[Helveg]

Hi everyone, this is going to be my longpost discussion thread on all things Python interface! Join me!

The neuron root module in Python is currently an empty redirect to the h HocInterpreter but as I see it, being the root namespace .. and empty, it could house an extended Python interface to the things inside of nrn and the things accessible through h.

I'd like to propose thinking about a Python-first interface that leaves behind all historical limitations that come with the C, HOC roots of NEURON and focus on what makes the Python interface great. That is ofcourse until it gets replaced by the next thing. Rust-NEURON anyone?

To get started here are some outlines and ground rules for my ambitions of this Python interface (let's debate!):

• It is written entirely in Python
  • Better introspection, tooling
  • More intuitive use of objects by users
  • Easier for users to inherit, extend and overload classes
  • Easier for Python users to contribute code
  • Easier to raise errors before messing with the internal state
  • Better documentation
• We extend the classes in nrn to preserve base functions but don't expose all of them directly to the user
• We put things available in h in a shiny new Pythonic coat.
• We instead expose a sanitized public API but under the hood use all things that were already present.
  • This way we can "hide" unPythonic, unidiomatic, architecturally problematic or easy to misuse parts of the h API
• We extend the existing interface with as much low hanging fruit we can come up with! It's time we put some fancy bells and whistles on the Python interface :)

To give you an idea I have something like this in mind (that thanks to @ramcdougal I can actually start work on now! See #1096 )

import neuron

section = neuron.Section()
>>> SyntaxWarning: Created a Section without a name.
syn = neuron.mechanisms.ExpSyn()
isinstance(syn, neuron.mechanisms.PointProcess)
>>> True
section.insert(syn, 0.5)
p = section.get_point_processes()
>>> [syn]
section.get_density_mechanisms()
>>> []

The take away point of this is that us Python users/devs can create a Python interface from Python and give it whatever API NEURON's Python community decides on, empowering voiceless Pythonistas to contribute code in the language they're using, and without dealing with a "bastardized" hoc interface and its inherent historical difficulties (given that we can do the things we want still using that same interface under the hood). I see it as a noninvase enrichment of NEURON with a modern Python interface.

[ramcdougal]

Some of this stuff is already directly accessible through the Section object, e.g.

>>> soma = h.Section()
>>> ic = h.IClamp(soma(0.5))
>>> soma.psection()['point_processes']
{'IClamp': {IClamp[0]}}
>>> 

[Helveg]

Yep! So that's the "low hanging fruit"/Pythonic/modern aspect of it, we'd be in a prime position to extend the interface with cleaner, more expressive and intuitive APIs that make what's already there easier to use and find (e.g. .get_point_processes() instead of .psection()['point_processes'])!

Another big one would be that we could much easier fully document this using sphinx and generate up to standard pretty interlinked apidocs! How would we ever document all the keys of psection in an equally clean way as the get_point_processes function? You can docstring that, ref that from anywhere in sphinx, ... and I'm going off on a "docs" tangent now but I'm sure we'll discover endless benefits to such a full-Python Python interface.

[ramcdougal]

For what it's worth, you could think of sec.psection() as an almost declarative representation of what is in the section, and so thus the following would give you an almost declarative representation of a cell:

def declarativeish_model(any_section_on_cell):
    return {str(sec): sec.psection() 
            for sec in any_section_on_cell.wholetree()}

[Helveg]

Yea, what's in psection can probably be a good start for all the properties on a Section API.

Speaking of Section API... What does a Section do?

• It's a cylindrical compartment
  • It has dimensions and cable properties.
  • It's also secretly a branch through space, so it has 3d points associated to it.
  • It can form chains with other Sections connected head/tail to head/tail
  • DensityMechanisms can spread out over its surface
  • PointProcesses can be inserted into it on any arclength location
  • You can record their V, I, ... on any arclength location

Anything else that the object Section ought to be responsible of, conceptually?

Things like _ref_v and pt3dadd and the likes also got me thinking of a naming convention of things... _ref_<var> should probably be tucked away from users in an OO interface, and shortened cryptical names can be kept for those who are used to them but we can also give them additional more expressives aliasses, since #1096 gives us full freedom over the object attributes.

[ramcdougal]

A section is not in general a cylinder. It's the union of potentially many frusta and many compartments (segments).