[ENH]Add Image Processing Folder, include shrinking functions which is content-aware image processing.

doc/topics/stimuli.rst

@@ -144,6 +144,33 @@ The same is true for a dictionary of pulse trains:
     Stimulus({'A1': BiphasicPulse(10, 0.45, stim_dur=100),
               'C9': BiphasicPulse(-30, 1, delay_dur=10, stim_dur=100)})
 
+Preprocessing stimuli
+---------------------
+
+:py:class:`~pulse2percept.stimuli.ImageStimulus` and
+:py:class:`~pulse2percept.stimuli.VideoStimulus` come with built-in
+preprocessing methods that include basic image operations:
+
+-  ``rgb2gray``: converting from RGB to grayscale
+-  ``invert``: inverting the gray levels
+-  ``resize``: resizing to a desired shape
+-  ``crop``: cropping a region of interest
+-  ``shift``, ``scale``, and ``rotate``
+
+as well as advanced image processing techniques:
+
+-  ``threshold``: thresholding the gray levels (e.g., local, Otsu, ISODATA)
+-  ``filter``: filtering (e.g., Sobel, Scharr, median, Canny edge detector)
+-  ``retarget``: content-aware image retargeting [AlAtabany2010]_
+
+In addition, images and videos can be automatically converted into pulse trains
+using the ``encode`` method.
+
+Last but not least, if your favorite image processing method is not yet
+supported, you can pass a custom function to the ``apply`` method.
+This will work as long as the custom function accepts and returns an image 
+(or video).
+
 Interacting with stimuli
 ------------------------
 

doc/users/references.rst

@@ -29,6 +29,8 @@ Studies referenced throughout the Documentation:
                 *Journal of Comparative Neurophysiology* 292:497-523,
                 doi:`10.1002/cne.902920402
                 <https://doi.org/10.1002/cne.902920402>`_.
+.. [Fleck1992] MM Fleck (1992). Some defects in finite-difference edge finders. 
+               *IEEE J PAMI* 14:337--345
 .. [Han2021] N Han, S Srivastava, A Xu, D Klein, M Beyeler (2021). Deep Learning–Based
              Scene Simplification for Bionic Vision. *Augmented Humans Conference* 2021,
              45–54. <https://doi.org/10.1145/3458709.3458982>`_.

examples/stimuli/plot_retarget.py

@@ -0,0 +1,79 @@
+# -*- coding: utf-8 -*-
+"""
+===============================================================================
+Shrinking inputs by content-aware image retargeting
+===============================================================================
+
+*This example shows how to use shrink by content-aware image retargeting for retinitis pigmentosa patients.*
+
+pulse2percept.shrink implements improved content-aware image retargeting from https://doi.org/10.1186/1475-925X-9-52
+
+You can shrink an image, a video, or an stimuli(ImageStimuli or VideoStimuli)
+
+"""
+##############################################################################
+# First, let's take a look at the video using VideoStimuli.play()
+#
+import matplotlib.pyplot as plt
+import pulse2percept as p2p
+import numpy as np
+
+# reader = get_reader('road.mp4')
+# video = np.array([frame for frame in reader])
+# stim = p2p.stimuli.videos.VideoStimulus(video[0:7].transpose(1,2,3,0))
+stim = p2p.stimuli.VideoStimulus("road.mp4")
+
+stim.play()
+
+##############################################################################
+# You can shrink only one frame of the video, which is an image
+# The following would only shrink the first frame of the video
+
+
+image = video[0]
+img = p2p.processing.single_image_retargeting(image, wid=300, hei=50)
+plt.imshow(img, cmap='gray')
+
+##############################################################################
+# It does not work quite well, because our function needs motion information
+# to determine what is important. You can take the next frame of the image as
+# another input to provide motion information
+
+img = p2p.processing.image_retargeting(video[0], video[1], wid=300, hei=50)
+plt.imshow(img, cmap='gray')
+
+
+##############################################################################
+# You can see that the people in the image is identified, and do not shrink a
+# lot.
+# The function also takes other parameters: N. boundary, L, num
+#
+# When calculating the motion map, the image will be divided into blocks of
+# size ``N*N``, and the motion map Wt[x, y] will be determined by if there is
+# change larger than the ``boundary`` in the block containing the pixel [x,y].
+#
+# We use seams to cut the input frame. ``num`` determines how many seams you
+# would like to have when doing the seam carving.
+#
+# To preserve continuity between rows, we would apply a moving average window
+# of size ``L*1`` to the importance matrix.
+#
+#
+#
+#
+#
+#
+# Then let's try shrink the video. You can directly shrink it as a stimuli.
+# It takes a long time.
+
+
+new_stim = p2p.processing.video_retargeting(stim, wid=300, hei=50)
+new_stim.play()
+
+
+##############################################################################
+# You can also shrink it when it is a video array
+
+new_video = p2p.processing.stim_retargeting(video[0:5], wid=300, hei=50)
+new_video_stim = p2p.stimuli.videos.VideoStimulus(np.dstack(new_video))
+new_video_stim.play()

pulse2percept/__init__.py

@@ -52,6 +52,7 @@
 from . import model_selection
 from . import percepts
 from . import stimuli
+from . import processing
 from . import viz
 
 __all__ = [
@@ -60,6 +61,7 @@
     'models',
     'model_selection',
     'percepts',
+    'processing',
     'stimuli',
     'utils',
     'viz'

pulse2percept/setup.py

@@ -29,6 +29,8 @@ def configuration(parent_package='', top_path=None):
     config.add_subpackage('models')
     config.add_subpackage('stimuli')
     config.add_subpackage('utils')
+    config.add_subpackage('processing')
+
 
     # Data directories
     config.add_data_dir('datasets/data')