Sum-product network (SPN) implementation and its application on image completion.
The complexity of the partition function is a key limiting factor in graphical model inference and learning. Sum-product network is a new kind of architecture which is tractable, compared to Bayesian network and Markov random fields. Recently, SPN have appealed to more and more researchers. This repo contains all files of my undergraduate thesis, a project to implement SPN and reproduce the results of image completion.
The target of this project is to reproduce the results of the application to image completion proposed in the original paper[1].
SPNs are directed acyclic graphs with variables as leaves, sums, and products as the internal node, and probability as weighted edges. This architecture is tractable and aims to resolve the limiting factor, the complexity of the graphical model. Any tractable graphical model can be cast into SPNs. However, SPNs are more general.
Two dataset are used in this program, following the application of orginal paper[1]:
-
Caltech-101: The dataset contains 101 categories of pictures of objects, with about
$40$ to$800$ images per category.[2] -
Olivetti: The dataset contains a set of face images taken between April 1992 and April 1994 at AT&T Laboratories Cambridge with each image in size
$64 \times 64$ [3].
The models learned from the dataset will be stored in results/<DOMAIN>/models, which DOMAIN will either be caltech or olivetti. And the images for left and bottom completion will be output to results/<DOMAIN>/completion. The original and completed images will be aligned by separation with $10$px gaps but no zero intensity.
- common
my_mpi: Use OpenMPI to support the messaging in a parallel program. It means that this program will use parallel architecture to accelerate computing.parameter: Control parameters for EM algorithm, SPN, and evaluation.timer: Manage the time to help calculate the time spent on computation.utils: Some helper functions to access time, print log, and do some numeric process.
- evaluation
dataset: Read and process data from the dataset. Input are images and will be divided into training and test sets.eval: Conduct evaluation over the dataset.image_completion: Conduct image completion, which is the application.run: Control the program, which is the main function.
- spn
decomposition: Decompose the regions.generative_learning: Conduct the learning process to generative the model.instance: Record the mean and variance of an instance, which is calculated from the dataset.node: Define the node, to provide the base class of the sum node and the product node.prod_node: Define the product node, derived fromnode.sum_node: Define the sum node, derived fromnode.region: Compute the mean and variance of regions in the picture(for this image completion application), as well as the MAP.SPN: Define the Sum-product network, including a root, functions of learning and applications. These functions are implemented via calling other modules.
This picture will show the call graph between modules of this program.
Under folder Implementation, run the following commands to compile the executable program:
openMPI is needed to provide the parallel computing environmet. Please visit open-mpi official website for details if not available.
To make the program read files correctly, please modify the file path in dataset.cpp, eval.cpp, and run.cpp to correct path before compiling.
Please use MPI wrapped g++ compilers, either openmpi-wrapped gcc or Intel compilers with OpenMPI library.
Compile run first and run the binary for results.
Compile eval second and run the bianry to evaluate the MSE.
Run the following commands to get the executable program:
make run # to run the learning process
make eval # to do evaluationRun the following commands to reproduce the experiment:
mpic++ -n 102 run -d C #for caltech, 2 groups with 50 slaves and 1 master per group
# or
mpic++ -n 51 run -d O # for olivetti, 1 group with 50 slaves and 1 masterResults will be stored temporally under Implementation/results.
Note that the master branch does NOT contain any results, please switch the cluster branch to get my results.
Here are MSE of some categories:
| Category | MSE in Left Half | MSE in Bottom Half |
|---|---|---|
| accordion | 7629 | 8291 |
| brain | 2799 | 3057 |
| butterfly | 4621 | 4390 |
| cup | 5434 | 5278 |
| Olivetti | 1084 | 1156 |
The following images are the comparison of the original and Poon's(left column), the original and Exp #2(middle column), the original and Exp #4(right column).
| Original vs Poon's | Original vs Exp #2 | Original vs Exp #4 |
|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
For models, please see mdl files in Implementation/results/<DOMAIN>/models
For completion data, please see dat files in Implementation/results/<DOMAIN>/completions
For more evaluation results, please see evaluation_mse.txt in Implementation/results
For more completed images, please use plot_img_64x64.m or plot_img_100x64.m in /Implementation/tools to output the images. Note that the file path should be manually set.
Note that the evaluation and the images in this README were output based on the results stored in the clutser branch.
[1] Poon, Hoifung and Domingos, Pedro. Sum-Product Networks: A new deep architecture. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2011. Code.
-
Implementation of SPN (Dec. 28, 2018)
-
Documents of Code (Jan. 31, 2019)
-
Implementation of rest part(Mar. 4, 2019)
-
Reproduce of the Original Application (Apr. 24, 2019)
-
[ ] Further Exploration and Optimization (Apr. 27, 2019) -
Interim Report (Apr. 4, 2019)
-
Thesis Draft(May 6, 2019)
-
Final Defence (May 23, 2019)
Yilin ZHENG
E-mail: 11510506@mail.sustech.edu.cn
Supervisors:
- Prof. Shan He(UBir)
- Prof. Ke Tang(SUSTech)
Inspector:
- Prof. Bo Tang(SUSTech)
Committee Members(SUSTech):
- Prof. Qi Wang (Committee chair)
- Prof. Jianqiao Yu (Committee member)
- Prof. Jialin Liu (Committee member)
Other:
- PhD. candidate Weifeng Li(UBir)
- PhD. Xiaofen Lu(SUSTech)
- Phd. Guiying Li(SUSTech)
MIT