GSoC 2014 Michael Mueller
Reimplementation of sunpy.wcs as sunpy.coordinates and Creating a ginga based data explorer / database browser
I am a current high school senior, and my academic interests lie primarily in math, physics, and programming. Mathematics has always excited me as a subject, and I have taken online classes in multivariable calculus and differential equations with a local community college, as well as linear algebra with Oklahoma State University, after exhausting my high school's curriculum. More recently, I have become interested in physics because it so heavily relies on the application of mathematical deduction to observation, and I enjoy learning about how mathematics applies to the natural world. In addition to programming, I enjoy running, reading, and playing or listening to music in my spare time.
I run Ubuntu Linux on my laptop, and my preferred editors generally vary by language. For standard text editing and programming in C++, I use emacs. However, I use Eclipse for Java projects and will use either emacs or IDLE when programming in Python. Although emacs has a bit of a learning curve, I find it very useful as an editor because its numerous commands and macros allow for faster and more powerful editing. I also often use IDLE simply because I've been using it longer than emacs and I like its syntax highlighting and interactive shell.
My programming background extends back to 7th grade, when I discovered MIT's educational programming tool Scratch. After playing around with Scratch and reading more about programming, I used online tutorials to teach myself Java. From making small computer games with friends to trying out programming challenges like Project Euler and Code Golf, I then continued to immerse myself in the world of programming and soon picked up experience with C++ and Python. Since then, I have continued to enjoy programming recreationally. Although I haven't often worked on large programming projects, one project I particularly enjoyed working on was the creation of an OpenGL-based 3D engine in C++ (viewable at https://github.com/amras1/opengl-engine). This project was exciting to work on because it involved learning about graphics programming, which contains interesting mathematical underpinnings (such as matrix transformations and quaternions). I also incorporated simple models of mathematical and physical phenomena in the engine, such as Lindenmayer systems (or L-systems), which allow for the rendering of fractal patterns which imitate such natural objects as plants, and particle systems, which can be used to create interesting effects like fireworks or the flow of a water fountain. Although I didn't implement very advanced versions of these effects, I enjoyed discovering new intersections between math, physics, and programming. These intricate relationships continue to excite me, and I hope to explore the applications of computer science to other fields in the future. I have also previously worked on an extensible zombie apocalypse text adventure game, which may be viewed at https://github.com/amras1/zombie-text-adventure.
I have been using Python in particular for about three years, and I find it very useful as a language whenever high-level programming is appropriate. Although there is somewhat of a performance hit in using Python compared to more mid-level languages like C or C++, its ease of use and natural syntax have allowed me to program more quickly and with less propensity for error. I particularly enjoy the most distinctive, or “Pythonic”, aspects of Python, such as list comprehensions, generators, and lambda expressions. In fact, I've often found that when I return to C++ or Java after using Python for a while, I become annoyed at having to translate one of these features into a more cumbersome syntax. In my opinion, Python's most useful language feature is the existence of iterables and functions relating to them. for i, elem in enumerate(elem_set): foo(elem, i) is far clearer and easier to use than the C++ equivalent for (std::set<int>::iterator it = elem_set.begin(); it != elem_set.end(); ++it) { foo(*it, it - elem_set.begin()); }, and standard library functions like map() and zip() make it much simpler to operate on elements of a container.
I am reasonably comfortable with git, having previously contributed to a friend's project on BitBucket and having begun contributing to Sunpy on GitHub. As of this writing, I have six merged pull requests for Sunpy (https://github.com/sunpy/sunpy/pull/854, https://github.com/sunpy/sunpy/pull/859, https://github.com/sunpy/sunpy/pull/861, http://github.com/sunpy/sunpy/pull/877, https://github.com/sunpy/sunpy/pull/883, and https://github.com/sunpy/sunpy/pull/901). These have mostly involved documentation and renaming, but #859 involved adding tests for each public function in the previously untested module util.util.
I have also become involved with Astropy, an astronomy library closely related to SunPy. I currently have three merged pull requests for minor issues (http://github.com/astropy/astropy/pull/2110,https://github.com/astropy/astropy/pull/2114, and https://github.com/astropy/astropy/pull/2142). These pull requests have involved adding documentation for time formats supported by Astropy, removing exceptions from comparison operators in the Time class, and documenting classes in astropy.io.ascii.core. I have also been working on a more significant open pull request implementing an HTML reader and writer in astropy.io.ascii, which may be viewed at https://github.com/astropy/astropy/pull/2160.
While I have played around with Qt in C++ in the past, I do not have much experience with the library. However, I plan on learning it during the community bonding period if my proposal is accepted. I am comfortable with GUI programming using the Swing library in Java, so I should be able to pick up the requisite knowledge fairly quickly.
Since the Sunpy library is intended for use by solar physicists and is therefore highly scientific in nature, a powerful framework for describing the position of points in 3-dimensional space is highly important. Thompson, W. (2009) describes a number of solar coordinate systems which are particularly convenient for use in solar physics. These systems range between Cartesian, Cylindrical, and Spherical representations and are each applicable in a variety of situations, varying based on the importance of the observer's position, whether angles or distances are most appropriate, etc. Currently, Sunpy contains a number of functions in sunpy.wcs.wcs, such as convert_hpc_hcc and convert_hcc_hg, for switching between various solar coordinate systems. However, this system is less than ideal as it simply uses numpy.ndarray for coordinate data and therefore misses out on useful coordinate functionality.
The project Astropy (closely related to the Sunpy project) contains a useful coordinate package in astropy.coordinates. However, as part of APE5 (Astropy Proposal for Enhancement #5), Astropy will be switching to a new coordinate framework involving the classes CoordinateFrame and CoordinateRepresentation, which allows for the more efficient creation of new reference systems and fixes previous issues with the astropy.coordinates package. It would be extremely helpful to be able to use this framework in the Sunpy library.
Accordingly, my proposal is to use Astropy's new coordinate framework to implement Heliographic, Helio-Centric, and Helio-Projective coordinates using the new inheritance model spelled out in APE5. In this model, high-level classes such as SphericalRepresentation and CartesianRepresentation inherit from CoordinateRepresentation and store actual spatial information, while lower-level classes combining both reference systems and frames inherit from CoordinateFrame. I intend to create subclasses from CoordinateFrame for these Heliographic, Helio-Centric, and Helio-Projective coordinate reference systems and add functionality for such reference frames as Heliocentric Aries Ecliptic, Heliocentric Earth Equatorial, and more.
At this point, my primary concern will be with the integration between these newly implemented reference systems and the future Astropy coordinate package, as well as the redesign of current functionality in sunpy.wcs. This will involve implementing transformations between solar reference systems and the Cartesian, Cylindrical, and Spherical coordinate representations, as well as adding general functionality for converting between Heliographic, Helio-Centric, and Helio-Projective coordinates. In addition, I will work on the general task of integrating the Astropy coordinate framework into the existing Sunpy data types, such as Map and LightCurve.
Finally, I plan to take even further advantage of the new Astropy coordinate system by implementing transformations between the celestial coordinate systems of Astropy and the solar coordinate systems I have implemented as part of this project. If there is enough time left over, I also hope to work on the general task of integrating the astropy.units package, which provides a powerful framework for handling the units of physical quantities, into Sunpy.
- Community bonding period (April 21 — May 18)
Read in detail about solar and other celestial coordinate systems to gain a solid understanding of these systems and of the algorithms for converting between them. Become more familiar with both Sunpy and Astropy, reading documentation (particularly regarding the changes accompanying APE5) and trying to fix issues on GitHub.
- May 19 — June 1 (2 weeks)
-
Begin writing classes
HelioProjective,HelioGraphic, andHelioCentricextendingCoordinateFrame, using the new Astropy inheritance framework as a guide. Use extensive testing to ensure that this implementation works as intended. Start implementing specific reference frames for these systems after discussing with my project mentor which frames will be of particular use. - June 2 — June 8 (1 week)
-
Finish the implementation of classes
HelioProjective,HelioGraphic, andHelioCentricand fix any issues or bugs that arise that this point. Once this is done, implement transformations between coordinate representations, such asSphericalRepresentation,CartesianRepresentation, etc. - June 9 — June 22 (2 weeks)
-
During these two weeks I will focus on extending the functionality of my new implementation. Use the bi-directional transformation graph in
astropy.coordinatesto implement conversion between Heliographic, Helio-Projective and Helio-Centric coordinates, replacing the current functions insunpy.wcs. - June 23 — June 29 (1 week)
-
Mid-term evaluations take place during this week. Since the project should be at least halfway completed, use this week to write up extensive documentation for the current functionality, write tests I might have missed in previous weeks, and work on any potential patches that might be submitted to the Astropy code base involving the coordinates package.
- June 30 — July 6 (1 week)
-
Explore the Astropy codebase in order to determine how conversion between existing celestial coordinate systems and the newly implemented solar coordinate systems might work. This part will involve reading more about mathematical conversions between coordinate systems and other nuances similar to the use of the small-angle approximation in converting heliocentric to helioprojective coordinates. Write complete documentation for these conversions and formulate a plan for implementing them.
- July 7 — July 13 (1 week)
-
Work on the implementation spelled out in the previous week, starting with the most common celestial coordinate systems like ICRS (International Celestial Reference System) and Galactic. Depending on the nature of the API accompanying APE5, the difficulty of this implementation might vary considerably.
- July 14 — July 20 (1 week)
-
Thoroughly test the previous week's implementation and add support for conversion involving other celestial coordinate systems. Work with the Astropy library (which might involve submitting patches upstream) in order to facilitate all of these conversions. Make certain that all conversions are well documented.
- July 21 — August 3 (2 weeks)
-
After performing unit testing and fixing any unforeseen issues that may arise, work on integrating the new coordinate framework into existing Sunpy data types like
MapandLightCurve. The full task of integration will likely be too extensive for me to cover by the end of the summer, so I plan on using these two weeks to plan out the structural changes involved in adapting to the new coordinate functionality, implement changes that can be done without a major redesign, and potentially draft an SEP (SunPy Enhancement Proposal) to facilitate future integration ofsunpy.coordinates. - August 4 — August 10 (1 week)
-
This week will serve as a buffer period in case earlier steps of the proposal require more time, unforeseen difficulties arise, etc. If everything runs smoothly and the buffer period is unnecessary, then work on integrating
astropy.unitsinto Sunpy. This will allow the Sunpy library to make use of functionality like unit conversion, decomposing units into irreducible parts, and dynamically creating new units which seamlessly fit into the existingastropy.unitsframework. There are also other opportunities I might be able to take on relating to this task, such as the integration of the highly important Astropy classQuantityinto the Sunpy code base. - August 11 (suggested pencils down date) — August 22 (final evaluation deadline)
-
Write documentation for all previous code and add any tests I may have missed during the main coding period. Look for potential bugs and fix any that arise.
Although the SunPy library already contains numerous features for dealing with solar data, a powerful and easy-to-use GUI for visual exploration of solar data currently does not exist. Luckily, the Ginga toolkit (https://github.com/ejeschke/ginga) already contains extensive functionality for graphically representing scientific image data. In particular, Ginga is built to use the FITS (Flexible Image Transport System) format, a highly prevalent format in solar physics, which further suggests that Ginga would blend very well with the SunPy library and could be adapted for a number of SunPy-specific uses.
Another major advantage of the Ginga framework is its flexibility, which stems from a highly configurable design pattern and will allow for easier implementation of useful features. The Ginga class structure follows a variant of the Model-View-Controller design pattern; the model (implemented in class BaseImage) performs the bulk of the viewer's algorithmic work, the view (in class ImageView) is responsible for displaying output to the user, and the controller (implemented both in ImageView and the Bindings class) feeds user input back into the model. In addition to providing a convenient organizational structure, this three-tiered approach is highly extensible, as new subclasses can be built for each component. More importantly, Ginga also supports a plugin system through which one can create either local or global plugins and use the Ginga library in conjunction with a chosen backend (either Qt or GTK) to add new features easily.
My proposal will involve using the Qt graphics library as a backend to create several Ginga plugins specific to SunPy. In particular, I first plan on implementing a database viewer plugin which will capitalize on the functionality of the sunpy.database package. The sunpy.database package currently allows for saving solar data that has been retrieved from an external service like the VSO (Virtual Solar Observatory). It would be very useful if this feature could be enhanced to allow for visual exploration of the information inside a database, so implementing this enhancement will be a major part of my project.
In addition, I plan on creating a Ginga plugin which will support solar coordinates using the coordinate system in AstroPy. Ginga conveniently encapsulates its own world coordinate system behind an object in its AstroImage class (which extends BaseImage), so it will be relatively simple to reconfigure Ginga to use a different World Coordinate System. If time allows, I hope to explore other possible plugins or extend Ginga's functionality, depending on what the project mentor might suggest or I happen to find.
- Community bonding period (April 21 — May 18)
-
Read the Ginga documentation and explore its class structure in order to get a preliminary sense of how its functionality might be extended via plugins. Read the pertinent documentation in
sunpy.databaseand experiment with SunPy's database functionality. Become proficient in programming with the Qt library by reading the documentation and trying out some smaller programs. - May 19 — May 25 (1 week)
-
Create a plan for implementing a global Ginga plugin that allows for the use of a solar world coordinate system using
astropy.coordinates. The Astropy coordinates package is expected to undergo a major redesign as part of APE5 (Astropy Proposal for Enhancement 5), so this part will involve figuring out these changes and potentially communicating with another GSoC student if another coordinate reimplementation project is selected. - May 26 — June 8 (2 weeks)
-
Implement the world coordinate system plugin spelled out in the previous week. Ginga is sufficiently flexible to allow for an easy shift in world coordinates with
AstroImage.set_wcsClass(customWCS)wherecustomWCSis a coordinate system class that implements certain important functions. Therefore, this plugin should present little difficulty from an integration standpoint and I can focus instead on ensuring the correct functionality of the coordinate system. - June 9 — June 15 (1 week)
-
Establish the basis for future code to add database functionality to Ginga. This will entail adapting the API in
sunpy.database, which allows users to save retrieved data, to a form that will be usable for visual representation of databases. Create functions designed to handle the metadata in local or remote databases and retrieve linked data from file, making use of theDatabase.query()method.sunpy.databasealso contains a caching feature, so that should be a useful part of this intermediate API. - June 23 — June 29 (1 week)
-
Since this is the week of mid-term evaluations and the project should be at least halfway completed, use this week to write up extensive documentation for the current WCS functionality and create tests I might have missed in previous weeks.
- June 30 — July 13 (1 week)
-
Use the intermediate API designed two weeks before to implement a local Ginga plugin for high-level viewing of database records. Make sure to communicate with the project mentor about how to format the GUI in Qt, so as to ensure that this and other plugins will be as useful as possible for SunPy's user base.
- July 14 — July 20 (1 week)
-
Use the intermediate database API to implement another local Ginga plugin (or simply extend the functionality of the first plugin) for previewing database records in greater depth and examining their associated data.
- July 21 — August 3 (2 weeks)
-
Look for other possible plugins to add to Ginga or for ways to extend its current functionality so as to make the tool more useful for SunPy users. For example, this might involve working on a tool for handling 3-dimensional data in FITS files.
- August 4 — August 10 (1 week)
-
This week will serve as a buffer period in case earlier parts of the project require more time or unforeseen difficulties arise. If everything has gone according to schedule, use this week to improve documentation and provide thorough testing.
- August 11 (suggested pencils down date) — August 22 (final evaluation deadline)
-
Write documentation for all previous code and add any tests I may have missed during the main coding period. Look for potential bugs and fix any that arise.
The last day of classes for me will be May 27 (the beginning of the second week of coding), so I will not be able to devote 40 hours for the first week. However, I anticipate that I will have very little, if any, schoolwork and I should be able to complete the set tasks for week 1. If not, then I can make up for missed work the next couple of weeks. Except for the first week, I will be fully capable of spending 40 hours per week on Google Summer of Code.
I have set up a blog for Google Summer of Code (http://muellergsoc.blogspot.com), which I intend to use weekly during the summer for progress reports. I will also be available for regular contact with my project mentor over the course of the summer through IRC, email, phone calls, or any other mutually convenient form of communication. I may be going to Cape Cod for a week or two over the summer, but I will have functioning wifi and plan on working on the same time schedule so this should not present a problem.
Note: all times in this proposal are EST (GMT-5).
