MoveToCode is an open source augmeted reality visual programming language (VPL) created by the Interaction Lab led by Maja J Matric. The VPL is used to study student kinesthetic curiostiy. The VPL can be thought of similar to Scratch in 3D.
MoveToCode is built for the Microsoft Hololens 2, as well as ARCore supported mobile devices, using the Mixed Reality Toolkit. Currently it is set up to work with the Mayfield Kuri robot but can be modified for any ROS supported robot via ROS# and rosbridge. A virtual, high fidelity model, Kuri robot is also currently availble with limited capabilities.
Doxygen documentation can be found here: https://interaction-lab.github.io/MoveToCode/html/index.html
- MoveToCode + a robot tutor using the Hololens 2 - this is when students coded text based exercises. We have since moved to phone based AR coding a robot (video coming soon).
- MoveToCode mobile captured from the iPad's point of view. In pairs, students first build mazes out of physical pieces of paper. Once the mazes match, students program the smaller robot to complete these mazes. Students must move around to complete the above tasks presenting programming in a physically situated environment.
M2C Mobile Elementary Pilot next day outreach changes: 88c7fcbeee11265823d3d979b8108003c1b9a505
M2C Mobile Elementary Pilot: b3780ab4b00eabf99acdcc934e9d9d88c6489ddb
Augmented Reality Appendages for Robots: Design Considerations and Recommendations for Maximizing Social and Functional Perception (Commit:d459dcc9a1fe267fa805064aabaf4a51934ca3f1 )
@inproceedings{goktan2022augmented,
title={Augmented Reality Appendages for Robots: Design Considerations and Recommendations for Maximizing Social and Functional Perception},
author={Goktan, Ipek and Ly, Karen and Groechel, Thomas Roy and Mataric, Maja},
booktitle={5th International Workshop on Virtual, Augmented, and Mixed Reality for HRI},
year={2022}
}
Commit for all papers below this: 0765f1d1ab46373f1bc1be90d9d5b07fafc2a533
@InProceedings{10.1007/978-3-030-71151-1_22,
author="Groechel, Thomas and Pakkar, Roxanna and Dasgupta, Roddur and Kuo, Chloe and Lee, Haemin and Cordero, Julia and Mahajan, Kartik and Matari{\'{c}}, Maja J.",
editor="Siciliano, Bruno
and Laschi, Cecilia
and Khatib, Oussama",
title="Kinesthetic Curiosity: Towards Personalized Embodied Learning with a Robot Tutor Teaching Programming in Mixed Reality",
booktitle="Experimental Robotics",
year="2021",
publisher="Springer International Publishing",
address="Cham",
pages="245--252",
isbn="978-3-030-71151-1"
}
@inproceedings{mahajan2020adapting,
title={Adapting Usability Metrics for a Socially Assistive, Kinesthetic, Mixed Reality Robot Tutoring Environment},
author={Mahajan, Kartik and Groechel, Thomas and Pakkar, Roxanna and Cordero, Julia and Lee, Haemin and Matari{\'c}, Maja J},
booktitle={International Conference on Social Robotics},
year={2020}
}
Versions of packages are pushed directly to the repository to avoid versioning issues, licensing permitting.
- ROS#; dwhit UWP branch (Apache 2.0 License)
- Mixed Reality Toolkit v2.5(MIT License)
- NugetForUnity (MIT License)
- Automatic Documentation Generation Unity Editor Plugin (MIT License)
The following are currently tested versions of required software. Higher versions of each software may work but have not been tested.
- Unity 3D v2021.2.3f
- .NET Framework 4.6 and Visual Studio 2019/VS Code
- Built for:
MoveToCode is open source under the MIT License
- Propeller Cap created by tamzidfarhan
For any questions, you can contact Thomas Groechel at groechel@usc.edu. Feel free to fork for your own projects and/or submit pull requests.