Research Intern - Speaking the World into Existence

Research Internships at Microsoft provide a dynamic environment for research careers with a network of world-class research labs led by globally-recognized scientists and engineers, who pursue innovation in a range of scientific and technical disciplines to help solve complex challenges in diverse fields, including computing, healthcare, economics, and the environment.

The Speaking the world into existence research effort aims to apply the recent progress in large language models (LLMs) to prompt-based creation of interactive three-dimensional (3D) scenes. Integrating LLMs with a game engine should enable not only faster development of 3D content in various domains, such as gaming, mixed reality applications or animated films, but also allow for spontaneous user-generated content in the course of an interactive experience. Our previous work has also demonstrated that giving a large Artificial Intelligence (AI) model the ability to act in a simulated environment with feedback has the potential to improve the outputs of generative models by grounding them in the real world. One of the fundamental research directions of this project is to make large multimodal models more reliable in the domain of human-scale activity, by not only incorporating what has been said about the world, but also testing results in a simulation of the world.

Beyond application in gaming and building compelling virtual worlds, easier creation of interactive 3D content and simulations by non-coding users also opens applications to education (e.g. allowing teachers to create immersive virtual reality (VR) lessons in a short time), rapid creation of interactive training scenarios (imagine a group of first responders spinning up a simulation in the 30 minutes it takes them to arrive at the location of a disaster so that they can be prepared for the potential difficulties at the location) and many other applications in various domains such as real-time generation of virtual environments for therapy or creative applications.

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

Research Interns put inquiry and theory into practice. Alongside fellow doctoral candidates and some of the world's best researchers, Research Interns learn, collaborate, and network for life. Research Interns not only advance their own careers, but they also contribute to exciting research and development strides. During the 12-week internship, Research Interns are paired with mentors and expected to collaborate with other Research Interns and researchers, present findings, and contribute to the vibrant life of the community. Research internships are available in all areas of research, and are offered year-round, though they typically begin in the summer.

For this Research Internship (Summer 2024), our group is seeking PhD Research Interns with a similar passion for research on applications of the newest generative AI models to creative domains, with particular interest in mixed reality and gaming, as well as improving large multimodal generations by grounding them in the real world. Intern responsibilities will include (1) helping to develop a research platform integrating large multimodal models within a rendering engine, (2) building compelling scenarios exploring the various potential uses of this new technology.

Qualifications

Required Qualifications

• Currently enrolled in a PhD program in Computer Science or a related STEM field. Candidates enrolled in a Master's program might also be considered.

• At least two years of research experience, including peer-reviewed publications, researching a topic closely related to the above description, such as artificial intelligence (AI) systems for mixed reality, natural language processing, deep learning, generative models, mixed and virtual reality, multimodal interaction, etc.

• At least one year of experience in building applications with the help of game engines, particularly Unity.

Other Requirements

• Research Interns are expected to be physically located in their manager's Microsoft worksite location for the duration of their internship.

• In addition to the qualifications above, you'll need to submit a minimum of two reference letters for this position. After you submit your application, a request for letters may be sent to your list of references on your behalf. Note that reference letters cannot be requested until after you have submitted your application, and furthermore, that they might not be automatically requested for all candidates. You may wish to alert your letter writers in advance, so they will be ready to submit your letter.

Preferred Qualifications

• Demonstrated ability to develop original research agendas.

• Able to collaborate effectively with other researchers and product development teams.

• Experience in applying multimodal machine learning in a real interactive system.

• Experience programming multimodal systems that interact with users, particularly by integrating multiple machine-learned components such as computer vision, speech recognition, dialogue handling, natural language generation, etc.

• Effective interpersonal skills, cross-group, and cross-culture collaboration.

• Ability to think unconventionally to derive creative and innovative solutions.

The base pay range for this internship is - Applied Sciences IC2 : USD $5,090 -$10,120 per month. There is a different range applicable to specific work locations, with the San Francisco Bay area and New York City Metropolitan area, and the base pay range for this role in those locations is USD $6,690 -$11,030 per month.

The base pay range for this internship is- Applied Sciences IC3 : USD $6,290 -$12,170 per month. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $8,060 - $13, 240 per month.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: ~~~

Microsoft accepts applications and processes offers for these roles on an ongoing basis.

Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (~~~) .