How real can the virtual get? Quite real. Photogrammetry is a technology that allows us to recreate real-life objects and spaces in the virtual world while retaining photorealistic qualities and life-size scales. This article will explore how this technology works, why it can become a powerful tool for educators, and how you can leverage it in your curriculum designs.
What is Photogrammetry?
The 3D software manufacturer Autodesk describes Photogrammetry “as the science and art of extracting 3D information out of 2D images. The process involves taking overlapping photographs of an object, structure, or space, and converting them into 2D or 3D digital models.” In lay terms, it essentially means that if you have sufficient 2D images of an object, from enough angles, with enough detail, specialized software can help you reconstruct the 3D geometry of the original object based on the information found in the 2D images.
How does it work?
The idea is to take pictures – many of them – of the object that you would like to replicate in virtual form. This can be a simple object on your desk, a piece of furniture in your room or even something as large as a street building. If you can photograph it, you can capture it. You are limited mainly by the number of angles accessible to you: if you can only photograph it from one perspective, your virtual replica will only show the pieces visible to you. If you can photograph the entire object in 360 degrees, then you will end up with a complete replica.
For example, if you are trying to capture a building and you can only photograph it from the street level, then your model will most likely not have a roof due to you having not taken any photos of the roof. However, if you were to deploy a drone or use other ways to photograph the top of the building, then your model would be complete.
The end result will depend on:
- The number of images you have taken: The more images you have, the more data the software will have to build the geometry with, and the more accurate it will be.
- The quality of the images taken: The higher the resolution of your images, the more detailed and accurate the result will be.
- The number of angles the images come from: The more angles you have of the object, the more complete the end result will be. Remember: the software can only reconstruct the geometry based on the images you provide it with. If there are angles missing in the images (such as the rooftop example mentioned earlier), the replica will also have those sections missing.
The process can be thought of like this:
Here, you can see an example of a meeting room that was photographed by the KnowledgeOne team:
Case study: Wing’s Noodle Factory
Now that we know what Photogrammetry is and what it can do, how can it be useful to us, as educators and instructional designers, to help enhance the learning experience?
Below is a case study of how our team used Photogrammetry to participate in the preservation efforts of a historical landmark located in Montreal’s Chinatown.
Background:
In 2021, activist groups in Montreal’s Chinatown sounded the alarm about real estate developers purchasing a significant amount of land within the historic neighbourhood. There were concerns about the developers’ intent, with many questioning how their plans could significantly alter the landscape and landmarks of the area. One of these landmarks is the Wing’s Noodle Factory building, located at the corner of the Coté and De La Gauchetière streets. This begs the question: why is a noodles factory a cultural and historical landmark?
The factory was not always a factory. Originally built in 1826, the 200-year-old building was constructed to house the British Canadian School, a secular, bilingual school set up for working-class children. It was Montreal’s first purpose-built school. James O’Donnell, the architect who designed Montreal’s iconic Notre Dame Basilica, donated his time to design the school building as a philanthropic endeavour.
After the school shut down in 1894, it was eventually occupied by the oldest company in Chinatown: Wing’s, founded in 1897 by Hee Chong Lee, an immigrant of China. The business is active to this day and has become well-known in many Asian communities across Canada.
First as a bilingual, secular school for working-class children, then as the factory of one of the oldest and most iconic Chinese-Canadian companies, the building embodies 200 years of Montreal’s anglophone, francophone and allophone history.
Activists urged the government to protect the building and the surrounding areas by granting it cultural heritage status.
Problem posed:
There was a risk that the building could be demolished or significantly altered by its new owners. This could have meant losing access to this piece of Montreal’s history.
Process for developing the solution:
One of the challenges we faced was discovering what technologies, such as Photogrammetry, could bring to the conversation.
We had given ourselves a set of objectives:
- If possible, the result needed to be as immersive as possible. The landmark could have disappeared if it were demolished, so retaining as much information as possible, in the rawest form possible, would have been important.
- The resulting output should have been versatile: the extracted information must have been usable and reusable.
- Cost or time efforts should not have been prohibitive: it shouldn’t have taken a very long time or cost a lot of resources to be a feasible solution.
As demonstrated earlier, Photogrammetry allows us to capture physical objects and spaces with photorealistic qualities and create virtual or digital copies at reasonable costs.
Therefore, our team went on-site in late 2021 and scanned the exterior of the Wing’s building using Photogrammetry techniques.
Results:
The result is a photorealistic 3D model of the Wing’s building:
This model can be viewed directly from your browser: it can be embedded into a webpage, and you can use your mouse to rotate the model, zoom in and zoom out. You can access it on your phone and use your finger to rotate and observe the model.
Augmented Reality (AR)
Revisiting the importance of versatility mentioned earlier, this model can also be viewed in Augmented Reality, an XR technology increasingly used in universities worldwide. This means the student could superimpose this model on their own physical space through their phone or smart device (if it is Augmented Reality capable) and observe it up close. We opted for a web-based AR approach to maximize accessibility, as the student won’t need to download an extra app to view the model. This can be done through the deployment of a framework: our team used Google’s Modelviewer, but there are also other ways on the market to achieve this.
Virtual Reality (VR)
Furthermore, Photogrammetry preserves photorealistic details and can also maintain scale. Therefore, this model can also be used in Virtual Reality, thus opening up the possibility of bringing the student “on-site” in VR. As the world struggles with the one pandemic (and potentially others in the future), we cannot disregard the power this use case can bring to any curriculum. Here is a proof of concept build that our team developed for VR: the building retains a photorealistic quality and is to scale, ensuring that the user would feel as if they were standing next to the actual building.
Takeaway
With this case study, our team demonstrated that Photogrammetry could be a potent tool in the toolkit of any educator or instructional designer:
We now have an eternal, immortal, photorealistic and three-dimensional piece of history that can be used in various formats and platforms. Should the building not exist in the future, for whatever reason, this snapshot now exists.
This snapshot can serve as a starting point for an infinite number of educational topics:
- The history behind the building: how did Anglophones and Francophones come together to set up a bilingual, secular school?
- The history of the Chinese-Canadian community in Montreal
- Chinese-Canadian entrepreneurship: from food to technology
- Ghettoization and renewal of Chinatown
Photogrammetry 101
You might ask yourself: so how approachable is this technology anyways? How expensive is it? Does it require super fancy tools? Technical expertise? The answers might surprise you. So here are some quick tips for you if you’re thinking about trying this out!
The Kit
The technology involves three main elements that do not necessarily require significant monetary investment or technological resources.
1) Camera
Remember that the result depends on the number of photos taken and the quality of the images. While high-end equipment will produce higher quality results, any photo-taking device will do the job, including the camera on most smartphones. If you are not sure that this is a technology you would like to invest in, do not let the tool hold you back: take out your phone and try it out!
2) Specialized Photogrammetry Software
The software is part of the magic formula and does most of the heavy lifting: it extracts the data from the photos to recreate the 3D replica. The industry-standard software is often a paid option, such as RealityCapture, but there are many free and open-source alternatives that can do the work. For example, Meshroom is one of many free applications that can produce photogrammetry renderings. If you search on the internet for “free photogrammetry software,” you will most likely find a long list of choices to try out, depending on your preferences.
2.5) 3D editing software
Sometimes, your model might not be very “clean” once rendered. This is bound to happen when capturing a high amount of information: powerful as it may be, the software might not be able to produce a 100% accurate rendering. If the model needs to be cleaned or modified, this can be done in most 3D editing software on the market today. One free and open-source tool that can be used is Blender. This will depend on your comfort level using 3D editing software.
3) Distribution
The content must be easily accessible and distributed once it’s completed. You can use some technical options, such as setting up webpages with frameworks (as mentioned earlier, we used Google’s ModelViewer to set up our model). Still, it can also be free and straightforward, like signing up to popular platforms such as Sketchfab, a website where users can easily upload and share 3D work by simply sending a URL.
For those who are interested in 3D printing… Yes, the result can also be 3D printed! (Some preparation work is needed, though!)
Tips
If you’re ready and equipped, remember these points before heading off!
- Point your camera at the object and start going around it while taking pictures.
- Make sure you go around the object and take photos from as many angles as possible (do not forget to take pictures from the top and the bottom whenever possible. Remember, any parts you don’t or can’t photograph will be missing in your result).
- Take more photos than you think you’ll need. For the Wing’s building, we took approximately 800 photos. You may need more or less depending on your subject, but having more is better than not enough!
- Make sure each photo overlaps with the previous photo as much as possible.
- Go for static lighting
- If indoors, ensure you have static lighting: do not move your light sources or your object once the process has begun. You want to avoid shadows randomly appearing or disappearing, or moving on your model throughout the process.
- If outdoors (such as capturing a building as we did), we recommend that you choose a cloudy day, as you want to avoid the sun creating harsh shadows on your object or have a cloud move through the sky in the middle of your process.
Conclusion
Photogrammetry is content-neutral: instructors and students can scan small things on their desks, medium things in their room, or even buildings on the street. As long as you can photograph it, you can capture it.
With talks of the advent of the Metaverse, an “alternate universe” to our reality, one can imagine the type of role a technology like Photogrammetry could play.
The virtual can be more real than we think.
Epilogue
In late January 2022, the Quebec government officially granted heritage status to the Wing’s Noodle Factory. This means that the owners of the buildings will need approval from the Ministry of Culture before performing any works on the building.
Related articles:
