The Georeferenced Video (GeoVid) system acquires and manages videos that are being tagged with sensor properties of the camera during recording (for example, but not limited to, geographic information). The collected and correlated sensor properties (meta-data) are then utilized for storing, indexing, searching, handling and presenting large collections of videos in novel and useful ways.
By considering video-related meta-data and cross-correlating all the information, videos can be better managed, searched and presented. The current GeoVid system implementation demonstrates a system that consists of several acquisition applications for mobile devices and a sample video search portal. One of the novel GeoVid processing aspects is the use of an estimation model of a camera’s viewable scene. For GeoVid video acquisition, the system uses automated annotation software that captures videos, their respective field-of-views (FOV) and other sensor information.
1. A mobile device can continuously transmit the sensor stream to a portal server, while at the same time recording and storing the video in its local memory
2. The small size of the sensor data allows for robust and fast transmission to, say, a portal, on which the information can be further processed and made available to other users
3. Keyframes are extracted from the video and represent snapshots that allow a user to gain a quick understanding of the visual content of a video.
4. Sensor data indexing allows for efficient search of identifying and selecting videos with certain properties e.g. videos or video segments that show a certain geographical area.
5. Large archives of sensor-rich videos may be browsed and presented in various ways, such as camera location, the camera viewing direction or the viewable scene, and Multiple camera trajectories representing different captured videos can be shown on a map simultaneously.
Imagine you are planning a trip and would like to check out all the places before deciding where to visit. Currently, you can do this with, for example, Google Streetview. However, there the scenery is static and you won’t get a good “feel” for the liveliness and activities in the area. Furthermore, users cannot contribute and images will probably only be refreshed every few years. With GeoVid, the experience is more immediate because
• users capture video content and upload it; or
• users upload a sensor-rich description of the video and the data can be requested on-demand;
• videos and/or sensor-data are available for search and viewing almost instantly, in real-time; and
• popular areas are covered from many different viewpoints, during day and night, rain or shine, etc.
In summary, one of the application areas for GeoVid is a social video network that allows users to participate and share videos by giving them a dynamic view of places around the world.
The invention allows sensor-rich video recording and management, i.e., video is automatically annotated with relevant meta-data information. An important aspect is that the meta-data is automatically collected, relieving the user from the task of entering annotation keywords to describe the video. Furthermore, the collected sensor data is non-ambiguous (i.e., objective), unlike keyword-annotation performed by users where ambiguous and non-relevant descriptions may be entered.
The sensor-enriched videos are suitable to be processed efficiently and effectively in a number of ways. For example, videos can be searched, managed and presented based on their geographic properties.
One benefit of the invented method is that sensor-annotation can be done very conveniently and cost-effectively on current-generation mobile devices (e.g., smartphones). These devices contain many or all of the necessary sensors and are now available at relatively low cost and in high-volume numbers. Therefore, users can capture high-quality videos with devices that they carry with them all the time. The GeoVid system, therefore, enables a large number of novel applications.
For example, GeoVid users are able to upload, search and view videos at different angles and directions in real-time. GeoVid includes advanced features based on augmented sensor information which makes the sharing experience more social, enjoyable, and technically scalable beyond simple video uploading and viewing.