AR, Drones, JoT: A Closer Look at Three New Kinds of Tech-Driven Reporting
,There are those who say that when you are occupied with creating and disseminating fact-based reports dealing with current events, that will always be called journalism – no matter what kind of tools you enlist. However, other people argue that there is a significant difference between writing a standard newspaper article and putting out a multilayered, interactive digital story that features text, pictures, videos, audio, and data repositories. While we hope that there continues to be a common denominator, we tend to agree that there are now indeed very different kinds of journalism, some of them driven by technologies that were in their infancy or not available just a decade ago. In the following post, we will take a closer look at three new (sub)categories of digital reporting that are particularly appealing to us and have played a significant role in recent projects: Augmented Reality (AR) Journalism, Drone Journalism, and Journalism of Things (JoT).
AR Journalism
The display of your phone shows a virtual infographic hovering over the coffee package on the supermarket shelf; here is your chance to learn more about the environmental footprint of the beverage. Right next to it: Virtual links to articles about coffee farming in Colombia that lead to your favorite news apps. And once you get back home, you are invited to use the white wall in your living room as a canvas for a documentary on the cultivated plant Coffea arabica, screened via your AR headset. This is just one of many examples of how AR can be used for journalistic purposes.
AR is a technology that enhances the user's view of his or her environment: Pieces of digital content (texts, photos, videos, 3D objects) are superimposed via a smartphone screen or headset. Natural and artificial perception are thus blended, giving audiences a whole new experience. AR has been around for a while (e.g in head-up displays), but it was not until late 2017 that it could make a bigger splash, thanks to powerful, inexpensive hardware and better software (like Apple's ARKit). In 2018, the Reuters Digital News Report predicted the technology would soon take "a major step toward widespread adoption", and a number of media organizations (including DW) have been experimenting with AR stories and applications ever since (check out DW's ARticles and You Draw It, or SN Digital's Mission To Mars).
There are basically three ways to use AR in journalistic storytelling:
In the most basic scenario, the smart device detects a simple marker (e.g. a photo in a magazine or a logo on a wall) and replaces it with AR content (e.g. an animation, a video, or an object).
In a more sophisticated scenario, the smart device uses computer vision and/or geo-tags to identify a certain object (or building or landmark) in order to display additional information (texts, pictures etc.). This is the coffee report described at the beginning of this section and sometimes referred to as "in-situ AR". The "You Draw It" example can maybe thought of as a variant of this scenario, with the smart device's camera turned around and the mix of ambiance and digital items turned into an AR DDJ story.
Another refined way of using AR is to have a smart device "project" a digital 3D model (e.g. of a person, an animal, a vehicle) in a marked-off area (a tile, a table, the floor of your living room) where you can subsequently look at it from all perspectives and/or interact with it. This is the Mars Rover example. The projected content can also take up the whole display, thus creating an augmented reality meets virtual reality (VR) effect ("a doorway to the Red Planet").
While AR is still a major technology trend in 2021–especially with regard to the recent metaverse discourse–AR journalism has clearly not gone mainstream yet. The reasons are manifold: A lack of experts and designated budgets in media organizations, technical issues and relatively complicated workflows, conservative managers, and–last but not least–not enough users who have the right devices and are really enthusiastic about the medium. Nevertheless, augmented storytelling will definitely play an important role in the near future, as all the groundwork has been laid already and big media/IT companies are investing an incredible amount of money and labor power (e.g. Facebook's Reality Labs division now has 10.000 employees).
Drone Journalism
Drones, once enormously expensive and reserved for military purposes, have arrived in the consumer market and civilian media sector in recent years. Today, tech-savvy journalists and storytellers mainly use them to capture fantastic natural panoramas–and shocking images from crisis areas. In fact, the technology offers a lot more.
The term drone journalism applies when compact UAVs (unmanned aerial vehicles = drones) are used for journalistic reporting or research. The drones are almost always equipped with photo/film cameras and sensors. The control can be manual, semi-automatic or fully automatic.
Drones are a welcome addition to the journalistic toolbox. They allow both a different view of people and events (bird's eye etc.) and a new visual language (e.g. through complex tracking shots) while production teams do not require the "Hollywood" budget and logistics previously needed to pull of something like this (think: dollies, trusses, helicopters). Drones can be used in places that are inaccessible to humans and regular crafts and vehicles, or only accessible at great risk (e.g., a destroyed city in a civil war country or the interior of an active volcano). LIDAR (light detection and ranging, a radar-like method for measuring distances, speeds, and atmospheric parameters) and other sensor systems also make drones attractive for very thorough data-driven journalism (drone journalism professor and Pulitzer award winner Matt Waite even believes that's where the future lies).
As drone journalism has become a relatively safe endeavor thanks to technical advances and best practice documentations, many media organizations have worked with UAVs, mostly for relatively "classic" photo and video reporting. For example, National Geographic sent a drone team to the South Pacific volcanic island of Ambrym, CBS provided images from Chernobyl, and the BBC flew drones over the Auschwitz death camp to mark the 70th anniversary of its liberation. DW experimented with novel aerial cinematography in the complex and tech-heavy MultiDrone project and also has a Daily Drone section on its website.
However, there still are still no major "drone news rooms" or a massive number of sophisticated, drone- and data-driven stories and reports as envisioned by Waite. As the technology will become even better/cheaper and more journalists start experimenting with it, the situation is likely to change, though.
Journalism of Things (JoT)
Of the three new categories of journalism introduced here, Journalism of Things (JoT) is arguably the most avant-garde and emancipatory. Its focus lies on the critical observation and creative use of the Internet of Things (IoT). The IoT stands for a world in which not only computers and smartphones, but also coffee machines, bicycles and industrial robots have IP addresses–and provide all kinds of data.
The JoT manifesto, first introduced in 2019, states that all things equipped with cameras/sensors/AI components and connected to the internet are able to do a lot of good–if handled the right way. Following the vision of the manifesto's authors, IoT devices can serve as interfaces and digital reporters, make journalism more tangible, foster accountability, communicate the world, fight disinformation, and shape proper discourse. Furthermore, there is the realization that there are not only digital, but also biological and chemical "things" to study and recruit. JoT is, of course, closely connected to data-driven journalism (DDJ), but goes well beyond the scope of this by now almost traditional discipline.
JoT is not exactly popular yet–because it is relatively complicated–but as microelectronics and sensor gadgets become ubiquitous and journalists lose their fear of handling them, the number of high-profile JoT projects is starting to grow. Germany seems to be a true JoT hotspot: For example, there is the Tagesspiegel's Radmesser, which relies on input by a community of readers and a lot of sensor data to document (and mitigate) dangers for cyclists in Berlin road traffic. In 2019, a JoT team hired by the WDR used sensors and interactive web technology to do a very different kind of live report on bee colonies and the topic of insect mortality (#bienenlive). luftdaten.info, set up by the Open Knowledge Foundation (an NGO), is a project dedicated to constructing DIY fine dust sensors and visualizing air pollution. In the same vein, the Financial Times equipped a number of reporters with so-called Flow Pollution Monitors, sent them to different metropolises and asked: How safe is the air that we breathe?
In 2020, an IoT/JoT landmark success was achieved with the Corona-Datenspende (Corona data donation), a project initiated by the Robert Koch Institute (RKI), Germany's federal government agency and research institute responsible for disease control and prevention. More than 500.000 volunteers transmitted data from their fitness trackers and smartwatches, allowing the RKI (and journalists) not only to learn a great deal about the spread of the pandemic, but also about other intriguing things, like the average resting heart rate of people in cities vs. that of people in rural areas.
DW is experimenting with IoT/JoT in the scope of the CALLISTO project. The topic will also become increasingly important with regard to DW Data.
Journalism for the 21st Century
As already explained, neither AR nor drone journalism nor JoT will anytime soon become the predominant form of reporting or have a massive impact on how ordinary citizens consume the news. They are, however, exciting means to open up new perspectives, dig deeper, inspire audiences, use technology for good, and make journalism fit for the 21st century.
With this in mind, we are looking forward to many more experiments and projects.
