If your autonomous car is headed into oncoming traffic, what should it be programmed to do? Swerve or hit the vehicle?

Chris Alcantara, Youjin Shin and Aaron Steckelburg, members of the graphics department at The Washington Post, recently published a three-dimensional immersive experience to model how an autonomous car works. Many car companies have said that they are close to releasing self-driving cars, but research and safety reports show that there are many things that the car’s sensors fail to pick up or react to in time.

Storybench spoke with Alcantara and Shin about how they came up with the idea for this experience and how they put it together.

How did this story come together?

Chris Alcantara: We started this project earlier this year and one of the things we were interested in looking at was the technology of autonomous vehicles. There has been a lot of coverage about this technology and how companies are saying they are getting close to building self-driving technology and getting cars on the road that are able to drive themselves. But we wanted to see whether this technology is working, what are some of its challenges and limitations, and because Youjin, Aaron and I are within the graphics department, we wanted to try doing this in a visual way.

No one else has gone into the details of how autonomous vehicles work and presented it in a visual way. Our mission was to make this a very immersive visually-based explainer on how they work in simple terms without getting into the technical jargon.

We wanted to give people something to look at and they could see and so when they pass by what they think might be an autonomous car, they would know that what’s at the top is a camera or that what’s on the side might be the radar detectors.

What made you decided to create a virtual experience as opposed to writing an article?

Alcantara: We are in the graphics department, so visual first is kind of like our mantra. There have been plenty of articles and plenty of text-based research on autonomous vehicles, but for a lot of people who many not have the time or patience to read through what would be a very complicated story or research paper, doing this visually is something we think could be beneficial for visual learners.

This was a challenge for us because we wanted to break the mold of our normal graphics template or the normal way we look at things and experiment with some new 3-D technology, modeling, illustrations and reporting to get this out in an interesting and visual way.

Youjin Shin: I also have a background in self-driving car technology and whenever I talk to people about it I realize that people do not really understand the limitations of autonomous vehicles and text-based articles do not explain the limitations well.

I thought that making an immersive experience could be better at helping people understand how the cars work.

What did the design process for this look like?

Alcantara: It has been a long road. A lot of the things we work on are fast-paced things, but for this we wanted to know exactly what was going to be on the page before we started building anything.

Youjin and I sat in a room with a white board and drew out what we wanted every single slide to say and show. Youjin, then, transferred it to a prototype to show our editors exactly what our vision was.

Once we had our prototype and the information we wanted to include, we did more reporting. That consisted of reading all the publicly available safety reports from autonomous vehicle companies and research on self-driving cars and how it compares to human drivers, and flying out to Chandler, Arizona to test drive one of Waymo’s cars.

We did a lot of research on these cars so it was good for us to go out there and see how the car is moving and how parts of it function. We also recruited another reporter to test another car in California. After that it was just building. Youjin and Aaron were responsible for getting a look and feel of the experience. Aaron drew a lot of the 3-D models and Youjin developed it. I helped with a lot of the writing and reporting. By the end of it we had a full 3-D immersive project.

In the meantime, it was going through a lot of editing, a lot of questions from our editors. A lot of going back and revising, going back and reporting, going back and revising, over and over and over again.

How much time would you say you spent on this project?

Alcantara: I would say maybe more than 6 months but that was in between breaking news, other projects that came up and we had to jump on and even vacations. It was a very long process, but our editors are very happy with it and we have gotten a lot of good feedback from within the autonomous vehicle industry as well as outside of it.

What feedback did you get from the autonomous vehicle industry?

Alcantara: We got a lot of feedback on Twitter and everywhere else saying that this is a great representation of the cars. And we have also gotten some feedback from people who have been a little bit skeptical about it who wondered whether some parts of it were misleading in some way.

But we stand by our story. Our objective was to educate people how this technology works and what limitations still preside as companies develop this technology.

What tools did you use in designing the prototype and the final product?

Shin: For the prototyping we first used a single paper to represent each slide where we would draw it out and write down what things we wanted to say in each slide. And then we moved it to the Invision app where we started designing it and used colors and some representation of visuals and then we started building 3D animations in three.js, a JavaScript 3D library.

How did you pitch this piece?

Alcantara: Last year our graphics director asked us what big projects we want to work on next year and autonomous vehicles was one that Youjin had pitched early on and the assistant managing editor all the way at the top of the food chain was very interested. Youjin had already gone through a lot of research and complied a lot of documents to kind of make the formal pitch to our editors. By the time we got to the white-boarding phase, we had approval to work on it.

What difficulties did you face in creating this piece?

Alcantara: The one hard part was trying to translate a lot of this stuff into simple language. There is a lot of jargon and things that we had to simplify down for our piece. Another hard part was writing the text. We worked with our editors 4 or 5 times to cut the text down so it could fit into the graphic and so that it wasn’t just a wall of text. We also had to make sure the text does not obstruct the graphic on mobile.

We edited it down so that there is just enough that we are informing people about it, but not where it is too overwhelming. We tried to do that while making some compromises like including tips and walkout text towards the end to give readers more background once they finish the piece.

Shin: Fact checking every single word we used was a challenge because it was all based on a lot of technical reports so wanted to make sure everything is correct. To do that we had to go back and look over all the reports and documents.

Another challenge was the technical development. We used a lot of sophisticated animation and so the file was very heavy. Every time we ran the program in the beginning the computer’s fan would start blowing very loudly.

Alcantara: This was a source of anxiety for us because we really had to work on scaling it down and make the webpage as light as possible so that anybody with a computer or a phone could enjoy this.

What limitations did this medium have?

Shin: We wanted to make it as immersive as possible but we had never created graphics like this before so it was really challenging for us to learn this technology. We also had not done this this page-by-page storytelling process.

We had to learn about this kind of prototyping from other people through things like Youtube videos in order to show our editors what we wanted it to look like in a clear way. The process of design and development and then reporting was super challenging for us.

Alcantara: There is not a lot of information to go off of in terms of what a vehicle sees or how it thinks, we only have what these companies have chosen to release to the public and research reports. Some companies released videos showing what their cameras see and we took video from inside the Waymo car of the screen that shows people how the car is perceiving its environment.

This has been an emerging field for a very long time and it is getting fast tracked. There are billions of dollars at stake with the market. The last thing companies want to do is share a lot of information or give competitors an edge because they could put themselves at a disadvantage.

If you could go back and redo this piece, what, if anything, would you change or do differently?

Shin: I would have documented everything digitally from the beginning. As we continued to edit the text we had written, we would have to go back each time to the documents to make sure that our wording is accurate. This was not efficient because there were a lot of reports.

Alcantara: It would have been helpful to maybe have one additional person to help with the reporting. An extra set of hands and eyes to look at this, offer feedback and offer some help would have been great. Three people on a project is not really common; not a lot of graphics teams have that luxury of having this many people working on a project.

Shin: We did not expect this project to be this big.

Alcantara: Yeah. When we were pitching this project at the beginning of the year we were saying we are going to have this project our by April, and it came out in September. As this project grew in scale it sucked up a lot of our time.

What did you think about this project as an overall experience?

Alcantara: The project was a very good learning experience for us in terms of learning how to report on something that is super complicated. Now we have ways of experimenting more with 3D technology.

Youjin learned a lot of new coding techniques in terms of how to make a lot of amazing transitions that we had not seen in a lot of other pieces and we will now be able to use for other projects.

Shin: After we published the article, we realized that a lot of experts in the autonomous vehicle industry or artificial intelligence field appreciated our piece and stated that we put the information out there in an understandable way.

Alcantara: This is a project that not only are we happy with but everybody else in the newsroom was pretty blown away by. We are looking forward to doing the next big thing, but for now we are going to take it easy and work on some smaller stuff.