The 2021 Kyle Rittenhouse case has put video evidence into the forefront of many discussions and debates around the United States and beyond. One of the most common questions from the trial, at least as it pertains to video evidence, is this, “does pinch and zoom actually alter the video evidence?”
While some people think it is ridiculous to say that zooming in alters the video, the issue is more nuanced and complicated. Authentication, the question of whether an exhibit accurately represents original data, is a legitimate legal debate with real legal consequences.
In this article, we will explore what happens when software zooms in on digital video evidence, explain several key terms, and discuss why it is so important for those who testify in court to understand how their software works.
Disclaimer: We are not involved in the Rittenhouse case in any way and have not reviewed the evidence in that trial. The information contained in this post is for informational purposes only, and is not a statement of law. This information is not intended to provide legal advice and should not be interpreted such. Readers should always confer with an attorney to obtain legal advice and consult with local, state, and federal ordinances and laws in their applicable jurisdiction. Our goal for this post is to provide an expert witness perspective and to summarize how our field is measured when we testify using commonly accepted methods, techniques, and tools (such as those available within Axon Investigate).
Let’s get started.
Table of Contents
Why Resize an Image?
What is an Interpolation Algorithm?
How Algorithm’s Impact What We See
How does Pinch to Zoom work?
Which Interpolation Method is Best for Playback?
Conclusion
Why Resize an Image?
Every image starts with a finite number of pixels, (small square shaped color values). Pixels do not have any more information in them other than a single color, and they cannot be resized without creating additional pixels or deleting pixels.
We live during an era where high-definition videos and photos are available at our fingertips. For instance, if you were to take a family photo with a modern iPhone, assuming proper focus/environmental conditions then you could likely have millions of pixels per person in the image.
This level of resolution is impressive, and EXPONENTIALLY more than we typically encounter in the video evidence world where “enhancements” are often requested by investigators.
Even during the day, typical digital video compression more often reveals that ‘enhancements’ are a Hollywood myth and are only valuable in limited scenarios. For example, it is very common for investigators to request enhancements of faces, license plates, and weapons where the entire object may only be reproduced with a few pixels.
To make things worse, the recording that needs to be enhanced is often created at night on a low frame rate surveillance system with infrared illumination where the data is heavily compressed.
:format(webp))
When producing a sports game replay, the program’s director can zoom in with remarkable clarity because the production is using high-definition cameras in well-lit environments, all the while using live optical zoom features. With most surveillance video evidence, we simply don’t have that technical visual infrastructure.
Because of this limitation, when investigators resize an image that already has so few pixels, interpolation can have a significantly negative impact on the result.
What is an Interpolation Algorithm?
When someone uses software to resize an image, they will be using an interpolation algorithm (method).
Images have limited pixels, and if you resize an image to make it larger, you cannot get extra data from within a single pixel. Instead, you must add more pixels.
For example, if I start with an image that is 352×240 and I double the size to 704×480, then I inherently must make new pixels. The “interpolation” algorithm decides what color to make those new pixels.
The most common interpolation methods are: Nearest Neighbor, Bicubic, and Bilinear.
Nearest Neighbor: This algorithm will duplicate all the original pixels and create identical pixel values, provided the resizing is done to an even integer, such as 200%, 300%, 400%, etc. It does not create any color values that were not in the original image. This algorithm exaggerates the hard edges in the original image but is an excellent method for highlighting exactly what the recording device captured. It is also a forensically sound interpolation method and appropriate for answering questions about suspect height, distances, etc.
Bicubic Interpolation: This algorithm will add new pixels with different colors based on a weighted average influenced by a grouping of pixels around each pixel target. It creates smoother-looking images that may appear more pleasing to the naked eye. This interpolation method is perfectly fine for demonstrative exhibits, but it is important to understand that the evidence has arguably been modified or changed.
Bilinear Interpolation: Like Bicubic, Bilinear also introduces new colors to newly created pixels with the weighted averages of the pixels around them. Unlike bicubic, the algorithm can be reversed and creates less rounding than bicubic does. It is best used for aspect ratio adjustments.
If you want to learn more about the different interpolation methods and see a specific example of why this matters, then you should watch part of the training video below:
How Algorithm’s Impact What We See
Depending on the interpolation algorithm used, specific questions such as, “In what direction was the gun pointed?” could certainly be impacted by using the wrong method.
One of our training classes contains an example of this exact scenario. In that example, the original image of a suspect contained only a few pixels around his hand area:
One of the experts resized the image using bicubic interpolation (image on the left, below) which added dozens of extra pixels and rounded out the shape of the hand area – giving it the appearance of a gun in the individual’s hand. That “fabrication” of shape is something done entirely by the interpolation algorithm:
How does Pinch to Zoom work?
During the Rittenhouse trial, the judge questioned, “whether using the pinch-and-zoom feature on an iPad would alter an image and he barred prosecutors from using that specific method while presenting video evidence to jurors.”
While some people expressed a concern that the judge’s ruling was unwarranted, Grant Fredericks of Forensic Video Solutions, (who was not involved in the case), explained that the basic problem with using the pinch and zoom method in court is that it does not produce a permanent record of what the jury was shown.
“Any demonstrative exhibit presented during a trial must be reproduced for the trial record. A copy of exactly what was presented to the jury must be available for potential future review by an appellant court.” Additionally, he describes that the pinch and zoom method employs an unknown interpolation algorithm, making it very difficult to quantify how much change to the image has occurred.
It’s important to understand the technical descriptions for the resizing process and to articulate how they can affect the evidence – because as we saw during the trial, different interpolation methods can in fact change the appearance of what is being resized.
Fredericks adds, “Careful thought must be given prior to trial for each and every video exhibit, including how and on what devices it should be displayed.” For more information about courtroom display of video evidence, check out his article in Police Chief Magazine.
Which Interpolation Method is Best for Playback?
Every video-related case is different. There are many different interpolation algorithms, and many of them have a place in the toolbox for different reasons.
As described above, the three most popular interpolation methods (Nearest Neighbor, Bicubic, and Bilinear) are available on the workflow tab of Axon Investigate within the resize node when converting files or exporting clips for court.
In addition, the Axon Investigate software has a simple toggle on the player of the Interrogate tab that can switch between a “Nearest Neighbor” rendering mode or a smoother rendering mode, depending on the question being asked during the examination. The simple toggle is highlighted below:
Are you looking to answer a forensic question, such as
What is the size?
What is the color?
How many pixels?
What is the distance?
Did something move?
If so, then we recommend using Nearest Neighbor rendering.
Are you simply tracking activity and performing a general investigation of what happened? Then we recommend using Smooth rendering for a prettier picture.
All interpolation processes are easy to track and replicate inside Axon Investigate, ensuring the evidence is reproducible and can be clearly communicated to the trier of fact.
Conclusion
There are several key lessons about video evidence from the Rittenhouse trial.
One important takeaway is that experts should be able to describe the technical actions they perform within forensic or investigative software, including being able to explain how their resizing or enhancement filters may have affected the visual evidence.
Dr. John R. Black, who utilized Axon Investigate to play the video evidence during the Rittenhouse trial, did an excellent job of explaining why and how he was using Axon Investigate. He specifically mentioned that he had completed the Axon Investigate Operator and Examiner certifications and demonstrated how the courses gave him the knowledge to testify with confidence on a global stage.
If you are interested in learning more about video evidence, we strongly recommend signing up for one of our online courses today.
:format(webp))
:format(webp))
:format(webp))
:format(webp))