Behavioral Detection Software: How Police Are Listening to You Part 3
Thibault Serlet
What Are Behavioral Recognition Systems?
Behavioral recognition systems (BRS) are one of the first law-enforcement applications of AI. BRS is an AI-based software which analyzes camera footage without human input. Increasingly, American and foreign law enforcement agencies are using BRS software that can analyze CCTV camera footage to detect various behaviors. There are benign applications for such technology (as pictured above). Police could use BRS to automatically detect all fights captured on camera within a city.
Most software which analyzes video footage is “stupid.” That said, the software is only capable of detecting specific pre-programmed behaviors. BRS takes surveillance software to a new level: it can be trained to detect new types of behavior and learns through statistical analysis.
Behavioral Recognition Systems is also the name of a corporation which invests and researches technology of the same name. They are the primary focus of this article, as most advancements in BRS have come from BRS labs.
How does BRS work?
In 2008, Behavioral Recognition Systems Corporation filed a patent for an AI that could use individual frames from any video to detect the behavior of the filmed individuals.
John Frazzini, President of BRS Labs briefly explained his patent. “Generally speaking, video analytics software receives video data from cameras and issues alerts based on very specific and narrowly defined human programmed rules that have failed to provide operational value in the video surveillance market. In strong contrast to those limited and deteriorating solutions, the patented technology of BRS Labs does not require any human pre-programmed rules, thereby providing an inherently scalable enterprise class software platform to the video surveillance market.”
The patent details several steps which the AI takes when analyzing video footage, although not in great detail.
First, BRS attempts to delineate groups of pixels from a picture into objects. Identifying objects and types of objects is difficult. In order to understand how BRS can learn to recognize objects, let’s detail the process BRS software might take to learn to identify cars.
If the BRS relied solely on using static pictures, visible non-objects such as shadows or reflections might get misidentified as objects. When BRS identifies a pattern of pixels, it will first attempt to see if the object persists between frames. This process is likely carried out to eliminate potential false positives. The diagram below shows how BRS eliminates the shadows cast by a tree by verifying if the shadow appears in multiple frames.
Once an object has been tracked over time across multiple video frames, the BRS then attempts to see if the object or parts of the object are already in its database. In the example of the BRS learning how to identify cars, lets assume the BRS has never encountered cars before.
Although the patent doesn’t go into detail about how BRS identifies objects, it likely uses some form of genetic appearance-based pixel searching. This method of object recognition technology searches for patterns of pixels such as edges, greyscale/color matching, and position recognition. This isn’t as simple as it sounds; whole cars are very hard to recognize. Both limousines and Humvees are cars, and yet they are very physically dissimilar. Instead of looking for whole objects, the BRS will look for the most easily recognizable parts of objects, and if enough parts repeatedly keep appearing, define a meta-object. A linear classifier will attempt to label new objects using an evolutionary database.
In the case of a car, examples sub-objects include circular patterns of black pixels which consistently appear near the ground (wheels), glowing pixel patterns in the front and back of cars (lights), etc…
After an object has been identified and classified, the BRS will then attempt to create a 3D model of what it sees. The process of creating a 3D model of a situation is fairly complex, and this brief summary will not do it justice.
Generating a 3D model from a single 2D static imagine is much more difficult than generating such a model from a 2D video. That is because static images only have calculable X and Y distances. In a 2D video, objects in motion appear to change size between pixels as the move relative to the position of the camera. The apparent change in size over time can be used to calculate distance.
Using the calculated distances, the BRS will attempt to identify a blob of pixels in the foreground, and a blob of pixels in the background. The computer assumes that the objects in the frame which remain static between frames are a background, while objects which move are a foreground.
By tracking 2 different blobs across at least 3 different frames depicting motion, the BRS estimates the positions of the identified objects, and thus creates a crude geographical 3D model of the scene.
Using methods similar to the BRS’ recognition of 2D objects, the BRS then groups elements in the 3D model into 3D objects. Once more, the objects are tracked frame by frame. If patterns of behavior
are detected in the geographical model, they are noted and stored.
How BRS Works: TL;DR
If police wanted to find all the muggings within LA, here is how LAPD would proceed to train the BRS. First, they would provide the BRS with as much footage of known muggings as possible. The BRS would turn the 2D objects into 3D models, then identify patterns of behavior within the 3D models as described above. Once common traits have been detected across the mugging footage, the BRS now has learned how to identify muggings.
Once the BRS has completed its training, it will then look for the patterns it identified in the footage earlier across all of LA’s CCTV footage. Now, any time the BRS detects a mugging taking place, it can notify LAPD.
When has BRS been used?
Police used BRS systems to police the 2012 Tampa Florida Republican protests.
It has also been confirmed that numerous train and rail services across the country have already deployed BRS on their CCTV systems. Amtrak, a California railroad line, is also known to use BRS on their cameras. Atlanta’s MARTA transit authority has also deployed BRS.
In addition to servicing local police departments, BRS has been deployed by businesses and governments worldwide. The Department of State deployed BRS systems across numerous hotels in Mumbai, India to fight terrorism after a series of bombings in 2009. BRS has already been deployed everywhere from nuclear power stations in Virginia to airports in the United Arab Emirates. It is likely that BRS-like systems will soon become a ubiquitous component of CCTV surveillance.
The Dangers of BRS
Human surveillance is strictly limited by manpower. When humans analyze data, more cameras gathering footage doesn’t amount to more actual monitoring. If anything, there is a data overflow, and diseconomies of scale kick in.
BRS has the potential to change that. In the hands of the American government, BRS poses a relatively small threat to humanity. American police will probably use this technology to prevent real crime, at least for now. Imagine that an evil government, such as the North Koreans, got their hands on such technology.
The North Koreans could use BRS-like systems to install a camera in every home, and literally bring about 1984. Dissidents would have no chances to escape the ubiquitous police state. What little opposition manages to sustain itself behind closed doors will die.
How to Fool BRS Surveillance
Because BRS relies on reading camera frames rather than more indirect methods of sensory input, there are few ways of easily avoiding BRS detection. There are, currently, numerous groups working to undermine the surveillance state.
Various design firms are creating clothing styles that can foil facial detection software. A Dutch artist is creating a T-shirt to foil facial recognition software. Fake 3D-printed faces may also hold the key to foiling facial recognition technologies. For lower budgets, this makeup tutorial explains how to easily hide one’s identity.
There also are numerous smart phone apps released and in development built to detect CCTV cameras. Some, such as the NYC surveillance camera project are massive databases of user-submitted camera locations.
Other apps, such as Privacy Electronic’s app, use infrared beams to detect nearby cameras.
Privacy is an arms race, with both sides making leaps and bounds. The outcome of the surveillance wars is, as of yet, unwritten. It is up to the brave and technical people of the world to find new ways to outwit the ubiquitous surveillance, and push back 1984. While the odds are currently stacked against privacy, history has a tendency to give moral victories to the most virtuous. Even more important than fighting against BRS-like systems using technology is winning the moral and ethical arguments for privacy.
Wanna learn more while keeping up with our latest blog posts? Check out our facebook page and twitter and stay connected.