The Minority Report: The future of security

(I posted this right after the Boston Marathon bombing.)

Following the events in Newtown, CT and at the Boston Marathon, the question of security, and especially how to prevent events like those from occurring reached the headlines, again.  Last Friday I was interviewed on the AVWEEK radio show, and one of the topics of discussion was an article claiming that the surveillance camera which provided the footage that helped catch the bombers (although, after the fact) was almost taken offline for service, which could have prevented the capture of those suspects. 

There are more and more cameras installed with live feeds available throughout the world.  Companies such as USTREAM (and now, apps such as Periscope) exist to give us access to that live feed.  The positive aspect of it is that we may get access to footage that will allow us to capture terrorists.  However, the negative aspect is that the amount of video stream provided by this increasing number of cameras cannot be monitored by security personnel in a way that will allow preventing terror events before they happen.  Or can it?

There is, and will always be a debate of whether surveillance cameras infringe on our constitutional right for privacy.  There will always be the debate of whether a surveillance camera in a public location does constitute such infringement, but this is not the topic of this article.  I will talk about the technological future of such surveillance.

One area of technological development is the field of smart cameras.  These are cameras that do not only provide a video feed for people to watch, but can actually analyze what they see using a technical discipline called computer vision, and provide some insight to what’s in that video stream.  Although sounds futuristic, we see some elements of this now.  When a regular pocket camera shows a square around someone’s head so you can focus the picture around that head, or when facebook shows you squares around people’s heads in an image to allow you to tag them, that’s computer vision. 

Many universities and research institutes work on different projects that make cameras “smarter”, allowing computers to analyze images and video streams and determine what’s in them.  Companies such as Image Vision Labs, located in our own Collin County, Texas, claim that computers can analyze 144 times more images than humans at the same given time.

In 2010, I attended the largest gathering of computer vision researchers, the IEEE CVPR (Computer Vision and Pattern Recognition) conference in San Francisco to see some of those projects.  One example was the ability to extract speed limit signs from a video stream coming from a dashboard camera to alert the driver to the allowed speed limit.  While the implications of the research presented were fascinating and far-reaching, almost every time I learned that it took almost a week for the computer program to analyze a 30-second video, and thus the analysis was not done in real time.  After all, learning that the speed limit was 40 MPH a week after the fact might be a bit irrelevant…   However, considering the fact that processing power doubles every two years (see a previous blog entry, "The Robots will take over by 2028"), without considering any improvement in the efficiency of those detection algorithms, and using 2010 as a base year, then by 2038 those algorithms will run in real time, and my car computer will tell me that it “sees” a speed limit sign as soon as I pass it.  This also means that by 2040 the same computer will be able to analyze the video stream from two cameras in real time, and by 2098 it will be able to analyze the video stream coming from all 1 billion cars that exist in the world (although, of course, by 2098 there will likely be more cars…). 

Furthermore, let’s assume that the image required to identify a person takes 1MB of storage.  Today I found on eBay a 2TB disk drive for $100.  Such a drive can hold the images of 2 million people.  The capacity cost of disk drive (as measured in GB/$) grows 60% every year (as I reported in my book, Bowling with a Crystal Ball).  This means that in 2027 we could buy a disk drive that could hold the images of all 7 billion people on earth for $100. 

Those dates seem very far in the future, but they really aren’t.  We will be able to buy that disk drive in our lifetime, and while a single computer that can analyze the video stream from all cars in the world will only occur in our grandchildren’s lifetime, the ability of a single computer to analyze the video feed from one camera in real time will occur ours.

One final reference before I reach the conclusion.  In 2006, Gene Frantz, then Texas Instruments’ most senior technical person and a friend of mine visited an Israeli company called CNOGA ( see article) that at the time showed the ability of a computer to analyze the video from a camera watching a person and provide vital signs, such as blood pressure, heart pulse rate, and even sugar level.  That company now sells a non-invasive, camera-based device that allows monitoring of those vital signs, especially useful for diabetic people who will not have to draw blood several times a day just to monitor their sugar level.

What does it all mean?

Smart surveillance cameras are already installed throughout the world.  Although they currently have limited functionality, and are installed in limited (although growing) numbers, this functionality will increase as both processing power and storage capacity increase.  The Boston surveillance cameras may have been instrumental when the video feed from them was watched carefully after-the-fact by personnel to catch the terrorists, but tomorrow’s networks of cameras and computers will be able to realize that someone has left a bag somewhere and fled the scene and alert security forces to prevent the act of terrorism before it occurs.  Cameras, networks, and computers like that could possibly identify that the Newtown murderer was up to no good as soon as he left his home (vital signs could have given his intentions away?  What he was wearing could have indicated the presence of the AR-15 that he carried?) long before he reached the school and alert the police.

Many times we have seen how reality follows sci-fi movies.  In our lifetime, I expect to see “The Minority Report” becomes the future of security.