Who knows me better than I know myself?

Who knows us better than ourselves? No seriously, who does? While the answer to ‘who' may not be surprising, the extent to which may be astonishing. 

We are aware of the vast amount of information is being collected by each our devices. While some of this happens without our explicit knowledge, most of the data is from text and images that we explicitly provide. Within each mobile application, there are various SDKs or javascript snippets that automatically collect and transmit data back to their servers. Desktop browsers and browser-plugins have the means to track and manipulate the content on any page that is browsed. When banking online, the browser and plugins have access to personal financial data including account numbers and balances. They can track online shopping and spend behavior, medical problems, sexual behavior, religious and spiritual beliefs. 

Accessing data through a web browser is inherently less secure than using an application because the content delivery and its consumption are controlled by two different entities with different objectives. A browser may allow plugins to be installed which can track and manipulate content; the website providing the content has little or no control over this. It is now possible to identify a user across devices through cross device tracking techniques; it is now possible to correlate a person's browsing history across different desktop and mobile devices. While I explicitly sign onto my Google account from multiple devices, there are other companies who are able to determine this through other means. For example, if I browse for an Amazon product on my phone, an advertisement for the same product is displayed on my other devices.

In addition to having information collected about us, we voluntarily provide significant amounts of information about ourselves mostly through uploading photos, email and messaging and storing documents online. 

Here is a photo taken in the early 1980’s, which was later scanned and uploaded to Google Photos. 

Google determined this picture was taken at NASA Johnson Space Center in Houston, Texas. This was determined without GPS information embedded in the photo. Google has created a visual representation of the entire world from all the uploaded photos, which has been folded into Google Maps and Google Street View. Every location, building, point of interest has been tagged, analyzed and memorialized. Using this Google can accurately locate where a picture was taken even without GPS coordinates. Additionally, image recognition algorithms have become so good, that Google is now able to identify people across different time periods. The three people in the foreground were accurately identified, thirty years from when the picture was taken. 

Here are some of the implications of these technologies, extrapolating from where we are today.

  1. Using image recognition, Google and Facebook are able identify people from their photos. From uploaded photos, they can create a list of real world connections between people even if others in the photos are not tagged. 
  2. Photos provide a wealth of other information including
    1. Locations where we live and the places we visit.
    2. Personal tastes and preferences including
      1. What we like to wear,
      2. What we like to eat and where,
      3. What we drive,
      4. What we watch, play, listen to as in sports, games, music and films.
    3. Determine a person's health by
      1. Using visual cues to estimate a persons weight over time,
      2. Tracking distance moved (walked, run, cycled),
      3. Tracking sleeping and waking times,
      4. Tracking number of visits to the doctor,
      5. Tracking heartbeat using a fitness tracker if available.
    4. Track relationships between people by analyzing the sentiment from the photos they upload.
  3. From capturing the mouse movements on the screen, determine if a user is right or left handed.
  4. Through the browser track personal financial status, including how much and where each person spends their money, what their bank balances are.
  5. Most personal correspondence has moved online through email and messaging applications. These products provide insights into our deepest and most intimate thoughts, emotions and sentiments.

These companies have a near 360 degree picture of who we are, what we wear and eat, where we travel, the state of our health, our spending patterns and our thoughts and feelings. They are constantly developing new algorithms and techniques to learn more about us with the current data sets. With advances in AI and ML, it is possible to correlate all this information to create deeper analysis. While currently this is used to serve more relevant advertisements, it could have other uses in the near future. For example, an AI assistant can to make recommendations and predictions based on personal knowledge. This AI assistant could recommend a family friendly car upon the arrival of a new child (if there is not on already) or suggest that a person who has been steadily gaining weight and missing work, to go see a doctor. 

However, this information could have more sinister uses. Imagine what could happen if knowledge of a person's physical address coupled with the their current location were to fall into wrong hands. Google, Facebook and Uber may know this directly while others like Amazon, UPS/FedEx could infer this from shopping or delivery patterns. Through Facebook or Instagram, the whole world may know when a person or family is away from their homes for an extended period.

Is the cat out of the bag?

Individually there are certain things we can do to minimize exposure like using native applications (e.g. mobile banking app and not the browser), using browsers in incognito mode and disabling browser plugins.

Legislation has to be strengthened so that ownership of data rests squarely in the hands of the consumers. Data collection should be separated from its usage, i.e. Permissions have to be individually and explicitly granted for collecting and using data. For example, it should be possible for a consumer to allow access to collect location information and use it for routing, but not for advertising. If this does not happen voluntarily, this has to be legislated and audited to verify compliance. Sharing of data between businesses should follow the same guidelines. 

We do not want to rewind to a time before the mobile internet and give up the conveyances that it provides. But while trading privacy for convenience the tradeoff should rest squarely with the consumer.