‘The exchange of private information, that is what drives our economy. But you come after me because you can’t arrest a land mass.’
- Malcolm Tucker, The Thick of It
The Tyranny of Data
“Social Credit” is from China; it’s the most sophisticated form of mass surveillance in the world. “Good behaviour” could refer to spending choices or what you say on social media. Behaving well could make it that much easier to get a loan, to get a prestigious job – etcetera. And vice versa, of course.
That makes Westerners wince, but arguably a subtler version is here in capitalist dressing. The premise of the lender Kreditech is that non-traditional data sources and machine learning serve for more precise credit decisions. Your output on social media could make it easier or harder to get a loan.
You can apply this to just about any information and any financial service: an IoT toothbrush could get you cheaper dental insurance, if you let it monitor your brushing habits.
Big data, people say, will empower consumers, because it empowers businesses who serve them. But it comes with a hidden assumption; that organisations will use the data to find out how they can give more value to their customers, rather than figuring out, to a more precise extent, how they can take value from them.
Rather, big data could exaggerate the current state of the market. Where a market is highly competitive and consumers are empowered, it is reasonable to assume that more data will make the service better. Every bit of information will go into improving your journey. If you are searching for a technical product which you don’t fully understand, your route through a vendor’s website will be guided by more experienced users, and their choices.
Conversely, market failures could become worse. Prices for medicines in market economies can be devastatingly high because, whatever they charge, buyers of the drug will have no choice but to pay more.
Medicine is an inelastic good, meaning that demand is less affected by price changes. But for all products, there could be elasticity curves on an individual by individual basis. Drugs which might make a moderate improvement to person A’s health could save person B’s life. With a personalised price index, big data could recognise that and make certain that person B is charged as much as they can afford. Where a customer desperately needs access to a product, the prudent economic choice is to drastically raise the price.
This has more serious implications when viewed through the matrix of finances. The example of “what is the most a person can pay for a loan” is a very serious one. The very principle of insurance is that lots of people buy a policy, for fewer people to claim. What happens when insurers can accurately guess the prospective claimants? Could “pre-existing poor genes” become an issue in U.S insurance markets as “pre-existing conditions” are today?
The idea that a vendor could know everything about you, including what you earn, how much money you are likely to have in the bank, and how much access you have to rival services, before they set the price - this cannot be considered a tool to empower consumers. This is a tool to negotiate against them.
Information and Labour
Amazon recently patented a wristband to track their workers’ movements around warehouses, to identify structural issues with the placement of goods.
It’s also a naked gambit for extra leverage over workers. Bosses can be more demanding if they can be more exact.
In a few years, Artificial Intelligence will be able to track individual objects across multiple cameras and the wristbands will be unnecessary. Indeed, computer screens, voice calls, outgoing emails, can all be parsed by machine learning bots for sentiment, success, manner, and other metrics. Metrics-led analysis of performance tied to structured incentives may creep out of sales and across an organisation.
This trend will benefit many. Performance will go recognised, and meritocracy is good. But for people who are bad at their jobs, or for those whose work is debilitating or mundane, this will function as a riding crop. This will empower bosses first, (the people collecting the data); it may or may not empower workers subsequently.
Data will revolutionise recruitment too. Soon, AI bots could scan thousands of CVs and determine skills. They will also be able to scan social media, and determine how good a “cultural fit” a candidate is for your organisation. (Cambridge Analytica was able to accurately predict a person’s skin colour, sexual orientation, and political affiliation, with 95%, 88% and 85% accuracy respectively based on a dataset of 68 “likes”.)[1]
A HR employee told The Guardian that it’s good practice for candidates to add “Oxford” or “Cambridge” to their CV in white text to pass the automated screening process.[2]
Of course, employers already check social media, just as they already read CVs. The difference is depth. An employer might glance over a Facebook profile and be turned off by clear red flags; a bot could cross-reference innocuous “likes” with data-derived assertions about propensity for success in unrelated tasks. An employer might check social media as a precaution; bots could do it as a matter of form.
The early internet empowered individuals in ways which rocked some industries. Now, big data from the web will empower organisations big enough to use it. They will use this to better reward candidates who drive their organisations forward and fire those who do not.
"The result of big data will be a more efficient realisation of market capitalism via mass data collection."
Everybody’s Doing It
What do data subjects think about all this?
In May, General Data Protection Regulation [GDPR] will make consent for data collection a prerequisite. The only issue is that a powerful enough market can make consent less meaningful. A warehouse worker arrives on their first day and loudly announces they do not consent to the wristbands monitoring their movements. What are they told?
Social media is possibly the biggest incursion into privacy in the West today; it’s also entirely voluntary. Mainstream commentary is still catching up with its implications. The Daily Mail wrote in February that ‘Facebook may be developing a system to automatically detect how rich or poor you are’.[3] But the Mail is reframing something people intuitively know. Facebook already has such a system; it’s called, Facebook. Data collection is the central idea of the profits arm of the site.
In 2013, Edward Snowden revealed that the US National Security Agency was collecting huge volumes of metadata on US citizenry (and an unchecked amount elsewhere). There was a backlash; but it was hampered by varying levels of technical literacy and the unsettled cultural question of “what is private?”
Today, in the UK, senior members of the Conservative Party openly threaten the most popular messaging app in the UK on the very basis that it’s encrypted. They can make this case because politically, privacy is a damp squib. Data collection only feels violating when one person is singled out. The public aren’t politicians or celebrities. Pro-privacy campaigns have been conspicuously elite; for example, celebrities embarrassed about affairs. Facebook has said that the Cambridge Analytica scandal has not affected use rates.
In Guarding Life’s Dark Secrets: Legal and Social Controls over Reputation, Propriety, and Privacy, Lawrence Friedman writes:
‘Privacy is a modern invention. Medieval people had no concept of privacy. They also had no actual privacy. Nobody was ever alone. No ordinary person had private space. Houses were tiny and crowded. Everyone was embedded in a face-to-face community. Privacy, as idea and reality, is the creation of a modern bourgeois society.’[4]
The important angle on mass data collection is not privacy; it’s data markets. The “information superhighway” has become an outdated metaphor.
"With the ability to sort such information, “the information supermarket” is a better one."
Privacy matters in the context of power; unequal power in any negotiation will affect the outcome. When you buy a smart home speaker, the data collection is less spooky than the fact that it’s an agent of another market player, engaged in a strategy to accrue leverage. Amazon may have been selling Alexa at a loss.5 This is significant; it indicates that Amazon thinks the data they will collect is at least as valuable as the amount they’re losing per sale.
Common wisdom about popular platforms goes that their strategy occurs in two phases; “attract”, and “extract.” Amazon’s mantra has been to simply set their “attract” metre to full; to not pay shareholders’ dividends; and to pass on their savings to customers. For its great service and value, customers must hope they stay in the “attract” phase forever, and continue to use their data to compete.
1 Grassegger H. and Krogerus, M. (2017) ‘The data that turned the world upside down’, Motherboard, 28 Jan [online]. Available at: https://motherboard.vice.com/en_us/article/mg9vvn/how-our-likes-helped-trump-win. (Accessed: 16 Feb 2018)
2 Buryani, S. (2018) ‘How to persuade a robot you should get the job,’ The Guardian, 4 Mar [online]. Available at: https://www.theguardian.com/technology/2018/mar/04/robots-screen-candidates-for-jobs-artificial-intelligence. (Accessed 15 Feb 2018)
3 Palmer, A. (2018) ‘Facebook may be developing a system to automatically detect how rich or poor you are: Patent reveals how the site could track your socioeconomic status,’ Daily Mail, 2 Feb [online]. Available at: http://www.dailymail.co.uk/sciencetech/article-5346733/Facebook-patent-uses-track-socioeconomic-status.html. (Accessed 16 Feb 2018)
4 Friedman, L. (2007) Guarding Life’s Dark Secrets: Legal and Social Controls over Reputation, Propriety, and Privacy. California: Stanford University Press, p. 258.
5 Hook, L. and Waters, R. and Bradshaw, T. (2017) ‘Amazon pours resources into voice assistant Alexa’, Financial Times, 17 Jan [online]. Available at: https://www.ft.com/content/876ede9c-d97c-11e6-944b-e7eb37a6aa8e. (Accessed 16 Feb 2018)