Technology companies wield significant and increasing power. Having achieved this privileged position, incumbent on them surely should be to use this power responsibly? Failing to do so gives more credence to calls for increased regulation and oversight of the technology industry.
Under fire recently is Apple, whose well-intentioned attempt to clean up its App Store has inadvertently resulted in taking down programs which ought to have remained.
The controversy started earlier this year when a group of Norwegian politicians succeeded in passing a motion to strengthen the country’s gambling rules.
Subsequent to this, the Norwegian Gaming Authority (Lotteri-og stiftelsestilsynet) made a formal request to Apple to restrict access to certain gambling apps to Norwegian users of Apple products.
What appears to have happened on Thursday is applications have been mis-classified, such as a Polish Magazine and Image Sharing service.
It appears that Apple had used an autonomous system to scan applications for content that appeared to be gambling related, and then selected the apps for removal based on the results returned from this algorithm.
There are two questions I feel we need to consider in evaluating this:
Firstly, to what extent should autonomous systems be left unchecked without the guide of a human hand to protect against error?
To answer this, we might consider either the types of operations we are willing to submit to automation, or instead ensure human intervention is retained where the consequences of the autonomous action are above a certain threshold.
There is no best practice on this matter today, but I feel that it would only be reasonable to review manually the outputs of such a scanning mechanism to ensure accuracy before the consequences were imposed. Given the potential for economic harm to software developers of mistake, it seems clear that on a results-based assessment, manual intervention ought to have been deemed necessary.
The second question is what are the mechanisms to enforce market fairness when a small number of large platforms like Apple control all market participation?
The old-world equivalent of the App Store might be the portfolio of art held in a gallery, or the literature printed by a publisher. While any one gallery owner might take offence at a particular piece, or publisher refuse to put to print an individual work, the liquidity in the distribution channels meant that one could either hedge relationship between intermediaries or shop around until one could build a relationship with a distributor who matched their niche.
Today though, the same choice is not available. Many developers are dependent on Apple or Google for their source of income. If an application is removed from listing, then serious economic harm can suffer. Similarly if a merchant were to be blocked by eBay or Amazon – it could literally shut down their business. In the old-world we created special legal requirements for those people and organisations on which we depended on – we called them fiduciaries. Isn’t it time that that status be updated to include Apple, Amazon, Google, eBay and others?
In cases where a regulatory request prompted the intervention from Apple it might appear hard on the surface to criticise them. Looking deeper though, it is clear that we have unconsciously moved into a dangerous paradigm where the mechanisms that we built to protect us from the abuse of power no longer work in the digital world.
Most countries have a well-developed system of checks and balances that have evolved to ensure that power is wielded fairly and disputes can be independently settled. The Financial Ombudsman Service is an example that protects British consumers against unfairness which might be caused by Banks. The Valuation Tribunal Service is another that guards citizens against arbitrary bureaucracy from councils. While Apple do have their own internal escalation channels, as do eBay, Amazon and Google – they are not independent, not transparent, and fall outside the purview of the democratic institutions that manage ombudsmen in other industries.
This is surely something that must be addressed with some urgency?
What we can all learn from this though is to ensure etiquette is retained as we increasingly automate. It would have been easy for Apple to have forewarned those they chose to cut off. A simple letter tipping off developers that their works might infringe Apple’s rules, or in this case, a regulator’s wishes, would have gone far to have avoided what has proven to be a costly and embarrassing mistake.
Interested in the effects of technology on society?
Sign up and get a free copy of my ebook here:
