Inside Palantir, the CIA-funded startup operating in the shadows

By Kulvinder Lotay

The location is Palo Alto, California. The electricity and excitement can be discretely felt in the atmosphere and surroundings, as Silicon Valley giants stand next to the plethora of tech startups du jour. In the shadows is a building better described as a SCIF – sensitive compartmentalized information facility.


As part of its security, the building is 'air-gapped' from any outside networks, with walls impenetrable by phone signals and radio waves, and biometrics confirming the identity of every individual to enter or exit the facility. Above the entrance, a minimalistic logo with emboldened letters next to it. Its data storage is blockchained, requiring digital passcodes held by dozens of independent parties whose identities are themselves protected. These are the headquarters of CIA-backed startup Palantir.  


Founded in 2004 by PayPal co-founder Peter Thiel and current CEO Alex Karp, Palantir, (a reference to J.R.R. Tolkien's palantiri — seeing stones — in The Lord of the Rings) was intended to take Big Data where no one else dared to go.  The 2002 film Minority Report depicts a police officer in an LAPD 'pre-crime' unit in 2054, where police are able to predict when someone is going to commit a crime before it happens.  Palantir makes the idea of 'pre-crime' in Minority Report a reality, taking Google-level engineering and applying it directly to law enforcement and government spying.  


There are suggestions that say without Palantir, suspects would have already moved on to a different location by the time the field agents received the information.   However, by using advanced data mining and analytics, Palantir is able to predict a the occurrence of a crime and its location seconds or even years ahead of the crime actually being committed.  This is a company with as much real-world power as Google, Facebook, Apple or Microsoft, but operates so far under the radar that it is special ops.


Initially, Palantir began by taking the algorithms and systems used in PayPal for fraud analysis and applied them to other, larger sectors.  Valued at $20 billion in 2015, it has now become the go-to company for mining massive data sets for intelligence and law enforcement applications.  Combining a slick software interface and programmers who parachute into their clients' headquarters, Palantir is able to sift through millions of digital records across multiple jurisdictions, spotting links and sharing data to make or break cases.  With up to 50% of its business in the public sector, their client list includes the CIA, FBI, NSA, CDC, the Marine Corps, Air Force, Special Operations Command, and the IRS.


Palantir's law enforcement technology is based on its Gotham platform, a system used to organize and analyze unstructured data, such as emails, spreadsheets, and reports that it also sells to businesses and governments.  In Newton, Los Angeles, their system aided in reducing crime by 15%, while homicides dropped by 59%, according to Wired.  Everyone is tracked, from corporate fraudsters — like Bernie Madoff, who was imprisoned with the help of Palantir — to potential terrorists — Palantir is reported to have played a crucial role in the tracking and killing of Osama bin Laden.



Beyond the pure practical use of such software, Palantir’s practices bring up the notion of the Orwellian-thought police state, opening the door to a multitude of concerns regarding the ethics of the use of such powerful technology.  Palantir claims that it has and continues to decline a large number of deals on the basis of them not meeting ethical standards.  There are various safeguards in place, including granular access controls and immutable logs, creating a trail that tracks everything a user does in Palantir, which can be audited.  Additionally, every forward-deployed engineer travelling to a client site is provided with a Batphone, which can be used at any time to anonymously report work on behalf of a customer they consider unethical.


While Palantir's military-grade surveillance tools aid a crucial part of the US government, the company's private-sector oriented arm, Palantir Metropolis, provides analytical tools to pharmaceutical firms, hedge funds, banks, and financial services firms that allow them to outcompete each other.  JPMorgan Chase stated that Palantir's software, "saved the firm hundreds of millions of dollars by addressing issues from cyberfraud to distressed mortgages."  Beyond government applications, Palantir is now also running Wall Street, so an IPO in the near future may not be out of the question.

As we see the rise of unique and powerful applications of data mining, it is becoming clear that there is a need for further oversight and special supervision.  In the meantime, Palantir is exactly what it says it is: a real life representation of the palantiri from The Lord of the Rings.  As said by writer Mark Bowden, "Palantir's software actually deserves the popular designation Killer App."

Kulvinder LotayComment