LegalTech 2015 — Not so much collaboration, cooperation or transparency

I attended LegalTech   2015 this year in New York. It was a great seminar and another well done project by ALM.

One wrinkle, though.

Lawyers, vendors and even judges continue to want to shove predictive coding down everyone’s throat. Not “technology assisted review (TAR)”, but specifically predictive coding.

With the upcoming changes to the Federal Rules of Civil Procedure; particularly those dealing with proportionality and cost shifting; I had hoped to go to LegalTech and hear a different message than I heard.

I was hoping to hear words such as “collaboration”, “collaborative cooperation”, “transparency” and “translucency”.  Not only did I not hear that from any, except the federal judges; I sat in a session while one lawyer said she might give process translucency and another lawyer stating he “is a no” on any process translucency, transparency or anything else that involves cooperating with an adversary in terms of the creation of training or seed sets or data reliance. The example he used was that of a solid, brick wall to sharing.

So much for cooperation and collaboration.

So what is “TAR”: “Technology-Assisted Review (TAR): A process for Prioritizing or Coding a Collection of Documents using a computerized system that harnesses human judgments of one or more Subject Matter Expert(s) on a smaller set of Documents and then extrapolates those judgments to the remaining Document Collection. Some TAR methods use Machine Learning Algorithms to distinguish Relevant from Non-Relevant Documents, based on Training Examples Coded as Relevant or Non-Relevant by the Subject Matter Experts(s), while other TAR methods derive systematic Rules that emulate the expert(s)’ decision making process. TAR processes generally incorporate Statistical Models and/or Sampling techniques to guide the process and to measure overall system effectiveness.”

And, predictive coding: “Predictive Coding: An industry-specific term generally used to describe a Technology-Assisted Review process involving the use of a Machine Learning Algorithm to distinguish Relevant from Non-Relevant Documents, based on Subject Matter Expert(s)’ Coding of a Training Set of Documents. See Supervised Learning and Active Learning.”

Predictive coding is the little sister to TAR’s much broader category of tools, including keyword sampling, emotive search, concept search, and similar document evaluations. Deciding which tools may be more effective, or which combination of tools will be used is a decision that requires both an intimate knowledge of the documents and an ultimate knowledge of the case itself.

Over 30 types of machine learning based classification tools currently exist and since different tools are optimally suited to different types of data, the results will always vary from project to project based on the data .

If predictive coding is insisted upon by a producing party because that party maintains it is the most efficient and cost savings approach, the validity and reliability of the process should be approached mutually.

Authors of a New York Journal article argued: “the various prior decisions revolving around the seed set emphasized the access of opposing counsel to the documents and rationale underlying the seed set. Such an attitude by the courts only encouraged intrusions by adversaries into the details and process of an opponent’s conduct of document review. As a result, counsel were rightly reluctant to adopt a document review system that would enable an adversary to entwine itself more readily into the internal decisions of counsel in the document review process.”

The rationale for determining privilege may continue to be a protected area for counsel, but the rationale for determining relevance and irrelevance to build the seed set should be something that both parties have knowledge and input about.  It should be for the Court to continue to keep a handle on the process to see that privilege and appropriate objection remains intact.

In Moore v. Publicis Groupe, Judge Peck outlined sharing seed sets and the analytics for the reliability with opposing counsel to review. By that, he seemed to recognize that if a balance of full discovery and proportionality were to be fairly balanced, unusual collaboration might be required. So, at least Judge Peck does not seem to view predictive coding as a process to be practiced behind a solid brick wall.

The nature of litigation involves a certain amount of warriors. Erecting brick walls to processes that, for some, still seems as much voodoo as science will never get the discovery process to a point of transparent collaboration or cooperation; nor will it ever allow us to maintain a level playing field, while controlling costs.