Post Process

Everything to do with E-discovery & ESI

Archive for the ‘Project Management’ Category

Around the block: 10/18/10

Posted by rjbiii on October 18, 2010

A few articles of note:

Unsurprisingly, to those who have been paying attention, some of Facebook’s apps transmit personally identifiable data. This breaks Facebook’s rules and raises many of the same privacy questions that has dogged the site in recent times. From the a WSJ article on the issue:

The problem has ties to the growing field of companies that build detailed databases on people in order to track them online—a practice the Journal has been examining in its What They Know series. It’s unclear how long the breach was in place. On Sunday, a Facebook spokesman said it is taking steps to “dramatically limit” the exposure of users’ personal information.

“A Facebook user ID may be inadvertently shared by a user’s Internet browser or by an application,” the spokesman said. Knowledge of an ID “does not permit access to anyone’s private information on Facebook,” he said, adding that the company would introduce new technology to contain the problem identified by the Journal.

The Ensigns blog has posted an interesting article on Search, perhaps inaptly entitled E-Discovery Search: The Truth, the Statistical Truth, and Nothing But the Statistical Truth. It is a very good primer on search, rather than on statistical methodology that one might surmise from the title. It is, however, a good article. An example is a passage on Latent Semantic Indexing:

What does “Latent” mean? Roughly speaking, it means “hidden.” And “Semantic” means, again roughly, “meaning.”

So, the phrase is actually descriptive of what we are trying to accomplish: find the hidden meanings (patterns) in a collection of documents, not because of the specific words we choose as input, but because of the other words in the documents containing the words we did choose and their “co-occurrence” with words in other documents, documents which do not contain our search terms. provides you 10 helpful tips for managing cases. In 10 Tips for Effective Litigation Case Management, there is more than just a nod to applying project management principles to help with ROI and making decisions, an approach of which I greatly approve. From the article:

The past decade has ushered in significant new challenges in litigation case management. These include: the explosion in electronic discovery, the increasing importance of cross-border cooperation in litigation and investigations, and the expectation that counsel will keep abreast of, and communicate to their clients, changes in relevant legal rules and precedent on a virtually real-time basis.

These challenges have been accelerated by the global financial crisis, which has led clients to become more comfortable asking for, and coming to expect, services and fee arrangements tailored to their unique needs and goals. We are in an era of increasing competition and increasingly sophisticated legal consumers. The goal must be maximizing client value without sacrificing quality service. In the end, after all, the business of law really is all about the client and achieving its objectives.

In brief, other topics include:

Posted in Articles, Privacy, Project Management, Search Engine Technology | Tagged: , , , , | Leave a Comment »

Getting the Measure of E-Discovery

Posted by rjbiii on February 23, 2010

I’ve been involved in putting together a regimen for measuring e-discovery processes for a client. Not the first time I’ve done it, but as I go through the exercise, I find myself reflecting on the metrics used.

One of my first observations is that the standard reporting portfolios for most platforms still need enhancement. In comparing various platforms and work flows, I become frustrated with the inability to obtain fairly basic reports out of the box. I’ll not give specific examples, because I’m not writing this to pick on anyone in particular. The importance of reporting for each phase of an e-discovery project (see the EDRM for a nice description of these phases) has increased with the recognition by those in the industry that better project management is needed in this space. There’s even a blog on the subject (that, incidentally, also discusses metrics). In order to manage the project correctly, however, you must have a handle on the current status of processing, review, and production. The platforms I’ve looked at fall short on that score. The usual solution is to tap into the back-end database to create the reports I want.

So a couple of things I look for when I examine technology are:

  1. The standard reporting portfolio for a platform; and
  2. Accessibility to the back-database for creating reports not provided.

Now, to some specific metrics. These can vary greatly, depending on work flow. A great result in one environment might be lousy in the next. Here, I’m considering a standard project with culling, attorney review, and production. When discussing loading, or ingestion, I look examine a number of things.

Ingestion Speed. This measures the volume of data that can be loaded into an application, and is expressed in volume over time (e.g., GBs / hour). It is not the most important metric, slightly slower ingestion speeds should not become a concern in a well-managed project. Large discrepancies in this, however, might serve to send attorney reviewers to the break room just a bit too often.

Ingestion Error PCT. This is important, and affects data integrity. This measures the inaccuracy of the ingestion process. Whenever a platform separates a component in a document (say a gif in the signature of an e-mail), it increases document count and leads to greater review times (or culling/processing times). Should a document not be correctly loaded, and goes missing from the review-set, then potentially relevant data is omitted from the review. Why measure inaccuracy rather than accuracy? My way of focusing on what should I think should be emphasized. Differences should be (relatively) small…so 99% to 97% doesn’t look like a big difference. But 1% to 3% does.

Culling Volume. This measure is an industry standard. It measures the total volume removed by the use of certain processes. File-type filtering, removing known files (or de-NISTing, as it is sometimes referred to), and date range filtering are three commonly used culling methods. De-duplication and “near” de-duplication are often factored in as well. Another method includes domain analysis and culling (flagging or removing junk and privileged domains). Culling volume can be expressed in terms of volume (obviously), using GBs or MBs, and it can be expressed as a percentage of the dataset (removed 30% of the documents).

Culling Error PCT. What percentage, if any, of documents culled would have been considered relevant or privileged? In other words, what culled documents should have been reviewed? Now, how do you obtain this figure? Only with a lot of work. You’d basically have to review the culled document-set. But it would be an interesting experience.

Review-set Precision. The percentage of relevant and privileged documents in the dataset that is presented for review by attorneys. This item greatly affects review time, which is by far the largest cost component on a project.

Search Term Analysis. This is not so much a measure of the technology used, but looks at the effectiveness of various components of the search protocol. It measures the effectiveness of each term and can be used to improve search criteria.

Review Rate. This metric is applied against both the review team as a whole and individual reviewers. It is expressed in DDH (document decisions per hour) and is vital in managing review. Faster is not always better, but the review as a whole usually has to move at a certain pace for deadlines to be met.

Reviewer Accuracy. Used to provide constructive feedback to reviewers when they make errors with respect to classifying (or “tagging”) documents. Obtained by using QC processes.

Production Rate (or Conversion Rate). Unless you’re really under the gun, this metric shouldn’t be a vital one for meeting deadlines, but it is important to know what a system’s throughput rate is for producing natives to tiffs and generating deliverables for exchange.

Production Accuracy. Not really a quantifiable measure, but should provide a general sense of how well production specs were followed and whether the material produced was what was requested by counsel to be produced.

This isn’t the full spectrum. There are a number of others available, but I think these provide a nice foundation for measuring effectiveness.

Posted in Best Practices, Project Management | Tagged: | Leave a Comment »

Electronic Discovery Blog talks Project Managment

Posted by rjbiii on July 28, 2009

The Electronic Discovery Blog posts a brief note on the importance of Project Management in the EDD space. From the blog:

Project management is likewise the key to success in e-discovery. Project management facilitates the coordination of the many parties involved, promotes communication between those parties, allows a record of the process to be created, and promotes discipline throughout the process. Project management fosters the creation of a repeatable process, and is an essential requirement for a successful e-discovery outcome.

Posted in Blog Posts, Project Management | Leave a Comment »