This report explores an evaluation framework to try to assess whether online legal aid tools and resources make a different in addressing gaps in access to justice. The researchers used the "Drake Equation", originally created in 1951 to list the factors that scientists should consider when determining whether there was extraterrestrial life. Through a series of interviews, a literature review, and an in-person session with access to justice experts, the researchers collected metrics and prepared a draft evaluation framework.
An important factor noted was "penetration" of online resources - the number of people using the resources divided by the number of people in the target audience. Elements of the framework, according to the researchers, are:
- "Targeted (T): People that the resource would ideally serve in the geography and legal topic covered
- Accessible: The percentage of T that are able to use the existing resources— for instance based on literacy, language or technology
- Found: The percent that find the resources—for instance, by being aware of the site, via a Google search or through a referral from a community partner
- Used: The percent that interact with the resources in some more substantive way—for instance, by navigating to the end, printing information or assembling a form
- Enabled Action: The percentage of people for whom the resource enabled some meaningful next step in the real world—for instance, filing a form, creating a referral or a decision by the constituent that it’s not worth their time to act
- Achieved Outcome: The percent that reach an outcome – which could be defined in many ways"
The researchers further identify seven different types of outcome measures:
- "Perception of Fairness: Do they help users feel that the process to get to an outcome was fair and they are able to have their say—as per a “procedural justice” lens?
- Education and Empowerment: Are users more able to help themselves through the process of this legal situation—and for future situations?
- Solution without Official Process: Is their problem solved without use of courts or lawyers (perhaps by choosing to do nothing or through informal methods)?
- Legal Outcomes: Are fewer cases defaulted because of the resources? Do the users receive better settlements that those who didn’t use the resources?
- Just Outcomes: Was the user able to get what a typical lawyer would call justice in the situation—or some other definition of what “justice” would mean in an ideal world?
- Life Outcomes: Did the resources reduce stress or provide a more stable family situation?
- Process Efficiency: Do the resources create time or cost savings for organizations within the legal aid ecosystem?"
The researchers hope to get feedback on the draft framework, and input on how it could be implemented and used.