Assessment of legal aid quality in Vietnam

          According to research, the quality of LEGAL AID cases is demonstrated through the following basic characteristics:
          + Legal basis: The implementation of LEGAL AID must fully comply with the provisions of the law on LEGAL AID and relevant legal provisions.
          + Service provider: the person performing legal aid must meet certain conditions and standards; must have legal knowledge, professional skills and ethical qualities as prescribed by law and be clearly aware of his/her profession when performing legal aid.
          + Providing legal services meets the legitimate needs of people receiving legal aid or as required by justice according to the provisions of law, in accordance with the psychological characteristics of gender, age, and occupation of people receiving legal aid (women, children, the poor, ethnic minorities, etc.). These are the desires formed from the legitimate rights and interests of people receiving legal aid that are protected by law and contribute to ensuring human rights and civil rights in access to justice and equality before the law.
          + The implementation of legal aid must meet the objectives of legal aid activities, helping people who receive legal aid protect their legitimate rights and interests, improve their understanding of the law, and their awareness of respecting and complying with the law. In addition, personal interests, community interests and social interests are unified and harmonious, so through the use of legal services, it will actively contribute to raising legal awareness, protecting justice, ensuring social equity, preventing and limiting disputes and violations of the law among the people.
          Factors affecting the quality of LEGAL AID
          The professional capacity of the LEGAL AID practitioner is the most important factor determining the quality of the LEGAL AID case; including the qualifications, professional skills and professional ethics of the LEGAL AID practitioner.
          a) Capacity of the person performing the legal aid: including the qualifications, professional capacity, and legal aid skills of the person performing the legal aid directly determines the quality of the legal aid case.
          b) Professional ethics: To provide good quality legal aid services, it is very important to behave properly, independently, honestly, and best protect the rights and interests of the person receiving legal aid... as stipulated in the Code of Ethics for legal aid practitioners.
          c) The level of awareness, psychology, and cultural traditions of people eligible for legal aid: Free legal services are intended to provide legal knowledge and use legal measures to protect the legitimate rights and interests of those eligible for legal aid. Therefore, the level of understanding and acquisition of legal knowledge as well as the ability to coordinate and support legal aid practitioners to help people protect their legitimate rights and interests depends largely on the awareness, psychology, and cultural traditions of people eligible for legal aid.
          Assessment of the quality of LEGAL AID cases
A completed LEGAL AID case is considered a product of the LEGAL AID implementation process of the LEGAL AID implementer. However, the quality of the case itself cannot always be affirmed. Therefore, to evaluate the quality of the case, there must be a quality standard used as a tool to evaluate and form a mechanism to evaluate the quality of the LEGAL AID case.
          Characteristics of LEGAL AID case quality assessment activities :
          - Is the activity of competent agencies, organizations and individuals;
          - The subjects of the assessment activities are completed LEGAL AID cases;
          - Assessment activities must comply with the procedures prescribed by law;
          - Evaluation activities must be based on certain evaluation criteria as prescribed by law.
          Subject of assessment of the quality of LEGAL AID cases
          Quality assessment agency: According to Circular No. 12/2018/TT-BTP, the competent authority to assess the quality of LEGAL AID cases includes:
          + Department of Legal Aid – Ministry of Justice
          + Department of Justice of centrally-run provinces and cities.
          - Person performing quality assessment: Circular No. 12/2018/TT-BTP does not specifically stipulate the standards and conditions for the person performing the case quality assessment, but assigns the right to choose the person performing the case quality assessment to the competent authority to decide.
          Scope of assessment
          - The Department of Legal Aid under the Ministry of Justice, as a central state management agency, has the authority to assess the quality of cases of organizations implementing legal aid (State Legal Aid Centers, organizations participating in legal aid) nationwide.
          - The Department of Justice under the People's Committee of a province or centrally-run city, as a local state management agency, has the authority to assess the quality of cases of legal aid-implementing organizations (State Legal Aid Centers, participating organizations) in its locality.
          When conducting quality assessment of legal aid cases, the scope of assessment must be clearly defined in the Quality Assessment Plan of legal aid cases according to the requirements of state management of legal aid at the central and local levels.
          Case types and case selection for quality assessment
          a) Type of quality assessment case
          Compared with the regulations of the Set of Standards for assessing the quality of legal aid cases issued together with Circular No. 02/2013/TT-BTP dated January 5, 2013 of the Minister of Justice, the regulations on the types of cases subject to quality assessment of Circular No. 12/2018/TT-BTP are narrower. Accordingly, there are only 02 types of cases subject to quality assessment: (1) Cases participating in litigation; (2) Cases representing outside litigation.
          - Select cases for quality assessment
          - Regular assessment: Cases may be selected either in their entirety or probabilistically for assessment according to the following criteria:
          + Complex and typical cases according to the provisions of law;
          + Other cases as required by management.
          - Ad hoc assessment: The following factors may be considered in case selection:
          + The case has recommendations or complaints from the person receiving legal aid or reflections from agencies, organizations, individuals, and mass media about the quality of the legal aid case or there is other basis to believe that the implementation of legal aid has not ensured quality according to the provisions of law;
          + The case has a request for compensation for damages from the person receiving the legal aid or their legal representative related to the legal aid case.
          How to evaluate quality
          When conducting a quality assessment of a legal aid case, the head of the legal aid case quality assessment team is responsible for announcing the decision to assess the quality of the legal aid case for the implementing organization subject to quality assessment. (The members of the assessment team are decided by the Legal Aid Department or the Director of the Department of Justice, and may include officers of the Legal Aid Department or the Department of Justice, legal aid practitioners, experienced lawyers, judges, investigators, prosecutors, legal experts, etc.). The assessment can be carried out as follows:
          - The head of the organization implementing the LEGAL AID and the LEGAL AID implementer whose case is being evaluated must quickly report on the contents as requested by the Delegation.
          - Review the report on the results of LEGAL AID activities, the report on the results of the quality assessment of LEGAL AID cases of the LEGAL AID implementing organization. It should be noted that the quality assessment is independent and does not depend on the results of the quality assessment conducted by the LEGAL AID implementing organization;
          - Directly check, research and review LEGAL AID case files.
          - Study the feedback, recommendations or opinions of the beneficiaries or their legal representatives (if any).
          - Consider feedback, recommendations or reflections from relevant agencies, organizations, individuals or the media (if any).
          - Contact and work to listen to the opinions and feedback of the beneficiaries of legal aid and relevant agencies, organizations and individuals on the quality of legal aid cases.
          - Consult with legal experts, lawyers, experienced legal assistants, litigators, etc.
          During the assessment process, the Head of the Delegation or a member of the Delegation may request responsible persons or relevant persons to provide additional information on unclear issues.
          In case of necessity, to check, verify information, documents or clarify issues related to the content of the case, request for legal assistance or compliance with legal regulations, ethics, skills, knowledge of the person performing legal assistance, the Head of the assessment team may decide to work with relevant agencies, organizations and individuals. The results of checking and verifying relevant information and documents must be presented in writing.
          - When evaluating the quality of the LEGAL AID case, the members of the delegation will evaluate and score according to the evaluation criteria to record on the scorecard.
          Criteria for assessing the quality of LEGAL AID cases
          According to Circular 12/2018/TT-BTP, quality assessment is based on the following criteria:
          Firstly, regarding the professional responsibility of the person performing legal aid, the professional responsibility of the person performing legal aid is one of the important factors that significantly affects the quality of the legal aid case. Therefore, it requires the person performing legal aid to comply with the law, professional ethics, be independent, respect objective truth, ensure the deadline, statute of limitations, and progress of work implementation; if necessary, must explain, guide, and provide timely information about the case to the person receiving legal aid to help the person receiving legal aid protect their legitimate rights and interests in a timely and effective manner.
          Second, regarding the implementation of legal aid cases by legal aid providers, legal aid activities are a type of free legal service, performed based on the request of the person receiving legal aid and the provisions of law. Therefore, legal aid providers perform legal aid based on the request for legal aid and the assigned scope. In addition, when performing legal aid for a specific case, to ensure the quality of the case, it is required that the legal aid provider must ensure comprehensiveness, timeliness, and legality through a number of activities such as: Collecting information, documents, and evidence related to the case; Researching and fully applying relevant legal provisions to ensure that the content of legal aid is correct and in accordance with the law; Participating in litigation activities or participating in non-litigation representation activities to promptly find solutions to protect the rights and legitimate interests of the person receiving legal aid, etc.
          Third, regarding the satisfaction of the person receiving legal aid, the satisfaction of the person receiving legal aid is first of all expressed through the assessment of the attitude, method, working style, etc. of the person performing legal aid. Therefore, when performing legal aid, the person performing legal aid must have the spirit, attitude, working style, and method of carrying out work... appropriate and proper to help the person receiving legal aid clearly understand the benefits that the person performing legal aid and the organization performing legal aid bring to them, at the same time help them perceive and understand the provisions of the law or bring material benefits from the settlement of the competent authority with the legal aid of the person performing legal aid, etc.
In addition to clearly defining the criteria for assessing the quality of LEGAL AID cases, Circular 12/2018/TT-BTP also stipulates the scoring structure (Article 16), specifically as follows:
          * Criteria on professional responsibility of the person performing LEGAL AID (30 points)

• Comply with the law, be honest, respect objective truth (10 points);
• Explain, guide, and provide timely information about the case to the person receiving legal aid (10 points);
• Ensure time and progress of case implementation (10 points).

          * Criteria for implementing LEGAL AID (60 points)

• Implement LEGAL AID in accordance with LEGAL AID requirements and assigned scope (05 points);
• Collect information, documents and evidence related to the case (10 points);
• Research and fully apply relevant legal regulations to ensure that the content of the legal aid is correct and in accordance with the law (20 points);
• Participate in litigation activities or participate in non-litigation representative activities to promptly find solutions to protect the rights and legitimate interests of the person receiving legal aid (20 points);
• The process of implementing LEGAL AID is fully demonstrated in the LEGAL AID case file (05 points).

          * Criteria for satisfaction of the beneficiary based on feedback from the beneficiary or relatives (10 points).
          Accordingly, this Circular also specifically regulates the classification of the quality of LEGAL AID cases as follows:

• Good quality case: Case with total score of 90 points or more;
• Fair quality case: Case with total score from 70 to under 90 points;
• Qualified case: Case with total score from 50 to under 70 points;
• Unqualified case: A case with a total score of less than 50 points or a case in which the person performing the legal aid or the organization performing the legal aid violates the prohibitions specified in Clause 1, Article 6 of the Law on Legal Aid.

          V. Comments and evaluations
          5.1. Evaluation methods, criteria and procedures
          Each LEGAL AID case has different contents and characteristics, so the evaluation system may include a combination of the following methods:

• Checklists indicate whether essential work for a case has been completed. For example, a checklist might include when a LEGAL AID beneficiary is interviewed, bail is applied for in a criminal case, evidence is collected in a civil case, alternative dispute resolution options are considered in a civil or administrative case, etc.;
• Data in the case management system (CMS) and other systems. The CMS will include important information such as the time the case was initiated, confirmation of receipt of court documents, relevant government regulations in administrative proceedings, adjournments, order and procedures for the case, etc. The checklist mentioned above can also be linked to the CMS so that the information in the checklist can be combined with other information in the CMS.
• LEGAL AID Practitioner Supervisor Report: The supervisor is often in the best position to assess whether a LEGAL AID case is being handled professionally. The supervisor should be in regular contact with the LEGAL AID practitioner, evaluate the work from the beginning of the case, and be able to comment on whether the practitioner has taken all required steps in a timely manner.
• Complaints: According to the United Nations Global Study on Complaint Mechanisms, legal aid is the most common tool used to ensure that legal aid services in a case are of appropriate quality. In many jurisdictions there is no systematic method for evaluating each case and the analysis is based on complaints filed by the legal aid recipient, other lawyers, court officials, judicial authorities or others.
• Information from the LEGAL AID beneficiary: The evaluation of a LEGAL AID case usually involves an interview or survey of the LEGAL AID beneficiary after the case has been completed. Although the LEGAL AID beneficiary may not be competent to make an informed assessment of the LEGAL AID work, it is an important sign of the LEGAL AID beneficiary’s confidence in the LEGAL AID system.
• Evaluation by other practitioners (peer evaluation): In some typical legal aid systems such as in the UK, experienced lawyers may be paid to observe court and review files etc. to evaluate the performance of a legal aid practitioner in a case. While this can be an effective method, it is expensive and care must be taken to ensure that those doing the evaluation have sufficient experience to be able to make a useful evaluation of another lawyer's work.
• Assessments by stakeholders, such as NGOs and judges. Since judges are likely to not give a favourable assessment to a lawyer who fights aggressively for their client (e.g. if it makes the case take longer or requires more work on the judge's part), and NGOs may not be fully aware of the case or issues, these assessments need to be carefully considered.
• Practitioner self-assessment. This tool may be unreliable because LEGAL AID practitioners may judge themselves to have provided better service than they actually did. However, this tool can be useful if combined with the assessment of the LEGAL AID practitioner's supervisor.

          5.2. Checklist
          A checklist can document the relevant steps taken by a legal practitioner in a case. Checklists can be designed as hard copies or online tools.
          Hard copy checklist : pros and cons
          If the checklists are created in hard copy form, they can be easily used by all LEGAL AID practitioners in different types of cases and regardless of whether they have access to smartphones, computers or internet services. However, LEGAL AID practitioners handle a large number of cases and all such hard copy documents need to be filed and stored securely.
          Electronic Checklists : Pros and Cons
          Electronic checklists or online tools enable the direct editing of checklists in real time over the Internet without the need to be filed and stored securely in an office or warehouse. However, checklists contain a lot of sensitive information, so digital security is an important issue. Electronic checklists are much more accessible to all those who may need the information, including the assessor. However, electronic tools require training and computer skills that need to be upgraded continuously and internet quality to implement, especially for remote areas.
          Below is an example of a survey of LEGAL AID beneficiaries' satisfaction with LEGAL AID work in criminal, civil and administrative cases used in Ontario, Canada:
 
 

Common Assessment Tools – Overview
Evaluation criteria	Evaluation factors	Level
Timeliness	Satisfied with the time it took to perform the service	3,5,7
Ease of access	Easily identify what the LEGAL AID recipient needs Easy to follow process If there is a problem, the person receiving the LEGAL AID is clearly instructed what to do. Easy to understand instructions	2,3,4,7
Employee Contributions	Staff know and understand the wishes of the person receiving LEGAL AID Staff try to solve problems beyond expectations The staff tried to solve the problem at all costs. Employees show dedication to work Highly skilled staff	3,4,6
Result	The person who received LEGAL AID achieved his wish. People receiving LEGAL AID are provided with clear and accurate information. People with disabilities are treated fairly.	3,5,6

 
          5.3. System assessment tools
          When evaluating a LEGAL AID system, the main task is to determine general indicators of LEGAL AID quality. Depending on the data source, they can be divided into objective and subjective indicators.
          Objective quality index
          Objective quality indicators are based on factual information about a specific activity, such as statistical data, official information from competent authorities, case law, etc. Their purpose is to assess the system's compliance with legal, financial, administrative requirements and standards. To establish objective indicators of the quality of the system, it is possible to assess:

• Legal aspects of LEGAL AID, i.e. compliance with national legal provisions and the application of LEGAL AID, international and EU standards, constitutional principles, national law;
• Financial and administrative efficiency of the LEGAL AID system (e.g., assessment of the efficiency of the use of funds and other resources in compliance with financial and administrative management principles and rules, etc.);
• Compliance of the LEGAL AID regulator and/or LEGAL AID practitioners with the requirements of legality, ethics, quality (etc.), good management principles, etc. prescribed by law;
• System compliance with good practice standards.

          Below we discuss assessment tools that aim to establish objective indicators of the quality of the LEGAL AID system:
          - Data analysis of the LEGAL AID system
          - General assessment of the quality of work of the LEGAL AID implementer
          - Synthesize and analyze official documents prepared by lawyers
          - Quantitative analysis of successful cases
          - Complaint analysis
          - Monitoring by non-governmental organizations
          - Evaluate the system according to good practice standards
          Subjective quality indicators
          The source of subjective quality indicators is the opinion of the legal aid recipients or the professionals (judges, prosecutors, police, etc.) who are exposed to legal aid in their daily work. The assessment can be done quantitatively, for example through representative surveys, or through qualitative methods such as interviews, focus groups; Alternatively, through the observation of meetings, where individual lawyers are not observed but the general data is analyzed and presented in a quantitative and personalized way.
          - Survey of people receiving LEGAL AID
          - Survey of legal professionals (e.g., judges, prosecutors, pre-trial investigators)
          - Lawyer survey
          5.4. Personal assessment of the person performing the LEGAL AID
          In contrast to systematic evaluations which collect and analyze aggregate data that point to certain general trends, individual evaluations aim to assess the quality of public services by analyzing individual cases of public services providers. Thus, compared to systematic evaluations, individual evaluations are specific. They highlight aspects of the quality of a particular public service provider and help identify specific and individual measures to improve quality. For example, this form of evaluation helps determine whether the services provided are of adequate quality in terms of representation and/or whether public resources are used effectively.
          In practice in different countries, the following forms of individual assessment can be distinguished:
          • Observe the work of the LEGAL AID performer;
          • Analyze legal documents drafted by legal practitioners;
          • Self-assessment of the work of the LEGAL AID implementer;
          • Audit the work of the LEGAL AID implementer.