Skip to main content

Policy Pulse: Baseline Study of Internet Connectivity for Last-Mile Schools in the COVID-19 Project: Quality Assurance and Data Management Processes

Marlene B. Ferido, PhD

Background and Context

Amidst the COVID-19 pandemic in 2021, the Asian Development Bank (ADB) committed to support the Department of Education (DepEd) through the EdTech Solutions for Last Mile Schools in COVID-19 initiative. This project provided critical funding to remote, underserved schools across four provinces in the Philippines.

To evaluate the program’s impact and inform future basic education policies, the ADB designed and funded a rigorous evaluation study. The research compared learning outcomes between 21 designated beneficiary schools and a control group of 21 non-beneficiary schools.

    • Baseline Survey (Early 2022): Conducted across the 42 participating schools, the baseline assessment gathered data from over 1,000 students (Grades 7–10). The instrument consisted primarily of a grade-specific aptitude test in Mathematics and English, supplemented by a questionnaire covering family background, home learning environments, learning attitudes, and study habits. To ensure ethical compliance, all data were strictly anonymized, utilizing only student Learner Reference Numbers (LRNs).
    • Follow-Up Survey (May 2024): Two years post-intervention, a follow-up survey was implemented. Because two of the original grade cohorts had already graduated from Junior High School, this timeline offered a critical final opportunity to evaluate the remaining two cohorts before another grade level would complete Grade 10.
    • Expanded Intervention: Additionally, the ADB extended supplementary support to these remote institutions that included enhanced internet connectivity via Starlink, digital modules and learning resources for teachers, and specialized teacher training. This expanded program was evaluated using the data collected during the May 2024 survey.

 Scope of Deliverables

This project encompassed the following deliverable-based Milestones:

    • Data Integration: Merging the 2022 baseline data with the July 2024 response data, alongside comprehensive documentation of the corresponding student matching success rates.
    • Instrument Development: Designing and finalizing a student questionnaire covering core areas such as basic demographics, study habits, non-cognitive skills, internet usage, and self-efficacy, alongside cognitive assessments in Mathematics and English for Grades 7–10.
    • Reporting: Submitting a consolidated report in November 2024 that detailed the student questionnaires, non-cognitive instruments, and cognitive assessments.

Summary of Processes

As a core component of the November 2024 report, this documentation outlines two essential processes implemented to safeguard the quality, security, and integrity of the study’s data:

    1. Quality Assurance Process: Established rigid protocols to ensure the security and administration integrity of assessments across the 42 target last-mile schools in the provinces of Abra, Bohol, Kalinga, and Zamboanga Sibugay.
    2. Data Management Process: Provided a structured framework for data cleaning and verification, ensuring that accurate test results were successfully generated from 3,106 student responses.

The Quality Assurance Process for Test Administration and Data Collection

The quality assurance process covered the full lifecycle of test administration and data collection. Key phases included material preparation, formatting, printing, packing, and distribution, alongside personnel orientation, final test administration, and the retrieval of materials from division offices.

    1. Pilot Testing and Test Revision

A pilot test was administered in a public high school in the Philippines to evaluate the cognitive assessments (Mathematics and English), the Supplemental Background Questionnaire (SBQ), and the Non-Cognitive Test. The results were used to identify and revise poorly performing items, ensuring the reliability and validity of the final instruments before large-scale administration.

    1. Formatting and Layout of Test Materials

All test materials were formatted to ensure consistency, clarity, and professional presentation. Each test booklet was assigned a unique control number to facilitate tracking and verification, and color-coded by subject to streamline distribution and collection.

    1. Printing

Test materials were produced under strict confidentiality protocols. Quality checks were performed throughout production to ensure precise alignment and adherence to uniform printing specifications.

    1. Packaging and Sending of Test Materials

Test materials were systematically packed by grade level, labeled in detail, and verified against inventory lists. Secure transport tracking was maintained during dispatch to the Schools Division Offices (SDOs) to guarantee accuracy, completeness, and security.

    1. Schools Division Office (SDO) Focal Person Orientation

An online orientation was conducted on January 3, 2025, for focal persons from the four participating SDOs. The session clarified their roles and responsibilities in overseeing test administration, managing logistics, maintaining security protocols, and supporting Test Administrators (TAs) in the field.

    1. Test Administrator (TA) Orientation

A series of online orientations was conducted for TAs across all participating divisions. These sessions equipped TAs with standardized administration procedures through a walkthrough of the assessment process, a review of the Test Administration Manual, and a question-and-answer segment. Supplementary reference materials were also provided for further preparation.

    1. Final Test Administration

Upon receiving the materials, SDO focal persons coordinated with schools to schedule and oversee the assessment. Each school completed testing within a designated two-day window, with TAs strictly adhering to established protocols to ensure standardized conditions across all sites.

    1. Retrieval of Materials from the Division Offices

The secure and complete return of all test materials from SDOs to ACTRC was managed through close coordination and constant communication with SDO focal persons. A trusted courier service was utilized, enabling real-time online monitoring via unique tracking numbers to ensure timely and accurate delivery.

Data Management in Generating Test Scores

This phase covers the end-to-end processing of assessment materials—from the initial to the final data generation that includes: review and sorting, scanning and digitization, Optical Mark Recognition (OMR), data cleaning, and psychometric scoring.

The primary objective of this phase is to ensure the absolute accuracy, reliability, and security of the final test scores. The operational workflow is structured into five key steps:

    1. Initial Review and Sorting of Assessment Materials

Upon receiving the physical test materials, ACTRC conducted a systematic verification and sorting process to guarantee complete data tracking. This involved:

    • Reviewing administrative documentation, including consent forms, vouchers, and Post-Administration Forms.
    • Verifying and cataloging student answer sheets.
    • Isolating test booklets for secure shredding in compliance with established data protection policies.
    • Utilizing a centralized online tracker for real-time monitoring of completeness and progress.
    1. Scanning and Digitization Process

To secure a permanent digital record and facilitate automated processing, School Consent Forms, Post-Administration Forms, and student answer sheets were systematically prepared for digitization.

    • Purpose: This step ensures precise data capture, electronic redundancy, and robust documentation.
    • Audit Trail: Post-Administration Forms were specifically used to document test material tracking, testing observations, attendance, and suggestions for improvement. These steps ensure accurate data capture, entry, and proper documentation.
    1. Optical Mark Recognition (OMR) and Data Cleaning

Digital answer sheets were processed using Remark Office OMR software to convert scanned student bubbles into raw digital data. To ensure data integrity, a rigorous cleaning protocol followed:

    • Error Identification: Automated scans were audited to detect and flag anomalies, such as multiple responses to single-choice items or blank fields.
    • LRN Verification: Learner Reference Numbers (LRNs) underwent manual review and rigorous cross-checking against physical classroom attendance records to eliminate student identification errors.
    • Outcome: This intensive validation pipeline yielded verified, well-organized, and clean datasets optimized for subsequent statistical analysis.
    1. Item Scoring and Statistical Analysis

Cleaned datasets were subjected to psychometric evaluation using statistical software to ensure instrument validity and consistency:

    • Item Analysis: Conducted to evaluate the performance of individual test questions, flag potentially problematic items, and empirically verify the accuracy of response answer keys.
    • Data Correction: Flagged cognitive test items were thoroughly reviewed and adjusted within the analysis software to maximize overall test reliability.
    • Non-Cognitive Coding: Responses from non-cognitive instruments were cleaned, appropriately recoded, and compiled for final reporting.
    1. Final Deliverables and Generated Files

The data management phase concluded with the generation of two foundational project outputs:

    • Comprehensive Codebook: A technical guide defining variable names, values, and scoring structures to facilitate seamless data interpretation.
    • Final Spreadsheets: Fully consolidated, clean datasets containing integrated cognitive scores, non-cognitive indices, and Student Background Questionnaire (SBQ) data.

Conclusion

The systematic quality assurance and data management processes documented in this report are more than procedural necessities; they are the foundation of credible evidence. By establishing absolute integrity at every stage, from secure field administration in last-mile schools to rigorous digital cleaning, these protocols ensure that raw data is successfully transformed into valid, uncompromised evaluations. Ultimately, these dual processes serve as an essential blueprint for large-scale assessments, transforming raw responses into defensible insights that could drive systemic educational reform and inform high-stakes policy decisions.