This issue focuses on the challenge of managing chronic pain. Commentaries present various treatment options, including opioids, nonopioid medications, surgery, and alternative therapies. Authors also address the problems of opioid misuse and abuse and discuss ways to lessen these risks. Original articles examine health professionals’ communication with adolescents about smoking, as well as children's immunization status as verified by practice records and by the North Carolina Immunization Registry.
Verifying Influenza and Pneumococcal Immunization Status of Children in 2009–2010 from Primary Care Practice Records and from the North Carolina Immunization Registry
Katherine A. Poehling, Lauren Vannoy, Timothy R. Peters
N C Med J. 2013;74(3):185-191.PDF | TABLE OF CONTENTS
Background The North Carolina Immunization Registry (NCIR) has been available since 2004. We sought to measure its utilization among practices that provide primary care for children who are enrolled in a prospective influenza surveillance study.
Methods This study included children aged 0.5–17 years who presented with fever or acute respiratory symptoms to an emergency department or inpatient setting in Winston-Salem, North Carolina, from September 1, 2009, through May 19, 2010. Study team members verified influenza and pneumococcal immunization status by requesting records from each child’s primary care practice and by independently reviewing the NCIR. We assessed agreement of nonregistry immunization medical records with NCIR data using the kappa statistic.
Results Fifty-six practices confirmed the immunization status of 292 study-enrolled children. For most children (238/292, 82%), practices verified the child’s immunizations by providing a copy of the NCIR record. For 54 children whose practices verified their immunizations by providing practice records alone, agreement with the NCIR by the kappa statistic was 0.6–0.7 for seasonal and monovalent H1N1 influenza vaccines and 0.8–0.9 for pneumococcal conjugate and polysaccharide vaccines. A total of 221 (98%) of 226 enrolled children younger than 6 years of age had 2 or more immunizations documented in the NCIR.
Limitations NCIR usage may vary in other regions of North Carolina.
Conclusion More than 95% of children younger than 6 years of age had 2 or more immunizations documented in the NCIR; thus, the Centers for Disease Control and Prevention 2010 goal for immunization information systems was met in this population. We found substantial agreement between practice records and the NCIR for influenza and pneumococcal immunizations in children.
Childhood immunization schedules have expanded over the past decade for children of all ages, including young children and adolescents [1, 2]. Children frequently obtain vaccines from multiple sources. Many children receive their first dose of hepatitis B vaccine during the birth hospitalization [3, 4], and some children receive immunizations from multiple providers . To minimize the dispersal of immunization records, the Centers for Disease Control and Prevention (CDC) has recommended the use of state-based immunization registries . North Carolina modified the Wisconsin Immunization Registry to develop the North Carolina Immunization Registry (NCIR), a secure, population-based, Web-based clinical tool that was implemented in 2004 . This study measured both the extent to which providers used the NCIR in 2009–2010 and the agreement between practice-based records and registry data for influenza and pneumococcal immunizations among children.
An influenza surveillance study prospectively enrolled children who presented with acute respiratory illness or fever to an emergency department or inpatient setting in 1 of 2 hospitals in Winston-Salem, North Carolina—including the region’s only children’s hospital—from September 1, 2009, through May 19, 2010. Eligible children resided in Forsyth County or 1 of 7 contiguous counties in North Carolina. After informed consent was obtained from a parent or guardian (along with child assent, when appropriate), children were enrolled in the study and permission was obtained to verify their influenza and pneumococcal immunization history by contacting their primary care practice and by reviewing the NCIR. (This study is distinct from our 2012 study that compared parental reports for the 2009–2010 seasonal influenza vaccine and the H1N1 vaccine to confirmation of vaccination status using either the NCIR or practice reports .)
For the current study, a facsimile was sent to the parent-identified primary care practice in the spring and summer of 2010; this facsimile requested verification of the influenza and pneumococcal immunization status for each child enrolled in the study. Influenza and pneumococcal immunization status were independently verified in the NCIR.
Study population. The study population comprised all children who were prospectively enrolled, had immunizations entered into the NCIR, and had their influenza and pneumocccal immunization status verified by a practice.
Influenza immunization status. In 2009–2010, influenza vaccine recommendations for the seasonal influenza vaccine differed from recommendations for the H1N1 monovalent influenza vaccine, particularly for children 9 years of age, for whom the recommendation was that they receive 2 doses of the latter but only 1 dose of the former [9-11]. The definitions of fully immunized, partially immunized, and not immunized for each vaccine are shown in Table 1.
Agreement between the practice-based records and registry-based records. Many practices responded to our request for immunization verification by providing a copy of the NCIR record. The expected agreement between the practice copy of the NCIR and our review of this registry should approximate 100%. Hence, we limited the assessment of agreement to children whose immunization status was verified using practice records alone. Because the influenza vaccine was recommended for children 6 months of age or older, the study population consisted of children 0.5–17 years of age. For these children, we extracted information from the practice-based record regarding doses of seasonal and H1N1 monovalent influenza vaccines in 2009–2010 and doses of pneumococcal conjugate vaccine (7-valent or 13-valent) or any pneumococcal polysaccharide vaccine, and we compared this information with the NCIR record. Because the 23-valent pneumococcal polysaccharide vaccine is recommended only for children 2 years of age or older who have medical conditions predisposing them to pneumococcal disease , we limited assessment of that vaccine to children 2–17 years of age. For each vaccine, we compared the number of doses listed in the practice-based records with the number of doses recorded in the NCIR to compute the percent agreement, expected percent agreement, and a simple (not weighted) kappa statistic (κ) with its P value. κ is a measure of inter-rater agreement that accounts for the likelihood that the observed agreement could occur by chance; this value can range from –1 (perfect disagreement beyond chance) to +1 (perfect agreement beyond chance). According to the categorization scheme of Landis and Koch , a κ value of 0.81–1.00 indicates almost perfect agreement, a κ value of 0.61–0.80 indicates substantial agreement, and a κ value of 0.41–0.60 indicates moderate agreement. For each vaccine, we also computed the sensitivity, specificity, positive predictive value, and negative predictive value for each child being classified as immunized or not immunized when the NCIR record was compared to practice-based records. Exact 95% confidence intervals were computed using the binomial distribution. All analyses were performed using the statistical package STATA 8.1 (College Station, Texas).
This study was approved by the Wake Forest School of Medicine Institutional Review Board with written parental consent and child assent when appropriate, and by an authorization agreement between the institutional review boards of Forsyth Medical Center and Wake Forest School of Medicine.
Of the 334 children enrolled from September 1, 2009, through May 19, 2010, the study population consisted of the 292 children (87%) who had influenza and pneumococcal immunizations verified by both the NCIR and practice records (Figure 1). More than three-quarters of study children were younger than 9 years of age, approximately half were male, half were black, and three-quarters of them resided in Forsyth County (Table 2). More parents reported that their child obtained care from a pediatric practice (76%) than from a family medicine practice (19%) or a health department (6%).
For 238 (82%) of the study children, the practice verified the child’s immunizations by providing a copy of the NCIR record. Compared to this group, children whose immunizations were verified with practice-based records were younger, less likely to be non-Hispanic white, more likely to reside in a county surrounding Forsyth County, and more likely to obtain care at a family medicine practice (Table 2). Of the 226 children younger than 6 years of age, 221 of them (98%) had 2 or more immunizations documented in the NCIR.
Parents reported that the 292 children in the study received care from 1 of 56 health care facilities, of which 30 (54%) were pediatric practices, 21 (34%) were family medicine practices, and 5 (11%) were health departments. The mean number of enrolled children per health care facility was 8 for pediatric practices (range, 1–60), 3 for family medicine practices (range, 1–12), and 2 for health departments (range, 1–4).
In this study, primary care practices administered the majority of seasonal and monovalent H1N1 influenza vaccine doses in 2009–2010. Among 182 verified doses of seasonal influenza vaccine, 85% were administered in their primary care practice, 10% were administered in a health department, 3% were administered in a practice other than their primary practice, and 2% were administered in a school or wellness center. Among 118 verified doses of monovalent H1N1 influenza vaccine, 75% were administered in their primary care practice, 17% were administered in a health department, 6% were administered in a school or wellness center, and 3% were administered in a practice other than their primary practice. In 2009–2010 the NCIR confirmed 172 (94.5%) of 182 verified doses of seasonal influenza vaccine and 115 (97%) of 118 verified doses of monovalent H1N1 influenza vaccine.
For 54 children whose immunization status was verified with practice records, we ascertained the level of agreement between the practice records and the NCIR. The kappa statistics for the 2009–2010 seasonal influenza vaccine and the H1N1 influenza vaccine ( κ = 0.63 and κ = 0.71, respectively; Table 3) were lower than those for pneumococcal conjugate and 23-valent pneumococcal polysaccharide vaccines ( κ = 0.92 and κ = 0.84, respectively; Table 3).
There were different reasons for the discrepancy between the practice report and the NCIR. For influenza vaccines, 2 different practices reported 1 dose of seasonal influenza vaccine and 1 dose of H1N1 monovalent influenza vaccine in the practice records but not in the NCIR, which negatively impacted the sensitivity and negative predictive value of the influenza immunization status in the registry (Table 4). Conversely, for the pneumococcal vaccine, 1 child had a pneumococcal conjugate vaccine and another child had a pneumococcal polysaccharide vaccine reported by the practice to the NCIR (per our review of the NCIR), but those vaccines were not recorded in the practice-provided verification. This discrepancy negatively impacted the specificity and positive predictive value of the pneumococcal immunization status in the registry.
This study demonstrates that the NCIR was commonly used in 2009–2010 by practices in Forsyth County and its 7 contiguous counties in North Carolina. Primary care practices provided immunization verification in the form of the NCIR record for most (82%) of the children enrolled in this study. Further, 221 (98%) of the 226 enrolled children younger than 6 years of age had 2 or more immunizations recorded in the NCIR. Thus, in this study population the NCIR achieved the 2010 CDC goal for immunization information systems, which specifies that at least 95% of children younger than 6 years of age should have 2 or more immunizations recorded in such a system .
For children whose immunization status was verified with practice records, we found substantial agreement between the practice records and the NCIR for seasonal and monovalent H1N1 influenza vaccines ( κ≥0.63) and almost perfect agreement for pneumococcal conjugate and polysaccharide vaccines ( κ≥0.84), per the categorization scheme of Landis and Koch . Hence there was high concordance for influenza and pneumococcal immunizations between the practice records and the NCIR.
We believe that the practices included in this study are representative of the practices serving Forsyth County and its 7 contiguous counties in North Carolina. Among enrolled children, 76% received primary care at a pediatric practice. An analysis of 2004–2007 data from the National Ambulatory Medical Care Survey reported similar findings, with general pediatricians performing 77% of all visits to primary care physicians among children 0–17 years of age .
Our results are comparable to those of previous reports for other immunization registries in the United States. We found that within 6 years of the North Carolina registry being implemented, 98% of children 0.5–5 years of age had 2 or more immunizations documented in the NCIR. In comparison, 92% of children aged 19–35 months were included in the KIDS Immunization Registry in Philadelphia, Pennsylvania , and 91% of children younger than 5 years of age were included in the Arizona State Immunization Information System . We also found that 94.5% of verified doses of seasonal influenza vaccine and 97% of doses of monovalent H1N1 influenza vaccine were reported in the NCIR; similarly, a regional immunization registry in Wisconsin captured 95% of all influenza immunizations during 2 consecutive influenza seasons (2006–2008) .
In 2009–2010, the NCIR required direct data entry by medical practice personnel, such that information flow occurred in only one direction. The NCIR is working to develop bidirectional communication with electronic health records that achieve design principles of Health Level 7 Standards [7, 18], and this expansion is expected to increase the proportion of all North Carolina children whose information is entered into the registry. Financial incentives for adopting electronic health records and meeting standards for meaningful use of these systems should significantly enhance the adoption of electronic health records in primary care practices throughout North Carolina. Once bidirectional communication between electronic health records and the NCIR is well established, use of this registry may increase not only for children but also for adults. Use of the NCIR to document immunizations among adults is potentially important given the expansion of the adult immunization schedule since 2002 [19, 20].
This study has several limitations. It may not reflect immunization registry usage throughout the state, because all children in this study resided in Forsyth County or 1 of its 7 contiguous counties. However, these counties include urban, suburban, and rural populations, thus reflecting the metropolitan diversity within North Carolina. Another possible limitation is that the few children who did not have an entry in the NCIR or whose practice did not verify their immunization status could have systematically differed from children who had their immunizations verified by both sources. Similarly, children were enrolled in the emergency department and inpatient setting and thus may have systematically differed from children who did not have an emergency department visit or hospitalization; however, being able to verify the immunization status of children who present to the emergency department or to an inpatient setting is important. Also, we may have underestimated the immunization status of children if they received an influenza or pneumococcal vaccine from a practice other than their primary care practice or from a location that did not enter the data into the NCIR. For example, pharmacists in North Carolina were granted temporary authorization to administer seasonal and monovalent influenza vaccines to children 14 years of age or older from October 9, 2009, through July 2010 . Finally, this project focused on only influenza and pneumococcal vaccines, not all recommended pediatric vaccines, and results for up-to-date status for all recommended immunizations may vary.
In summary, the NCIR was widely used to document immunizations for children residing in Forsyth County, North Carolina, and its 7 contiguous counties. There was substantial agreement between practice-based records and registry records for influenza and pneumococcal vaccinations. The NCIR is a valuable resource in the effort to defend public health through control of vaccine-preventable diseases.
We thank all of the participating children and their families, as well as the practices and health departments that made this study possible. We thank the anonymous reviewers whose comments and suggestions enhanced this manuscript.
Financial support. This work was supported by the National Institute of Allergy and Infectious Disease (R01 AI079226) and the Wachovia Research Fund. The views expressed in this article are solely those of the authors and do not necessarily represent the official views of the National Institute of Allergy and Infectious Diseases or the US government.
Potential conflicts of interest. All authors have received research support from BD Diagnostics (Sparks, Maryland).
1. Committee on Infectious Diseases, American Academy of Pediatrics. Recommended childhood and adolescent immunization schedules—United States, 2012. Pediatrics. 2012;129(2):385-386.
2. Committee on Infectious Diseases, American Academy of Pediatrics; Advisory Committee on Immunization Practices of the Centers for Disease Control and Prevention; American Academy of Family Physicians. Recommended childhood immunization schedule—United States, 2002. Pediatrics. 2002;109(1):162.
3. Mast EE, Margolis HS, Fiore AE, et al. A comprehensive immunization strategy to eliminate transmission of hepatitis B virus infection in the United States: recommendations of the Advisory Committee on Immunization Practices (ACIP). Part I: immunization of infants, children, and adolescents. MMWR Recomm Rep. 2005;54(RR-16):1-31.
4. Zhao Z, Murphy TV, Jacques-Carroll L. Progress in newborn hepatitis B vaccination by birth year cohorts—1998–2007, USA. Vaccine. 2011;30(1):14-20.
5. Joffe GP, Rodewald LE, Herbert T, Barth R, Szilagyi PG. Scattering of primary care: doctor switching and utilization of health care by children on fee-for-service Medicaid. J Urban Health. 1999;76(3):322-334.
6. Centers for Disease Control and Prevention (CDC). Immunization information systems progress—United States, 2006. MMWR Morb Mortal Wkly Rep. 2008;57(11):289-291.
7. The North Carolina Immunization Registry (NCIR). North Carolina Department of Health and Human Services Web site. http://www.immunize.nc.gov/providers/ncir.htm. Updated January 12, 2012. Accessed January 10, 2013.
8. Poehling KA, Vannoy L, Light LS, et al. Assessment of parental report for 2009-2010 seasonal and monovalent H1N1 influenza vaccines among children in the emergency department or hospital. Acad Pediatr. 2012;12(1):36-42.
9. Fiore AE, Shay DK, Broder K, et al. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2009. MMWR Recomm Rep. 2009;58(RR-8):1-52.
10. Centers for Disease Control and Prevention (CDC). Update on influenza A (H1N1) 2009 monovalent vaccines. MMWR Morb Mortal Wkly Rep. 2009;58(39):1100-1101.
11. Vaccine against 2009 H1N1 influenza virus. Centers for Disease Control and Prevention Web site. http://www.cdc.gov/h1n1flu/vaccination/public/vaccination_qa_pub.htm. Accessed January 10, 2013.
12. Nuorti JP, Whitney CG; Centers for Disease Control and Prevention. Prevention of pneumococcal disease among infants and children—use of 13-valent pneumococcal conjugate vaccine and 23-valent pneumococcal polysaccharide vaccine—recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2010;59(RR-11):1-18.
13. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.
14. American Academy of Pediatrics. Profile of pediatric visits: annualized estimates by source of payment, patient age, physician specialty, well vs sick visit, office setting, practice ownership, physician employment status, and geographic location, 2004-2007, and annualized estimates of visits to office-based physicians vs general dentists by infants and one-year-olds, 2004-2007. http://www.medicalhomeinfo.org/downloads/pdfs/ProfileofPediatricVisits2010.pdf. Accessed January 10, 2013.
15. Kolasa MS, Chilkatowsky AP, Clarke KR, Lutz JP. How complete are immunization registries? The Philadelphia story. Ambul Pediatr. 2006;6(1):21-24.
16. Stecher DS, Adelman R, Brinkman T, Bulloch B. Accuracy of a state immunization registry in the pediatric emergency department. Pediatr Emerg Care. 2008;24(2):71-74.
17. Irving SA, Donahue JG, Shay DK, Ellis-Coyle TL, Belongia EA. Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort. Vaccine. 2009;27(47):6546-6549.
18. Centers for Disease Control and Prevention. HL7 Version 2.5.1 Implementation Guide for Immunization Messaging. Release 1.3. http://www.cdc.gov/vaccines/programs/iis/technical-guidance/downloads/hl7guide-2011-08.pdf. Updated August 15, 2011. Accessed January 10, 2013.
19. Centers for Disease Control and Prevention (CDC). Recommended adult immunization schedule—United States, 2002-2003. MMWR Morb Mortal Wkly Rep. 2002;51(40):904-908.
20. Centers for Disease Control and Prevention. Recommended adult immunization schedule—United States, 2013. http://www.cdc.gov/vaccines/schedules/downloads/adult/adult-schedule.pdf. Accessed March 31, 2013.
21. Important information about influenza/H1N1, Thursday, October 1, 2009. Effective October 9, 2009, pharmacists will have temporary authority to administer seasonal and H1N1 flu vaccine to patients 14 and older. http://www.ncbop.org/H1N1info2009.htm. Accessed January 23, 2013.
Katherine A. Poehling, MD, MPH associate professor, Department of Pediatrics and Department of Epidemiology and Prevention, Wake Forest School of Medicine, Winston-Salem, North Carolina.
Lauren Vannoy, BS project manager, Department of Pediatrics, Wake Forest School of Medicine, Winston-Salem, North Carolina.
Timothy R. Peters, MD associate professor, Department of Pediatrics, Wake Forest School of Medicine, Winston-Salem, North Carolina.
Address correspondence to Dr. Katherine A. Poehling, Department of Pediatrics, Wake Forest School of Medicine, Medical Center Blvd, Winston-Salem, NC 27157 (firstname.lastname@example.org).