Appendix A — Appendix: Data Review and Upload
A.1 Introduction
Prior to publishing analysis and interpretation of water quality data, we will ensure that all data that meets QA/QC standards outlined in the current project Quality Assurance Project Plan (QAPP) and is accessible in the appropriate public repository.
Water quality data from this project is ultimately destined for the Environmental Protection Agency’s Water Quality Exchange (EPA WQX).
The Quality Assurance Project Plan (QAPP) for this project describes data management details and responsible parties for each step of the data pipeline from observation to repository. The 2021 data preparation and review process is published here as an appendix as an example of the process applied annually to each year’s data.
Year 2021 QA/QC is included here as an example. Other project years QA/QC may be accessed at https://kenai-watershed-forum.github.io/kenai-river-wqx-qaqc/.
A.1.1 Year 2021 Water Quality Data
In this appendix we will collate 2021 laboratory data from several sources into a single spreadsheet document with a consistent format. The desired end format is a spreadsheet template provided by the EPA Water Quality Exchange. These template files are available to download from the EPA at https://www.epa.gov/waterdata/water-quality-exchange-web-template-files.
Once the data is collated, it will be evaluated according to a Quality Assurance Checklist (template example provided by the Alaska Department of Environmental Conservation Soldotna office). Field observations that do not meet the quality assurance standards described in the evaluation checklist will be flagged before being uploaded to the EPA WQX.
Data that has been uploaded to the EPA WQX is evaluated biannually by the Alaska Department of Environmental Conservation (ADEC) in their Integrated Water Quality Monitoring and Assessment Report1. The integrated report evaluates available water quality data from the previous five years against Alaska water quality standards and regulations (ADEC 2020).
A.1.1.1 2021 Water Quality Data AQWMS Formatting
The code scripts in this document assemble water quality data from the three analytical laboratories that partnered with Kenai Watershed Forum for this project in 2021:
SGS Laboratories (Anchorage, AK)
Soldotna Wastewater Treatment Plant (Soldotna, AK)
Taurianen Engineering and Testing (Soldotna, AK)
A.1.1.1.1 2021 Metals/Nutrients Lab Results (SGS Labs)
*Note: the chain of custody documents for SGS Laboratories are integrated into the above downloadable PDF files.
Warning: package 'ggplot2' was built under R version 4.3.3Warning: package 'data.table' was built under R version 4.3.3A.1.1.1.2 2021 Fecal Coliform Lab Results (Soldotna Wastewater Treatment Plant (SWWTP)/Taurianen Engineering)
A.1.1.1.3 2021 Total Suspended Solids Lab Results (Soldotna Wastewater Treatment Plant (SWWTP))
A.1.2 2021 Provisional Results, Prior to QA/QC Review
Results last updated 2024-03-04
The above data sources have been collated in to a single .csv file (available for download) into a format compatible with the EPA Water Quality Exchange. These data have not yet been evaluated against QA/QC standards following guidance in the current project Quality Assurance Project Plan.
A.1.3 2021 Data QA/QC Evaluation
Prior to upload to the EPA WQX, all water quality data must be checked against a series of standard questions in order to evaluate how quality assurance / quality check (QA/QC) requirements are met. The draft Data Evaluation Checklist Template (available for download below) outlines these questions:
A.1.3.1 Pre-Database
A.1.3.1.1 Overall Project Success
Note: the sequence of questions is structured such that data will be gradually flagged at each step based on the criteria described.
1.) Were the appropriate analytical methods used for all parameters?
Yes. Analytical methods from the approved 2020 QAPP were employed.
2.) Were there any deviations from the sampling plan?
All sites were visited as planned on 5/11/2021 and 7/27/2021. Most intrinsic water quality parameters measured with instruments (pH, dissolved oxygen, conductivity, turbidity) were not measured, which is a deviation of from the QAPP.
3.) Were field duplicates, blanks, and/or other QC samples collected as planned?
To see a table comparing planned vs actual results for 2021, download and view the csv file linked above.
From the above table we can see that there are deviations between planned results and actual results available. These reasons for the deviations are known and are attributable to two causes:
Cause 1: The Spring 2021 Chain of Custody (COC) from KWF to SGS was completed erroneously. The COC specified for EPA Method 200.8 analyses (dissolved metals) to be complete for all sites (when they should have stopped upstream of Morgan’s Landing RM31), and it also specified for EPA Method 200.7 analyses (total metals) to stop upstream of Morgan’s Landing (when they should have been performed for all sites in the project).
As a result, for Spring 2021 total metals data will be unavailable for sites upstream of the Morgan’s Landing RM31 site.
Cause 2: For Summer 2021, the SGS laboratory performed the EPA Method 200.8 analyses (dissolved metals) for all 27 analytes available for the method; instead of just the smaller subset of analytes as requested. (E.g., KWF received extra data for free. In this case., there are no consequences of deviating from the planned analyses).
4.) Do the laboratory reports provide results for all sites and parameters?
The laboratory reports provide results for all sites, and for all parameters, with the exceptions outlined above in question #3.
5.) Is a copy of the Chain of Custody included with the laboratory reports?
We worked with three separate laboratories in 2021:
SGS Laboratories, Anchorage, AK
- Chain of Custody documents are included within the PDF laboratory reports linked above earlier in this appendix.
Soldotna Wastewater Treatment Plant, Soldotna, AK
- Chain of Custody documents are on file with Kenai Watershed Forum for fecal coliform and total suspended solids for 5/11/2021, and for total suspended solids on 7/27/2021.
Tauriainen Engineering & Testing, Soldotna, AK
- An individual document for each sample reports the time and date of delivery and analysis for each sample. These documents are included with the PDF laboratory reports linked above earlier in this appendix.
6.) Do the laboratory reports match the Chain of Custody and requested methods throughout?
The laboratory reports match the Chain of Custody and requested methods, with the one exception discussed in question #3.
For summer 2021, the SGS laboratory performed the EPA Method 200.8 analyses for all 27 analytes available for the method; instead of just the smaller subset of analytes as requested. (E.g., KWF received extra data for free. In this case., there are no consequences of deviating from the planned analyses).
7.) Are the number of samples on the laboratory reports the same as on the Chain of Custody?
The quantity of sample bottles sent to the laboratories matches the number of analyzed samples for samples collected and delivered on 5/11/2021 and 7/27/2021.
8.) Was all supporting info provided in the laboratory report, such as reporting limits for all analyses and definitions?
We worked with three separate laboratories in 2021:
SGS Laboratories, Anchorage, AK
- SGS provided data as PDFs which included reporting limits, as well as in the form of an Electronic Data Deliverable where this information is also included in column format.
Soldotna Wastewater Treatment Plant (SWWTP), Soldotna, AK
- SWWTP provided data in the form of .xls files. Reporting limits are described in this project’s current Quality Assurance Action Plan.
Tauriainen Engineering & Testing, Soldotna, AK
- Tauriainen provided data in the form of PDF documents. Reporting limits are described in this project’s current Quality Assurance Action Plan.
9.) Are site names, dates, and times correct and as expected?
- Yes, after post-season correction documented in this report. Notes: In 2021 Kenai Watershed Forum used pre-printed waterproof labels on all sample bottles, reducing opportunity for field and lab transcription errors. Remaining site name transcription errors from laboratories were corrected in post-season data review, using scripted code that did not modify the original report.
10.) Were there any issues with instrument calibration?
Instruments to measure intrinsic water quality parameters (sondes; to measure pH, dissolved oxygen, conductivity, turbidity) were not employed in 2021.
Teams did use hand-held probes to record water temperature on-site. Prior to field use, the hand-held probes were verified as measuring within the accuracy level defined in the QAPP using an ice bath in the Kenai Watershed Forum laboratory.
11.) Did the instruments perform as expected?
- The hand-held water temperature probes performed as expected in 2021.
12.) Was instrument calibration performed according to the QAPP and instrument recommendations?
- Water temperature is a parameter that is “verified” rather than calibrated. The hand-held water temperature probes were verified as measuring within the accuracy level defined in the QAPP using an ice bath in the laboratory, according to instrument recommendations.
13.) Was instrument verification during the field season performed according to the QAPP and instrument recommendations?
- The hand-held water temperature probes were verified as measuring within the accuracy level define in the QAPP using an ice bath in the laboratory, according to instrument recommendations.
14.) Were instrument calibration verification logs or records kept?
- Yes. These records are held at Kenai Watershed Forum, 44129 Sterling Hwy, Soldotna, AK.
15.) Do the instrument data files site IDs, time stamps and file names match?
- Instrument files were not employed in 2021. Measurements from hand held probes were recorded on waterproof paper field forms.
16.) Is any insitu field data rejected and why?
- No insitu data is rejected from 5/11/2021 or 7/27/2021.
17.) Were preservation, hold time and temperature requirements met?
- Yes. Summer and Spring 2021 holding time requirements were met for all samples. See downloadable files below. Laboratory result documents indicated no compromises of preservation and temperature requirements.
Joining with `by = join_by(epa_analysis_id)`18.) Are dissolved metal quantities less than total metals quantities?
Dissolved metals results are not consistently less than total metals results, as would be logically anticipated (see post from Flowlink Environmental website for description and rationale):
“Dissolved metal concentration is determined by filtering a water sample through 0.45 uM filter. Water that passes through the filter is analyzed for metals and the result is reported as dissolved metal concentration. Total metal concentration is determined by analyzing the unfiltered sample.
“Total metal concentration = Dissolved metal concentration + Particulate (insoluble) metal concentration.”
In 2021, there are results available for both dissolved and total metals for three elements: Ca, Fe, and Mg; for sites at and downstream of RM 31.
Dissolved metals results are available for 7/27/2021 (summer) only, and not for 5/11/2021 (spring); see question #3 in this sequence.
Results:
Calcium: all available observations show dissolved metals > total metals
Iron: all available observations show dissolved metals < total metals
Magnesium: nearly all available observations show dissolved metals < total metals, with the exception of three sites, all near the Kenai River mouth
As of Fall 2023 it is unclear why the values measured for total metals is greater than dissolved metals for the cases described above.
See discussion XX in “Summary” section of this report for plans in future years to address the issue of dissolved metals values being greater than total metals values in some cases.
19.) Are the duplicate sample(s) RPD (Relative Percent Difference) within range described in QAPP?
In 2021, field duplicate samples were collected at two sites on both 5/11/2021 and 7/27/2021:
- No Name Creek (Tributary, RM0)
- Funny River (Tributary, RM30)
Following the example in the Alaska Department of Environmental Conservation 2021 Kenai River Metals Field Report (Apsens and Petitt 2022): “… a set of paired samples was evaluated for RPD only if:
- a.) one or both of the samples were above the Limit of Quantitation (LOQ); AND if
- b.) one or both of the samples were at least two times the LOQ.”
A csv file of Relative Percent Difference values organized by site, date, and parameter may be downloaded below:
From the above downloadable table, we can see that a limited subset of paired values from field duplicates are eligible to be used for RPD calculations for 2021. Pairs of measurements must meet LOQ criteria described above (“a” and “b”) in order to be used. For 2021 data, of 69 paired field duplicate observations, 24 of them meet these criteria and thus may be used for RPD calculations.
RPD values range from 0 % to 165.22 %.
In the 2020 Quality Assurance Action Plan, Relative Percent Difference values for most parameters are set at 20%, with the exception of “Phosphorus, total” and “Nitrates (NO2+NO3)” at 25%, and “Suspended solids, total” at 5%.
Using the available RPD values against the above criteria, four paired observations exceed project precision goals in 2021. Three of these four observations are for fecal coliform, and one is for total phosphorus.
Most other parameters have at least one available RPD value that is within project precision goals defined in the QAPP.
The parameters that do not have at least one RPD calculation available, due to the LOQ constraints described above, are the following:
- Arsenic
- Cadmium
- Chromium
- Copper
- Lead
- Zinc
Other notes
In 2021 the full suite of dissolved metals was run for method 200.8 on samples from 7/27/2021. However we intend to only submit data for parameters that had a pre-defined QA plan as outlined in the QAPP, which is the case for the following dissolved six metals: Mg, As, Cd, Cr, Cu, Pb, and Zn.
Summary of Discussion on RPD (Relative Percent Difference) Values for 2021
We attempted to calculate RPD values for all parameters collected in 2021 at two field sites on two dates. For each pair of duplicate observations, one of the following descriptions apply:
- It was possible to calculate RPD values because:
- a.) one or both of the samples were above the LOQ; AND one or both of the samples were at least two times the LOQ
- Project data for this parameter will be submitted unflagged to EPA WQX if RPD values meet project goals outlined in the QAPP
- a.) one or both of the samples were above the LOQ; AND one or both of the samples were at least two times the LOQ
- It was not possible to calculate RPD values because:
- a.) one or both of the samples were not above the LOQ; OR one or both of the samples were not at least two times the LOQ
- Project data for this parameter will be submitted unflagged to EPA WQX if both paired values are non-detect
- a.) one or both of the samples were not above the LOQ; OR one or both of the samples were not at least two times the LOQ
Best practices for interpreting and applying RPD data in order to flag/not flag the overall dataset are currently being finalized. KWF communicated with the Alaska Department of Environmental Conservation on August 22, 2023 via an inquiry on this topic. The memo includes a visual flow chart of the logic for when data is flagged, and is available for download below.
Final determinations on flagging / not flagging 2021 data submitted to the EPA WQX
All 2021 field results will be submitted unflagged to the EPA WQX unless discussed below.
Fecal Coliform
Given that 3 out of 4 RPD values for fecal coliform values are >60%, field data values for the 2021 dataset may vary from actual environmental conditions. The 2020 KWF QAPP does not specify an RPD value for fecal coliform, it only specifies control checks for sterility and temperature. We used EPA Method 9222D (drinking water) for fecal coliform analysis methods, which does not specify an RPD value. However EPA general recommendations for pathogens in sewage sludge recommends and RPD of 20% (link).
Given the above considerations, KWF will submit 2021 fecal coliform data to the EPA WQX as “flagged”. Fecal coliform data will be evaluated in this report and assessed against state standards, but given the high RPD values among duplicate samples, 2021 data will likely not be considered in the context of potential regulatory exceedences.
TO DO: Calculate and visualize RPD values for 2014-2022 and compare, also compare to RPD values in previous comprehensive report.
Selected Metals (As, Cd, Cr, Cu, Pb, Zn)
- Because these parameters do not have an associated RPD value from 2021 field data, they will be submitted to EPA WQX as “flagged.”
- Note: beginning in Summer 2022, KWF initiated a series of changes to the QAPP that will better facilitate QA/QC decisions regarding Total Metals and Dissolved metals results. These changes include use of a dissolved metals field blank, and changing from field filtration to lab filtration as standard practice.
Joining with `by = join_by(activity_start_date, characteristic_name)`20.) Were there any laboratory discrepancies, errors, data qualifiers, or QC failures (review laboratory duplicates, matrix spikes and blanks)?
Laboratory discrepancies and QC failures from the SGS lab are described on page 2 of the spring and summer PDF document reports, which are available for download below.