S-01 — Implementation Guide for the Application of Statistical Sampling Requirements

S-01—Implementation Guide for the Application of Statistical Sampling Requirements (PDF, 80 KB, 22 pages)

Category: STATISTICAL METHODS
Bulletin: S-01 (rev. 1)
Document(s): S-S-01; S-S-02; S-S-03; S-S-04
Issue Date: 2011-11-23
Effective Date: 2011-11-23
Supersedes: S-01

Table of Contents

1.0 Purpose

1.1 The purpose of this bulletin is to provide guidance on the interpretation and application of Measurement Canada's Acceptance Sampling Plans, S-S-series. These guidelines are intended for use by Authorized Service Providers (ASPs) accredited according to S-A-01 requirements, as well as Measurement Canada (MC) staff.

1.2 This Guide explains the terms used in the sampling plans and provides practical guidance on sampling inspection. The requirements have been numbered for ease of reference and are not intended to be used as checklists for audit purposes.

2.0 References

2.1 The Acceptance Sampling Plan is detailed in the following documents:

  1. Bulletin S-02, Implementation Schedule for the Application of Statistical Sampling Requirements in S-S-01, S-S-02, S-S-03 and S-S-04
  2. S-S-01 - Specifications for Random Sampling and Randomization
  3. S-S-02 - Measurement Uncertainty and Meter Conformity Evaluation Specifications
  4. S-S-03 - Prerequisites to the Use of Sampling Inspection
  5. S-S-04 - Sampling Plans for the Inspection of Isolated Lots and Short Series of Lots

2.2 The sampling inspection requirements are based on international standards. Users are encouraged to refer to:

  1. ISO 2859-10, Sampling Procedures for Inspection by Attributes - Part 10: Introduction to the ISO 2859 Attribute Sampling System.
  2. ISO/TR 8550, Guide for the Selection of An Acceptance Sampling System, Scheme or Plan for Inspection of Discrete Items in Lots.
  3. Guide to the Expression of Uncertainty in Measurement, BIPM/IEC/IFCC/ISO/IUPAC/IUPAP/OIML.
  4. ISO 10576-1, Statistical Methods - Guidelines for the Evaluation of Conformity with Specified Requirements - Part 1 - General Principles.

2.3 Users are also encouraged to refer to other important documents such as:

  1. S-E-01 - Specifications for the Calibration, Certification and Use of Electricity Calibration Consoles
  2. S-E-02 - Specifications for the Verification and Reverification of Electricity Meters
  3. S-G-02 - Specifications for the Verification and Reverification of Diaphragm Meters
  4. P-S-04-A - Model Procedure for Sampling Inspection of Isolated Lots
  5. P-S-04-B - Model Procedure for Sampling Inspection of Short Series of Lots

3.0 Terminology

3.1 In the documents referenced in this Guide, the terms "production" and "presentation" are used interchangeably.

3.2 In the documents referenced in this Guide, the terms "implemented" as applied to corrective action, means that corrective action has been identified and carried out, but not necessarily validated.

3.3 The ISO standard uses the terms "device" and/or "item". In the documents referenced in this bulletin, both terms refer to a meter.

3.4 Terms and Acronyms

AMV

means "Accredited Meter Verifier"

AQL

means "acceptance quality limit"

ASP

means, "Authorized Service Providers"

GUM

means "Guide to the Expression of Uncertainty in Measurement"

ISO

means "International Organization for Standardization"

JWP

means "Joint Working Group"

LQ

means "limiting quality"

MADT

means "measure of absolute deviation from target"

MC

means "Measurement Canada"

MUT

means "meter under test"

n, N

means "sample size" and "lot size"

SPC

means "statistical process control"

Type A evaluation

means a method of evaluation of uncertainty by statistical analysis of a series of observations (reference GUM).

Type B evaluation

means a method of evaluation of uncertainty by means other than the statistical analysis of a series of observations (reference GUM).

4.0 Principles Governing Acceptance Sampling Inspection and Plan Design

4.1 During the review process (2006 to 2009) the Sampling Project Joint Working Group (JWG), which was comprised of industry and MC representatives, considered a number of national and international sampling plans to use as the foundation element in the development of the acceptance sampling plan. The base plan that was selected is the internationally accepted ISO 2859-2 plan. The design parameters associated with this plan were consistent with the level of conformity that the JWG wanted to achieve, which is generally described as a relative confidence level that a certain percentage of devices under test are conforming.

4.2 With the ISO plan, the design parameter assumed a consumer risk of 10% probability of lot acceptance. The various limiting quality (LQ) tables in ISO 2859-2 which form the basis of S-S-03 and S-S-04 were established on that probability. During the review process, a check of the ISO 2859-2 tables revealed inconsistencies. They are associated with the LQ = 2.0% table for lot size 51 - 90 and the LQ = 3.15% table for lot size 51 - 90. In both cases, the probability of acceptance of a lot is significantly over the 10% design parameter that was defined by the JWG. Consequently, the JWG reached a unanimous consensus on the following options to address these inconsistencies:

  1. users should use sample sizes associated with one of the larger lot sizes, or
  2. alternatively, the following sample sizes/lot size combination can be used:
    1. S-S-03, Table A.1, for LQ = 2.0% and lot size 51 - 90, use sample size of 58 units;
    2. S-S-04, Tables A.1 and A.3, for LQ = 3.15% and lot size 51 - 90, use sample size of 48 units.

The JWG has also recommended that this inconsistency be communicated to the ISO 2859-2 committee for final resolution.

4.3 In addition to the above comments, users of S-S-series plans should be aware that the acceptance quality limit (AQL) value tends to increase with the acceptance number and that the probability of acceptance at the LQ point may be greater that 10%. Furthermore, ISO plans are designed with constant proportions (i.e. percentages). Given those features of ISO 2859 based plans, users should consult the operating characteristics of the individual plan to ensure their process quality objectives are met. The following are some considerations and recommendations:

  1. S-S-04 tables B.1, B.2 and B.3, sample sizes are not associated with lot size. Users should consult the operating characteristics of the individual plan to ensure process quality objectives are met.
  2. S-S-04 table B.1, sample sizes for the third row exceed 10% probability of acceptance at the LQ value. To maintain consistency with the other plans it is recommended the sample size for the third row of Table B.1 be increased to 210, 105 (option 1) and 82, 41 (option 2) and in Table B.3 it be increased to 201, 105.
  3. Tables A.3 and B.3, use LQ of 3.15 for smaller lot sizes and LQ of 2.0 for larger lot sizes. ISO plans do not necessarily provide a good transition to a tighter LQ value as lot size increases. For nonperformance observations, the following sample sizes for tables A.3 and B.3 of S-S-04 are recommended:
    1. table A.3, lot size 1201 to 3200, it is recommended that the sample size be increased to 150 (single) and 95, 95 (double);
    2. table B.3, second row, it is recommended that the sample size be increased to 150, 75.

5.0 Guidelines on S-S-02

5.1 General Guidelines for the Determination of Measurement Uncertainty in Conformity Assessment (S-S-02, section 4.1.1)

5.1.1 With reference to section 4.1.1 of S-S-02, the determination of measurement uncertainty typically involves the following general steps:

  1. Identification of the potential sources of uncertainty.
  2. Determination of standard uncertainty (us) for each component.
  3. Determination of the relationship between the components (statement of the reduction equation).
  4. Identification and evaluation of covariance terms.
  5. Determination of the sensitivity coefficients for each component and creation of a uncertainty budget table.
  6. Combination of the standard uncertainties and sensitivity coefficients to determine the combined uncertainty (uc) in accordance with the ISO Guide to the Expression of Uncertainty in Measurement (GUM).
  7. Determination of the expanded uncertainty (U).

5.1.2 Evidence should be provided in the statement of uncertainty indicating the completion of the items listed in GUM, section 7.2

5.2 General List of Measurement Uncertainty Contributors in Gas Meter Conformity Assessment (S-S-02, section 4.1.2)

With reference to section 4.1.2 of S-S-02, the following is a partial list of measurement uncertainty contributors that may influence conformity assessment of a typical gas measuring apparatus. Their applicability depends on the design of the measuring apparatus and the choice of the reference standard. The recommended method for their determination is presented with each of the contributors listed.

  1. Repeatability of device under test:
    1. Repeatability of device under test (Type A evaluation)
  2. Resolution uncertainties:
    1. Resolution of the pressure, temperature and barometer indications (process instruments) (Type B evaluation) (square distribution)
    2. Pulse counting resolutions of the reference standard and device under test (Type B evaluation) (triangular distributor)
  3. Environmental performance uncertainties (influence quantities):
    1. Temperature and pressure sensitivity of reference standard calibration/performance (Type B evaluation) (square distribution - limited info)
    2. Temperature and pressure sensitivity of the process Instrumentation (Type B evaluation) (square distribution - manufacturer's data)
  4. System design and operation uncertainties:
    1. Put in / Take out reproducibility of the device under test (Type A evaluation) (normal distribution - by experiment)
    2. Long-term stability (LTS) of process instrumentation (Type A evaluation or Type B evaluation) (normal distribution - SPC charts)
    3. Sensitivity of reference standard's performance to operating rate (Type B evaluation)
    4. Influence of the location of sensors relative to the desired measurement point (Type A evaluation or Type B evaluation)
  5. Uncertainties of the calibration of the reference standards contained in the measuring apparatus (often referred to as fossilized uncertainty):
    1. Calibration of reference standard (Type A evaluation or Type B evaluation) (normal distribution)
    2. Calibration of process instrumentation (Type A evaluation or Type B evaluation) (normal distribution)
  6. Operator performance:
    1. Reproducibility of operator's ability to read instrumentation - parallax errors (Type A evaluation or Type B evaluation) (square distribution or normal distribution)
  7. Uncertainties from assumptions:
    1. Compressibility and Z / Z assumptions (Type B evaluation) (square distribution)
    2. Equation of state and algorithms (Type B evaluation) (square distribution)
    3. Temperature and pressure effects - changes in the connecting volume (Type B evaluation) (square distribution)
    4. The effect of an alternate pressure, temperature or test medium on the performance of the device under test (Type A evaluation or Type B evaluation)

5.3 General List of Measurement Uncertainty Contributors in Electronic Electricity Meter Conformity Assessment (S-S-02, section 4.1.2)

With reference to section 4.1.2, the following is a partial list of measurement uncertainty contributors that may influence conformity assessment of a typical electronic electricity meter. Their applicability well depend on the design of the measuring apparatus, the particular meter types being evaluated and the meter functions being evaluated. The recommended method for their determination is presented with each of the contributors listed:

  1. Electricity Meter Calibration Console
  2. Electricity Meter Under Test

5.3.1 Measurement Uncertainty Contribution from an Electricity Calibration Console

The major contributors to uncertainty from the electricity calibration console are:

  1. Test Burden (S-E-01, section 7.1.5). Test burdens for each meter type (single phase, polyphase, self-contained, etc.) intended to be evaluated on the console shall be established by using the burden evaluation data gathered under section 7.1.5.
  2. Burden Effect (S-E-01, section 7.2). Uncertainty figures for the test burdens established under 7.1.5 shall be determined using the data gathered under section 7.2 of S-E-02. (Type B evaluation)
  3. Number of Meter Under Test (S-E-01, section 7.3). For multiposition calibration consoles an uncertainty figure shall be established from the data gathered under section 7.3 of S-E-02. (Type B evaluation)
  4. Variation of Position (S-E-01, section 7.4). For multiposition consoles an uncertainty figure shall be established for the variation of console errors between positions. This will be determined using the data from section 7.4 of S-E-01. (Type B evaluation)
  5. Switching Effects (S-E-01, section 7.7). This is applicable to fully automatic and semi-automatic consoles. The uncertainty figure shall be established using the data in section 7.7 of S-E-01. (Type B evaluation)
  6. Regulation (S-E-01, section 7.6). This is a Type B evaluation and is applicable to verification of demand meters.
  7. Console reference meter. The uncertainty of the console reference meter shall be established by Type A evaluation.
  8. Interchangeable console reference meters (S-E-01, section 7.9). This is a Type A evaluation and is applicable to consoles which are equipped with interchangeable reference meters only.

5.3.2 Meter Under Test

5.3.2.1

The meter under test is a Type A evaluation and is established by the meter verifier for the specific meter being tested.

5.3.2.2

When assessing demand meters the uncertainty due to meter reading resolution is also a consideration in establishing the total meter uncertainty.

5.4 General Guidelines for the Determination of the Effects of Differences Between the Test Conditions and Usage Conditions of the Device Under Test (S-S-02, section 5.1.4 (d))

5.4.1 For gas measuring devices, the effect of the use of an alternative pressure, temperature or test medium on the performance of the device under test compared to normal operating conditions needs to be assessed (future).

5.4.2 For electricity measuring devices, the effect of the use of a measuring apparatus as an alternative source on the performance of the device under test compared to normal conditions needs to be assessed (future).

5.5 General Comments on Measurement Uncertainty and MADT

5.5.1 For diaphragm gas meters the measurement uncertainty may be a consideration with MADT (median) determination. Generally, however, the uncertainty of the difference between the high-load and low-load tests will be much smaller than the overall measurement uncertainty. This is because there is tendency of any bias errors and long-term drift terms to cancel each other in such a comparison. This results in an uncertainty that approaches the square root of 2 times the test repeatability. Where it is established that the repeatability of the process uncertainty is much smaller than the tolerance band, the uncertainty can be regarded as insignificant.

5.5.2 For electronic electricity meters the measurement uncertainty may be a consideration with MADT (mean) determination. Since MADT is determined on the basis of absolute deviation values, the uncertainties that would apply in an MADT determination tend to cancel each other out. This results in an uncertainty that approaches the square root of N (number of test points used in determining MADT) times the test repeatability. Where it is established that the repeatability of the process uncertainty is much smaller than the tolerance band, the uncertainty can be regarded as insignificant.

6.0 Guidelines on Homogeneity, Lot Formation and LQ Value

6.1 The purpose of random sampling is to ensure that the sample can be representative of a homogeneous lot. Requirements associated with lot homogeneity can be found in the device verification specification.

6.2 If an Authorized Service Provider (ASP) wishes to combine, in one lot, various models or vintages of meters, it is the ASP's responsibility to submit a request to MC with documentation supporting the claim that the models are homogeneous. The device owner is responsible for keeping records. Since the meters usually remain in the owner's inventory for many years, it is advisable that a record system be established to ensure the documents can be retrieved in the future.

6.3 The various actions and switching rule decisions that are outlined in S-S-03 and S-S-04 apply to original inspection and homogenous grouping.

6.4 Quality assurance (QA) Managers and Auditors will need to confirm that appropriate dispositions and corrective actions have been taken and that qualification requirements have been met regarding lot formation and LQ selection. Specifically, Lot Splitting (lots subdivided arbitrarily in order to create a long run), LQ Jumping (moving to different LQ values without a management plan) and Changing Mid Stream (changing a sampling strategy part way through an inspection process) are not permitted under S-S-03 and S-S-04.

6.5 To ensure lot sizes are within a consistent range, a factor from 0.5 to 1.5 of the historical (or typical lot size) can be employed. This would be a move of up/down one row in the ISO 2859-2 table.

6.6 In choosing an inspection strategy, organizations should consider the factors identified in ISO/TR 8550 and ISO 2859-10. Even though there is mathematical equivalence between single, double and multiple plans; many factors should be considered when implementing a plan. These factors include consideration of the plan's complexity, use of 100% inspection, number of sampling inspections, ease of drawing samples, quality history, etc. Complexity can arise when differing or additional samples are needed, or when events lead to nonconformance disposition and corrective action. Accredited Meter Verifiers (AMVs) should have the organizational ability to carry out the inspection strategy as well as to properly document these activities and related processes.

Note: It is recommend that operations initially start at the baseline LQ of 3.15% and single sampling inspection under S-S-04, Annex A. Consideration can be given to other sampling options once operational experience is gained.

7.0 Guidelines on S-S-03

7.1 Guidelines for "historical evidence" as noted in S-S-03, section A.3.7 include the following:

  1. ASPs currently sampling under LMB-EG-04 is qualified for sampling the devices currently listed in their accreditation manuals. Historical evidence can only be used once per homogeneity grouping as a qualifier at initial implementation.
  2. Historical evidence should be based on data for the facility and device under review. Sampling evidence from other organizations is not sufficient for qualification purposes. The intent is to confirm the integrity of both the device being sampled and the process being used by applicants.

7.2 With respect to limited device quantities, users have the option to use 100% inspection or use the sampling plan indicated in S-S-03 for limited volume purchases.

7.3 When applying the acceptance sampling plan to a brand new technology, or after the implementation of a new process, the following steps should be followed:

  1. First, perform a 100% inspection of the quantity specified in S-S-03.
  2. Second, ensure the qualification requirements that are specified in S-S-03 have been met.

7.4 Refer to section 11.4 below, to determine the classification of observations.

8.0 Guidelines for Process Related Corrective Action Resolution

8.1 Documentation is a key part of any corrective action resolution process. The documentation should include procedures that describe the processes for detection, feedback control and for the correction of conditions adverse to quality that have affected the device or the process. The procedures should include the following:

  1. Stopping rules and actions in the event that lots continue to fail.
  2. Events that would cause discontinuation of sampling.
  3. Escalation procedure.

8.2 The Corrective Action Resolution should typically include the following types of activities:

  1. Conduct immediate disposition.
  2. Conduct root cause analysis.
  3. Implement and evaluate corrective actions.
  4. Confirm and approve resolution.

8.3 Effective corrective action would normally be implemented prior to presentation of the next lot. However, since some actions may involve other parties (such as the supplier) or capital expenditures, the complete implementation of corrective action may take more time. Such time delays should not be used to delay any necessary actions. It is important that the ASP ensures the quality of the process; consequently, the ASP should provide procedures to achieve that goal.

8.4 Corrective action may include discontinuation of sampling inspection under S-S-04 Annex A or B. If that is the case, a re-qualification of the production process under S-S-03 is required.

8.5 Conditions that should result in corrective action may include:

  1. Failures, malfunctions and deficiencies in production processes, equipment or software.
  2. Inadequate procedures and documentation.
  3. Inadequate control of work.
  4. Non-compliance with procedures.
  5. Scheduling problems, or
  6. Process not statistically capable of producing product in accordance with the specifications.

9.0 Guidelines on Stopping and Escalation Procedures

9.1 Stopping rules based on a sequence of lots cannot address all outcomes. Without limiting other possibilities, the ASP should have documented procedures to address occurrences such as process disruptions or repeated failures due to the same or similar causes. The investigation and resulting recommendations could result in discontinuation of sampling inspection under annexes A and B of S-S-04.

9.1.1 If a cumulative number of lots in a sequence are not accepted on original inspection, then stopping rules (such as the following) should be implemented and inspection discontinued pending action to improve quality:

  1. In the event users are operating at LQ = 8.0% and have non-acceptance of 3 out of 5 (or optionally 2 out of 3) or fewer consecutive lots, then sampling can continue at 3.15%, unless evidence suggests that sampling is no longer appropriate (re-qualification would then be necessary).
  2. In the event users are operating at LQ = 3.15%, and have non-acceptance of 3 out of 5 (or optionally 2 out of 3) or fewer consecutive lots, then sampling would be suspended. Corrective action and process re-qualification would be initiated. To re-qualify the process, S-S-03 would be utilized.

9.2 Information obtained from sampling several lots in sequence may indicate that process concerns are warranted. That should result in using 100% inspection or the S-S-03 specification to invoke more rigorous sampling procedures. Sampling under S-S-03 provides consumer protection against poor quality; however, there is a higher risk to the producer of having an acceptable lot judged unacceptable. Indication of possible deterioration in product quality is a signal to initiate corrective action and ensure corrective action is in fact taken.

The following are some techniques which can be used to monitor the process:

  1. the use of data obtained from meters and lots which are the same or similar in nature;
  2. the use of tables and graphics such as frequency distributions and histograms;
  3. the use of charts such as the Pareto or defect concentration;
  4. the use of appropriate control charts such as CUSUM chart;
  5. a trend analysis; and
  6. the application of stopping rules to monitor a series of same, or similar lots.

9.3 100% inspection should be used until process related concerns have been addressed.

10.0 Guidelines on Sampling Plan Change Selection

10.1 Should there be a significant change in production (change in production infers a new process or substantial change), then process re-qualification procedure under S-S-03 should be utilized.

10.2 With regard to the Isolated Lot Plan, S-S-04, Annex A:

  1. There are no timeline restrictions.
  2. A significant change in process requires re-qualification.

10.3 With regard the Short Series of Lots Plan S-S-04, Annex B:

  1. The maximum tolerable time period between successive lots is six weeks. After a single stoppage over six weeks, then re-start at n1.
  2. For delivery time periods expected to be over six weeks, revert to isolated lot option, Annex A.
  3. Minimum of five lots presented in sequence. If there are insufficient lots, revert to isolated lot option, Annex A.
  4. A significant change in process requires re-qualification.

11.0 Guidelines on Short Series and Isolated Lot Inspection

11.1 S-S-04, Annex A (isolated lots) is the default plan and may be used for all sampling inspection. Typically, an accredited manufacturer would utilize S-S-04, Annex B (short series of lots) when production quantities are large and quality stability has been demonstrated over a series of lots. For an ASP processing limited volume purchases, refer to section 7.2 above.

11.1.1 Under Annex A, there is potential for a wide range of activities. Lots may be presented in isolation or the ASP may elect to inspect all lots under this annex. To ensure consistency between Annex A and B of S-S-04, the following guidelines should be followed:

  1. If an isolated lot is presented, any action is limited to what is stated in section 4.4 of ISO 2859-2. If there are follow-on lots, corrective action on the process or system would be similar to that stated in Annex B.
  2. If an isolated lot were presented, the way to deal with a rejected lot would be limited to what is stated in section 4.4 of ISO 2859-2. If there are follow-on lots, subsequent lots would be dealt with as per the requirements of Annex B (i.e. LQ of 8.0 lot may be inspected under LQ of 3.15 or 100% or undergo requalification of the process).
  3. If process performance has degraded significantly, sampling under Annex A or B should be discontinued, necessitating re-qualification as per S-S-03.

11.2 ISO references include: ISO/TR 8550, 5.2c and ISO 2859-10, sections 2.5 and 3.17.

11.3 Regarding short series of lots, if a lot is not accepted due to the observed number of performance or non-performance characteristics, the sequence for a short series of lots is typically as follows:

A 100% conforming product is the goal, coupled with corrective action and continual improvement activities when nonconformities are found. The focus is on acceptable levels of marginally conforming product by setting narrow test limits within the device specification.

Lot # 1 - not accepted

Lot # 2 - screened and found satisfactory

Lot # 3 - sampled using sample size n1

If the above sequence cannot be successfully accomplished, process requalification may be necessary. A non accepted lot may be resubmitted for sampling inspection under Annex A at the same LQ value.

11.4 To ensure sample meters are classified in a mutually exclusive manner, classification is performed by using the decision sequence shown in Appendix A.

11.4.1 If counting a meter in several categories would cause failure, it would be reasonable and prudent to fail the lot. Example, if the plan is Ac1 = 1, Ac2 = 5, and during inspection one type 1 meter and five type 2 meters are observed, but the type 1 meter is also type 2, it would be reasonable to fail the lot (since six type 2 meters would cause failure). Reference ISO 2859-10, section 3.3.

12.0 Guidelines on Outgoing Quality

12.1 The objective for outgoing quality is that it be the same regardless of whether the meters are processed by sampling or by 100% inspection. The outgoing quality requirement of the product of LQ and N applies to marginally conforming units. This requirement is considered to be met if a lot is accepted under sample inspection. If meters are inspected (lots inspected by 100% inspection or lots inspected but not accepted by sampling), nonconforming meters and marginally conforming meters shall be removed or repaired to ensure the outgoing quality standards are met.

12.2 In the case of 100% inspected diaphragm gas meters, regardless of the number of meters presented, compliance with the test limit and MADT requirements of section S-G-02, section 6.3.9 ensures the outgoing requirements of S-G-02 are met with respect to marginal conformities (type 1 and type 2) and performance nonconformities.

Note: Meters that do not meet the test limit or MADT requirements or have a nonconformity cannot be sealed.

12.3 In the case of electronic electricity meters: when processing lots, there are situations in which the test limit requirements are insufficient to ensure the outgoing quality requirements of S-E-02, Annex A have been met.

One such situation is when the test limit equals the verification specification limits. Example calculations are shown in Appendix E.

12.3.1 To avoid excessive administration costs for meters which only undergo 100% inspection, are not new and cannot be calibrated (cannot be renewed), there is a provision in S-E-02, section A.7e for the outgoing quality requirements to be considered met. This provision is valid only when the k value is at least 3 and the MADT limit is at most 50% of the specification limit.

13.0 Guidelines on Rounding and Significant Figures

13.1 Discrimination refers to the degree of exactness with which the quantity is stated (i.e. the smallest readable unit or the measurement resolution). The discrimination of a quantity refers to the right-most significant digit. The meter verification specification defines the measurement discrimination.

13.2 During calculations, caution should be taken to ensure errors are not introduced due to the lack of figures. To avoid introducing error, rounding should only be done at the end of the calculation process. The term "one step" as used in S-S-02, refers to a procedure in which rounding is only performed once during calculations.

14.0 Appendices

14.1 The sequence in Appendix A shows how to classify observations.

14.2 Appendices B, C, D and E contain calculations as well as selected questions and answers based on S-S-04, S-E-02 and S-G-02.

  1. Appendix B contains selected sampling plan application questions and answers.
  2. Appendix C contains selected conformity assessment and classification questions and answers.
  3. Appendix D contains selected screening inspection questions and answers.
  4. Appendix E contains outgoing quality example calculation.

15.0 Revisions

The purpose of revision 1 is to:

  • reflect amendments to S-S-02 (rev. 1), S-S-03 (rev. 2), S-S-04 (rev.2) and S-E-02 (rev. 3). Various amendments were made throughout this document;
  • make various improvements throughout to correct grammar, document titles and to facilitate readability and flow of material;
  • add to section 3.4 the terms AQL, GUM, MC, MUT, n, N and SPC;
  • change 4.2b(I) sample size from 57 to 58;
  • add section 4.3 to address the sample size recommendations set out in S-S-04, Annexes A and B;
  • incorporate wording in sections 5.3, 5.4 and 5.5 relating to the application of measurement uncertainty to electricity meters;
  • add to section 6.4 a prohibition against changing a sampling strategy mid stream;
  • add section 6.6 to address selection of complex inspection strategies;
  • remove incorrect phrase "small volume" in sections 7.2 and 11.1;
  • add assessment techniques to section 9.2;
  • add section 11.3 and 11.4 to address lot sequences and classification;
  • add section 12, "Outgoing Quality Requirements";
  • add Appendix A, which contains the classification sequence;
  • add Appendices B, C, D and E, which contain calculations, questions and answers.

16.0 Additional Information

For additional information regarding this bulletin, please contact the Senior Program Officer responsible for statistical techniques. For more information regarding Measurement Canada and its programs, visit our Web site.

Guy Dacquay
Manager, Utility Metering Division
Program Development Directorate

Appendix A - Classification of Observations in S-S-03 and S-S-04 Sampling Plans

To ensure meters are classified in a mutually exclusive manner, classification must be performed according to the following decision sequence for each sample meter:

  1. Does the meter have a nonconforming performance characteristic? If so, classify the meter accordingly. Go to step 2.
  2. Does the meter have a nonconforming non-performance characteristic? If so, classify the meter accordingly. Go to step 3.
  3. Does the meter have a type 1 marginally conforming performance characteristic? If so, classify the meter accordingly. Go to step 4.
  4. Does the meter have a type 2 marginally conforming (MADT) characteristic? If so, classify the meter accordingly; go to step 6. If not, go to step 5.
  5. Classify the meter as conforming if there are no nonconforming and marginally conforming characteristics. Go to step 7.
  6. Determine the category counts for the meter. Classify the meter according to the most serious defect. Go to step 7. Note: classifications falling under 1 are most serious, followed by 2, etc.
  7. Is there another sample meter? If so, proceed to the next sample meter and go to step 1. If not, go to step 8.
  8. Once the category counts are determined for all the meters in the sample, lot acceptability is determined by comparing each nonconforming and marginally conforming unit count to the associated acceptance number of the sampling plan.

Note: The acceptance number is always zero for nonconforming performance characteristics.

Appendix B - Selected Sampling Plan Application Questions and Answers Based on S-S-04, S-E-02 and S-G-02

B.1 For all of the answers (shown) - meters inspected under 100% inspection or sampling inspection must meet the outgoing quality requirements as specified in the meter verification specification. Measurement uncertainty is included in all conformity assessments.

B.2 Questions and Answers

Q.1 A utility purchases 2000 meters as a homogeneous isolated lot. It selects and inspects a random sample of the required size using single sampling (LQ of 3.15%) and discovers a marginally conforming type 1 unit in the sample. What is the appropriate sample size and what is the correct decision regarding the lot?

A.1 Single sampling chosen under LQ = 3.15%

n = 125 (for N = 501 to 3200)
Ac1 = 1
Ac2 = 19
Ac3 = 1
Ac = 0 for performance nonconformities
Since, dmc1 <= Ac1 - therefore, lot is accepted

Q.2 A utility purchases 2000 meters as a homogeneous isolated lot, selects and inspects a random sample of the required sizes using double sampling (LQ of 3.15%) and discovers two marginally conforming type 1 units in the first sample and none in the second. What were the appropriate sample sizes and what is the correct decision regarding the lot?

A.2 Double sampling chosen under LQ = 3.15%

n1 = 80 (for N = 501 to 3200); Ac1 = 0, Re1 = 2 Ac (performance) = 0
Ac2 = 11, Re2 = 18
Ac3 = 0, Re3 = 2

Since, dmc1 > = Re1 therefore lot is rejected on the first sample

n2 = 80 (for N = 501 to 3200); Ac1 = 1, Re1 = 2 Ac (performance) = 0
Ac2 = 22, Re2 = 23
Ac3 = 1, Re3 = 2

Note: The lot was rejected on first sample, a second sample need not have been selected or inspected.

Q.3 A utility purchases 2000 meters as an isolated lot. All meters are homogeneous with respect to all characteristics and production, except for 500 of them, which have an additional attribute type characteristic. What are the valid sampling inspection approaches? Assume only single sampling will be used.

A.3a Split lot into two fully homogeneous sub-lots of N1 = 1500 and N2 = 500. See figure 1.

Single sampling chosen under LQ = 3.15%
Sampling plan for N1 = 1500 is: n = 125, Ac1 = 1, Ac2 = 19, Ac3 = 1

Sampling plan for N2 = 500 is: n = 80, Ac1 = 0, Ac2 = 11, Ac3 = 0

For all samples, Ac = 0 for performance nonconformities.

Inspect each sample for all characteristics and decide on acceptance of each sub-lot accordingly.

A.3b Alternate solution: for the inspection of all common characteristics, leave the lot together. Treat meters with the additional characteristic as a sub-lot. See figure 2.

Single sampling chosen under LQ = 3.15%
Sampling plan for N1 = 2000 is: n = 125, Ac1 = 1, Ac2 = 19, Ac3 = 1

For all samples, Ac = 0 for performance nonconformities.

Sampling plan for N2 = 500 is: n = 80, Ac3 = 0

Select a random sample of 125 from N1. Assume the sample from N1 happened to include 30 from N2. An additional random sample of 50 is therefore needed from N2. Refer to A.5 below for sample selection options.

Figure 1
An isolated lot of 2000 meters, 500 of which have an additional attribute type characteristic, split into two homogeneous sub-lots of 1500 meters and 500 meters.
Figure 2
An isolated lot of 2000 meters, 500 of which have an additional attribute type characteristic, as a lot of 2000 meters for the common characteristics and a sub-lot of 500 meters for the additional characteristic.

Q.4 Describe Operation of the Plan

Lot size: 1000, single sampling chosen under LQ = 3.15%

Assume +/-1.00% specification limit; 0.50 MADT limit
Assume +/-1.60% specification limit; 0.80 MADT limit
Assume console displays to 2 decimal places

A.4 Required sample size: 125 and Ac1 = 1, Ac2 = 19, Ac3 = 1, Ac (performance) = 0

Randomly select 125 meters from the lot. Inspect all quality characteristics of the 125 meters

All inspected meters shall meet 100% inspection requirements.

Case 1, specification limit: +/-1.00%

Reject Lot

if 2 or more meters found with extended performance errors greater than +0.83% but not greater than +1.00% or less than -0.83% but not less than -1.00%

if 20 or more meters found with type 2 (MADT) performance errors greater than 0.50%

if one or more meters found with extended performance errors outside +1.00%

if two or more meters found with non-performance nonconformity

Case 2, specification limit: +/-1.60%

Reject Lot

if 2 or more meters found with extended performance errors greater than +1.33% but not greater than +1.60% or less than -1.33% but not less than -1.60%

if 20 or more meters found with type 2 (MADT) performance errors greater than 0.80%

if one or more meters found with extended performance errors outside +1.60%

if two or more meters found with non-performance nonconformity

Q.5 Describe Operation of the Plan

Lot size: 1000; single sampling chosen under LQ = 8.0% for performance characteristics

Assume +/-1.00% specification limit; 0.50 MADT limit
Assume +/-1.60% specification limit; 0.70 MADT limit

Assume console displays to 2 decimal places

A.5 Performance Characteristic Inspection

Required sample size: 50 and Ac1 = 1, Ac2 = 6, Ac (performance) = 0
Non-performance characteristic inspection: Required sample size: 125 and Ac3 = 1

Method for selecting the sample meters: Several methods can be used. The key point is that the 50 samples to be inspected for performance and nonperformance attributes must be randomly spread throughout the lot, and the 125 samples for nonperformance tests (of which the 50 are presumably a part) must also be randomly spread throughout the lot.

Randomly select 50, then randomly select another 75 meters (no overlap) from the balance of the lot. Inspect all quality characteristics of the first sample of 50 meters and all non-performance quality characteristics of the second sample of 75 meters.

Alternatively, randomly select 125 meters (leave them unordered) from the lot. Test the first 50 in the unordered list (unsorted list) for all quality characteristics and the other 75 for non-performance quality characteristics.

All inspected meters shall meet 100% inspection requirements.

Case 1, specification limit: +/- 1.00%

Reject Lot

if 2 or more meters found with extended performance errors greater than +0.67% but not greater than +1.00% or less than -0.67% but not less than -1.00%.

If 7 or more meters found with type 2 (MADT) performance errors greater than 0.50%

if one or more meters found with extended performance errors outside +1.00%

if two or more meters found with non-performance nonconformity

Case 2, specification limit: +/-1.60%

Reject Lot

if two or more meters found with extended performance errors greater than +1.08% but not greater than +1.60% or less than -1.08% but not less than -1.60%

If 7 or more meters found with type 2 (MADT) performance errors greater than 0.70%

if one or more meters found with extended performance errors outside +1.60%

if two or more meters found with non-performance nonconformity

Case 2, specification limit: +/-1.60%

Reject Lot

if 2 or more meters found with extended performance errors greater than +1.08% but not greater than +1.60% or less than -1.08% but not less than -1.60%

If 7 or more meters found with type 2 (MADT) performance errors greater than 0.70%

if one or more meters found with extended performance errors outside +1.60%

if two or more meters found with non-performance nonconformity

Q.6 A utility purchases 2000 meters as a homogeneous isolated lot. It selects and inspects a random sample of the required size using single sampling under LQ = 8.0% for performance characteristics. What options are available as valid sampling inspection approaches?

A.6a Performance Characteristic Inspection

Required sample size is 80 and Ac1 = 3, Ac2 = 11, Ac (performance) = 0
Non-performance characteristic inspection:
Required sample size is 125 and Ac3 = 1

Since the sample sizes are not the same, refer to the method outlined in answer A.5 (above) to select the sample meters.

A.6b Alternative Solution

Use single sampling for performance characteristics and double sampling for non-performance characteristics.

Performance characteristic inspection: same as above.

Non-performance characteristic inspection:

Required sample size for N1 is 80 and Ac3 = 0, Re3 = 2; and, n2 is 80 and Ac3 = 1, Re3 = 2

In this case, the sample sizes are the same for performance characteristics and for the first sample of non-performance characteristics. If the first sample of non-performance characteristics indicates a second sample is required, select from the unsorted list an additional sample of 80 from the balance of the lot.

Appendix C - Selected Conformity Assessment and Classification Questions and Answers Based on S-S-04 (rev.2), S-E-02 (rev. 3) and S-G-02 (rev. 1)

C.1 Given the following scenarios for diaphragm gas meters, what is a meter's conformity assessment and classification during sampling inspection?

C.2 For these examples, assume a specification limit of +/- 1.60, a test limit of +/- 1.00 (% error) and sampling is under a plan with an LQ of 3.15%, with a compressed specification limit of +/- 1.3360 and a MADT (median) limit of 0.8000 and 0.6000. Assume a small lot size (zero acceptance numbers) and an uncertainty value of 0.20.

Note: No sample meter or 100% inspected meter can be placed into service unless it meets all specified requirements.

C.3 Questions and Answers

Q.1.1 Meter with two test points, high load (HL) and low load (LL), had errors: +0.30, +1.70; all non-performance tests rated as acceptable.

A.1.1 Meter has type 1 and type 2 marginal conformities and one nonconformance; meter rated as a performance nonconforming unit and counted as a "dp"; meter cannot be sealed.

Q.1.2 Meter with two test points, HL and LL, had errors: +0.30, +1.70; non-performance tests: incorrect name plate markings.

A.1.2 Meter has type 1 and type 2 marginal conformities and two nonconformances; meter rated as a performance nonconforming unit and counted as a "dp"; meter cannot be sealed.

Q.1.3 Meter with two test points, HL and LL, had errors: +0.40, +1.10; all non-performance tests rated as acceptable.

A.1.3 Meter cannot be sealed as values over test limit criteria.

Q.1.4 Meter with two test points, HL and LL, had errors: 0.00, +1.40; non-performance tests: incorrect name plate markings.

A.1.4 Meter has type 1 marginal conformity and one nonconformity; meter rated as a non-performance nonconforming unit and counted as a "dnp"; meter cannot be sealed.

Q.1.5 Meter with two test points, HL and LL, had errors: +0.60, +1.40; all non-performance tests rated as acceptable.

A.1.5 Meter has type 1 and type 2 marginal conformities; meter rated as a marginally conforming unit and counted as a type 1 meter cannot be sealed as values over test limit criteria.

Q.1.6 Meter with two test points, HL and LL, had errors: +0.50, +0.70; all non-performance tests rated as acceptable.

A.1.6 Meter rated as conforming unit.

C.4 Given the following scenarios, for electronic electricity meters, what is the meter conformity assessment and classification during sampling inspection?

C.5 For these examples, assume specification limit of +/- 1.00 (% error) and sampling under LQ of 3.15% plan, with compressed specification limit of +/- 0.8350 and MADT (average) limit of 0.5000. Assume small lot size (zero acceptance numbers) and uncertainty value of 0.10 (k = 1.6449) and 0.18 (k = 3.0000).

Note: No sample meter or 100% inspected meter can be placed into service unless it meets all specified requirements.

C.6 Questions and answers

Q.2.1 Meter with three test points, HL, LL and power factor (PF), had errors: +0.40, +0.50, +0.95; all non-performance tests rated as acceptable.

A.2.1 Meter has type 1 and type 2 marginal conformities and one nonconformance; meter rated as a performance nonconforming unit and counted as a "dp"; meter cannot be sealed.

Q.2.2 Meter with three test points, HL, LL and PF, had errors: +0.40, +0.50, +0.95; non-performance tests - incorrect name plate markings.

A.2.2 Meter has type 1 and type 2 marginal conformities and two nonconformances; meter rated as one performance nonconforming unit and counted as a "dp"; meter cannot be sealed.

Q.2.3 Meter with three test points, HL, LL and PF had errors: +0.40, +0.60, +0.60; non-performance tests - incorrect name plate markings.

A.2.3 Meter has type 2 marginal conformity and one nonconformance; meter rated as one non-performance nonconforming unit and counted as a "dnp"; meter cannot be sealed.

Q.2.4 Meter with three test points, HL, LL and PF, had errors - +0.50, +0.60, +0.75; all non-performance tests rated as acceptable.

A.2.4 Meter has type 1 and type 2 marginal conformities; meter rated as a marginally conforming unit and counted as a "type 1"; meter cannot be sealed as values over test limit criteria.

Q.2.5 Meter with three test points, HL, LL and PF, had errors - 0.00, +0.20, +0.10; all non-performance tests rated as acceptable.

A.2.5 Meter rated as conforming unit.

Appendix D - Selected Screening Inspection Questions and Answers Based on S-S-04, S-ES-E-02 and S-G-02

D.1 Questions and answers

Q.1 Electricity meters are processed as a series of lots under S-S-04, Annex B. A lot of 100 electricity meters undergo screening inspection. What should the number of marginally nonconforming units in the previously screened lot be so that product of (LQ * N) is not exceeded?

Assume +/- 1.00% specification limit and console displays to 2 decimal places.

A.1a Scenario A - LQ = 3.15% plan, expanded uncertainty of 0.10 (k = 1.6449) only (0.0315 x 100) = 3 meters permitted with an observation falling in range of +/- (0.83 - 0.10) to (1.00 – 0.10).

That is, at least 97 meter observations fall within +/- 0.73% and balance fall within +/- 0.90%.

A.1b Scenario B - LQ = 8.0% plan, expanded uncertainty of 0.10 (k = 1.6449) only (0.08 x 100) = 8 meters permitted with an observation falling in range of +/- (0.67 - 0.10) to (1.00 – 0.10).

That is, at least 92 meter observations fall within +/- 0.57% and balance fall within +/- 0.90%.

Under both scenarios, only (0.20 X 100)= 20 meters permitted with MADT observation exceeding 0.50%. Refer to S-E-02 to determine whether sample meters can be sealed.

Q.2 Gas diaphragm meters are processed as a series of lots under S-S-04, Annex B. A lot of 100 meters undergo screening inspection. What should the number of marginally nonconforming units in the previously screened lot be so that product of (LQ * N) is not exceeded?

Assume +/- 1.60% specification limit and console displays to 2 decimal places.

A.2a Scenario A - LQ = 3.15% plan, expanded uncertainty of 0.30 (k = 1.6449) only (0.0315 x 100) = 3 meters permitted with an observation falling in range of +/- (1.33 - 0.30) to (1.60 – 0.30).

That is, at least 97 meter observations fall within +/- 1.03% and balance fall within +/- 1.30%

and only (0.20 X 100)= 20 meters permitted with MADT observation exceeding 0.80%.

A.2b Scenario B - LQ = 8.0% plan, expanded uncertainty of 0.30 (k = 1.6449) only (0.08 x 100) = 8 meters permitted with an observation falling in range of +/- (1.08 - 0.30) to (1.60 – 0.30).

That is, at least 92 meter observations fall within +/- 0.78%. The balance fall within +/- 1.30% and only (0.20 X 100)= 20 meters permitted with MADT observation exceeding 0.70%.

Under both scenarios, refer to S-G-02 to determine whether sample meters can be sealed.

Appendix E - Outgoing Quality Example Calculation

In the case of electronic electricity meters inspected under S-E-02 by 100% inspection: when processing meters there are situations in which the test limit requirements are insufficient to ensure the outgoing quality requirements of S-E-02, Annex A have been met. This occurs for example when the test limit equals the verification specification limits.

Example

100 meters are presented for 100% inspection under LQ = 3.15% plan. Assume expanded uncertainty of 0.10 (k = 1.6449) and 0.18 (k = 3.0000).

Consequently:

only (0.0315 x 100) = 3 meters are permitted with an observation falling in range of +/- (0.83 - 0.10) to (1.00 – 0.18)

At least 97 meters observations fall within +/- 0.73% and balance fall within +/- 0.82% and all meters have MADT observations within +/- 0.50%.