3.8. Annex C: modified questions in the process questionnaire#

The questionnaire is performed by a list of questions applicable for each phase of the development and manufacturing in order to estimate the process factor \(\Pi_{\text{process}}\) for quality and technical control over reliability in the product life cycle as defined in Section Section 3.4.1. However, some of the questions are not perfectly applicable to space applications. This is why this present annex contains the modified questions of the questionnaire adapted to space applications according to the content of Section Section 3.4.3.3.4. Recommendations for the satisfaction level of the modified and non-modified questions of the questionnaire are proposed in Section 3.7.

Questions n°14, n°71 and n°114 are modified only for the “Operation and maintenance” phase and are not modified for the other phases. Other questions are modified for all phases of use.

The question n°14 is modified for the “Operation and maintenance” phase according to the following table. It is not modified for “Integration into equipment” and “Integration into system” phases.

Table 3.8.1 Modified question n°14 for Operation and Maintenance.#

Recommendation

Perform final inspections and tests on launch pad.

N°

14

Phases in which the recommendation is applicable

OPERATION AND MAINTENANCE

Weight

10.4

Further description

Perform all final inspections and tests on launch pad in accordance with the quality plan and/or written procedures. The quality plan and/or procedures for final inspections and tests on launch pad must require that all specified inspections and tests, including those defined for product reception, are carried out and that the results are conforming with requirements. Before shipment, make sure that: All activities specified in the quality plan and/or written procedures have been satisfactory accomplished, The associated data and documentation are available and accepted.

Audit question

Have all final inspections and tests on launch pad been carried out in accordance with the quality plan and/or written procedures?

Level 1

No final inspection or test on launch pad.

Level 2

Final inspections and tests on launch pad are carried out, but they are not described in strict procedures or in a quality plan.

Level 3

Final inspections and tests on launch pad are carried out, They are described in strict procedures or in a quality plan.

Level 4

Final inspections and tests on launch pad are carried out in accordance with the quality plan and/or written procedures. The quality plan and/or procedures for final inspections and tests on launch pad require that all specified inspections and tests, including those specified for product reception or during its manufacturing, are done and that the results are conforming with the requirements. It is checked before shipment that: All activities specified in the quality plan and/or in written procedures have been satisfactorily accomplished Data and the associated documentation are available (follower sheet type document that records the configuration, operations carried out and observed anomalies) and accepted.

The question n°28 is modified for the “Specification” phase according to the following table.

Table 3.8.2 Modified question n°28.#

Recommendation

Define product failure.

N°

28

Phases in which the recommendation is applicable

SPECIFICATION

Weight

10.3

Further description

Precisely define what will be considered as a product failure (possibilities of acceptable degraded modes).

Audit question

What is considered as a product failure?

Level 1

No description of the product failure was defined in the contract. The customer has not given any list of feared events. The customer has not defined any degraded mode. The manufacturer has not defined these elements for his study.

Level 2

The manufacturer has produced the description of the product failure and (or) the list of feared events, and (or) product degraded modes without formal validation by the customer.

Level 3

The manufacturer has produced the description of the product failure and (or) the list of feared events, and (or) product degraded modes with formal validation by the customer.

Level 4

Product failures are perfectly identified in the contract. The list of feared events is supplied in the contract. Degraded modes are also described in the contract.

The question n°39 is modified for the “Design” phase according to the following table.

Table 3.8.3 Modified question n°39.#

Recommendation

Have and manage a table of skills required by activity, including names.

N°

39

Phases in which the recommendation is applicable

DESIGN

Weight

24.5

Further description

Make sure that the skills required for an activity are assigned to the operators and periodically verify that training is appropriate for the activities.

Audit question

Is there any management of skills?

Level 1

No monitoring of suitability for training.

Level 2

The operators have skills required for an activity but are not monitored in training

Level 3

The operators have training required for an activity, but there is no periodic verification that training is suitable for the activities.

Level 4

Training is periodically monitored and updated. There are regular evaluations of the suitability of training to satisfy the company’s objectives.

The question n°44 is modified for the “Design” phase according to the following table.

Table 3.8.4 Modified question n°44.#

Recommendation

Write a collection of business recommendations dealing with manipulation and storage operations at the user.

N°

44

Phases in which the recommendation is applicable

DESIGN

Weight

7.7

Further description

Make sure that there is a list of business recommendations on manipulation and storage operations at the user. This collection must be enriched by feedback from operations.

Audit question

Is there a collection of business recommendations for manipulation and storage operations at the customer?

Level 1

No collection of recommendations and no procedures for processing of feedback from operations.

Level 2

There is non formalised and unmanaged collection of recommendations. Feedback from operations is dealt but not systematically.

Level 3

There is a formalised collection of recommendations but it is not necessarily applicable to the project (not referenced to the project) and is not validated. Feedback from operations formalised in a base that is not managed and is not much used in design.

Level 4

Formalised collection of recommendations, validated and referenced to the project. Formalised and validated feedback from operations, referenced to the project, useable and acting as design input data to improve reliability.

The question n°61 is modified for the “Design”, “Manufacturing of board or subassembly” and “Integration into equipment” phases according to the following table.

Table 3.8.5 Modified question n°61.#

Recommendation

Identify and implement means of protecting subassemblies.

N°

61

Phases in which the recommendation is applicable

DESIGN, MANUFACTURING OF BOARD OR SUBASSEMBLY, INTEGRATION INTO EQUIPMENT

Weight

7.3, 7.3, 7.3

Further description

List and implement protection means to avoid reducing the reliability of the subassembly.

Audit question

Have means of protecting subassemblies during some production activities been identified and implemented?

Level 1

No particular protection means is identified.

Level 2

Protection means are identified but are partially applied in different activities.

Level 3

Protection means are identified and their application is verified.

Level 4

Protection means are identified following management of lessons-learnt and experience and their application is verified.

The question n°68 is modified for the “Support process activities” phase according to the following table.

Table 3.8.6 Modified question n°68.#

Recommendation

Identify risks related to Reliability at subcontractors.

N°

68

Phases in which the recommendation is applicable

SUPPORT PROCESS ACTIVITIES

Weight

7.2

Further description

During the design phase (with the subcontractor) identify risks associated with the reliability of the subcontracted product.

Audit question

Have risks related to reliability of products at subcontractors been identified?

Level 1

No reliability risk analysis is made with the subcontractor during the design phase (no specific provision)

Level 2

No reliability risk analysis is made with the subcontractor during the design phase, but a risk identification was made during the job. These risks are not managed.

Level 3

The reliability risk analysis was made during the design phase and is explained in a non-formal analysis. These risks are not managed.

Level 4

The reliability risk analysis was made during the design phase and is described in a formal document. Identified risks are regularly updated.

The question n°71 is modified only for the “Operation and maintenance” phase according to the following table. This question is not modified for the “Integration into equipment” and “Integration into system” phases.

Table 3.8.7 Modified question n°71 for Operation and Maintenance.#

Recommendation

Control documentation.

N°

71

Phases in which the recommendation is applicable

OPERATION AND MAINTENANCE

Weight

5.6

Further description

Store and keep product and process documentation available during operation process. Keep the list of documentation updated with respect to evolutions (ex: number of reconfigurations, number of station keeping manoeuvers, …) Train operators to follow instructions and recommendations. The whole documentation related to operations has to be up to date and available for operators. When documents are provided, analyse the validity of this product documentation. Be in possession of process control documentation. Specify technical documentation for each process (ex: avoidance manoeuvers, reconfiguration from ground, …). Provide this technical documentation and make it usable.

Audit question

Is documentation well controlled? Does it take account of all operational configuration changes?

Level 1

No documentation specific to the operational configuration or processes, there are no plans to provide special documentation.

Level 2

Documentation specific to the operational configuration or processes does exist, however it is not always effectively updated, there is no analysis about the validity of documents.

Level 3

Documentation specific to the operational configuration or processes does exist, it is updated periodically in a planned manner, the validity of the documents used is not analysed.

Level 4

Documentation specific to the operational configuration or processes does exist, it is updated periodically in a planned manner, the validity of the documents used is analysed. Precise procedures are applied for storage and preservation of the documentation.

The question n°72 is modified for the “Operation and maintenance” phase according to the following table.

Table 3.8.8 Modified question n°14 for Operation and Maintenance.#

Recommendation

Control product monitoring and maintainability in operation.

N°

72

Phases in which the recommendation is applicable

OPERATION AND MAINTENANCE

Weight

17.6

Further description

Control the capability of products to detect failures, control failure detection means, allow maintainability in operation.

Audit question

How are product monitoring and maintainability in operation controlled?

Level 1

No monitoring or maintainability performed

Level 2

There is a minimum surveillance by alarms

Level 3

The FDIR process is mainly automatic.

Level 4

The FDIR process requires both automatic reaction and manual intervention from ground. The FDIR principles are described in the FDIR User Manual. The operators have been trained for such procedures.

The question n°76 is modified for the “Support process activities” phase according to the following table.

Table 3.8.9 Modified question n°76.#

Recommendation

Control monitoring and measurement devices, metrology of measurement instruments and industrial means.

N°

76

Phases in which the recommendation is applicable

SUPPORT PROCESS ACTIVITIES

Weight

7.8

Further description

Control monitoring and measurement devices, the metrology of measurement instruments and industrial means. Control the verification, calibration and rating of measurement instruments and test and trial benches used by the company. APMs are tied to national standards.

Audit question

What procedure is there to control monitoring and measurement devices, the metrology of measurement instruments and industrial means?

Level 1

There is no verification, calibration and rating procedure for measurement instruments and test and trial benches in the company.

Level 2

There is a verification, calibration and rating procedure for measurement instruments and test and trial benches in the company, but without any evidence of application.

Level 3

There is a verification, calibration and rating procedure for measurement instruments and test and trial benches in the company, and it is applied.

Level 4

Verification, calibration and rating of measurement instruments and test and trial benches used by the company is controlled (accreditation, certification, etc.); APMs are tied to national standards.

The question n°104 is modified for the “Support process activities” phase according to the following table.

Table 3.8.10 Modified question n°104.#

Recommendation

Appoint a person responsible for reliability studies.

N°

104

Phases in which the recommendation is applicable

SUPPORT PROCESS ACTIVITIES

Weight

8.5

Further description

Appoint a person responsible for reliability for each project, who will guarantee that product reliability objectives are achieved. This person shall report on progress with studies and problems encountered.

Audit question

Has a person responsible for reliability studies been appointed?

Level 1

No person responsible for reliability studies has been identified.

Level 2

There is a person responsible for reliability studies in practice, but no record of his appointment is available or does not have been trained or does not have experience in the reliability field.

Level 3

A reliability study manager has been formally named, he has been trained and has experience in the reliability field.

Level 4

A reliability study manager has been named and integrated into the project, he is trained and has the required experience in the field. He makes regular reports on progress with studies in meetings or in reports.

The question n°114 is modified only for the “Operation and maintenance” phase according to the following table. This question is not modified for the “Integration into equipment” and “Integration into system” phases.

Table 3.8.11 Modified question n°114 for Operation and Maintenance.#

Recommendation

Possess a check up procedure.

N°

114

Phases in which the recommendation is applicable

OPERATION AND MAINTENANCE

Weight

5.7

Further description

The state of the satellite is regularly checked through tests implemented in the operation procedure.

Audit question

Is the state of the satellite (and its subsystems) checked?

Level 1

No check is scheduled and performed.

Level 2

No check is scheduled but can be performed if deemed necessary.

Level 3

Checks are scheduled and performed following a procedure. There is no recordings of what is done.

Level 4

Checks are scheduled, performed following a procedure. Every intervention is documented. The operator is trained for this procedure.

The question n°157 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.12 Modified question n°157 for Ruggedising.#

Recommendation

Provide training and manage maintenance of skills for implementation of the product

N°

157

Phases in which the recommendation is applicable

RUGGEDISING

Weight

7

Further description

Train users to make sure that the product is always used correctly

Audit question

Have users (for use) received training on the product? Is this training repeated and updated to satisfy needs?

Level 1

No training associated with the product

Level 2

There is some initial training or training of only some users

Level 3

There is a complete training but with no skills management

Level 4

There is a complete training. Skills management assures that all users have been trained and that the training is up to date

The question n°158 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.13 Modified question n°158 for Ruggedising.#

Recommendation

Check that procedures specific to the product and rules specific to businesses are respected by an appropriate monitoring system

N°

158

Phases in which the recommendation is applicable

RUGGEDISING

Weight

7

Further description

Implement means to supervise and control users in use of the product, to be able to identify deviations and deal with them.

Audit question

Are inspection means (process, recording means) sufficient for the supplier to assure that rules for the use of the product are well respected by users?

Level 1

No inspection means

Level 2

Existence of a few monitoring and inspection means

Level 3

Existence of non-exhaustive or informally used inspection means. Deviations are not systematically dealt with.

Level 4

There are complete and formalised inspection means. Deviations are dealt with.

The question n°159 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.14 Modified question n°159 for Ruggedising.#

Recommendation

Design dependable electrical protection devices.

N°

159

Phases in which the recommendation is applicable

RUGGEDISING

Weight

4

Further description

Identify protection needs. Design electrical protection devices. Make sure that they are testable.

Audit question

How are electrical protection devices designed?

Level 1

Operating dependability principles are not applied to electrical protection devices

Level 2

Operating dependability principles are applied to electrical protection devices in some cases

Level 3

Operating dependability principles are applied to electrical protection devices. There are verifications that these devices are operating correctly throughout the life of the product.

Level 4

Operating dependability principles are applied to electrical protection devices. There are verifications that these devices are operating correctly throughout the life of the product. This approach is described in a procedure.

The question n°160 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.15 Modified question n°160 for Ruggedising.#

Recommendation

Study and handle risks of the product under test being deteriorated by failures of its test means

N°

160

Phases in which the recommendation is applicable

RUGGEDISING

Weight

4

Further description

Minimise the risk of deterioration by taking it into account in the design of the test means and the tested unit, develop appropriate prevention means.

Audit question

How are risks of the product under test being deteriorated by failure of its test means dealt with?

Level 1

No study of failures of the test means

Level 2

Some known failures are taken into account

Level 3

These risks are analysed in the design of the test means and unit tested. Appropriate prevention means are set up.

Level 4

These risks are analysed in the design of the test means and unit tested. Appropriate prevention means are set up. This approach is described in a procedure.

The question n°161 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.16 Modified question n°161 for Ruggedising.#

Recommendation

Identify and use appropriate prevention means of preventing reasonably predictable aggressions

N°

161

Phases in which the recommendation is applicable

RUGGEDISING

Weight

4

Further description

Search and prevent effects of reasonably predictable aggressions (UV, light, radiations, single atomic oxygen, etc.)

Audit question

Are reasonably predictable aggressions taken into account?

Level 1

Aggressions are not taken into account

Level 2

Well known aggressions are taken into account

Level 3

Aggression cases are searched for and are taken into account

Level 4

Aggression cases are searched for and are taken into account. This approach is described in a procedure or a standard

The question n°163 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.17 Modified question n°163 for Ruggedising.#

Recommendation

Include production and storage environments in the product environment specifications

N°

163

Phases in which the recommendation is applicable

RUGGEDISING

Weight

4

Further description

Extend environment specifications so that they also cover production and storage situations (and not only usage cases). The controlling parameter for storage could be the duration.

Audit question

How are production and storage environments taken into account in the product environment specification?

Level 1

Production and storage environments are not specified

Level 2

Production and storage environments are taken into account if they are known

Level 3

Production and storage environments are taken into account in environment specifications

Level 4

Production and maintenance environments are systematically taken into account. Environments are formalised in a documentation

The question n°167 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.18 Modified question n°167 for Ruggedising.#

Recommendation

Carry out a process analysis of implementation operations

N°

167

Phases in which the recommendation is applicable

RUGGEDISING

Weight

4

Further description

Carry out a process FMECA (Failure Modes, Effects and Criticality Analysis) for implementation operations

Audit question

How are risks of errors in carrying out implementation operations analysed?

Level 1

No analysis of risks of errors in carrying out operations

Level 2

Well known anomalies and drifts are collected and used in the design of the product

Level 3

The process FMECA for implementation operations is done in some cases and is used for the product design

Level 4

The process FMECA for implementation operations is systematically done and is used for the product design

The question n°169 is modified for the “Ruggedising” phase according to the following table.

Table 3.8.19 Modified question n°169 for Ruggedising.#

Recommendation

Write complete procedures for all product implementation operations

N°

169

Phases in which the recommendation is applicable

RUGGEDISING

Weight

7

Further description

Provide users with documentation describing procedures to be applied for each situation

Audit question

Is there any documentation that describes all product implementation operations?

Level 1

No documentation, or superficial documentation

Level 2

There is documentation covering some of the needs

Level 3

There is complete documentation, but its exhaustiveness cannot be proven

Level 4

The documentation exists and is complete. The composition of the documentation is described in a procedure that guarantees that it is complete