Discussion Post – Ethical Dilemma

Discussion Post – Ethical Dilemma

Discussion Post – Ethical Dilemma

Discussion post – 2 to 3 paragraphs

In this discussion, you will debate an ethical dilemma around test development while relating it to validity and reliability concepts.

ORDER ORIGINAL, PLAGIARISM-FREE ESSAY PAPERS HERE

Consider the following scenario: A private school utilizes a test with well-established predictive validity for admissions. However, members of a particular group tend to score low on this test. In your initial post, answer the following question:

  • Is it ethical for the school to continue to use the test? Why or why not?

Attached are 2 articles that can be used to assist complete the assignment.

  • attachment

    0013164410391782.pdf
  • attachment

    jgme-d-11-00075.1.pdf

    A Primer on the Validity of

    Assessment Instruments Gail M. Sullivan, MD, MPH

    1. What is reliability?1

    Reliability refers to whether an assessment instrument gives

    the same results each time it is used in the same setting with

    the same type of subjects. Reliability essentially means

    consistent or dependable results. Reliability is a part of the

    assessment of validity. Discussion Post – Ethical Dilemma

    2. What is validity?1

    Validity in research refers to how accurately a study answers

    the study question or the strength of the study conclusions.

    For outcome measures such as surveys or tests, validity refers

    to the accuracy of measurement. Here validity refers to how

    well the assessment tool actually measures the underlying

    outcome of interest. Validity is not a property of the tool

    itself, but rather of the interpretation or specific purpose of

    the assessment tool with particular settings and learners.

    Assessment instruments must be both reliable and valid for

    study results to be credible. Thus, reliability and validity must be

    examined and reported, or references cited, for each assessment

    instrument used to measure study outcomes. Examples of

    assessments include resident feedback survey, course evaluation,

    written test, clinical simulation observer ratings, needs

    assessment survey, and teacher evaluation. Using an instrument

    with high reliability is not sufficient; other measures of validity

    are needed to establish the credibility of your study.

    3. How is reliability measured?2–4

    Reliability can be estimated in several ways; the method will

    depend upon the type of assessment instrument. Sometimes

    reliability is referred to as internal validity or internal

    structure of the assessment tool.

    For internal consistency 2 to 3 questions or items are

    created that measure the same concept, and the difference

    among the answers is calculated. That is, the correlation

    among the answers is measured.

    Cronbach alpha is a test of internal consistency and

    frequently used to calculate the correlation values among

    the answers on your assessment tool.5 Cronbach alpha

    calculates correlation among all the variables, in every

    combination; a high reliability estimate should be as close to

    1 as possible.

    For test/retest the test should give the same results each

    time, assuming there are no interval changes in what you are

    measuring, and they are often measured as correlation, with

    Pearson r.

    Test/retest is a more conservative estimate of reliability

    than Cronbach alpha, but it takes at least 2 administrations

    of the tool, whereas Cronbach alpha can be calculated after

    a single administration. To perform a test/retest, you must

    be able to minimize or eliminate any change (ie, learning) in

    the condition you are measuring, between the 2

    measurement times. Administer the assessment instrument

    at 2 separate times for each subject and calculate the

    correlation between the 2 different measurements.

    Interrater reliability is used to study the effect of

    different raters or observers using the same tool and is

    generally estimated by percent agreement, kappa (for binary

    outcomes), or Kendall tau.

    Another method uses analysis of variance (ANOVA) to

    generate a generalizability coefficient, to quantify how

    much measurement error can be attributed to each potential

    factor, such as different test items, subjects, raters, dates of

    administration, and so forth. This model looks at the overall

    reliability of the results.6

    5. How is the validity of an assessment instrument determined?4,7,8

    Validity of assessment instruments requires several sources

    of evidence to build the case that the instrument measures

    what it is supposed to measure.9,10 Determining validity can

    be viewed as constructing an evidence-based argument

    regarding how well a tool measures what it is supposed to

    do. Evidence can be assembled to support, or not support, a

    specific use of the assessment tool. Evidence can be found in

    content, response process, relationships to other variables,

    and consequences.

    Content includes a description of the steps used to

    develop the instrument. Provide information such as who

    created the instrument (national experts would confer

    greater validity than local experts, who in turn would have

    more validity than nonexperts) and other steps that support

    the instrument has the appropriate content.

    Response process includes information about whether

    the actions or thoughts of the subjects actually match the

    test and also information regarding training for the raters/

    observers, instructions for the test-takers, instructions for

    scoring, and clarity of these materials.

    Relationship to other variables includes correlation of

    the new assessment instrument results with other

    performance outcomes that would likely be the same. If

    Gail M. Sullivan, MD, MPH, is Editor-in-Chief, Journal of Graduate Medical Education.

    Corresponding author: Gail M. Sullivan, MD, MPH, Editor-in-Chief, Journal of Graduate Medical Education, 515 N State St, Suite 2000, gsullivan@nso1.uchc.edu

    DOI: 10.4300/JGME-D-11-00075.1

    E D I T O R I A L

    Journal of Graduate Medical Education, June 2011 119

     

     

    there is a previously accepted ‘‘gold standard’’ of

    measurement, correlate the instrument results to the

    subject’s performance on the ‘‘gold standard.’’ In many

    cases, no ‘‘gold standard’’ exists and comparison is made to

    other assessments that appear reasonable (eg, in-training

    examinations, objective structured clinical examinations,

    rotation ‘‘grades,’’ similar surveys).

    Consequences means that if there are pass/fail or cut-off

    performance scores, those grouped in each category tend to

    perform the same in other settings. Also, if lower performers

    receive additional training and their scores improve, this

    would add to the validity of the instrument.

    Different types of instruments need an emphasis on

    different sources of validity evidence.7 For example, for

    observer ratings of resident performance, interrater agreement

    may be key, whereas for a survey measuring resident stress,

    relationship to other variables may be more important. For a

    multiple choice examination, content and consequences may

    be essential sources of validity evidence. For high-stakes

    assessments (eg, board examinations), substantial evidence to

    support the case for validity will be required.9

    There are also other types of validity evidence, which

    are not discussed here.

    6. How can researchers enhance the validity of their assessment instruments? First, do a literature search and use previously developed

    outcome measures. If the instrument must be modified for use

    with your subjects or setting, modify and describe how, in a

    transparent way. Include sufficient detail to allow readers to

    understand the potential limitations of this approach.

    If no assessment instruments are available, use content

    experts to create your own and pilot the instrument prior to

    using it in your study. Test reliability and include as many

    sources of validity evidence as are possible in your paper.

    Discuss the limitations of this approach openly.

    7. What are the expectations of JGME editors regarding assessment instruments used in graduate medical education research? JGME editors expect that discussions of the validity of your

    assessment tools will be explicitly mentioned in your

    manuscript, in the methods section. If you are using a

    previously studied tool in the same setting, with the same

    subjects, and for the same purpose, citing the reference(s) is

    sufficient. Additional discussion about your adaptation is

    needed if you (1) have modified previously studied

    instruments; (2) are using the instrument for different

    settings, subjects, or purposes; or (3) are using different

    interpretation or cut-off points. Discuss whether the

    changes are likely to affect the reliability or validity of the

    instrument.

    Researchers who create novel assessment instruments

    need to state the development process, reliability measures,

    pilot results, and any other information that may lend

    credibility to the use of homegrown instruments.

    Transparency enhances credibility. Discussion Post – Ethical Dilemma

    In general, little information can be gleaned from single-

    site studies using untested assessment instruments; these

    studies are unlikely to be accepted for publication.

    8. What are useful resources for reliability and validity of assessment instruments?

    The references for this editorial are a good starting

    point.

    References

    1 American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association; 1999.

    2 Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004;38(9):1006–1012.

    3 Beckman TJ, Ghosh AK, Cook DA, Erwin PJ, Manderkar JN. How reliable are assessments of clinical teaching?: a review of the published instruments. J Gen Intern Med. 2004;19(9):971–977.

    4 Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments. Am J Med. 2006;119(2):166e7–166e16.

    5 Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ. 1997;314:572.

    6 Brennan RL. Generalizability Theory. New York, NY: Springer-Verlag; 2001. 7 Downing SM. Validity: on the meaningful interpretation of assessment

    data. Med Educ. 2003;37(9):830–837. 8 Downing SM, Haldyna TM. Validity threats: overcoming interference

    with proposed interpretations of assessment data. Med Educ. 2004;38(3):327–333.

    9 Kane M. Validating high-stakes testing programs. Educ Meas Issues Pract. 2002;1:31–41.

    10 Kane M. The assessment of professional competence. Eval Health Prof. 1992;15(2):163–182.

    E D I T O R I A L

    120 Journal of Graduate Medical Education, June 2011

     

     

    T he following are corrections to the June 2011 issue.

    1. Sullivan GM. A primer on the validity of assessment

    instruments. J Grad Med Educ. 2011;3(2):119–120.

    On p 119, the sentence should read: Cronbach alpha

    calculates correlation among all the variables, in every

    combination, and generates one number that the closer it is

    to 1, the higher the reliability estimate.

    2. Salem JK, Jones RR, Sweet DB, Hasan S, Torregosa-

    Arcay H, Clough L. Improving care in a resident practice

    for patients with diabetes. J Grad Med Educ.

    2011;3(2):196–202.

    The Figure legends should read:

    Figure 1 Description of Sample Selection for Outcomes

    Analysis

    Figure 2 Timeline for Implementation of Interventions

    3. Saeed F, Majeed MH, Kousar N. Easing international

    medical graduates entry into us training. J Grad Med Educ.

    2011;3(2):269.

    The lead author’s name is Fahad Saeed, MD.

    4. Sweeney A, Stephany A, Whicker S, Bookman J,

    Turner DA. Senior Pediatric Residents as Teachers for an

    Innovative Multidisciplinary Mock Code Curriculum.

    J Grad Med Educ. 2011;3(2):188–195.

    The Figure 3 label for the seventh column is:

    Communicating Effectively.

    5. Le-Bucklin KT, Hicks R, Wong A. Impact of a

    Teaching Rotation on Residents’ Attitudes Toward

    Teaching: A 5-Year Study. J Grad Med Educ.

    2011;3(2):253–255. Discussion Post – Ethical Dilemma

    The Results section of the Abstract should read:

    Results: Four categories showed significant

    improvement, including feeling prepared to teach

    (P , .0001), having confidence in their teaching ability

    (P , .0001), being aware of their expectations as a teacher

    (P , .0001), and feeling that their anxiety about teaching

    was at a healthy level (P 5 .0037). There was an increase

    in the level of enthusiasm, but the P value did not reach a

    significant range (P 5 .12). The level of enthusiasm started

    high and was significantly higher on the pretest than every

    other tested category (P , .0001).

    Footnote c to Table 2 should read: P value as calculated

    using the Mann-Whitney U test.

    E R R A T A

    446 Journal of Graduate Medical Education, September 2011

     

    << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Dot Gain 30%) /CalRGBProfile (None) /CalCMYKProfile (U.S. Sheetfed Coated v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Error /CompatibilityLevel 1.4 /CompressObjects /Off /CompressPages true /ConvertImagesToIndexed false /PassThroughJPEGImages true /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.1000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize false /OPM 1 /ParseDSCComments false /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Remove /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 150 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 600 /ColorImageDepth 8 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /FlateEncode /AutoFilterColorImages false /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.40 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 600 /GrayImageDepth 8 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /FlateEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.40 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly true /PDFXNoTrimBoxError false /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox false /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (Euroscale Coated v2) /PDFXOutputConditionIdentifier (FOGRA1) /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /False /CreateJDFFile false /SyntheticBoldness 1.000000 /Description << /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e002c00200075006d002000650069006e0065006e00200042006500720069006300680074002000fc00620065007200200064006900650020005000440046002f0058002d0033002d004b006f006d007000610074006900620069006c0069007400e4007400200065007200680061006c00740065006e00200075006e00640020005000440046002d0044006f006b0075006d0065006e007400650020006e00750072002000640061006e006e0020007a0075002000650072007300740065006c006c0065006e002c002000770065006e006e0020007300690065002000fc0062006500720020006400690065007300650020004b006f006d007000610074006900620069006c0069007400e400740020007600650072006600fc00670065006e002e0020005000440046002f00580020006900730074002000650069006e0065002000490053004f002d004e006f0072006d0020007a0075006d002000410075007300740061007500730063006800200076006f006e0020006400690067006900740061006c0065006e00200044007200750063006b0076006f0072006c006100670065006e002e0020005700650069007400650072006500200049006e0066006f0072006d006100740069006f006e0065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e0020005000440046002f0058002d0033002d006b006f006d00700061007400690062006c0065006e0020005000440046002d0044006f006b0075006d0065006e00740065006e002000660069006e00640065006e002000530069006500200069006d0020004100630072006f006200610074002d00480061006e00640062007500630068002e00200044006900650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f0062006100740020006f0064006500720020006d00690074002000640065006d002000520065006100640065007200200034002e003000200075006e00640020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /FRA <FEFF004f007000740069006f006e00730020007000650072006d0065007400740061006e007400200064002700e900760061006c0075006500720020006c006100200063006f006e0066006f0072006d0069007400e9002000e00020006c00610020006e006f0072006d00650020005000440046002f0058002d003300200065007400200064006500200063006f006e0064006900740069006f006e006e006500720020006c0061002000700072006f00640075006300740069006f006e00200064006500200064006f00630075006d0065006e007400730020005000440046002000e000200063006500740074006500200063006f006e0066006f0072006d0069007400e9002e0020005000440046002f0058002000650073007400200075006e00650020006e006f0072006d0065002000490053004f00200064002700e9006300680061006e0067006500200064006500200063006f006e00740065006e00750020006700720061007000680069007100750065002e00200050006f0075007200200065006e0020007300610076006f0069007200200070006c0075007300200073007500720020006c006100200063007200e9006100740069006f006e00200064006500200064006f00630075006d0065006e00740073002000500044004600200063006f006e0066006f0072006d00650073002000e00020005000440046002f0058002d0033002c00200063006f006e00730075006c00740065007a0020006c00650020004700750069006400650020006400650020006c0027007500740069006c0069007300610074006500750072002000640027004100630072006f006200610074002e00200049006c002000650073007400200070006f0073007300690062006c0065002000640027006f00750076007200690072002000630065007300200064006f00630075006d0065006e007400730020005000440046002000640061006e00730020004100630072006f0062006100740020006500740020005200650061006400650072002c002000760065007200730069006f006e00200034002e00300020006f007500200075006c007400e9007200690065007500720065002e> /JPN <FEFF005000440046002f0058002d00336e9662e0306e30ec30dd30fc30c87528304a30883073658766f84f5c62107528306b4f7f75283057307e30593002005000440046002f00580020306f30b030e930d530a330c330af30b3002030f330c630f330c4590963db306b304a3051308b002000490053004f00206a196e96306730593002005000440046002f0058002d003300206e9662e0306e658766f84f5c6210306b306430443066306f0020004100630072006f006200610074002030e630fc30b630ac30a430c9309253c2716730573066304f30603055304430024f5c62103057305f00200050004400460020306f0020004100630072006f0062006100740020304a30883073002000520065006100640065007200200034002e003000204ee5964d30678868793a3067304d307e30593002> /PTB <FEFF005500740069006c0069007a006500200065007300740061007300200063006f006e00660069006700750072006100e700f5006500730020007000610072006100200063007200690061007200200064006f00630075006d0065006e0074006f007300200050004400460020006500200065006d0069007400690072002000720065006c0061007400f300720069006f007300200073006f00620072006500200063006f006e0066006f0072006d0069006400610064006500200063006f006d0020006f0020005000440046002f0058002d0033002e0020005000440046002f0058002000e900200075006d0020007000610064007200e3006f002000640061002000490053004f00200070006100720061002000740072006f0063006100200064006500200063006f006e0074006500fa0064006f00200067007200e1006600690063006f002e002000500061007200610020006f00620074006500720020006d00610069007300200069006e0066006f0072006d006100e700f50065007300200073006f00620072006500200063006f006d006f00200063007200690061007200200064006f00630075006d0065006e0074006f0073002000500044004600200065006d00200063006f006e0066006f0072006d0069006400610064006500200063006f006d0020006f0020005000440046002f0058002d0033002c00200063006f006e00730075006c007400650020006f0020004700750069006100200064006f002000550073007500e100720069006f00200064006f0020004100630072006f006200610074002e0020004f007300200064006f00630075006d0065006e0074006f0073002000500044004600200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002c002000520065006100640065007200200034002e00300020006500200070006f00730074006500720069006f0072002e> /DAN <FEFF004200720075006700200064006900730073006500200069006e0064007300740069006c006c0069006e006700650072002000740069006c00200061007400200072006100700070006f007200740065007200650020006f006d0020006f0076006500720068006f006c00640065006c007300650020006100660020005000440046002f0058002d00330020006f00670020006b0075006e002000700072006f0064007500630065007200650020005000440046002d0064006f006b0075006d0065006e007400650072002c002000680076006900730020006400650020006f0076006500720068006f006c0064006500720020007300740061006e00640061007200640065006e002e0020005000440046002f005800200065007200200065006e002000490053004f002d007300740061006e0064006100720064002000740069006c00200075006400760065006b0073006c0069006e0067002000610066002000670072006100660069006b0069006e00640068006f006c0064002e00200059006400650072006c006900670065007200650020006f0070006c00790073006e0069006e0067006500720020006f006d0020006f007000720065007400740065006c007300650020006100660020005000440046002d0064006f006b0075006d0065006e007400650072002c00200064006500720020006f0076006500720068006f006c0064006500720020005000440046002f0058002d0033002c002000660069006e00640065007200200064007500200069002000620072007500670065007200760065006a006c00650064006e0069006e00670065006e002000740069006c0020004100630072006f006200610074002e0020005000440046002d0064006f006b0075006d0065006e007400650072006e00650020006b0061006e002000e50062006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200034002e00300020006f00670020006e0079006500720065002e> /NLD <FEFF004700650062007200750069006b002000640065007a006500200069006e007300740065006c006c0069006e00670065006e0020006f006d00200064006500200063006f006d007000610074006900620069006c006900740065006900740020006d006500740020005000440046002f0058002d003300200074006500200063006f006e00740072006f006c006500720065006e00200065006e00200061006c006c00650065006e0020005000440046002d0064006f00630075006d0065006e00740065006e002000740065002000700072006f006400750063006500720065006e002000640069006500200063006f006d007000610074006900620065006c0020007a0069006a006e002e0020005000440046002f0058002000690073002000650065006e002000490053004f002d007300740061006e0064006100610072006400200076006f006f00720020006800650074002000750069007400770069007300730065006c0065006e002000760061006e002000670072006100660069007300630068006500200069006e0068006f00750064002e002000520061006100640070006c0065006500670020006400650020006700650062007200750069006b00650072007300680061006e0064006c0065006900640069006e0067002000760061006e0020004100630072006f00620061007400200076006f006f00720020006d00650065007200200069006e0066006f0072006d00610074006900650020006f00760065007200200068006500740020006d0061006b0065006e002000760061006e0020005000440046002d0064006f00630075006d0065006e00740065006e002000640069006500200063006f006d007000610074006900620065006c0020007a0069006a006e0020006d006500740020005000440046002f0058002d0033002e0020004400650020005000440046002d0064006f00630075006d0065006e00740065006e0020006b0075006e006e0065006e00200077006f007200640065006e002000670065006f00700065006e00640020006d006500740020004100630072006f00620061007400200065006e002000520065006100640065007200200034002e003000200065006e00200068006f006700650072002e> /ESP <FEFF0055007300650020006500730074006100730020006f007000630069006f006e00650073002000700061007200610020007200650061006c0069007a0061007200200075006e00200069006e0066006f0072006d006500200073006f0062007200650020006c006100200063006f006d007000610074006900620069006c006900640061006400200063006f006e0020005000440046002f0058002d003300200079002000670065006e006500720061007200200064006f00630075006d0065006e0074006f007300200050004400460020007300f3006c006f00200073006900200073006f006e00200063006f006d00700061007400690062006c00650073002e0020005000440046002f005800200065007300200075006e002000650073007400e1006e006400610072002000490053004f0020007000610072006100200065006c00200069006e00740065007200630061006d00620069006f00200064006500200063006f006e00740065006e00690064006f00200067007200e1006600690063006f002e002000500061007200610020006f006200740065006e006500720020006d00e1007300200069006e0066006f0072006d00610063006900f3006e00200061006300650072006300610020006400650020006300f3006d006f00200063007200650061007200200064006f00630075006d0065006e0074006f0073002000500044004600200063006f006d00700061007400690062006c0065007300200063006f006e0020005000440046002f0058002d0033002c00200063006f006e00730075006c007400650020006c006100200047007500ed0061002000640065006c0020007500730075006100720069006f0020006400650020004100630072006f006200610074002e0020004c006f007300200064006f00630075006d0065006e0074006f00730020005000440046002000730065002000700075006500640065006e00200061006200720069007200200063006f006e0020004100630072006f00620061007400200079002000520065006100640065007200200034002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /SUO <FEFF004e00e4006900640065006e002000610073006500740075007300740065006e0020006100760075006c006c006100200076006f006900740020006d00e400e4007200690074007400e400e40020005000440046002f0058002d0033002d00790068007400650065006e0073006f0070006900760075007500640065006e0020006a00610020006c0075006f00640061002000730065006e0020006d0075006b006100690073006900610020005000440046002d0061007300690061006b00690072006a006f006a0061002e0020005000440046002f00580020006f006e002000490053004f002d007300740061006e006400610072006400690073006f006900740075002000670072006100610066006900730065006e002000730069007300e4006c006c00f6006e0020006500730069007400790073006d0075006f0074006f002e0020004c0069007300e40074006900650074006f006a00610020005000440046002f0058002d0033002d00790068007400650065006e0073006f00700069007600690065006e0020005000440046002d0061007300690061006b00690072006a006f006a0065006e0020006c0075006f006e006e00690073007400610020006f006e002000410064006f006200650020004100630072006f0062006100740020002d006b00e400790074007400f6006f0070007000610061007300730061002e0020005000440046002d0061007300690061006b00690072006a0061007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f006200610074002d0020006a0061002000520065006100640065007200200034002e00300020002d006f0068006a0065006c006d0061006c006c0061002000740061006900200075007500640065006d006d0061006c006c0061002000760065007200730069006f006c006c0061002e> /ITA <FEFF00550073006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000760065007200690066006900630061007200650020006c006100200063006f006e0066006f0072006d0069007400e0002000610020005000440046002f0058002d003300200065002000630072006500610072006500200064006f00630075006d0065006e00740069002000500044004600200073006f006c006f00200069006e0020006300610073006f00200064006900200063006f006e0066006f0072006d0069007400e0002e0020005000440046002f0058002d0033002000e800200075006e006f0020007300740061006e0064006100720064002000490053004f00200070006500720020006c006f0020007300630061006d00620069006f00200064006900200063006f006e00740065006e00750074006f0020006700720061006600690063006f002e002000500065007200200075006c0074006500720069006f0072006900200069006e0066006f0072006d0061007a0069006f006e0069002000730075006c006c006100200063007200650061007a0069006f006e006500200064006900200064006f00630075006d0065006e00740069002000500044004600200063006f006e0066006f0072006d0069002000610020005000440046002f0058002d0033002c00200063006f006e00730075006c00740061007200650020006c0061002000470075006900640061002000640065006c006c0027007500740065006e007400650020006400690020004100630072006f006200610074002e0020004900200064006f00630075006d0065006e00740069002000500044004600200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000520065006100640065007200200034002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e500200072006100700070006f007200740065007200650020006f006d0020005000440046002f0058002d0033002d006b006f006d007000610074006900620069006c00690074006500740020006f00670020006c0061006700650020005000440046002d0064006f006b0075006d0065006e00740065007200200062006100720065002000680076006900730020006b006f006d007000610074006900620065006c002e0020005000440046002f005800200065007200200065006e002000490053004f002d007300740061006e006400610072006400200066006f00720020006700720061006600690073006b00200069006e006e0068006f006c006400730075007400760065006b0073006c0069006e0067002e00200048007600690073002000640075002000760069006c0020006800610020006d0065007200200069006e0066006f0072006d00610073006a006f006e0020006f006d002000680076006f007200640061006e0020006400750020006f007000700072006500740074006500720020005000440046002f0058002d0033002d006b006f006d00700061007400690062006c00650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006b0061006e002000640075002000730065002000690020006200720075006b00650072006800e5006e00640062006f006b0065006e00200066006f00720020004100630072006f006200610074002e0020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50070006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200034002e00300020006f0067002000730065006e006500720065002e> /SVE <FEFF0041006e007600e4006e006400200065006e00640061007300740020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006e00e40072002000640075002000760069006c006c00200072006100700070006f007200740065007200610020006f006d0020005000440046002f0058002d0033002d007300740061006e00640061007200640020006f0063006800200073006b0061007000610020005000440046002d0064006f006b0075006d0065006e007400200073006f006d0020006600f6006c006a00650072002000640065006e006e00610020007300740061006e0064006100720064002e0020005000440046002f0058002000e4007200200065006e002000490053004f002d007300740061006e00640061007200640020006600f6007200200075007400620079007400650020006100760020006700720061006600690073006b007400200069006e006e0065006800e5006c006c002e0020004d0065007200200069006e0066006f0072006d006100740069006f006e0020006f006d002000680075007200200064007500200073006b00610070006100720020005000440046002d0064006f006b0075006d0065006e007400200073006f006d0020006600f6006c006a006500720020005000440046002f0058002d003300200068006900740074006100720020006400750020006900200061006e007600e4006e00640061007200680061006e00640062006f006b0065006e0020006600f600720020004100630072006f006200610074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e0020006b0061006e002000f600700070006e006100730020006d006500640020004100630072006f0062006100740020006f00630068002000520065006100640065007200200034002e003000200065006c006c00650072002000730065006e006100720065002e> /ENU (Settings for the Rampage workflow.) >> >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice