Saville Wave professional styles 2014

Test ReviewWave Professional Styles

(WAVE)

The British Psychological Society © 2014. All rights reserved. No part of this publication may bereproduced or transmitted, in any form or by any means, without permission.

This test has been granted registration as a psychological test by the British PsychologicalSociety, Psychological Testing Centre.

Permission has been granted to the distributor / publisher named above to distribute copies of thisreview in paper or PDF file format so long as such copies are not amended or changed in any wayfrom the original version published by the BPS.

1

Wave Professional Styles (WAVE)Reviewers: Barry Cripps

& Stephen Fisher

Consulting Editor: Paul Willner

Senior Editor: Iain Coyne

GENERAL INFORMATION AND DESCRIPTION OF THE INSTRUMENT

Test Name: Wave Professional Styles

Authors of the original test:

P. Saville, R. MacIver and R. Kurz

License for Distribution Issued To:

Saville Consulting UK Ltd

Local test distributor / publisher:

Saville Consulting UK Ltd

Publisher of the original version of the test: Saville Consulting Group Ltd

Date of Publication of Current Review/Edition: 2005

Date of Publication of the Original Test: 2005

Date of Current Review: 2014

ISBN: 9781854336910

General Description of Test: The Saville Consulting Wave Styles assessments are part of amulti−dimensional assessment suite for use in a wide variety of applications (selection, individualdevelopment, coaching, leadership assessment, team development, organizational talent audit, businessgrowth, self−awareness and career planning). The Styles assessments were developed to separate talentsfrom underlying predispositions or motives, and measure motivation, talent, competency, potential andpreferred culture in one questionnaire. The Styles questionnaires were designed specifically for theInternet. They operate as both trait and type instruments and rely on a hierarchical model of workperformance that is aligned to the Big Five personality factors and the Great Eight competencies.Wave adopts a hierarchical structure called the Wave Performance Culture Framework, made up ofBehaviour (work behavioural styles and competencies), Ability (assessing a person’s abilities to performcertain intellectual or cognitive tasks), and Global (describing broad overall effectiveness characteristics ofperformance at work such as applying expertise, accomplishing objectives and demonstrating potential).The instrument measures an individual’s ‘Professional Style’ from a total of 216 items representing 108‘Facets’, each of which includes 2 items probing motive (will do) and talent (can do) components. TheFacets are hierarchically organized into 36 ‘Dimensions’, which are further grouped into 12 ‘Sections’(Evaluative, Investigative, Imaginative, Sociable, Impactful, Assertive, Resilient, Flexible, Supportive,Conscientious, Structured and Driven), which combine to create 4 ‘Clusters’ (Thought, Influence,Adaptability and Delivery). Four ‘Response Style’ summary scales are also available. A compact Saville

Wave Professional Styles © 2014 BPS of 28

Consulting Wave Focus Styles questionnaire comprises 72 items measuring 36 behavioural facets thatwere selected for their strong validity. The Handbook provides very little information about this short form ofthe instrument. The Handbook provides detailed descriptions of clusters, sections, dimensions and facets.Throughout the Handbook, the four Clusters are colour coded for ease of reference.Wave is computer administered, scored and normed. The Professional Styles questionnaire takes about 40minutes to complete, while Focus Styles is typically completed in 13 minutes. Completing the Waverequires test−takers to respond to a combination of ‘Likert’ type ratings together with forced−choice items.The scoring integrates rating and ranking responses and results in a combined profile that highlightsdifferences between ipsative and normative scores on the profile.12 different computer−generated reports are available. These are:− An ‘Expert Report’, intended for trained users, providing a full set of profiles, ‘Competency PotentialProfile’ and ‘Predictive Culture/Environment Fit’.− A ‘Personal Report’ providing a summary of results intended for the test−taker.− A ‘Line Manager Report’, intended for trained or untrained stakeholders such as potential managers. Itsuse is supervised by a trained user.− The ‘Interview Guide’ provides areas of interest based on Saville Consulting’s Wave Professional Styles,Focus Styles and Work Strengths model that might be pursued in interview.− ‘Summary Development’ and ‘Premium Development Reports’ provide suggestions concerning anindividual’s development needs.− A ’Types Report’ classifies individuals according to Saville Consulting Wave Typology.− A ‘Team Roles Report’ indicates an individual’s most and least preferred role in a team.− A ‘Leadership Report’ predicts situational leadership effectiveness.− A ‘Reflections Report’ provides a profile designed to give an opportunity for test−takers to reflect on theirworkplace styles.− A ‘Sales Report’ includes selling styles and sales potential indicator profiles.− An ‘Entrepreneurial Potential Report’ provides a set of profiles based on the ‘Entrecode®’ model.The Styles assessments were developed as work relevant, high validity, international tools suitable for arange of job roles across different industry sectors. They can be used with most adult working populationsand are available in 28 languages.

Classification

Content Domains:

Personality − StatePersonality − TraitMotivation

Intended or main area(s) of Use:

Work and OccupationalCounselling, Advice, Guidance, and Career Choice

Intended mode of use (conditions under which the instrument was standardised and validated):

Controlled unsupervised administration. Control over conditions (timing etc) and some control of identify ofthe test taker (e.g. tests administered over the Internet but only to known individuals − password restrictedaccess)

Description of the populations for which the test is intended:

This questionnaire can be used with most working adult populations. It is most commonly used for therecruitment and development of professionals, managers and graduates, as well as other occupations. Thequestionnaire is available in 28 languages other than English, with norm data generated for specificcountries and occupational groups.


Number of scales and brief description of the variable or variables measured by the test:

The WAVE Wave Professional Styles questionnaire comprises 108 ‘Facets’ grouped hierarchically into 36‘Dimensions’ 12 ‘Sections’ and 4 ‘Clusters’. Four ‘Response Style’ summary scales are also included.The 4 Clusters are ‘Thought, ‘Influence’, ‘Adaptability’ and ‘Delivery’, which comprise the following‘Sections’: −− Thought: Evaluative, Investigative, Imaginative− Influence: Sociable, Impactful, Assertive− Adaptability: Resilient, Flexible, Supportive− Delivery: Conscientious, Structured, DrivenAt the base of the hierarchy ‘Sections’ are comprised of the following ‘Dimensions’:− Evaluative: Analytical, Factual, Rational− Investigative: Learning Orientated, Practically Minded, Insightful− Imaginative: Inventive, Abstract, Strategic− Sociable: Interactive, Engaging, Self−promoting− Impactful: Convincing, Articulate, Challenging− Assertive: Purposeful, Directing, Empowering− Resilient: Self−assured, Composed, Resolving− Flexible: Positive, Change Oriented, Receptive− Supportive: Attentive, Involving, Accepting− Conscientious: Reliable, Meticulous, Conforming− Structured: Organized, Principled, Activity Oriented− Driven: Dynamic, Enterprising, Striving.

Items format:

Likert ratingsForced choice, mixed scale alternatives (ipsative)

Number of test items:

There are a total of 216 items, two for each of the 108 Facets.

Administration modes:

Computerised Web−based application − unsupervised/self−assessment• Computerised Web−based application − supervised/proctored•

Response mode:

Computerised

Time:

Preparation: N/AAdministration: 35 mins. Advised time; 40 mins. Modal completion time.Scoring: N/AAnalysis: N/AFeedback: 30 minutes − 2 hours

Different forms of the test:

Focus Styles is a short form with 72 questions covering 36 of the 108 Professional Styles facets, groupingthem in the same higher order clusters and sections. For both versions, there are parallel forms forsupervised and unsupervised adminstration.


Measurement and Scoring

Scoring procedure for the test:

Computer scoring with direct entry of responses by test takerBureau−service − e.g. scoring by the company selling the test

Scores:

Respondents are presented with blocks of six items and indicate their responses on a 9−point Likert scale.Item rankings are initially determined by examining the rating value assigned to each of the six items, withties broken by a further forced−choice question. Each item therefore receives a normative score rangingfrom 1 to 9 and an ipsative score ranging from 1 to 6.Scale scores are normed separately at each level of the hierarchy. The final overall Wave Styles trait scorefor any scale at any level in the Wave model is based on both the normative and ipsative scores. Separatemeasures are given for ipsative and normative responses, motive and talent responses and facet ranges,which give details of the 108 Facets. A Facet scale is composed of two items (ipsative and normativescores combined) and Dimension scores are thus composed of 6 items (3 facets with 2 items each; withipsative and normative scores combined), while Motive and Talent scores at the dimensional level arecomposed of 3 items (one for each facet). Ipsative and normative scores at the dimensional level are basedon differences from derived ipsative and normative scores. Competency Potential scores and CulturePrediction scores are also calculated, which are complex ratings of different Facet scores based on theircriterion−related validity. Scores are also given for Acquiescence, Consistency, and Normative−Ipsativeand Motive−Talent Agreements, based on averaging responses across the profile and standardising them.Sten scales are used throughout for reporting of normed scores.

Score transformation for standard scores:

Normalised − scores obtained by use of normalisation look−up table•

Scales Used:

Standard scores:

Stens•

Computer−Generated Reports

Are computer generated reports available with the instrument?:

Yes•

Number of Computer Generated Reports available:12

Do distributors offer a service to correct and/or develop computer generated reports?:

Yes•

Report Name:Expert report


Brief description of Reports..

Media: Integrated text and graphics•

Complexity: Complex•

Report Structure: Scale based•

Sensitivity to context: One version for all contexts•

Clinical−actuarial: Based on empirical/actuarial relationships•

Modifiability: Not modifiable•

Degree of 'finish': Publication quality•

Transparency: Mixture of clear/concealed linkage between constructs, scores and text•

Style and tone: Directive and guidance•

Intended recipients: Qualified test users•

Report Name:

Personal report



Complexity: Simple•

Report Structure: Scale based•

Sensitivity to context: Pre−defined context−related versions•




Transparency: Clear linkage between constructs, scores and text•

Style and tone: Descriptive•

Intended recipients: Test takers•

Report Name:

Line Manager Report




Complexity: Medium•

Report Structure: Criterion based•







Intended recipients: Qualified system users• Third parties•

Report Name:

Entrepreneurial Potential Report




Report Structure: Construct based•







Intended recipients: Qualified test users• Test takers• Qualified system users• Third parties•

Report Name:

Types Report





Report Structure: Construct, factor and criterion based•








Report Name:

Summary Development Report










Style and tone: Guidance•

Intended recipients: Qualified test users• Test takers• Qualified system users•

Report Name:

Premium Development Report












Intended recipients: Qualified test users• Test takers• Qualified system users•

Report Name:

Leadership Report












Report Name:

Interview Guide





Report Structure: Scale and criterion based•






Style and tone: Directive•

Intended recipients: Qualified test users• Third parties•

Report Name:

Reflections Report





Sensitivity to context: One version for all contexts•






Intended recipients: Qualified test users• Test takers•

Report Name:

Sales Report













Report Name:

Team Roles Report












Supply Condition and Costs

Documentation provided by the distributor as part of the test package:

User manualSupplementary technical information and updates (e.g. local norms, local validation studies etc)Technical (psychometric) manualBooks and articles of related interest


Methods of publication:

PaperLive Internet (test runs in a web browser)Internet download

Start−up costs:

£670 to transfer from another ‘Level B’ instrument.No start−up costs for bureau system.£2240 for own system implementation (Saville Consulting Oasys Platform).

Recurrent costs:

There are no licence, lease or system maintenance fees.

Prices for a report generated by user installed software:

Where the Oasys online administration system has been purchased:Expert Report £55.80Personal Report £25.20Line Manager Report £33.50Entrepreneurial Report £44.70Types Report £44.70Summary Development Report £55.80Premium Development Report £111.60Leadership Report £89.40Interview Guide £44.70Reflections Report £44.70Sales Report £44.70Team Roles Report £44.70

Prices for a report generated by the Internet services:

Expert Report £83.70Personal Report £37.80Line Manager Report £50.30Entrepreneurial Report £67.10Types Report £67.10Summary Development Report £83.70Premium Development Report £167.40Leadership Report £134.10Interview Guide £67.10Reflections Report £67.10Sales Report £67.10Team Roles Report £67.10

Test−related qualifications required by the supplier of the test:

Accreditation in general personality and assessment: measures of typical behaviour, attitudes andpreferences

•

Test specific accreditation•


Professional qualifications required for use of the test:

Holder of BPS Certificate of Competence in Occupational Testing Level B•

Evaluation of Test Materials

Key to symbols:

Inadequate

No longer used

Adequate/Reasonable

Good

Excellent

[N.r.i.o.r](for updates only) Item was not rated in original

review

Quality of the explanation of the rationale, the presentation and thequality of information provided:

Overall rating of the Quality of the explanation of the rationale:

i) Theoretical foundations of the constructs:

ii) Test development procedure:

iii) Thoroughness of the item analyses and item analysis model:

iv) Explanation of content validity:

v) Summary of relevant research:

Adequacy of documentation available to the user (user andtechnical manuals, norm supplements etc):

i) Rationale:

ii) Development:

iii) Standardisation:

iv) Norms:

v) Reliability:

vi) Validity:


Quality of the Procedural instructions provided for the user:

i) For test administration:

ii) For test scoring, norming etc: N/A

iii) For interpretation and reporting:

iv) For providing feedback and debriefing test takers and others:

v) For providing good practice issues on fairness and bias:

vi) Restrictions on use:

vii) References and supporting materials:

Quality of the materials:

i) General quality of test materials (test booklets, answer sheets, testobjects, software, etc):

ii) Test quality of the local adaptation (if the test has been translated andadapted into the local language):

N/A

iii) Ease with which the test taker can understand the task:

iv) Ease with which responses or answers can be made by the test taker:

v) Quality of the items:

Reviewer's comments on the documentation (comment on rationale, design, test development andacceptability):

RationaleThe primary assessment goal of Saville Consulting Wave Styles is to accurately forecast people’sperformance at work throughout the employee life−cycle. The 744 page Handbook explains in detail allaspects of the instrument. Much of the document is rightly devoted to the nature of the ‘Professional Styles’model with detailed explanations of the relationships between the scale descriptions and theirinterpretation. A supplementary manual referring to the ‘Leadership Report is also available. Thecomprehensive nature of the ‘Handbook’ is both its strength and weakness: without extensive training onthe model the less sophisticated test user might find it difficult to digest. There is no discussion of the prosand cons of using the full (Professional) or short (Focus) version of the test.DesignThe documentation for the Saville Consulting Wave Styles is professional and detailed, and makes gooduse of colour and diagrams. The instrument is built around an integrated theoretical model and innovativeonline format integrates rating and ranking responses and results in a profile that highlights differencesbetween ipsative and normative scores. The Handbook clearly explains these features, and includes adviceon how to use the instrument for different purposes (e.g. selection or team development) and illustrativecase studies. The Handbook also includes explanations of basic statistical and psychometric concepts,which will be useful for the non−expert reader, and an adequate reference list, albeit that referencingthroughout the manual is somewhat patchy. Considering its length, the Handbook would benefit greatlyfrom an index. A ‘quick−reference guide’ would also be helpful. Part 1 of the Handbook, ‘Overview’, goes


some way to fulfill this need but lacks a summary of some of the basic psychometric data that arepresented in detail in later sections. Also omitted is information concerning the Oasys online administrationsystem, which is offered as an alternative to the more expensive email/phone and internet bureau service.Test developmentDevelopment of the Wave Professional Styles was based on both inductive and criterion−related validity atboth the predictor and criterion ends of assessment. Neither method focused on empirical validity until latein the development process when a validation−centric approach to item selection identified items or itemgroupings (facets) that had the best criterion−related validity, rather than selecting items on the basis ofhow they correlate with other items or how they contribute to a scale’s internal structure. The authors claimthat an advantage of this approach is the ability to create short and concise but highly valid scales. Chapter16 of the Professional Styles Handbook thoroughly outlines the development stages, beginning withpreliminary concept trials and running through to supporting materials. However, little information ispresented about the item and factor analyses that were conducted during development.AcceptabilityThe questionnaire is clearly presented on screen with easily understandable instructions suitable for thosefamiliar with everyday computer use. Items in the test are relevant to work and likely to be acceptable tothose being assessed for employment or career development. Readability statistics are satisfactory and anadequate ‘test−takers’ preparation guide is available to download. In summary the candidates’ experienceof taking the questionnaire should be a good one. From a test user viewpoint, the Handbook is impressivein its size and appearance, and appears to contain all of the necessary information, but it is not an easyread.

Norms, Validity & Reliability

Evaluation of technical information:

Norms or reference group information:

Overall adequacy:

i) Appropriateness for local use, whether local or international norms:

ii) Appropriateness for intended applications:

iii) Sample sizes:

iv) Procedures used in sample selection: Norm tables based onoperational use, UKManagement, USManagement, InternationalManagement MixedOccupational, Graduates,Australian Professionals andManagers

v) Quality of information provided about minority/protected groupdifferences, effects of age, gender etc:

Reviewers' comments about the norms:

The Wave Styles ‘Handbook’ provides details of several norm groups: two sets of data were collected forthe standardization group (‘Invited Administration’ and ‘Supervised Administration'), and all others were


collected under ‘Invited Administration’ conditions. Normative data for different samples were collectedbetween 2005 and 2009. Details of each sample are given in Appendices to the ‘Handbook’. These includea breakdown of the gender, cultural background, education, work function and work experience of eachsample, but the tables are not stratified by any of these variables because differences in the mean scoreswere relatively small. Norms (mean and SD) are provided at the Dimensions level of the Wave ProfessionalStyles hierarchy only, not at the Cluster, Section or Facet levels, and not for Focus Styles.When generating reports the norm group used is determined by the Oasys administrator; there is no facilityfor the user to nominate a norm group. The reports provided by the Oasys system describe performance interms of Sten scales. These are more informative for the reader than raw scores, which are not available tothe test user, but no information is provided about the mapping of raw scores into Sten scales, so theextensive norm tables in the Handbook are useful only for reference, not for interpreting outputs.Norms generated from the standardization of the instrument are based on 1153 ‘Professionals’. Femalescomprised 53% of the standardization sample, with 5% of the sample reported as being from Indian,Pakistani, Chinese, Asian Other, Black Caribbean Black African or mixed cultural backgrounds. Highestqualifications ranged from 12% postgraduate to 2% no qualifications. 37% had a degree (12%postgraduate, 25% undergraduate), 15% professional qualifications, 7% HNC/HND or equivalent, 35%school level qualifications and 4% Other qualifications. Over a third of the sample had been employed formore than 20 years with fewer than a third working for less than 10 years.The other norm sets are −UK (total N=33730): Senior Managers and Executives (n=4,276); Professionals & Managers (n=9,884);Mixed Occupational Group (n=10,953); Graduates (n=4,021); Individual Contributors (n=3,190).US (total N=4271): Senior Managers and Executives (n=597); Professionals & Managers (n=1,849); MixedOccupational Group (n=2,143); Graduates (n=685); Individual Contributors (n=323).International (total N=13,333): Senior Managers and Executives (n=953); Professionals & Managers(n=2,600); Mixed Occupational Group (n=3,095); International Graduates (n=1,423); Individual Contributors(n=2,202).Australia: Professionals & Managers (n=474).The ‘Handbook’ reports that Wave Styles is being produced in over 25 languages and users are invited tocontact their local office for details. It is not clear however whether the instrument is simply being translatedinto other languages or whether norms are also being generated and standardized.Despite reservations concerning the possible need for re−standardizing the test for some groups there areappropriate norms for most applications of the questionnaire. However, it should be noted that the normsamples were heavily biased toward more educated participants.

Validity:

Overall adequacy:

Construct Validity (overall adequacy):

i) Designs used: Correlations with otherinstruments and performancecriteriaIntra−scale (item−restcorrelations)Exploratory Factor AnalysisConfirmatory Factor Analysis

ii) Sample sizes:


iii) Procedure of sample selection: Incidental

iv) Median and range of the correlations between the test and othersimilar tests:

v) Quality of instruments as criteria or markers:

vi) Differential Item Functioning (DIF) analyses: N/A

Criterion−related validity: overall adequacy:

i) Description of the criteria used and characteristics of the populations: ConcurrentPredictive

ii) Sample sizes:

iii) Procedure of Sample selection: Incidental

iv) Median and range of the correlations between the test and criteria:

Reviewers' comments about validity:

The authors discuss the importance of validity and provide numerous examples of the patterns ofcorrelations of Wave Style measures and other established tests. Quite rightly the emphasis is on is onexplaining to the reader what the Wave scales are measuring.Results from ‘Project Epsom’ a study involving 308 participants tested with the Wave Professional Stylesquestionnaire are quoted as evidence of construct validity. Factor analyses support an alignment of theWave Dimensions with the Great Eight work performance styles suggested by Kurz and Bartram (2002),with very high item−factor loadings and very little cross−loading. Subsequent analyses show correlations ofthese performance styles and five other instruments, the Saville PP, OPQ®, NEO−PI−R, Hogan PersonalityInventory, 16PF, DISC, DISCUS and MBTI. Although extensive evidence of construct validity is quoted aclear picture suitable for interpretation by an average user does not always emerge. In particular, separatetables for each of the 36 Wave dimensions show that they vary greatly in the extent of correlations withother instruments (from zero to significant correlations with almost all other scales); this is not discussed.There is a similar lack of discussion in relation to the factor analysis aligning the standardization sample tothe Great Eight performance styles.Concurrent validities are again derived from the ‘Project Epsom’. Additional US (n=399) and Mexican(n=120) evidence is also presented. In Project Epsom, the Wave Professional and Focus Styles and all ofthe other instruments used were evaluated against the SHL Great Eight competency framework. Meancorrelations of Wave Professional Styles with external ratings of work performance competencies were0.20 and 0.22 at the Dimension and Section levels, respectively, but these values rose to 0.38 and 0.44when corrected for criterion unreliability. Correlations with measures of global overall performance,adjusted for criterion unreliability, were .49 for Professional Styles and .38 for Focus Styles. On both formsof the Wave, correlations for measures of Wave−derived Competency Potential scores were marginallyhigher than Styles scores. Again, the results are presented with minimal discussion.Evidence of predictive validity is less extensive, this being largely restricted to a study of 108 individualsrated externally on Wave Professional Styles and the ‘Wave Performance 360’ questionnaire. The mediancorrelation with external ratings of work performance competencies was 0.17, rising to 0.33 afteradjustment for reliability. These data are presented at the Section level only owing to the smaller samplesize.In addition to scale validities separate concurrent and construct validity coefficients are quoted for themeasures derived for Types, Team Roles, Leadership, Reflections, Sales, and Entrepreneurial Potential.


Evidence of construct validity is based on correlations with either the Hogan Development Survey (n=306)or with the competency scores derived from the OPQ32 (n=308). Concurrent validity is based oncorrelations with external ratings of performance.Overall, Wave shows good evidence of both construct and criterion validity. However, whilst adequateevidence of concurrent validity is presented Wave could benefit from more predictive validation studiesagainst a more extensive set of criteria.

Reliability:

Overall adequacy:

i) Data provided about reliability: Reliability coefficients for anumber of different groupsStandard error ofmeasurement given for anumber of different groups

Internal consistency:

i) Sample size:

ii) Median of coefficients:

Test retest stability:

i) Sample size:


Equivalence reliability:

i) Sample size:


Reviewers' comments on Reliability (comment on confidence intervals for reliability coefficientsand provide Spearman Brown equivalents for a 30−item scale):

At the Dimension level, Internal consistency statistics based on Cronbach’s Alpha and using the 1,153standardization sample are presented for the combined normative−ipsative for both ‘Invited Access’ (range.58 to .86, median .76) and ‘Supervised Access’ administrations (range .65 to .87, median .78). A USsample of 2,102 at the ‘Dimension’ level demonstrates alphas ranging from .52 to .84 with a median of .74.Although not high, these are acceptable levels of consistency. Higher levels of consistency (medians > .8)are reported at the Section level of the hierarchy. Test−retest reliability data for the Wave ProfessionalStyles are quoted for a sample of 100 ‘Invited Access’ participants with an 18 month period between testadministrations for both ‘Dimension’ (range .58 to .85, median .74) and ‘Section’ (normative scoring: range.76 to .86, median .79) levels. Six month test−retest reliabilities are also reported for the thirty−six facetscales that are used in the Wave Focus Styles assessment (n=214) with results ranging from .58 to .84(median .72). Although involving smaller samples than used in the internal consistency studies thetest−retest results can be viewed as being good. Alternate form reliability coefficients for normative, ipsativeand combined normative−ipsative dimensions are quoted for both ‘Dimension’ and ‘Section’ levels. Thealternate forms of the instrument are two independent sets of 216 questions that are used in different waysof administering the questionnaire – ‘Invited Access’ where candidates take the test unsupervised and‘Supervised Access’. The median ‘alternate form’ of combined normative and ipsative ‘Dimension’ scores


was .87. ‘Section’ combined scores yielded a median reliability of .92 with a minimum of .86. It is not clearwhether the order of administration of the two methods of administration was controlled. A further studywas conducted to compare the 36 Wave Professional Styles Dimensions with the 36 Facets of the WaveFocus Styles (n=308). This study reported a median correlation of .71 (range .50 to 84), indicating, perhapsunsurprisingly, that the short form of the Wave (Focus Styles) is less reliable than the long (ProfessionalStyles) version. As with validities in addition to scale reliabilities further figures are quoted for the derivedmeasures, in the respective chapters of the Handbook. − For People Types and Task Types, alternativeform reliabilities were .95 and .93 and test−retest reliabilities were .84 and .83. − For eight Team Roles allalternative form reliability coefficients exceeded .91 and Test−Retest reliabilities ranged from .79 to .85(median .80). − For twenty−four Leadership Style scales alternative form reliabilities ranged from .81 to .95(median .88) and test−retest reliabilities ranged from .49 to .85 (median .77). − For six Leadership Basescales alternative form reliabilities ranged from .82 to .92 (median .92) and test−retest reliabilities rangedfrom .74 to .84 (median .78). − For nine Reflections scales alternative form reliabilities ranged from .87 to.94 (median .90) and test−retest reliabilities ranged from .74 to .85 (median .81). − For twenty Sales scalesalternative form reliabilities ranged from .86 to .95 (median .91) and test−retest reliabilities ranged from .70to .85 (median .79). − For twenty−seven Entrecode® Entrepreneurial Potential Scales alternative formreliabilities ranged from .70 to .94 (median .85) and test−tetest reliabilities ranged from .57 to .84 (median.73).

Quality of Computer−Generated Reports

Report Name:

Expert report

i) Scope or coverage:

ii) Reliability:

iii) Relevance or Validity:

iv) Fairness, or freedom from systematic bias:

v) Acceptability:

vi) Practicality:

vii) Length−number of printed pages:10

viii) Length index (number of pages (including composite and derived scales) divided by number of scales),multiplied by 10 and expressed as an integer:2

Report Name:Personal



ii) Reliability:



v) Acceptability:

vi) Practicality:



Report Name:Line Manager


ii) Reliability:



v) Acceptability:

vi) Practicality:




Report Name:Entrepreneurial


ii) Reliability:



v) Acceptability:

vi) Practicality:



Report Name:Interview Guide


ii) Reliability:



v) Acceptability:

vi) Practicality:




Report Name:Summary Development Report


ii) Reliability:



v) Acceptability:

vi) Practicality:



Report Name:Types Report


ii) Reliability:



v) Acceptability:

vi) Practicality:




Report Name:Team Roles


ii) Reliability:



v) Acceptability:

vi) Practicality:



Report Name:Leadership


ii) Reliability:



v) Acceptability:


vi) Practicality:



Report Name:Reflections Report


ii) Reliability:



v) Acceptability:

vi) Practicality:



Report Name:Sales Report


ii) Reliability:




v) Acceptability:

vi) Practicality:



Report Name:Premium Development

Overall adequacy of computer−generated reports:


ii) Reliability:



v) Acceptability:

vi) Practicality:

vii) Length−number of printed pages: 29

viii) Length index (number of pages (including composite and derivedscales) divided by number of scales), multiplied by 10 and expressed asan integer:

8

Reviewers' comments on the quality of computer generated reports:

Wave can produce 12 computer generated reports:Expert Report , Personal Report, Line Manager Report, Entrepreneurial Report, Types Report, SummaryDevelopment Report, Premium Development Report, Leadership Report, Interview Guide, ReflectionsReport, Sales Report and Team Roles Report.The reports are clearly presented and laid out and make good use of colour. Personal and Line Managerreports, which provide lay feedback, are provided free if the Expert Report is purchased. The other reports,which provide more detail, are available to purchase individually. The Handbook contains an example ofeach type of report (other than the Premium Development Report). All reports carry a clear ‘health warning’that the contents are based on the test−taker’s self−report. The Handbook contains a short section ofguidance on how to detect and explore potential faking and distortion.The Development Reports provide advice to improve performance at work, under four headers: buildingstrengths, possible overplayed strengths (‘watch fors’, with action points), development tips and managinglimitations. However, it is not obvious how this advice is derived from the psychometric data. The InterviewGuide provides two to four questions (with follow−ups), derived from the psychometric assessment, on


each of the Sections of the Wave hierarchy. All of the other reports rely heavily on the presentation of dataas profile scores and although each has an explanatory paragraph many users might prefer more by way ofa narrative summary account. Adequate training on Wave would seem to be essential if most is to be madeof these reports as the selection and interpretation could be seen as being key to their successful use. Thisis particularly the case for the Expert Report, which presents detailed information on each of the 36dimensions with little ‘unpacking’ or comment. Although the ‘Handbook’ suggests a degree of training isrequired to use the reports this is not always made clear in the report itself.In summary the comprehensive range of reports available to the user is a strength of Wave, especially ifthe user is technically sophisticated enough to undertake the processing of the information provided.

FINAL EVALUATION

Evaluative report of the test:

The version of Wave Professional Styles under review was published in 2005, and the current version ofthe Handbook was published in 2012. Wave consists of 216 questions, which are organized hierarchicallyas 4 ‘clusters’, 12 ‘sections’, 36 ‘dimensions’ and 108 ‘facets’, each of which contains 2 questions, oneabout motive (will do) the other about talent (can do). The short form of the test (Wave Focus Styles) usesa single 2−question facet corresponding to each of the 36 dimensions. The scoring is based on aninnovative combination of normative and ipsative methods. The outputs of the test include individualprofiles at different levels of the hierarchy, a typology of 16 combinations of people and task types, andanalyses of divergence between normative and ipsative scores, and between motives and talents. It is atribute to the authors that responses to just 216 items can produce so much information.The test is intended to provide information that could be useful for a wide variety of occupationalapplications at any stage of the employment cycle. If someone is looking for a ‘one stop shop’ for a ‘typical’assessment of occupational behaviour, Wave attempts to do just that. It is therefore likely to be particularlyattractive to the busy test−user who may not have time to select a battery of tests, perhaps from differentpublishers. However, Wave Styles produces a wealth of output and a considerable investment in time isrequired to use it to the full.The documentation on the test – a 744 page Handbook – is impressive, though much of the technical datapresented would benefit from a more extensive narrative to spell out the implications of the data presentedin a multiplicity of well laid−out tables, and the absence of an index in a volume of this size is a seriouslimitation. The 12 computer−generated reports are similarly attractive on the eye, but again present awealth of information that would be easier to digest with more textual interpretation. It is not obvious fromthe Handbook which reports are produced for the short (Focus) version of the test and how, if at all, theFocus and Professional reports differ.Wave is administered and scored electronically. There is no paper version, and no access to the rawscores, leaving the user wholly dependent on the computer−generated reports. Psychometric information inthe reports is presented as Sten scores, which are clear and easily comprehensible, but do not relate in atransparent way to the norms presented in the Handbook.Psychometrically, the test has been well constructed. The test has good reliability, and norms arepresented for many different well−described occupational groups, although how the standardization samplewas recruited is a little unclear. It is disappointing that more evidence of criterion based validity is notpresent, not because insufficient studies are quoted but rather because of the narrowness of the criteriaused. There is a heavy reliance on ratings and ratings mainly on one hierarchical competency framework.Many will be happy with this, but it would have been reassuring if additional criteria were also employed,including more evidence of predictive validity. Having said this there is every reason to believe that Wavewill give good results in its intended areas of use.The test is available in a number of languages, but it is not clear whether separate standardizations havebeen carried out for non−English speaking groups and caution should be exercised if use with differentcultures is contemplated.A user’s confidence in the use of Wave will to a large extent depend on the comprehensiveness of thetraining given by the publisher. A thorough grounding in the nature of the measures available together withthe allied performance frameworks appears essential. Equally important is training on the feedbackprocess. Here the ‘Handbook’ provides useful guidance.


In summary Wave Professional Styles is a good ‘omnibus’ measure with a wide range of applications, butcomprehensive training on the underlying model is essential for its safe use.

Conclusions:

Wave Professional Styles is a well−developed and well−researched test that is suitable for use in all thecontexts for which it was designed. However, Wave could benefit from more predictive validation studiesagainst a more extensive set of criteria.The Wave is well presented on−line and easy for the test taker to use. It produces in a single test theinformation that might be gained from the administration of several other instruments and will be a valuabletool in the kitbag of trained professionals. The computer−based reports are comprehensive and detailed,albeit many of them are hard work to read because they include little by way of narrative to explain the datapresented.A note of caution concerns the vast amount of data that is generated from a single test sitting. Safe use ofthis test will be enhanced by comprehensive training on the models on which the Wave is based and theperformance frameworks with which it is associated.

Recommendations:

Suitable for use in the area(s) of application defined by the distributor, by test users who meet thedistributor's specific qualifications requirements

•

SUMMARY EVALUATION OF THE TEST

Content Domains:

Personality − StatePersonality − TraitMotivation

Intended or main area(s) of Use:

Work and OccupationalCounselling, Advice, Guidance, and Career Choice

Intended mode of use (conditions under which the instrument was standardised and validated):

Controlled unsupervised administration. Control over conditions (timing etc) and some control of identify ofthe test taker (e.g. tests administered over the Internet but only to known individuals − password restrictedaccess)

Test Description:

Test Name: Wave Professional Styles

Local test distributor / publisher: Saville Consulting UK Ltd

Date of Current Review: 2014

Date of Publication of Current Review/Edition: 2005

Constructs Measured: The 108 Facets are hierarchically organized into 36‘Dimensions’, which are further grouped into 12‘Sections’ (Evaluative, Investigative, Imaginative,Sociable, Impactful, Assertive, Resilient, Flexible,Supportive, Conscientious, Structured and Driven).


These combine to create 4 ‘Clusters’ (Thought,Influence, Adaptability and Delivery).

Administration Mode: Computerised Web−based application −unsupervised/self−assessmentComputerised Web−based application −supervised/proctored

Response Mode: Computerised

Instrument Evaluation:

Characteristics Evaluation

Quality of Documentation

Quality of Materials

Norms and reference groups

Construct validity

Criterion−related validity

Reliability−overall

Number of Computer−Generated Reports 12

The British Psychological Society © 2014. All rights reserved. No part of this publication may be reproducedor transmitted, in any form or by any means, without permission.

Debug mode: **0 (none), 1 (basic), 2 (advanced), 999 (edit)


Saville Wave professional styles 2014

Leadership & Management

Transcript of Saville Wave professional styles 2014