VERBAL PROBLEM ANALYSIS EXERCISES

Many interviewers prefer to ask a candidate to analyze and solve a complex problem during the interview. This is a waste of valuable interview time and in most cases is not job related because the candidate would have more time in the actual job setting to analyze and solve the problem. The problem analysis exercise method described in the tool kit administers this type of exercise prior to or after the actual interview.     Determining the level of Problem Analysis Ability for the Target Job Your job analysis has determined that the position requires the ability to analyze some sort of problem. The question then becomes “Analyze what problem and what level of problem analysis ability is necessary upon entry?” Listed below are some examples: Technical Supervisory Administrative Policy and Procedures Organizational Planning Staffing    Developing the exercise This exercise requires the candidate to analyze a problem similar to those that are performed on the job. Thus, the exercise must only measure abilities that are necessary upon entry to the job. The hiring authority (SME) is responsible for developing, and preparing the instructions for completing the exercise. The SME then completes the problem analysis assignment, which is used to develop the scoring key.     Developing the scoring key Based on the SME’s completed exercise the scoring key is developed primarily by using the SME’s judgment and the job analysis. The exercise is usually scored along two general criteria: 1) recognition and level of analysis of problem; and 2) quantity and quality of problem solution. Rating scales are used to rate each of the criteria to obtain a numeric score. The rating scales should be defined by benchmarks for each criterion. Benchmarks are suggested answers to the questions that are linked to the rating scale. Benchmarks provide an objective and consistent reference for assessing the candidate’s responses. There should usually be at least three suggested answers for each sub-criterion: a superior, a satisfactory, and an unsatisfactory response to develop benchmarks, using the guide below, ask SMEs to create answers that fit the different levels of the rating scale.   5— What would one expect or want an outstanding candidate to give as the best possible answer? 4— 3— What is an acceptable answer that one would expect a qualified candidate to give? 2— 1— What would one expect as a poor answer from a candidate who has little or no knowledge or skill on this job requirement?     Problem Analysis Exercise Administration Depending on the difficulty of the exercise, candidates should be scheduled ten minutes to one hour before the interview. The exercise administration should take place in a quiet room or cube containing a pad of paper and other office supplies. It is suggested that the exercise be administered by someone who is not taking part in the interview process. Depending on the level of difficulty of the exercise, the exercise administrator may or may not read the specific instructions for the exercise to candidate. In higher level exercises it is recommended that the candidate just be handed a written set of instructions to read and interpret on their own. When the candidate has five minutes remaining to complete the exercise, the exercise administrator may wish to give the candidate a five-minute warning. When the time limit has expired or the candidate finishes the exercise, the exercise administrator will collect the candidate’s instructions and allow the candidate to take their notes into the interview room. It is highly recommended that the problem analysis exercise be the first question in the interview.     Exercise Scoring If a panel scores the exercise and interview they should be trained. After scoring all candidates, the panel should review their individual ratings as a group and make sure that there is a consensus on each criterion. For example, on a scale of 1 to 5, if one rater gave a 4, one a 3, and the other a 1 on the same criterion, this might indicate some rater bias or misunderstanding of the candidate’s comments. All raters should be no more than one point away from all other raters’ scores. For example, it would be acceptable to have two 3s and a 2 given on the same criterion. If raters find that there is not a consensus on a criterion, they should discuss the reasons for their ratings and attempt to reach a consensus. The results of the interview and exercise may be combined at this point.

Privacy Statement | Accessibility | Contact HR | Contact ADOA