|
Exam Program Staff Changes
The BJCP Exam Directorate recently made some staffing changes during January 2014 to March 2014 to assist the Directorate in the tremendous growth in the number of exams.
- Associate Director Charlie Gow retired
- Assistant Exam Director Susan Ruud is retiring after training her replacements.
- Sandy Cockerham moved from Associate Director to Assistant Exam Director
- Gail Milburn is a new Assistant Exam Director
- Jim Wilson moved from Associate Director to Exam Director
- Bill Bopp became a new Associate Exam Director
- David Lytton became a new Associate Exam Director
- Josh Weikert became a new Associate Exam Director
Sandy Cockerham and Gail Milburn will together handle the Assistant Exam Director role previously handled by Susan Ruud. They will work out the split of the duties between themselves.
AHA Support for Exams
As part of an agreement with the AHA for support the exam program we will be transitioning to a process where local exam administrators will mail the hard copy of their exams to the AHA for processing. The AHA will then scan the exam papers and provide the scanned images to the Exam Directors. The Exam Directors in turn will be providing scanned images to the individual graders. Except for the administrator mailing the originals to the AHA there will be no mailing of hard copies of exams. At least one cycle through the postal (mail delivery) system will be eliminated. This change will eliminate some of the administrative load for the Exam Directors, allowing them to spend more time on grading activities. This change should provide a slight improvement in exam turnaround times. The AHA will not have access to information that associates a given exam with an individual and they will not have access to details about the questions to be used in a written exam prior to the exam being given.
Judging Ethics
Some of the existing true/false questions on the various exams focus on judging ethics. After an issue around a judge’s use of social media to comment on beers while judging it was suggested that additional questions be added to clarify this area. This area is something that should be covered in the existing Judging Procedures Manual (JPM) at /judgeprocman.php. Once the JPM has been updated, we will add additional questions to the exams. At this time the BJCP is still in need of volunteer(s) to take on updating that document.
Exam Program Document Changes
Some changes have been made to the Exam Program Document based upon updates within the Directorate over the past few years.
Exam Program Changes
The exam staff has been discussing a lot of different ideas that might help improve the processing of exams. At this time most of these ideas are still being discussed and little has been finalized. Some things that are moving along include:
- A new summary sheet for collecting information from the proctors was generated.
- There is ongoing discussion of changes to the proctor’s scoresheet to guide the proctors to providing the information the graders need.
- Of the ideas discussed one will be tested on a future set of exams. The idea being tested is to minimize the feedback provided on the RTP (Response To Participants) form that the graders complete for Beer Judging Exams. The idea is to not provide detailed descriptions of what was right or wrong in the exam but rather to provide just some tables that are automatically generated by the EGF (Exam Grading Form) as the graders evaluate the examinee’s scoresheets. The generation of detailed feedback on each beer is time consuming. The review of the detailed feedback is even more time consuming. Eliminating this part of the feedback might improve exam turnaround times.
- A minor change, from the examinee’s perspective, is in the grading of Beer Written Proficiency Exams. These exams are a fundamental part of existing judges advancing to National and higher ranks. Going forward we will try to have at least one of the graders of these exams be a Master or Grand Master judge – a judge that has already scored at the Master-level on the equivalent exam.
- We discussed requiring that Mead Exam graders have attained the equivalent of a National or higher score on the mead exam (at least a composite score of 80) but at this time we have way too few exam graders that meet that requirement. We would not be able to support the existing mead exam demand if we imposed such a requirement.
- Also related to the Mead Exam, we had a Mead Judge (someone who has never taken the beer exams) with a mead exam score at a National or higher level express interest in grading mead exams. Doing so requires us to update some documents since we currently say all graders have a National or higher rank but mead judges do not have a rank. Mead Judges in this category would only be allowed to grade mead exams.
Impact of the New Style Guidelines
The new BJCP Style Guidelines will impact all the various exams in some way. The new guidelines will not immediately become the basis for exams. A transition period will be needed though the period may not be the same for all the exam types. More details on the transition process will be announced after the new guidelines are finalized.
- The Beer Judging Exam will only be impacted in that the graders will have to be familiar with the new guidelines. The exam administration documents may need minor revisions due to style changes. The examinees will need to know the new styles.
- For the Beer Written Proficiency Exam, the styles associated with the S0 question may need to be revised. The graders will need to be familiar with the new guidelines. The examinees will need to be familiar with the new style descriptions.
- For Beer Entrance Exam (online exam), the question base will have to be revised to correspond to the new guidelines. All the existing questions will have to be checked for correctness vs. the new guidelines and new questions will need to be added for new styles. The examinees will need to be familiar with the new style guidelines.
- For the Mead Exam, the essay portion will need revision to correspond to the new guidelines. For both portions the graders will need to be familiar with the new guidelines. The examinees will need to be familiar with the new guidelines. Our desire is to separate the Legacy Mead Exam into an online Mead Entrance Exam (similar to the Beer Entrance Exam) and a Mead Judging exam as part of the transition to the new guidelines. While much more effort is required, the developing pool of mead questions now stands at over 2000 questions.
Training
There is a desire to provide several types of training for exam grading. First is training for actual graders. Second is training for Associate and Exam Directors – in these roles the staff is doing more reviewing and guiding of graders than actual detailed grading. To handle the exam load ADs and EDs need to be able to do a quick top-down evaluation of exams then guide the graders to the appropriate scores. There is some initial training material for the first area in development. Hopefully it will be ready for review by the exam staff soon so that it can be rolled out in a beta-type setting before a broad release.
Software Access for Graders
Due to the BJCP’s non-profit charitable organization status we are eligible for free access to Microsoft’s Office 365 suite of web-based Office applications. Any grader without access to MS Office will be provided with online access to Word and Excel. This is intended to help standardize the tools used by the grading pool and simplify the multiple levels of review and revision necessary for the EGF and RTPs.
Quarterly Written Exam
Some areas are having difficulty finding six or more judges eligible willing to take the Beer Written Proficiency Exam. In many cases organizers think they have enough people but then they drop out as the exam date gets near. Frequently the drop outs take the site below the minimum needed to offer the exam. This effectively punishes those that want to follow through and take the exam. From the perspective of grading exams, sets of exams smaller than the minimum of six is a very inefficient use of the limited grading resources so we do not want to have sites that are below the minimum size.
In an attempt to alleviate some of the problems we are going to experiment with offering a Written Proficiency exam once a quarter in a way that does not have a size minimum. However, there are going to be a number of additional restrictions that go along with this exam. It will not replace the existing ability to schedule Beer Written Proficiency exam. The restrictions may not make this approach the solution for everyone but we hope it will satisfy a number of potential examinees.
Details
The Exam Directors will select one and only one date within a quarter for this special Beer Written Proficiency Exam. The date will be publicized at least a quarter in advance via the online BJCP exam calendar.
Existing exam graders will be allowed to offer the Written Proficiency Exam to eligible judges on the designated date. Potential exam takers will need to arrange to work with an exam grader well in advance of the exam date. It is expected that if there are multiple interested takers in a metropolitan area that they will all work with a single grader – this is essential in helping minimize the overhead for coordinating this exam.
The potential exam takers will need to pay for their exam at least two months before the exam date. If they drop out later there will be no refund of the exam fees.
The takers will pay the grader that is the exam administrator and then the grader will submit the fees to the BJCP via PayPal – no checks to the BJCP will be allowed. The payment in advance helps determine how many examinees there really will be – essential for scheduling grading resources.
The time of day for the exam will be coordinated so that everyone is taking the exam simultaneously, no matter their location, i.e., time zones will be taken into account to coordinate the time.
The designated Exam Director will communicate the exam forms to all the exam administrators approximately a week before the exam date. After the exam is completed, the individual exam administrators will mail the completed exams to the designated Exam Director.
The Exam Director will wait for all the exams to arrive from all the sites before starting the process through the normal grading cycle. Because of the extra coordination and potential delays waiting for all the exams to arrive it is expected that the grading process for these exams will take longer than normal.
The fee for this exam will be higher than normal to account for the addition overhead associated with having multiple sites. The fee will be $25 per examinee.
There will be no option to change the date or the designated time of day for the exam. If the restrictions do not fit an individual judge’s schedule the judge will have to try for a different exam site or a different quarterly exam
###
|
