Is there such a thing as an accurate practice test?

As I prepare for the written part of my CCIE, I have started reading the Cisco Press Official Exam Certification Guide.  As most everyone who buys Cisco Press books knows, the Exam Cert Guides come with practice tests.  Historically, these tests have been the source of many a gripe on the NetPro forums.

I don’t know why I thought the test for the CCIE would be different.  But for some reason my brain was thinking that Cisco would hold their test supplier (Boson) to a higher standard for the CCIE.  My hopes were raised further as I first installed the practice tests engine.  This time, it was different from all of the rest.  It actually used the Boson Exam Environment instead of the Cisco Press specific environment like the CCNA and CCNP tests did.  My hopes soared as I saw that the test wasn’t actually on the CD, but downloaded directly from Boson.  That must mean that this content is fresh, right?

So I fired up the exam and chose the layer-2 subjects that I have been studying this week to get a feel for how well I was absorbing the material.  Just a few questions into the test, my hopes were dashed and I came crashing down to reality.  One of the questions showed a diagram of 4 switches and was asking STP questions referencing ports P1 and P2 on SW1.  The problem being, the diagram had labeled all of the ports on SW1 and none of them were labeled P1 and P2.  That short of makes it hard to answer the question when you don’t know what ports they are talking about.

So I pressed on from there and quickly ran into a question about negotiated trunking.  The scenario said that a switch with the stated port configs was connected to 3560s on each port and asked which ports would establish a trunk.  So I chose only the ports with Dynamic Desirable or Trunk On (with negotiation still enabled).  According to the test engine, I was wrong.  So I had it show me the answers to see what I missed.  That’s where I find out that the test thinks that the default port setting on a 3560 is Dynamic Desirable.  Shortly after this, I just quit the test because I was mad.  Partly I was mad at Cisco for allowing Boson to supply bad practice tests, and partly I was mad at myself for being silly enough to believe that this time would be different.

Well, this morning I felt like giving it another try.  I even found the update option and clicked on it.  To my surprise, it actually said that it was downloading a couple of things.  My hopes raised again.  But, whatever the updates did, they did not fix the 2 questions that I described above.  Back to reality I guess…  I did send notice to Boson about their mistakes.  But something tells me that if the test has been around for as long as the book (1 year), then Boson really isn’t fixing exam content issues.