Sunday, December 6, 2015

Pay no attention to the “AP Honor Roll”

On a pretty regular basis we see reports like this—this one apparently prompted by a College Board press release. I wish we saw fewer of them.

If there were a metric that could accurately show that our educational policy choices were enabling more kids to succeed at challenging coursework, that would be worth celebrating. But the number of students taking AP courses, and the number of students getting a 3 or above on an AP exam, are absolutely awful, utterly useless proxies for anything worth measuring.

First, whether AP courses—courses geared entirely toward preparing for tests created by the College Board—are the most valuable type of advanced coursework is entirely debatable. That debate is less likely to happen if people take “honors” like these at face value.

Second, there are ways to raise AP participation and score numbers that are academically unsound—but become incentivized when people start valuing this kind of “honor.” For example, a district can simply open the doors to AP courses regardless of whether the students are ready for the course. The district can then encourage only the most successful students to take the AP test in that subject. Many students might be poorly served by that kind of course, but the College Board gets more business and the district gets an “honor”!

Our district has celebrated this kind of accolade in the past (examples here, here, and here). In our district, some students—more than just the rare outlier—are invited to take AP courses in the first semester of their freshman year in high school. Are those really “college-level” courses? If so, are they really right for high school freshmen? The answer might well be “no” to both questions.

The fact is: we don’t know what the “right” amount of AP participation is. The last thing we should do is start chasing isolated numerical indicators, which is just a recipe for unintended consequences. That’s all the more true when those indicators are in service of thinly disguised advertising for companies like the College Board.


IC Local said...

What do you think then of the University of Iowa's Belin Blank center for gifted programing that ranks Iowa high schools on the amount of AP classes given?

Amy Charles said...

Chris - Amen.

IC Local - Belin Blank does some tremendously good programming itself, and I have trouble believing that they're really ranking high schools (if they are ranking high schools) by AP courses offered. Doesn't make much sense. They're bright people there and aware of both the problems with AP madness and the fact that non-AP teaching can be as good or better than AP courses; depends very much on the teacher and the supportiveness of the school/district.

I also suspect that AP is not the admissions magic so many parents believe it to be, particularly for elite schools. My long perigrinations in universityland have taught me that (a) the price is not the price; (b) they are still in fact looking for very bright, very interesting kids, and collecting AP charms on your bracelet ... that's nice, but I don't believe it's the magic they're looking for. As a parent with a kid off to college eventually, and as someone who works with university students, I'm really not worried about my kid's taking these classes -- and if they're substituting crushing workloads for thought, exploration, development of genuine interests, then I'd rather she not take them at all. I have no idea which of my students have taken AP courses - any or many - but I can tell you which ones are bright and likely to make successful and interesting careers.

Here's what I think the admissions offices are looking for in the cutoff pile, after which it's largely a dice throw: Does the kid have acceptable test/GPA scores for our rank? Is this kid obviously a [name of school] student? Will this kid help our stats in some way? Is this an extremely bright, very interesting, articulate, sane-enough kid who's likely to burnish our reputation for the next forty years? Is there faculty backing this kid, maybe the kid already worked in a lab or did summer research? <= notice that "has the kid taken a stack of AP courses" is not on this list.

In other words: tell the kids (and the parents) to forget the freakout about the two letters and actually be smart and interesting. If the courses are genuinely interesting and worthwhile, and the kid wants to take them, then why not. If the courses are not available at this school or that, there is a marvelous thing called the internet that's riddled with course materials and lectures, and there's a wonderful thing called libraries where you can read books on subjects that interest you. We also have a university, right here, full of academic types who live for the bright and interested student, and who are frequently happy to help a kid along.

iclocal said...

Amy Charles,
Here is the link for the 2015 Iowa AP Index as created by the Iowa's Belin Blank Center:


"Each year since 2005, the Belin-Blank Center has released the Iowa AP Index to recognize the Top 50 Iowa accredited public and nonpublic high schools for providing Advanced Placement opportunities to Iowa's high school students. The 2015 Iowa AP Index is based on May 2014 AP exams and May/June 2014 graduation data."

Amy Charles said...

(rubs forehead)

That doesn't mean they're ranking the high schools in anything but number of AP courses provided. "Top 50 for X" is not the same thing as "top 50 overall". I'm sure you could make a Top 50 Iowa accredited high schools for volleyball, too, but that wouldn't mean they were the fifty best high schools in the state. (Except in volleyball.)

Should Belin Blank issue such a list? To the extent that it encourages an AP fixation, maybe not; as an index for school shoppers, why not -- one kind of choice among many.

Please stop freaking about AP courses and imagining the label is that important. I could write you one of those things if you paid me to do it. So could dozens, maybe hundreds, of other people in this district.

iclocal said...

Amy Charles,
I am not 'freaking out' as you put it over AP tests. I was just merely pointing out that the Belin Blank center does rank schools regarding AP tests--I didn't say they were the best schools over all in the state. I just thought it was important to note this since Chris's post was about AP lists.
As someone who attended CHS and graduated from Iowa I know the differences between high school level classes and college. I can honestly say many of my high school classes in the ICCSD were more challenging than some of the classes I took at Iowa. As you noted above a lot of it depends on the teacher. It also depends on the college too.
I don't understand why the AP lists bothersome to some. A district should be offering advanced classes to students. The only thing I find upsetting about it is that the district does not pay for the test or the extra materials needed for the classes. They should at the very least be offering to pay for the FRL students in the district to take the tests.

Chris said...

Anonymous and Amy — I know that the Belin-Blank center does a lot of good work, and I’m sure its “AP Index” is well-intentioned, but I think rankings of that kind do not provide useful information and end up doing more harm than good. The index is based solely on the sheer number of AP exams taken per graduating student. According to the site, “A high AP Index is a reflection that a school has developed a culture that is supportive of student participation in AP courses and exams.” But all a high rating shows is that the high school succeeded in getting students to take many AP tests. It’s certainly possible to do that in educationally unsound ways. And it doesn’t even factor in how well the students did on the tests!

The problem is that, no matter how you might feel about the value of AP courses, the number of AP courses per student is a “highly corruptible indicator” of exactly the kind described here. The concern is that the more we value the AP Index rankings (and other accolades that are driven largely by AP participation), the more incentive there is for schools to raise their ranking in ways that don’t actually serve their students—in the same way that a school could raise its graduation rate by simply mailing every student a diploma.

Reductive indicators like the AP Index are a form of partial data, and, when you’re making policy, partial data is often worse than no data at all. That’s especially true when the partial data consists of data about benefits without data about costs, which often seems to be the pattern. Unless we know what (if any) values were sacrificed for the sake of raising those AP indicators, we can’t know whether a high ranking is a good sign.

Chris said...

Sorry, I meant to address that to *iclocal* and Amy.

Chris said...

As for the arguments against the AP model, a recent example is here -- worth at least considering.

Amy said...

Oh, I see.

The students who take classes like the blogger describes are the ones who wind up, eventually, in my classes, and they're wound tighter than Jack Nicholson in One Flew Over the Cuckoo's Nest. And they're superprepped and they ask the most inane and trivial questions, and eventually they get so upset about the fact that the course wanders to interesting places not on the syllabus that they show up in my office on the verge of tears, wanting to know what they have to do to succeed.

And my answer, invariably, is to stop being ridiculous about grades and actually use their heads. And think. And understand that adult life is not a set of modules, and that they would do well to actually know and care about something when they get there. We spend about 45 minutes on that, during which time I invariably hear about their families and a mean teacher or two, and by the end they're very, very angry that nobody's talked to them like this before, when they've known all along that they've had stupid and exhausting ways of learning things that leave them with nothing, nothing, nothing.

Here's why I hate courses like this: they train the kids so far out of doing anything thoughtful or interesting that I have to spend ridiculous amounts of time teaching them the sorts of things a 9-year-old should know. Like, "Do you know what this sentence you just read means?" (No.) "What can you do about that?" (uh...Google?) "Google what?" (The thing?) "Google doesn't know what you mean by that. What specifically are you asking Google? What words are you typing into the search engine?" (Silence as they all stare intently at their screens.) So we go through word by word and identify the words they don't understand, and then there's all this magic where they LOOK WORDS UP. And then DO SOMETHING with the definitions. Like extract understanding so that they can figure out what the sentence means, and then notice that the sentence has a context, which, bewilderingly, also seems to have meanings. It's unbelievably painful.

They are utterly, utterly puzzled by exercises like this. The notion of understanding what you're reading, rather than convincingly rearranging phrases in papers or on tests, is utterly foreign to most of them. And it's because of these stupid exercises in pretend education. Even more shocking to them: the noises they're studying are actually about real and important things. And they're worried throughout because they can't see how any of the conversation connects to grades.

You're provoking me on purpose, aren't you, Chris.

Amy said...

Sorry, Chris, I'm taking up your entire blog.

I've been thinking about this tonight as I've been listening to some Mendelssohn and reading a book I first tried to read in 1997, a book about Modernism by a godsend of a high school teacher named William Everdell. There's a lot I don't know about classical music -- most of it -- and as I was thinking about composers and performances I should listen to, I realized I'd done a very smart thing in my 20s: I'd spent a tremendous amount of time listening to classical music and jazz, for no reason other than I wanted to know what it was. Blew a ton of money I didn't have on CDs, composers and artists I'd heard of and knew nothing about. And listened, over and over. I don't have time anymore, I think, before I die, to understand music the way my classical musician friends do, or math the way my mathematician friends do. But I still have a long time. The pianist Leif Ove Andsnes has a great attachment to Grieg, and I trust him; maybe I will start with Grieg.

The thing about these courses, and K12 in general, is the terrible and sad idea that "education" is this thing you shovel into the kids for 18, maybe 22 years, and by god you'd better stuff everything you want them to know in there by then, because they'll never learn a damn thing again afterwards. Which is an idea for the insane, particularly since the poor children can't even read, most of them, for the first third of those years. All you can do, in the first 18 years, is give them some notion of what you know there is to know, and maybe a little bit about things nobody knows, and teach them to be precise and honest about what they do and do not know, and give them some hints about what kinds of questions might be important ones, and put the sounds of intelligence, beauty, wisdom, and discord into their ears, so that they know what these things sound like. And guide them to a perennial recognition that they aren't done, it isn't wise to rest there.

When I left high school, I didn't know when the Civil War had been, or what it was. I didn't know what the Bill of Rights said, or what imaginary numbers were (I'm pretty sure I still don't understand that one and that it's a much more grownup idea than we make it out to be in children's algebra classes), or most anything. But I did know a lot already about things I cared about: many writers, and what turned out to be a golden age of illustration, and photography, and design. Fortunately, I was given what is probably another sixty or seventy years to learn about other things. And because I had the opportunity to learn seriously when I was a little child, I refused to be hurried along in grade school or college: if you told me I had to read twelve novels in a semester I'd say no, that's ridiculous and I won't do it, you can't read them well at all that way, let alone enjoy them. The hurry serves the institution, not the student.

I can stop there. Isn't it terrible? No credit at all for teaching a child how to learn, except that many years later the students come back and say thank you! The thought pleases me very much. Here's the problem, Chris: this machine will never do the thing you want it to do. Isn't built for it. Its operators need credit right away, with measuring sticks.

iclocal said...

The Belin Blank center love AP classes! Here are some tips on how to develop an AP culture from their latest blog entry: https://belinblank.wordpress.com/2015/12/15/encouraging-ap-developing-an-ap-culture/

Amy said...

Do they (you?), or are they merely recognizing that it's a point of fixation for many? The blog post gives tips for "encouraging an AP culture"; doesn't say anything about why that might be desirable beyond the exceptionally vague "greater opportunities for students".

I imagine they're talking about college admissions; can't think what else they might refer to. College admissions officers want to see that the kids have done some serious work, but they know what AP courses are, and no, they don't regard AP as magic. They also know that there are schools that don't have them, and schools that had them and got fed up; it's a quick/dirty "kid worked hard", but there are certainly other ways of showing that you, the college applicant, have a significant interest in something and have put serious and well-directed time and energy into it.

As someone who works with students putting together applications, I can tell you that you can spot a well-qualified student remarkably easily. On paper, if they're reasonably able writers, you have a sense within the first page; in person, within the first few minutes of conversation. After that it's about budgets, acceptance rates, and fit. So I really don't know what BB's talking about with the opportunities.

I'm pretty sure, incidentally, that I don't want an "AP culture". I would very much like a library culture, and a culture of serious, critical, witty, and far-ranging conversation, and a culture of driving the kids towards specificity and depth of thought. There was great value in midcentury scientific culture, and we could stand to read and hear those people again. AP is a product; it does a limited thing. Syllabi and curricula are not culture, no matter whose box they come out of; cultures produce syllabi and curricula.

Anonymous said...

Just an FYI. Just pumping up numbers of students taking the course/test is NOT enough to achieve honor roll status. Increasing number(%) of students scoring a "3" or better(until the school reaches 70%) is also criteria. So success in the course also in the mix.