“You’re not as conservative as I thought!” I hear this all the time. I’ve learned that when media outlets seek me out, it is best to let them know early on that I cannot serve as right-wing “balance” in a quest for a “diversity” of black voices, and that my politics are best described as cranky liberal.

But that’s not the reputation that precedes me. Why the confusion? Because I write about race politics, and when you venture into that territory, today’s narrow view of what’s acceptably liberal quickly distorts ideological labels.

There are two reasons for this. One is that until recently I was a Senior Fellow at the Manhattan Institute, a free-market think tank, and I continue to be a Contributing Editor for its magazine. Understandably, few know that one does not have to be a Republican to be associated with them. I was hardly the only Democrat in the fold. The question is why the Manhattan Institute would take on someone with my views, and why I felt comfortable under their aegis.

And that leads to the second reason for the lack of fit between public conceptions of where I fit in and what I actually write and say. The views I have that are reflexively classified as conservative today – and are welcome in a conservative think tank -- are ones that black leaders fifty years ago would have considered brie-and-Zinfandel liberal.

For example, I’m often depicted as opposing affirmative action. That’s not true. I simply think it should be aimed at the disadvantaged. My beef is with creating separate standards of evaluation based on skin color alone in a quest for “diversity.” This is not a conservative view. John Hope Franklin, the dean of black historians, was appalled when he learned while testifying for the University of Michigan in its Supreme Court case that exactly this was the modus operandi of the school’s preferences program.

I also think the limitation of welfare to five years, with a focus on job training, was a good thing. Only from the sixties to 1996 was welfare much different from this, becoming an open-ended program with little commitment to job training. Given the proposal for this sea change in the early sixties, March on Washington organizer Bayard Rustin considered it absurd. The new welfare decimated black communities; today, any number of decidedly un-conservative inner city single mothers readily attest that the welfare reform of 1996 was a good thing.

And there’s another reason I get classified as a conservative. To me, racism should be treated as a conquerable inconvenience, not an implacable obstacle. I think a focus on battling something as abstract as “societal” racism is a distraction from actually helping people be less poor with programs like Geoffrey Canada’s Harlem Children’s Zone. This, again, has nothing to do with the philosophies of Edmund Burke or William F. Buckley. Erstwhile NAACP head James Weldon Johnson said in 1917:

I will not allow one prejudiced person or one million or one hundred million to blight my life. I will not let prejudice or any of its attendant humiliations and injustices bear me down to spiritual defeat. My inner life is mine, and I shall defend and maintain its integrity against all the powers of hell.

Right. But not politically, I assume. Why do we see black strength when someone says it a hundred years ago but strange conservatism if someone says the same thing today?

It’s because in the sixties, radicalism guilted the American establishment into processing race issues separately from others. Yes, radicalism – but I mean just that, with no “overtones” here of socialism or anarchism.

Take affirmative action. To champion diversity over skill level is, as human history goes, highly unusual, and remains a fragile argument even today. It is, despite its espousal by university presidents, a radical proposition. To encourage poor people to sign up for welfare and stay on it indefinitely is, similarly, a radical proposition. It was fought for explicitly under that label by Columbia’s Francis Fox Piven and Richard Cloward in the sixties, when their aim was to break the government’s budget and force redistribution of wealth. To imagine a society where no one harbors any racist biases at all is also quite radical, every bit as much as imagining a society devoid of anger, physical imperfection, or inequalities of skill levels.

Part of being a good-thinking modern American, then, is to process classically radical views as centrist when it comes to a particular topic: race. Under this lens, my views become “conservative.”

There is a pendulum aspect to this. The signs are everywhere that in fifty years, young people will be perplexed that in our era views like mine were considered right-wing – or that it even made news that Barack Obama wouldn’t want his daughters to be given, of all things, racial preferences.

For now, however, we will make do with the complexity involved in classifying someone who voted in the past three Presidential elections for Nader, Kerry, and Obama as a political conservative. Maybe we can just chalk it up as a new kind of diversity.

John McWhorter is a contributing editor at City Journal and The New Republic and is a lecturer at Columbia University. His latest book is Our Magnificent Bastard Tongue: Untold Stories in the History of English.