If you’re in the journalism world on Twitter, you’ve probably been seeing a lot of tweets with the #newschallenge and #knc hashtags over the last few days. It’s because the Knight News Challenge application deadline is looming. (The drop-deadline is Dec. 2, 7 a.m. EST/UTC -5. In other words, there’s still time to finish your application. Get cracking.)

This season, I’ve been helping the Knight Foundation do News Challenge outreach. Almost a year ago, I was sitting in a room with about 20 other people, reviewing approximately 2,300 applications that had come in for the 2009-2010 Knight News Challenge. Our job was to whittle down the applications to 500, and then to 50, which were then passed to a second group of reviewers (the Foundation’s preferred term for judges) before being forwarded to the Knight Foundation board for final cuts and consideration. All the work the reviewers put in was guidance for the board. The board decided who would receive grants, and in what amounts. Ultimately, a dozen projects were funded. You can learn more about them here. The 2009-2010 Knight News Challenge official report by Chris Connell was published June 15, 2010.

For me, the review process was eye-opening and highly educational. Friends and colleagues have not asked about what I learned, but rather what it was like, and obliquely, what we reviewers were looking for. As the 2010-2011 application process comes to a close, now seems like a good time to write this post.

In full disclosure, the Knight Foundation did not ask me to write this. I do so — and this is very important — without knowing anything about how the current application review process will be run. This time, things could be very different. The procedure changes year to year, as do the reviewers, the landscape, the technologies, the applications themselves, and the sense of what’s important and what constitutes innovation when it comes to digitally informing communities.

So take this for what it is: One person’s experience as a reviewer the last time this thing happened.

First, some myth-busting. The reviewers weren’t all journalists. They came from a wide variety of professional backgrounds. Some had worked all their careers in the non-profit world. Others came from journalism academia. Some fell squarely in the technologist/developer camp. Still others came from the startup world as entrepreneurs, often with technical backgrounds. A few were former Knight News Challenge grant-winners. A few had been reviewers the prior year, but in the final round, not the initial rounds. Not all of us were American, or American-born. There was a fairly broad spectrum of ages, from people in their 20s to those whose careers were perhaps twice that. And if I remember correctly, the group was almost evenly split between men and women of varied ethnic backgrounds.

I don’t have a copy of last year’s application, which is different from the current one, but I do remember we were asked to use the application criteria as our guide: The projects had to use open source technology. They had to serve a geographic community. They had to be “innovative,” and we were free to use individual judgement as to its definition. There was one more criterion, but it escapes me right now.

In any case, so long as the application met the criteria, we were free to decide whether it was worth forwarding to the next evaluation rounds.

We worked in teams of three such that each application was read at least once by three people. The Knight Foundation, via random assignment, parceled out the applications. We were encouraged to read other applications too, so long as our assigned batches were finished first. Imagine you’re a high school teacher grading multi-part essays — and last year there was no cap on length. You start to get the picture.

Applications were reviewed, discussed, re-evaluated among teams. In cases where an application referred to something outside one team’s expertise, the group would ask the other reviewers to review and comment. Inevitably, someone with the necessary knowledge stepped in. If any reviewer felt there was a conflict of interest, they’d recuse themselves from the review. Those without conflict would step in. We’d leave notes for each other. If we really liked or disliked something, we’d say so and give our reasons why. If we needed clarification, we were allowed to email the applicant ourselves and ask for it. If an application a reviewer truly loved didn’t make a cut, they could petition others for reconsideration. Sometimes it worked.

Hour by hour, day by day, we started noticing patterns and trends. Some applicants had great ideas but no proof they could actually carry them off. Some applicants had solid ideas but the funding request was out of proportion to the project. Some applications were outlandish, outside the scope of the News Challenge, restatements of already-funded grant winners (there were a number of proposals that were, in essence, Spot.Us), or dull rehashes of ideas whose time had passed.

What we were looking for were proposals that could clearly state a need, meet the geographic requirement, and describe an innovative, plausible, workable solution. The “innovative” hurdle was, and I think still is, the highest. In part, it’s because the standard of innovation is subjective by design. But the proposal doesn’t have to be unicorns and magic, it just has to be a really good idea that hasn’t been approached your way before. To paraphrase Jose Zamora, the Knight Foundation’s journalism associate: There are wheels. There’s luggage. But the Rollaboard? Now that’s innovation.

There are a lot of good ideas for the News Challenge out there, and a lot of talented people capable of making them real. Maybe this time it will be you. Good luck.