Iron Chef... Let's Talk

[gbprime]

Actually... let's brainstorm.

Plainly put, the Iron Chef competitions on this board are exceedingly cool. They are, however, exceedingly slow. (The current round gave 1 week to submit builds and has been awaiting judging scores for 5 weeks now... still not done.) This phenomenon is spilling over to the E6 "Appetizer Edition" (1 week for builds, 3 weeks and counting for judging).

Now I realize that real life gets in the way of having time for both entering and judging these competitions. And I certainly wouldn't begrudge any judge who is lagging due to medical reasons, which has happened. But here's the pattern... 1 week for submission, weeks and weeks for judging.

So... back to brainstorming. And since we need a new chair(wo)man for next go around, this is a good time for it.

How can we streamline these competitions so the community has less time to wait?

[dextercorvia]

Good questions... You said brainstorming, so I'll throw out there a timeline for judging. We expect contestants to finish within a certain amount of time, judges could be held to similar standards. We could limit the number of submissions -- I would be much more likely to jump in and judge between 5 contestents, rather than 9.

[Dusk Eclipse]

The problem is that the number of contestant vary a lot between contests; IIRC we have had competition with more than 14 entries and some with less than 5, so if we have to give a deadline for Judges I believe we should take into account the number of builds submitted.

[Maho-Tsukai]

My proposition is simple. If it is taking to long for people to judge then don't allow everybody and their mother be a judge. Democracy and freedom are nice but if people are not making their deadlines then the commission should select a pannel of designated judges who both have expertice in the field and can reliably judge compititions in a timley fashion. Organization is key and the more systemized this is, the more efficent it will be. Generally, the more chaotic something is the less efficent it will be, though allowing it to be open to anybody offers more possible opinions and things that a selected pannel of judges could not, it offers less structer and reliability and generally will create a slow, ineficent process. Thus, if you want to make things faster and more efficent you would do well to have tighter organization and a pre-set pannel of reliable judges rather then letting anybody that wants to be a judge have their word in.

[Dusk Eclipse]

Some of the regular judges sometimes want to compete, or they might not have time to devote to the Iron Chef contest. Besides since they are not receiving any kind of compensation for their efforts (apart from the gratitude from contestants and spectators alike) we can't expect for them to make time just to judge a forum competition.

[Maho-Tsukai]

That is true, but there still should be more of a stremelined system then "anybody can hop in and judge whenever they want." Perhaps people who want to judge ask, and then there is a filtering system that weeds out unreliable people. The judges would thus be different each contest, but the filtering system would weed out those who don't have the time to judge. Such a filtering system is as simple as posting in each thread that a judge who cannot meet the deadlines will not be allowed to weigh in and give their opinion, or something simmilar. Instead of making the contest wait for those who are too slow, simply deny those too slow from being in the contest. I know thats crule but a buisness dose not wait for a late worker to get running. Sometimes effiency can seem ruthless, but thats just how effiency is. If you make it clear to people in the OP that the deadlines won't wait for anybody then there should be no hurt feelings or blame as the tardy judges only have themselves to blaim when they are booted out as they disreguarded the statements in the OP.

[KoboldCleric]

I'm not sure if there is a way to make it quicker due to the nature of the competition. I just finished my judging for the latest e6 competition late last night (I'll be posting the scores when I get home from work after giving them a least one over) It took me 4 days and upwards of an hour with each build (started Sunday, ended late last night). Had there been more builds or as with the regular Iron Chef, 14 more levels, it probably would have taken much longer.

Knowing that the contestents may each have spent a good few hours on their builds, it wouldn't feel right not to research into each combination and playtest a bit of their signature tactics and such. I also didn't own every book used, so in the interest of a fair judging I had to borrow a couple, which also meant some delay as I procured the books in question. The final tallies ended up looking similar to my inital gut reaction to the builds, but there were a few significant changes once I read into the combination or playtested a bit .. so I wouldn't have felt comfortable spending less time than I did.

Perhaps giving 2 weeks to submit builds with the stipulation that you must submit your sources by the 1st week (to a max of 10 or some other arbitrary number) would help, as then the judges would have a better idea of how to prepare and what to expect. You could take off a point from elegance for every sourse change to enfore the rule. Other than that, I'm not sure how you can speed it up ... you can't really penalize judges for taking time, they're not competing for anything :/

[Larpus]

What about using attendance slots?

Such as there is a post stating that a new round is coming up and that there'll be X slots for contestants and Y for judges, everyone that wants in shall respond and commit to take part in the challenge, let's give them a week or so to enter, once the slots are used up, no one else can enter for the round, but the show can go on with less than asked for contestants.

Then things proceed the normal way, but keeping the time limit for judges.

Something like that...

[Zonugal]

Perhaps we could look at a way of making the actual act of judging quicker. I know in the past some judges have written a lot for each build so one might simply ask if that is needed/requested? We could always ask for simpler judging style (which does carry the possible problem of judging coming off as more gut-feeling or such, but...).

[Akal Saris]

Perhaps we could look at a way of making the actual act of judging quicker. I know in the past some judges have written a lot for each build so one might simply ask if that is needed/requested? We could always ask for simpler judging style (which does carry the possible problem of judging coming off as more gut-feeling or such, but...).

This, this, this.

I've seen some judging that was almost longer than the character submission itself, for goodness sake. And that is simply unnecessary - just tell the players your honest opinion and offer the most obvious suggestions for improvement, and move on.

What's the hold-up with the current Iron Chef? I thought it was an issue with the chairperson being temporarily banned or something...?

[dextercorvia]

This, this, this.

I've seen some judging that was almost longer than the character submission itself, for goodness sake. And that is simply unnecessary - just tell the players your honest opinion and offer the most obvious suggestions for improvement, and move on.

What's the hold-up with the current Iron Chef? I thought it was an issue with the chairperson being temporarily banned or something...?

Only two judges have reported, one is awol, and with an interim chair, decisions aren't being made as quickly.

Also, I agree that streamlining the judging is a good idea. It's something I struggled with, I would read a build and try to get a feel for what it was doing, but often knowing whether it works means referring back to all of the feats and features. I have an easier time knowing what is legal in e6, since the scope is narrower. IIRC, Sheet checking was one of the major bottlenecks in ToS, and those were 13th level characters, not 20th.

[gbprime]

We could limit the number of submissions -- I would be much more likely to jump in and judge between 5 contestents, rather than 9.
How much quicker does the judging go based on the number of entrants? Isn't the holdup the time being taken to judge?


so if we have to give a deadline for Judges I believe we should take into account the number of builds submitted
Better. This would work if we had more than a minimum number of judges. Example... we have 15 people say they're going to submit builds, 8 actually make the deadline. If we can work it the same way with judges... 6 people say they'll judge, 3 actually do so by the deadline.

The trick therefore, is to increase the pool of judges somehow without having "panelists" who are locked into not submitting an entry. Because I do agree that the option to submit or judge is cool.

[gbprime]

Perhaps we could look at a way of making the actual act of judging quicker. I know in the past some judges have written a lot for each build so one might simply ask if that is needed/requested? We could always ask for simpler judging style (which does carry the possible problem of judging coming off as more gut-feeling or such, but...).

Good idea, or make a checklist of things to judge by so that the process of judging is less painstaking. But if we standardize judging criteria by writing up a checklist of sorts, then people will start making builds that play to the criteria to maximize points. We might lose creativity that way.

[Hand_of_Vecna]

My first instinct is for a regular panel of judges, not the same 5 people all the time of course but a pool from which judges come like most shows with an Iron Chef like formula often do. Honestly I don't think it would even need to have much in the way of standards other than hasn't flaked as an Iron Chef judge without good reason.

Alternatively just have a reasonable deadline for judging and ask for substitute judges after that. I'd suggest trying for substitute judges with proven track records of judging promptly just because people are already waiting for something over due.

Edit: Oh and I'm definatly against having a maximum number of entries since generally alot more people sign up than submit entries.

and I third the idea of signing up more than the minimum number of judges then going with what there is after a judging deadline.

[The Gilded Duke]

I'd be against attendance slots, making a character is difficult, and I have often backed out of contests mid way through. I think generally we get maybe half the entries that people say they will post?

I think recently the contests have just been too obscure, I would have been willing to judge the recent one, but I have never read sword and fist, and I am not really that interested in going back to 3.0 material for a contest.

Also, might be obscure, but I think a pathfinder Iron Chef would be amazing.

[Private-Prinny]

What's the hold-up with the current Iron Chef? I thought it was an issue with the chairperson being temporarily banned or something...?

It's not temporary, I'm afraid. Jumilk, his fiancee, is finishing this round, since she has the same information that he did. As for future rounds, I volunteered to have the torch passed back to me unless someone else who hasn't had a chance to chair yet wants a turn.

On streamlining the judging process: What if we came up with a giant list of possible additions and deductions for each category, and judges picked a small set of the ones they feel are most important and judge the builds on those aspects? For example, we could come up with 30 different interpretations of what could make a build "Powerful", or "Original", and a judge would pick the 10 that they placed the most importance on. That way, we have a slightly more rigid rubric for the judges to work within, and the judges can still inject a slight amount of personal bias.

Granted, some of these criteria are bound to disagree with or outright contradict each other, which is a good thing, since a build scored highly by one judge might be scored harshly by another, which eliminates the possible problem of a large margin in scores between builds.

And finally, of course, any judges would be advised not to share the criteria that they picked, even with other judges, in the interest of stopping contestants from trying to pander to the judges panel. Sharing your choice of criteria with the chairman should be optional as well.

[Zaq]

It's not temporary, I'm afraid. Jumilk, his fiancee, is finishing this round, since she has the same information that he did. As for future rounds, I volunteered to have the torch passed back to me unless someone else who hasn't had a chance to chair yet wants a turn.

On streamlining the judging process: What if we came up with a giant list of possible additions and deductions for each category, and judges picked a small set of the ones they feel are most important and judge the builds on those aspects? For example, we could come up with 30 different interpretations of what could make a build "Powerful", or "Original", and a judge would pick the 10 that they placed the most importance on. That way, we have a slightly more rigid rubric for the judges to work within, and the judges can still inject a slight amount of personal bias.

Granted, some of these criteria are bound to disagree with or outright contradict each other, which is a good thing, since a build scored highly by one judge might be scored harshly by another, which eliminates the possible problem of a large margin in scores between builds.

And finally, of course, any judges would be advised not to share the criteria that they picked, even with other judges, in the interest of stopping contestants from trying to pander to the judges panel. Sharing your choice of criteria with the chairman should be optional as well.

Yeah, I don't like the sounds of that. I can't see that happening in anything resembling an elegant manner.

I almost feel like if we standardized the entries a little more, we'd be on the right track. Yes, it's a lot of fun to see the creative presentations and everything, and in no way am I suggesting we get rid of those, but if every contestant had to present a concise and straightforward summary of why they took what they did when they did and what they plan to do with it, well, I know that I'd probably judge a little faster. Just think of it as an extension of the already semi-standardized (damn you, skill points!) build table.

Of course, I'm well aware that implementing my idea relies on just as much squishy language as Prinny's does, so maybe I'm not helping . . .

I do agree that shifting the entrant : judge ratio a little bit in favor of the judges would be likely to help. Any ideas how we can facilitate that?

[WinWin]

The Original Iron Chef Character Optimization Competition (http://brilliantgameologists.com/boards/index.php?topic=1701.0)

Simplicity has it's advantages. Adding more rules and complexity will not make a competition move more quickly, nor will it attract the interest required to maintain such a competition.

Ditch the fluff. Focus on the crunch. Get rid of contestion of judging, seriously; Judge once, no takebacks. You get it wrong or make a mistake, too bad. That cuts out a massive part of the workload required right there.

[Pechvarry]

As an observer who reads all the submitted builds, I cannot count the number of times that I didn't understand what the build actually did and what made it cool. I usually pick up on half the tricks the contestants used by reading the judgings -- that is to say, the people who had to go through and completely reconstruct the builds, sifting through all of the abilities used, the ramifications of their uses together.

In short, the submissions themselves could do better with concise wording dedicated to all of the build's interplay. We have a decent chunk of this, but they make a lot of assumptions of what abilities do what and just how much. More importantly, it's usually geared towards "use of secret ingredient" where I'm led to believe that something awesome is working with X quality of the secret ingredient. I'm reading the secret ingredient, I know what it can do. That doesn't mean I automatically understand what ability is being paired with it.

[BobVosh]

Ok I'm going to attempt to put my thoughts on this.

I like the fluff, it doesn't really add any time to judging. It doesn't really add any time to building the character. So cutting that won't be a help. Sometimes a fluff character is the one I like the most in the entire competition. I freely admit when my players derail me from whatever I am doing I go and grab characters from these competitions as ready made NPCs.

Everytime I have judged (so far 3 times) I spend about 3 hours on each build. Sometimes longer. I check to make sure everything works as stated, and if unsure/don't have the books I get someone to help. This is easily the longest part of the competition for me. I'm not willing to cut it short as I feel it is contrary to the entire point of the exercise. Also it is mean to the people who spent so long cooking up an amazing meal.

In order to make this faster a lot of players have begun putting in locations and explanations on how stuff works. This has been an incredible help. In order to speed up actual judging I don't have a solution, as I don't feel it would be fair to either the players or the judges to make them have a deadline.

However I do have another solution: run two at a time. One at the beginning of a month, and one at the 15th. Let the judges be anyone who wants to, as we currently do but they can only be judging in one of the two at any time. This means if it gets done before whatever the cut off would be (8th and 22nd by my reckoning) the judge would have a week to finish his as well if he wants to keep doing it.

Also if the entries explained pretty much everything they took that might speed it up a lot.

[arguskos]

As someone who judged, I believe, seven of these competitions, I never found judgment all that time consuming. What did take time was checking to make sure it all 100% worked, and navigating the hordes of people saying "your method is badwrong!!!1!!1!!" Avoid the pitfall of judging the judges, and ask everyone to include an explanation of their tricks and whatnot, and you have a far more streamlined process.

As a side note, I have thought about returning, but I've got a lot spinning right now and can't find the time.

[Thurbane]

I agree that judges need some sort of deadlines, but these should reflect the number of entries and a couple of other factors.

Having said that, and while I realize that judges are very selflessly volunteering their time for the benefit of the community, there seem to be a lot of "dog ate my homework" excuses for judges running way over time. The number of HDD/MS Office crashes have been what I would consider very much above the statistical norm.

I would ask if people volunteer to judge, don't do so on a whim. Realize up front that you are offering to volunteer for what is a time intensive task, and that many people (contestants and fans) are going to be awaiting your judging. Also, if you realize early on, after volunteering, that real life or other considerations are going to make timely judging impossible, don't be afraid to own up to it.

[Amphetryon]

Would it be reasonable to set a timeline for judging based on the number of contestants? To throw a number out there, if we said two days per entry as a fair deadline for judging, would that seem fair to everyone involved?

[Kesnit]

Would it be reasonable to set a timeline for judging based on the number of contestants? To throw a number out there, if we said two days per entry as a fair deadline for judging, would that seem fair to everyone involved?

I like that as a base, but maybe there should be a flat buffer of 1 or 2 days. When I've judged, I know I can do several a day, but the judges that write novels to explain their judging take a lot longer. If someone misses a day, that's more entries they have to judge the next day.

[Thurbane]

Yeah, maybe 2 days plus 2 more per entry.

Either that, or say 1 flat week, or 2 days per entry, whichever is greater.

What's the record for number of entries? Because once you hit 15, you're looking at a month of judging...

[potatocubed]

I think the key to success is a judging deadline, although not having judged I couldn't even guess what a reasonable time period would be.

I'd also be inclined to remove the challenges - I can see how that might lead to people missing out on trophies occasionally, but it just seems to devolve into 'but but but' all the time. If a judge routinely makes mistakes then disinvite them from judging - otherwise just accept their rulings.

Contestants, on the other hand, should be prepared to call out their build's special features ahead of time and justify anything that might look odd. Like "I know you can't normally take Cheesomancer without being a ghostwise half-grey human trollblooded ghoul, but the feat OMGHax taken at first level makes an exception."

[Heliomance]

I'm happy to chair again also. Don't have time for a longer post now, will say more later.

[gbprime]

We're a week further into waiting on judging for our two running Iron Chefs. Anyone else have idea-fodder for our new chairman (whoever that turns out to be)?

[Zonugal]

Perhaps we could look at reshaping the purpose, and underlying conception, behind the Iron Chef competitions. Ideally they are maintained & performed as to emulate the television show but there is a breakdown in this almost faithful adaptation. And that is the judging. On Iron Chef the judges taste the meals and give a very quick blurb (and score) to it without going into a complete examination (to be fair the contestants are fully viewed & watched so they have that advantage over us character builders). But having built some characters for different Iron Chef competitions it was rarely my goal to build a character so dependent on the rules, but rather just a character who has some perks/talents.

Perhaps the judging could be re-geared to focus it not on the mechanical aspects but more on a quick look of the build, and than giving their brief thoughts (and scores) concerning it. Looking at the general basis given for how judges operate (originality, power, elegance and use of the secret ingredient) only one of them (power) needs the higher look at rules (and thus a problem of judging taking upwards of a month).

But how do we remedy this one, singular problem? I feel it is found in the cooperation of the builders with the judges in outlining, before submitting, their characters talents/strengths. Explicitly say what the character's goals/purposes is and how, through techniques or abilities, achieves those goals/purposes. But this only works if the judges streamline their inspection method and to a degree trust the builders. After all most of the competition lies on what the build is (classes, prestige classes and how they both mix with those abilities) as opposed to the numbers within it.

Now an element of fear washes over some in that what may befall the competition is judges using purely gut reactions for their scores, but if we can recall our inspiration that is how the culinary judges on Iron Chef do it...

[gbprime]

IMO, the big hurdle to judging is being familiar with all the rules that have been pulled from all corners of 3.5, and owning copies of all the books they come from. (The former is actually more of a barrier since 3.5 has sunsetted.) After that point, some judges are breif, some are more in depth. And I don't think we should fault a judge for being too far out on either end of that scale.

Personally, I'm one of those rules magnets. I don't have them all memorized (and I'm certainly not always correct), but I'm familiar enough with them to know where to find them on short notice and re-read them in the context of a build. For me, that means it takes about an hour or two to judge each submitted build and produce a few paragraphs of formulated opinion. Personally I wouldn't want to go faster than that (these builds are chock full of "not so obvious" sometimes) nor would I want to spend much more time on them (if I didn't "get" your build after staring at it over a pile of rulebooks for 2 hours then I'm not going to "get" it at all), but neither would I pass judgement on a judge that does choose a faster or slower approach.

But... 6 weeks and counting to get three judges to weigh in on nine builds? Length of time considering each build isn't the issue there. What this says is that the judges are not starting to judge within a few days of the entries being posted.

That could just be real life getting in the way. Or not. But the SOLUTION IMO, is to get more judges. We start with 12 contestants, we end up with 6 builds. So we need to start with 5 or 6 judges, and then close judging after 2 weeks.

...

So here's my brainstorm... how do we give judges more incentive to judge? The winning contestants get bragging rights and the occasional cool trophy, so they have plenty of incentive.

Is prize support too wacky an idea? Would an online retailer willing to give discounts to the winners and judges just break the system by flooding the thing with entries? Would it kill the spirit of the competition?

[dextercorvia]

IMO, the big hurdle to judging is being familiar with all the rules that have been pulled from all corners of 3.5, and owning copies of all the books they come from. (The former is actually more of a barrier since 3.5 has sunsetted.) After that point, some judges are breif, some are more in depth. And I don't think we should fault a judge for being too far out on either end of that scale.

Personally, I'm one of those rules magnets. I don't have them all memorized (and I'm certainly not always correct), but I'm familiar enough with them to know where to find them on short notice and re-read them in the context of a build. For me, that means it takes about an hour or two to judge each submitted build and produce a few paragraphs of formulated opinion. Personally I wouldn't want to go faster than that (these builds are chock full of "not so obvious" sometimes) nor would I want to spend much more time on them (if I didn't "get" your build after staring at it over a pile of rulebooks for 2 hours then I'm not going to "get" it at all), but neither would I pass judgement on a judge that does choose a faster or slower approach.

But... 6 weeks and counting to get three judges to weigh in on nine builds? Length of time considering each build isn't the issue there. What this says is that the judges are not starting to judge within a few days of the entries being posted.

That could just be real life getting in the way. Or not. But the SOLUTION IMO, is to get more judges. We start with 12 contestants, we end up with 6 builds. So we need to start with 5 or 6 judges, and then close judging after 2 weeks.

...

So here's my brainstorm... how do we give judges more incentive to judge? The winning contestants get bragging rights and the occasional cool trophy, so they have plenty of incentive.

Is prize support too wacky an idea? Would an online retailer willing to give discounts to the winners and judges just break the system by flooding the thing with entries? Would it kill the spirit of the competition?

I'll admit that the first time I judged, I was lured by the possibility of a judge's ribbon. Unfortunately, that was the competition where Jumik's laptop failed.

My judging method is very close to yours. It takes me a couple of hours per build. E6 was more forgiving, I think I averaged about 1hour/build. I agree that closing judging after 2 weeks and taking whatever judgings have been offered to that point as official.

[Private-Prinny]

So here's my brainstorm... how do we give judges more incentive to judge? The winning contestants get bragging rights and the occasional cool trophy, so they have plenty of incentive.

Is prize support too wacky an idea? Would an online retailer willing to give discounts to the winners and judges just break the system by flooding the thing with entries? Would it kill the spirit of the competition?

I think this is probably a step in the right direction. This and enforcing a hard limit on time allowed for judging should cut down on the time taken for each round.

As for prize support, it would probably have to be all virtual. In order to speed up the trophy distribution and increase variety, we could possibly recruit more than one artist, who will make a draft of the trophy, and allow the winner to choose which one he wants engraved. Any prize with monetary value would be hard to obtain, and may flood the competition with people who are simply attempting to get a prize, which would lead to longer judging periods (again).