CZ:Managing Editor/2012/004 - Approval of Editor-authored articles when no appropriate nominating Editors available

Statement of problem

Existing policy regarding article approval does not take into consideration the not uncommon circumstance in which an Editor has predominantly authored an article to a stage ready for consideration of Approval but no other Editor is available to consider nominating it for Approval, owing a dearth of truly active Editors in any of the article’s Workgroup categories.

The question: Until Citizendium grows to the stage when the current dearth of Editors no longer exists, or no longer is of severity to present a serious problem finding appropriate Editors to nominate, or reject the nomination of, articles for approval, should Citizenium allow Editors with established track-records nominate for Approval the articles they predominantly authored, with following provisos:

1. The article belongs to one of the Workgroup categories for which the Editor has Editor status.
2. The Approvals Manager judges the Editor's responses to comments from authors-at-large to be satisfactory in terms of edits to the article and to rebuttals to critiques.
3. The Editorial Council concurs with the Approvals Manager's judgment.

Existing applicable policy

Charter

• Article 36

• The Managing Editor has the following duties:

1. to ensure by means of executive decisions that the principles and policies of the Citizendium are effectively and coherently observed; such decisions shall be based on established policy where defined;
2. to make interim decisions on behalf of the Editorial and Management Councils when established policy does not provide guidance; these decisions shall be overridden by the establishment of relevant policy;
3. to represent the Citizendium in its relations with external bodies, such as the mass media, and academic or non-academic institutions.

Decisions by the governing bodies

Editorial Council

From "CZ:Approval process" (http://en.citizendium.org/wiki/CZ:Approval_Process):

If the editor has worked on it [an article] herself as an author, he/she asks another editor to approve it; or, if there are several editors all doing significant work as authors on the article, then at least three of them can agree to approve it. (These rules are to prevent a single person from approving his or her own work without involving review by experts who were not authors.)

Those rules do not take into consideration the circumstance described above, under "Statement of problem".

Seeking outside reviewers is unsatisfactory at the present time (July 2012) inasmuch as the prestige ranking of Citizendium is not high enough to offer incentives to busy and career-oriented specialists to peer-review articles for Citizendium. In the future, the use of outside reviewers may be a practical option.

The Editorial Council’s regulation, EC:R-2011-027/ Approval process (http://ec.citizendium.org/wiki/EC:R-2011-027) likewise does not set policy for the particular circumstance described above, under "Statement of problem".

Draft decision

The text below is what I plan to decide in this case. Feel free to edit the text if you think this improves it. If your edits require discussion, please use the dedicated section below. Editing and discussion in this "Draft decision" section shall stop 24h after my last edit to it.'

Yes: Until Citizendium grows to the stage when the current dearth of Editors no longer exists, or no longer is of severity to present a serious problem finding appropriate Editors to nominate, or reject the nomination of, articles for approval, Citizenium will allow Editors with established track-records to nominate for Approval the articles they predominantly authored, with the following provisos:

1. The article belongs to one of the Workgroup categories for which the Editor has Editor status.
2. The Approvals Manager judges the Editor's responses to comments from authors-at-large to be satisfactory in terms of edits to the article and to rebuttals to critiques.
3. The Editorial Council concurs with the Approvals Manager's judgment.

That affirmative decision will enable the Approval Process to accelerate without compromising article quality given the provisos stated that provide safeguards by both the Approval Manager and Editorial Council.

Indeed, quality might improve, as the Editor's reputation is more at stake.

Editors whose articles are being considered, and authors-at-large, may comment on the Talk Page of the article. An Editorial Council member who happens to be an Editor-in-consideration is expected to recuse himself/herself from concurring or not with the Approval Manager's judgment.

It is intended that this decision stay in effect until most Workgoups have adequate numbers of truly active Editors to support the Approval Process, or until overridden by the Editorial Council.

Discussion of Draft decision

When reading or editing this section, please keep in mind that the current version of the draft decision might be different from the one referred to by previous commenters.

This decision does not examine the purpose of "certification", which of course is to make sure that such articles are accurate, complete and well expressed. When there are very few or perhaps no editors at all that can be found to undertake a review, the logical thing to do, and the one most consistent with the objectives of certification, is to shelve this process in any situation where insufficient editors can be found.

The effect of "certification" is to make any revision of an article so cumbersome as to discourage any attempts at improvement. A revision of a certified article requires formal procedure, and where few or no experts are available, resistance to change prevents any improvement beyond fixing of typos. These observations are not hypothetical; examination of past attempts to revise certified articles shows that revisions are very, very difficult already, even where several contributors engage.

To "certify" an article based upon the assumed "track record" of reputable authors without expertise is very ill advised. John R. Brews 14:46, 8 July 2012 (UTC)

To make a point, I might nominate these articles for certification, all authored by myself, and in most cases on topics where I have no expertise. I am confident that most of them would become certified under the proposed decision. John R. Brews 14:55, 8 July 2012 (UTC)

To respond to your last point, this decision applies only to draft articles written predominantly by an Editor who nominates the article himself/herself. Because you are not an Editor in any of the Workgroups whose categories the articles you listed belong, the scenario you describe does not apply. You may suggest any of those articles be considered for approval, but it requires a ToApprove Editor in one of the article's Workgroup categories to advance the Approval Process.

Indirectly your point contributes, however, as I will now edit the 'Draft decision' to include the specific proviso that the article written predominantly by the 'self-approving' Editor belong to one of the Workgroup categories for which the Editor has Editor status. That would disqualify an Editor, say, with only Health Sciences Editor status as a ToApprove Editor for an article that Editor wrote in a non-Health Sciences category. That will give the decision a third safeguard to the two already stated.

I will respond to your other comments after I make that change. Anthony.Sebastian 22:52, 8 July 2012 (UTC)

+++++

Regarding your comment, "To "certify" an article based upon the assumed "track record" of reputable authors without expertise is very ill advised.": I would agree with you if track record were the only proviso. The decision provides safeguards much beyond that.

Regarding your comment, "The effect of "certification" is to make any revision of an article so cumbersome as to discourage any attempts at improvement.": I cannot agree with that. Approved articles can be improved upon in its accompanying draft version, and re-approvals are often easier to obtain, certainly not discouragingly cumbersome. Anthony.Sebastian 23:23, 8 July 2012 (UTC)

Hi Anthony: Your suggestions for changes are an improvement, though not sufficient to reassure me. It is not possible for me to agree with you that changing approved articles is certainly not discouragingly cumbersome, as that claim contradicts my own direct experience. John R. Brews 03:28, 9 July 2012 (UTC)

My overall reaction to expediting the approval process is that this is a very bad idea, as it will serve two functions for certain: (i) making it hard to improve articles, and (ii) making approved articles even of lower quality than they are now.

What is the upside to this proposed change? It would seem that the only upside would be to present the appearance of CZ having actually in place an approval process that guarantees more reliable articles than, say, WP. I see absolutely no reason to believe any such thing. John R. Brews 03:33, 9 July 2012 (UTC)

Hi John: Thank you for your challenging arguments. Of course, any of our Approved articles could be improved in some way or another, and it might be useful to think about those ways sometime. But an article subject to improvement can still provide a core of reliable and comprehensible information, and that is one of the big things we strive for in our articles. Approving developed articles that provide the core of the subject in a reliable, coherent, well-written way both contributes to our mission and gives incentives/rewards to our authors who are responsible for our productivity.

As usual, your remarks inspire ideas. We might consider rewording the banner above the text of an Approved article, which now reads:

"Article approved by an editor from the listed workgroup. The Biology Workgroup is responsible for this article. While we have done conscientious work, we cannot guarantee that this article is wholly free of mistakes...Help improve this article further on the draft page!"

I'm not feeling creative enough at the moment to rewrite it, but I suspect it could be rewritten to be more informative about the relationship between the locked version and its accompanying draft version. And more. Anthony.Sebastian 04:28, 9 July 2012 (UTC)

←Anthony: Thanks for your patience here. I belief the crux of the argument in favor of this proposal is your remark:

"Approving developed articles that provide the core of the subject in a reliable, coherent, well-written way both contributes to our mission and gives incentives/rewards to our authors who are responsible for our productivity."

I believe that approval of articles that "provide the core of the subject in a reliable, coherent, well-written way" do serve the mission of CZ. Unfortunately, we are not in a position at the moment to identify such articles because we do not have the expertise required.

For example, although we have a Music Group that might be aroused to propose my recently written articles Pitch (music), Tone (music) and Note (music), the Music Group has not so far made any commentary upon these articles, even though directly solicited on their Talk pages. They seem to have been inactive for years. That is also the case for finding editors for virtually all the articles on this list. On this basis, the proposal would have exactly no effect upon the status of these articles.

On the other hand, Set theory became an "approved article" based upon the opinions of a few collaborative and none-too-critical parties, and attempts to make simple improvements became impossible. The very few experts in command of this article have locked it up, a reflection of the natural tendency of authors to defend their prodigy against all revision.

CZ simply is in no position to implement its goals in this regard. It faces two problems: there aren't experts available to assess content in most areas, and there are so few contributors interested in most topics that approval is too readily obtained (or refused, as the case may be).

The motivation you suggest for approving articles:

"gives incentives/rewards to our authors who are responsible for our productivity"

may apply to some authors who have been almost entirely responsible for a particular article, and feel a sense of authorship. However, for those who wish to add to existing articles, possibly the majority of contributors, the opposite is the result.

The approved article status is best left unused under these circumstances where it cannot be satisfactorily implemented. John R. Brews 16:03, 9 July 2012 (UTC)

John, I commend you, with admiration, for your perseverance. I have no illusions that this decision, which the Editorial Council can override, will solve all the problemmatic aspects of the Approval Process. The problems you describe should not be ignored. If someone as productive as you are has problems with the Approval Process, they most emphatically should not be ignored. CZ seems to be now at a heightened level of self-reorganization. This decision, if it passes EC muster, might turn out to be just a temporary component in a trajectory to rethinking "Approval" at a fundamental level of inquiry, as part of that self-reorganization. Anthony.Sebastian 21:43, 9 July 2012 (UTC)

Decision

Until Citizendium grows to the stage when the current dearth of Editors no longer exists, or no longer is of severity to present a serious problem finding appropriate Editors to nominate, or reject the nomination of, articles for approval, Citizenium will allow Editors with established track-records to nominate for Approval the articles they predominantly authored, with the following provisos:

1. The article belongs to one of the Workgroup categories for which the Editor has Editor status.
2. The Approvals Manager judges the Editor's responses to comments from authors-at-large to be satisfactory in terms of edits to the article and to rebuttals to critiques.
3. The Editorial Council concurs with the Approvals Manager's judgment.

That affirmative decision will enable the Approval Process to accelerate without compromising article quality given the provisos stated that provide safeguards by both the Approval Manager and Editorial Council.

Indeed, quality might improve, as the Editor's reputation is more at stake.

Editors whose articles are being considered, and authors-at-large, may comment on the Talk Page of the article. An Editorial Council member who happens to be an Editor-in-consideration is expected to recuse himself/herself from concurring or not with the Approval Manager's judgment.

It is intended that this decision stay in effect until most Workgoups have adequate numbers of truly active Editors to support the Approval Process, or until overridden by the Editorial Council.

Post-decision comments

How is "an adequate number of editors" to be decided? Instead, this decision should be reviewed on a regularly scheduled basis. John R. Brews 04:52, 10 July 2012 (UTC)

It seems to me that any "decision" of this nature requires a general vote and cannot be implemented at the will of the Managing Editor, nor by the Editorial Board by itself. John R. Brews 04:56, 10 July 2012 (UTC)

The Councils can over-rule any decision the Managing Editor makes, and those decisions should be compatible with the Charter and existing rules, but in the absence of any policy from the Councils, the Managing Editor is free to make interim decisions. So Anthony can do this unless the EC says otherwise. Currently there is no movement among EC members to block the him in this matter. Furthermore, the EC would be able to authorise this too, as it has responsibility for Editors and content.

Of course, I am just stating here what the Managing Editor can do. You could argue that this is a major change and should be put to a vote, but in that case we would be here another six months. John Stephenson 08:35, 10 July 2012 (UTC)

The problem seems to be the low level of activity on the EC. They don't seem to have made any decisions on content issues, which is what they're there for, since last September, and no comments on such issues seem to have been posted on their wiki for weeks. They replaced the old approval system with a new one, which seems pretty vague to me, and left it to Approvals Manager(s) to interpret. Now Anthony-as-ME is giving instructions to Anthony-as-AM as to how to do so. Not really what was envisaged, I suspect, but maybe the best that can be managed right now.

There was a similar period when the MC wasn't functioning properly. Maybe this discussion should be moved to the forum. Peter Jackson 10:22, 10 July 2012 (UTC)

From what is said here, Anthony is free to make this change on his own. It is, so to speak, largesse on his part to invite any comment about it whatsoever. As they say in the army, however, it is better to rule by consent than by decree, even if it is not required by the rules.

Anthony has not argued with any persuasiveness that easing the approval process will be useful, the only pro suggested being that it will encourage some authors to make more contributions. The "side-effect" of course is that approval makes changes to articles nigh impossible. So, apart from pinning medals on a few authors, this proposal raises the prospect that a few determined soles will be able to dictate what CZ says on some topics, and that will serve to dissuade participation not only by authors, but by readers too.

At the moment, anyone can contribute to this discussion, but no-one has. So the lack of interest is general, and going to the Forum seems unlikely to attract additional interest. John R. Brews 14:31, 10 July 2012 (UTC)

I've raised the question in the Forum to see what others think. John R. Brews 15:50, 10 July 2012 (UTC)

(Since I was travelling (participating at a converence) I can only comment on this issue now. Sorry)
As I see this, this decision is based on a misunderstanding of the Approval process as defined by EC:R-2011-027. (The sentence cited from CZ:Approval Process refers to the approval process valid until 2011.)

• The EC regulation does not exclude an Editor (not even an Author) of an article to start the Approval process.
• Main Authors of the article are also invited to put a review statement on the /Approval subpage.
• It is in the discretion of the Approval Manager (and/or the EC) to judge whether the reviews (including the statements of the authors) suffice to approve the article.
• Even in the extreme case that there is only the author's statement certification is possible if the author can be considered a reliable expert. (This should be considered independent of Workgroup affiliation.)
• However, since correctness is not the only criterion applicable for approval, usually other Editors (including non-experts) should check the article for readability, consistency, etc. Thus an Approval based on a single statement by the article's Author should happen rarely.

Certification is only the last, and purely formal, step by that the AM declares that he is satisfied by the statements on the /Approval subpage.
Therefore, this Decision does not create any new possibility. --Peter Schmitt 00:39, 11 July 2012 (UTC)

Peter, I may have misunderstood. Do you interpret EC:R-2011-027 as no longer requiring, in the ToApprove or Approved sections of the metadata page, the name of an Editor from at least one of the Workgroup categories of an article before the AM can certify the article for Approval?

The reason I ask is that if the ToApprove or Approved sections of the metadata page does not show the name of an Editor from at least one of the Workgroup categories of an article under consideration for Approval, and the AM certifies the article for Approval, then Citizendium states, implicitly, that Approved Articles need not have the support of a qualified expert. —Anthony.Sebastian 01:48, 11 July 2012 (UTC)

The Metadata template was designed with the "old" (provisional) approval process in mind and does not fit the "new" process. Adapting the template is difficult because any change would also affect existing pages, and it involves the complicated subpages template and its cluster. But it is only a formality. Therefore I have suggested (and still suggest) that instead of an approving Editor the certifying AM (or/and a link to the /Approval subpage) is used. (For some time now, I have been thinking about how to resolve this, but I have not yet found a solution that satifies me.) --Peter Schmitt 16:56, 11 July 2012 (UTC)

The present procedure does appear to allow approval without the OK of an editor or even any author participating in the relevant Work Group. The present procedure also suggests final approval is at the personal discretion of an "Approval Manager", or in the event none has been appointed, then the "Secretary of the Editorial Council". This officer's ruling is final, and has no necessary connection to any guidelines or rules: the officer simply has to decide whether in their personal opinion the case has been satisfactorily made.

So, in fact, Peter is correct in suggesting the present process is even more lax than you or I thought it to be, or deem reasonable. John R. Brews 14:40, 11 July 2012 (UTC)

Yet, the present Approval process states up-front, in A-2: "Citizendium Editors shall, on the basis of their competence and expertise, be responsible for the Approval of articles selected according to guidelines established by the Editorial Council." (Emphasis added)

If an Editor is be responsible for the Approval of articles, on the basis of expertise, as the statement asserts, surely that Editor's expertise must bear on the topic of the article, ensured by his/her Editor status being in at least one of the article's categories.

I doubt Peter intended article Approval acceptable without expert support. Anthony.Sebastian 16:53, 11 July 2012 (UTC)

The Welcome Page also states it up-front: "We have 16,262 articles at different stages of development, of which 164 are expert-approved. (emphasis added) We have just 164 approved articles. Anthony.Sebastian 17:01, 11 July 2012 (UTC)

Quite correct, Anthony. The Approval process does not relax the requirements but makes them more precise. Workgroup affiliation often is too loose and sometimes too narrow. Certification requires to check if expertise acknowledged in general includes a specific topic. Thus it is more restrictive than the previously used criteria that only check formally if Workgroups coincide (while both the article's and Editor's WG assignment may be misleading or even wrong). The certifying AM only acts as a kind of a lawyer, has to justify his decisions, and is responsible to the EC. --Peter Schmitt 17:14, 11 July 2012 (UTC)

A wider scope for decision?

OK then. It would seem that the language of EC:R-2011-027 states on one hand that "Editors shall, on the basis of their competence and expertise, be responsible for the Approval of articles" and yet, on the other hand, puts the responsibility entirely in the hands of the Approval Manager, who is instructed to decide "whether the reports justify either acceptance or rejection of Approval". These reports come from all and sundry, and no guidance is provided as to how they are to be evaluated, or what weight is to be given to "experts", or how an "expert" is to be identified or ranked, or how clarity is to be balanced against specialist mumbo jumbo. Maybe, Anthony, your Decision here should address some wider issues? John R. Brews 17:49, 11 July 2012 (UTC)

The AM is in the same position as the Managing Editor of a scientific journal: He has to collect referee's reports, to read and evaluate them, and finally to decide on their basis whether to accept or reject a paper, or if additional reviews have to be commissioned. Such a decision can never be completely objective, but -- if administered with responsibility -- it is the best possible. No "guidance" can avoid this, and any "automatic" procedure will be inferior and much more prone to serious mistakes. --Peter Schmitt 00:27, 12 July 2012 (UTC)

Peter: The parallels with the Editor-in-Chief of a journal are there, but also there are differences. Two that occur to me are (i) the Editor-in-Chief invites the reviewers on the basis of their very special expertise on each article subject, and doesn't sift through a grab-bag of volunteer reports of uncertain quality, and (ii) journals are usually very specialized to begin with: they don't have anything remotely approaching the breadth of CZ. Even within this specialization, sub-editors skilled in narrower subject areas manage their areas of expertise and send the reports from experts that they have invited along with their own assessment of these reports to the Editor-in-Chief.

The point is that CZ does have a different situation, and the strategy of journals in handling their issues is not applicable directly to an open-access CZ, even supposing it had the manpower and the available linkages to the community of specialist reviewers. So some original thought about the peculiarities of CZ process might improve it. John R. Brews 11:46, 12 July 2012 (UTC)

As I've said earlier, my favorite proposal is to shelve the approval process entirely at this stage of CZ development. CZ just hasn't the ability to perform a sensible approval process at this time. John R. Brews 12:09, 12 July 2012 (UTC)