0
$\begingroup$

I know that it is possible to ask specific questions about cryptography in this site, and that generic questions are discouraged as it is not clear how to answer them (effectively general questions become a form of vandalism where time is wasted without any value gains to humanity).

However, I still think that there is a huge room to ask for the specific question that is "peer review my idea". This is specifically broad, and users are open to attack the idea from whichever angle they deem useful.

I know that this is a bit different than the traditional format, but I think there is a great need for it, specially that many academic conferences/journals are too overloaded, up to a point where the acceptance or the rejection rate of a paper has approached 50% in some cases, independent of quality (as some academics told me) [correction: by 50% I mean sort of random chance and less dependence on quality due to how conferences/journals are overloaded].

So perhaps *.stackexchange.com sites can take some of that load in order to offer some kind of preliminary peer review for preprints, before authors go on to submit their ideas to academic conferences or journals.

I think implementing this idea here can be as easy as simply adding the peer-review tag, only! So this is a huge return on investment.

I think this is very easy to implement, and will have a large return value to humanity. I imagine that such idea would eventually get adopted into several other *.stackexchange.com sites. I even think that preliminary online peer review would eventually become a standard practice before anyone goes on to submit an idea to a conference/journal (unless the person is so sure he doesn't need a preliminary stackexchange peer review).

Optionally, and eventually over the course of time, we may end up slowly perfecting and improving this idea (depending on demands of users) by adding additional changes, such as:

  • When a question gets the peer-review tag, the author will no longer have the ability to accept answers, since it is meaningless in such a context.
  • When a question gets the peer-review tag, it will have additional rating buttons (in addition to up/down votes), such as: accept because ..., accept after making changes ..., reject because ....

Update: this is not only about papers, but is about ideas in general, including apps.

$\endgroup$
3
  • 1
    $\begingroup$ Just fyi, the acceptance rate of the three top crypto conferences has not been above 50% since at least 1992. In fact there's only a single conference for which we have the data that had an acceptance rate above 50% and that was CRYPTO 1988. They've consistently hovered around a 25% acceptance rate since about 1994. $\endgroup$
    – Maeher
    Nov 18, 2020 at 15:55
  • 2
    $\begingroup$ Please do note that there is a difference between mods and the community management (CM) team. In the end this would have to go through the CM team to get implemented by the SO developers the way you describe it. Also note that a lot of times the contents of the scheme or protocol will continuously be changed and added to, making it a moving target. $\endgroup$
    – Maarten Bodewes Mod
    Nov 18, 2020 at 17:03
  • $\begingroup$ @Maeher Oops, I mean random chance with reduced dependence on quality. $\endgroup$
    – caveman
    Nov 19, 2020 at 10:27

2 Answers 2

Reset to default
3
$\begingroup$

I think implementing this idea here can be as easy as simply adding the peer-review tag, only! So this is a huge return on investment.

Technically yes. For the moderators / the SE company it would be a somewhat simple task to support this.

I know that this is a bit different than the traditional format, but I think there is a great need for it, specially that many academic conferences/journals are too overloaded, up to a point where the acceptance or the rejection rate of a paper has approached 50% in some cases, independent of quality (as some academics told me).

Yes, high prestige conferences easily reach a 50%+ rejection rate for papers. That's what makes them prestigious.

So perhaps *.stackexchange.com sites can take some of that load in order to offer some kind of preliminary peer review for preprints, before authors go on to submit their ideas to academic conferences or journals.

We can't offer peer-reviews of papers. This is simply too much to ask for from answerers. And because every contribution here is voluntary, a lot of these reviews will be left untouched because they imply so much work. This leads to frustration on the end of those seeking reviews as they don't get them. It also leads to frustration on the end of the answerers who now have to "dig" through more questions to find those they feel like answering.

To give you an impression from one of our academics:

I don't think I've ever managed to review a paper that wasn't an obvious crackpot paper in an hour. - Maeher

And 1 hour is certainly already on the higher end of time investment for the usual StackExchange answer.

Of course, the usual advice applies: Questions about very specific parts of papers are allowed. Things like "does doing X have any advantages over standard techniques" should be ok. But not full-blown open-ended peer reviews.

$\endgroup$
2
  • $\begingroup$ Yes, but (1) I am not suggesting deep peer review. Just rough preliminary ones. E.g. someone comes, throws a comment and walks away. Plus, the gist of the idea will be written in stackexchange, and no large PDFs will be used. So no strong commitments since this is a general preliminary check. (2) I added a comment about my 50% statement to clarify what I actually meant. I don't disagree with you here, it's just that I meant something else when I said "50%" (my mistake). $\endgroup$
    – caveman
    Nov 19, 2020 at 10:35
  • $\begingroup$ @caveman If the review requests are focused enough, they should make for fine questions anyways. If you need "too long" descriptions chances are you'll slide into the same problem of overlong questions attracting way fewer answers. $\endgroup$
    – SEJPM
    Nov 19, 2020 at 15:35
0
$\begingroup$

What we see on this site is peer reviewed by definition. In cryptography, it is dangerous to stop the conversation. Someone hits a "like" button and the conversation stops, the discovery ends--not a good idea.

In the context of the history of cryptography, the person who disagrees, the one who tells everyone else that they are wrong, even the one with mad ideas, can sometimes be valuable, if only to keep everyone else on their toes.

$\endgroup$
1
  • $\begingroup$ Why would the discussion end? It wont be an accepted answer. There will be a distribution of comments on it. The reader will explore the opinions on the idea. $\endgroup$
    – caveman
    Dec 1, 2020 at 4:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .