# (When) are reference requests actually on-topic here?

I recently realized that we have some contradictory documentation on our site (and, probably, contradictory beliefs and opinions among users) about the topicality of questions asking for off-site references. In the interests of consistency, and of ensuring that we don't offer contradictory advice to new users, I'd like to propose that:

1. we take this opportunity to discuss and vote on what our actual policy regarding such questions should be, so that we can get everybody on roughly the same page, and

2. having done that, we update the help center, the list of canned off-topic close reasons, the reference-request tag wiki and any meta posts linked from those pages to all say the same thing.

### The current situation

When voting to close a question as off-topic, one of the three "canned" close reasons that one can choose currently reads:

Requests for literature, software or similar recommendations are off-topic here. For details, see: Do we want “literature recommendations” and similar “list/subjective questions”?

The close reason text links to our very first meta question, whose top-voted answer, written by Stack Exchange Director of Community Development Robert Cartaino, states in no uncertain terms that "You don't want this here." Together, the close reason and the linked meta thread certainly give, at least to a casual reader, the impression that all requests for off-site resources are categorically off-topic here.

On the other hand, we also have a tag (with 263 unclosed questions, as of this writing) whose tag wiki, while citing the very same meta thread from 2011, actually describes a much more nuanced policy that only open ended "big list" style requests (like "What's your favorite introductory crypto book?") should actually be considered off-topic, whereas requests that can be definitely answered (like "Where is scheme X officially defined?") should in fact be allowed.

Finally, the "What topics can I ask about here?" page in our help center, which is supposed to be the definitive description of what's on-topic at this site, currently makes absolutely no mention at all of literature or reference requests in any way.

Thus, we're in the somewhat uncomfortable situation of having the site interface (specifically, the "vote to close" dialog) advising people to close reference requests as off-topic for a reason that:

• is not backed up by the supposedly definitive description of our site scope in the help center, and
• can be interpreted in a way that seems to contradict the tag wiki and the current consensus on meta.

### History and past discussion

Both the current canned close reason and, a bit paradoxically, the tag wiki link to our very first meta question from July 2011:

The top-voted answer to this question, by Robert Cartaino, could be fairly summarized with the single word "no." However, it's worth noting that Robert's answer specifically talks about open-ended "Big List™ questions" like "what books should I read", not about e.g. requests for a specific reference document.

Nonetheless, that meta thread does appear to have created the impression, at least among some users, that all requests for off-site resources were forbidden here. This impression is evident in the subsequent meta question, posted in September 2013, about the existence of the tag:

The consensus established in that discussion appears to be that, while open-ended "big list" questions (like "What's your favorite introductory crypto book?") indeed are and should remain off-topic, specific reference requests that can be definitely answered (like "Where is scheme X officially defined?") should in fact be allowed.

The same issue was again raised here on meta (by a new user, who again appears to have been under the impression that all reference requests were considered off-topic here) in June last year:

The answers to that question again cite and affirm the consensus from 2013 that specific reference requests are on-topic here.

The advice currently given in the reference-request tag wiki matches and appears to be based on the consensus established in those two meta discussions, although, oddly enough, the tag wiki doesn't actually link to either of them, but only to the one from 2011.

Meanwhile, earlier last year, we voted on our canned "off-topic" close reasons:

The current "requests for recommendations" close reason (quoted above) came in second, at 10 votes for and one against, replacing an earlier similar canned close reason that used to read "reference recommendations" instead of "literature, software or similar recommendations".

In fact, one of the stated reasons for the changed wording was to "distance it from on-topic questions". In my personal opinion, however, that attempt was not particularly successful (as also indicated by the June meta question) — the close reason, as currently written, still appeasr to suggest that all requests for references to off-site literature are off-topic.

This close reason is quite commonly used; within the last 90 days, it was used to close 47 questions, or 12.5% of all closed questions during that period (source, 10k+ only). Many new users will see it, either on their own closed questions or on those asked by others. Established users with access to the closing and flagging tools will take it as guidance on which questions should be closed. If it does not clearly communicate our actual policy, it should IMO definitely be fixed.

### What should we do?

As there appears to be some genuine disagreement on what exactly our policy regarding such questions is or should be, I'm going to take SEJPM's advice and post my own proposal below to be voted on. Of course, if anybody has a better suggestion, please feel welcome to post your own answers too.

Following SEJPM's suggestion, I'd like to propose that, in one month from now (on 29th May 2017), the top-voted answer should be considered to reflect current community consensus, and enacted accordingly, provided that it has reached a score of at least +6 with at least twice as many upvotes as downvotes.

In any case, whatever we actually decide the policy should be, I do hope that we'll resolve the apparent inconsistencies between the canned close reasons, the help center and the relevant tag wikis. In particular, if we wish to include references from those places to meta discussions, we should take care to actually link to something that clearly documents all aspects of the current consensus, not just to an old discussion that has since been clarified and amended elsewhere.

### Results

After a month of discussion and polling, it looks like the proposal to allow (some) open-ended reference requests and close the rest as duplicates enjoys community consensus as defined above, with a score of 8 votes for (and none against) as of this writing. In the interest of clarity, let me attempt to summarize the key points of the winning proposal below, as I understand them:

• Requests for specific literature references (e.g. "Who invented X?" or "Where is Y officially defined?") are on-topic, as described here and here. Such questions should generally be tagged with the tag.

• Open-ended literature recommendation requests (e.g. "What introductory crypto books would you recommend?") are generally off-topic, as described here, except as noted below.

• As a specific (new!) exception to the previous rule, the community may choose to establish and curate specific "canonical" Q&A pages for commonly asked literature recommendations, similar to e.g. "The Definitive C++ Book Guide and List" on Stack Overflow. Other open-ended recommendation requests should be closed as duplicates of the appropriate canonical question, where one exists.

Such canonical questions (and their answers) should be marked as Community Wiki by ♦ moderators (ordinary users can only do that for answers), both to indicate that they are collaboratively maintained by the entire Crypto.SE community, and also to prevent votes on them from artificially inflating their original authors' reputation. (Votes on CW posts do not give any rep.)

No specific procedure (yet) exists for proposing or creating such canonical recommendation questions, but it should be done "in a coordinated way" where strong demand for such a Q&A clearly exists. If any disagreement on the appropriateness and desirability of a particular question arises, the matter should (as usual) be discussed on meta.

• The current canned off-topic close reasons and the help center should be updated to make all this clear, e.g. as suggested here (but with the "canonical recommendation Q&A" exception, as described above, also included in the help center text).

Any corrections or clarifications to the summary above are welcome.

• I'm gonna add a few other answers so people don't have to be bothered with creating them and have more than a single answer to vote on. The next step would be to get hard criteria into your question to get a clear and well-documented community-decision within a reasonable time-frame. My proposal would be: 1 month time (including mod-feature), top-voted answer gets accepted and enacted, but has to have at least +6 with at least 2x as many up- as down-votes. – SEJPM Apr 28 '17 at 21:08
• Oh and we would need a clarification on how to use meta for "random bypassers" to clarify that "down-vote = I disagree" and "up-vote = I agree". – SEJPM Apr 28 '17 at 21:26
• Just a heads-up - we've passed 29th May 2017. ;) – e-sushi Jun 1 '17 at 11:29
• @e-sushi: Thanks for the heads-up. :) Looks like SEJPM's suggestion to allow some canonical open-ended requests is the winner. I've marked it as accepted and added a summary of its key points into the question above; now we just need to implement it. – Ilmari Karonen Jun 2 '17 at 15:05

## Allow (some) open-ended requests and close the rest as duplicates

This is effectively the same proposal as Ilmari made in his opening post and his answer with one key difference: We allow exactly one open-ended book or whatever recommendation and curate it (if possible).

This would bring us more in-line as to how this matter is handled on other StackExchange communities, especially Stackoverflow which has such lists for C++ and C books for example.

All ressource modifications would be the same as in Ilmari's answer and we handle the cases where we think and agree that they can be useful for a large number of people as manual exceptions to the outlined rules or such questions are created in a coordinated way as community wikis if we discover a strong demand for such a recommendation list (potentially with prior meta consensus).

New questions would be closed as duplicates of the existing canonical recommendation Q&As and questions which don't have enough "potential" as reference recommendations interesting for the masses get closed with the custom reason.

• This might at least be worth trying. The only real issue I see with this suggestion is that it could create a temptation to close recommendation requests as duplicates of the "canonical" question(s) even when the answer(s) to the canonical question don't actually provide any recommendations matching the OP's specific request. Arguably, that could be less helpful than just closing such requests as off-topic, because we'd basically be telling the OP "your answer is right here!" even though it really isn't there (or at least isn't presented in a way that would make sense to the OP). – Ilmari Karonen Apr 28 '17 at 22:05
• Basically, I'm imagining a scenario where we, say, set up and curate a canonical list of introductory crypto books for beginners. A new user comes and asks a question, saying something like "I'm a 21-year-old math student with a strong background in topology and algebra, and I'd like to learn about elliptic curve crypto. I've read your canonical book list, but I still have no idea what book I should start with." Now what do we do? Close as off-topic? Just close as a dupe anyway, even though we know it's not helpful? Leave open? Answer in comments? Ninja-answer before closing? Something else? – Ilmari Karonen Apr 28 '17 at 22:25
• I'm all for a community wiki dedicated to reference request. I've been trying to find the names of whoever invented SIDH and I'm not coming up with anything. – nope May 1 '17 at 4:09
• @floorcat: "Who invented SIDH?" sounds like the kind of non-open-ended request for a specific reference that would be on-topic under my proposal (and what arguably is current policy, too), without needing a canonical CW question. (Also, Wikipedia says De Feo, Jao, and Plut, 2011.) – Ilmari Karonen May 1 '17 at 11:58
• I do like your proposal @IlmariKaronen , but I also like the idea of the questions and answers being wiki's by default. That could bump into issues with the Crypto.SE site not being a wiki, but I think we do need to allow some reference requests. – nope May 1 '17 at 15:15
• FWIW, if I understand this proposal correctly, this recent question (requesting a list of introductory crypto books using a concrete (non-asymptotic) approach to security proofs) looks to me like a perfect example of a question that might be a candidate for turning into a canonical CW reference list, if this proposal passes. – Ilmari Karonen May 24 '17 at 19:34
• @IlmariKaronen Prepared it for that purpose. Just waiting for this poll to end to reopen that as community wiki (in case this answer wins, if not it'll simply remain on-hold). – e-sushi May 24 '17 at 19:53
• @e-sushi when we upvote a question (or answer) that has became a community wiki, the original author doesn't receive reputation points? – Hilder Vítor Lima Pereira May 31 '17 at 6:09
• @Vitor That's correct — neither from upvotes, nor from downvotes. The SE system handles "community" Q&As as if the community wea a user, so no one gains reputation from up/down-votes when this are converted to wiki. – e-sushi May 31 '17 at 8:44
• Another question: by "curate it", does SEJPM mean to transform the question and its answers in community wikis? I'm asking it because recommendations questions/answers use to have a lot of upvotes, so, they can somehow deregulate the actual scores. Also people may start to create recommendations questions with the only goal of getting those upvotes. – Hilder Vítor Lima Pereira May 31 '17 at 9:27
• @Vitor I think the idea was to have exactly one answer with such Qs. If said answer only has content from one author or 90%+ from one author I don't see a problem with rep awarding. Otherwise a CW does sound like the best option. By "curated" I mostly meant that those lists need to be up-to-date. – SEJPM May 31 '17 at 9:45
• Well, the problem is that authors of really good answers to "normal" questions may receive only few upvotes while one that (typically) just copies some links to books' web page is expected to receive several upvotes. I'm saying it based on my experience at math.exchange, where reference request questions are usually highly upvoted and their questions as well. Much more than the "normal" questions on the same topic. For example, "Good book for self study of a First Course in Real Analysis" is among the 30 top questions in a total of 65025 questions about real analysis (35 / 65025 = 0.00046) – Hilder Vítor Lima Pereira May 31 '17 at 10:35
• Looks like this answer is the winner. I've tried to summarize its key points in my edit to the question above, but please correct me if I misinterpreted anything. – Ilmari Karonen Jun 2 '17 at 14:56
• @IlmariKaronen looks good to me, well done! – SEJPM Jun 2 '17 at 15:03

## Only open-ended recommendation requests should be off-topic.

The current "requests for recommendations" canned off-topic close reason should be changed so that it's clear that specific, definitely answerable reference requests are on-topic here. I would suggest something like the following wording:

Open-ended requests for literature, software or similar recommendations are off-topic here. For details, see the reference-request tag wiki.

A note to the same effect should be included on the "what's on-topic here" page in the help center, e.g. as follows:

### Can I ask people to recommend their favorite crypto books, websites, software, etc. for me?

No, such open-ended recommendation requests are a poor fit for the Stack Exchange Q&A model, and are considered off-topic here. If you have at least 20 rep points here (or on any other Stack Exchange site), you may ask for such recommendations in chat.

However, requests for specific literature references (e.g. "Where is scheme X officially defined?") are on-topic here. See the reference-request tag wiki for more information.

The tag wiki, which the proposed close reason and help center text above both link to, can stay substantially as it is now. (In any case, unlike the close reasons and the help center, it's freely editable by anyone, subject to review for users with less than 5k rep.) However, it should be edited to include links to the more recent meta threads here and here.

• Note for posterity: While this answer did not win the poll, its specific suggestions were incorporated by reference into the most upvoted answer, which in addition proposed the option to create a limited number of community-maintained canonical open-ended recommendation questions. – Ilmari Karonen Jun 2 '17 at 15:01

## No reference-requests and -recommendations are on-topic

This would be the policy as allegedly practiced already by some members of our community: Don't allow any reference recommendations / -requests.

As for the potential why: Quite often they are only one quick search away and mostly only encourage link-only answers (what are you going to answer with except for a link to the paper and a proper citation maybe!?) and thus don't make good answer-material.

The custom close reason would have to be updated to

Pure reference-requests and reference-recommendations are off-topic here as they more often than not encourage link-only answers.

The help center would need a short similar text similar to the above close reason or my argumentation and the tag wiki for would need a similar update and clarification of when it should be used.

• This answer has my down-vote (because as the current majority author I can't down-vote it). – SEJPM Apr 28 '17 at 21:23
• I'd really appreciate contributions to make the updates on the help center and the wiki more explicit and perhaps some proposal on when to still use the tag. I couldn't really find a place for this tag with this implementation, but others may have smart ideas. – SEJPM Apr 28 '17 at 21:25