If you give a paper at a university — most universities — you spend most of an hour and a half on an oral presentation of the paper. Any remaining time can be spent answering questions from people most of whom have just heard your ideas for the first time.
This reminds me of the sort of Bible study popular in the Navigators, a Christian group for college students (and also the military and some other places). In most other Bible studies I've been in, we read the passage together for the first time at the study. But in Navigators, we made a point of having each one of us read and analyze the passage individually before coming to the study.
I think it helped us get into a lot more detail, and if I remember correctly, several of my friends agreed. It's probably true across a lot of other fields of study too.
IIRC, the Institute for Advanced Study at Princeton had a weekly physics seminar which you were only allowed to attend if you were prepared to give the talk. At the start of the meeting, a name would be drawn from a hat to give the talk (on a pre-announced topic). I believe that Feynman said this in one of his books.
I attended many two-hour seminars in the philosophy and government departments for some years at LSE. This invariably involved the speaker reading aloud for at least an hour (zzzzzzz). Only during the questions did it usually become useful and interesting. I would much prefer a 15-minute summary with the option to read the paper in advance.
I think code reviews and this kind of seminar may have similar incentive problems, leading to a proliferation of implementations that are less effective for their stated purpose.
Code reviews are supposed to be about improving the code, on the principle that many pairs of eyes are better than one. They are particularly supposed to be about finding defects, rather than improving the code or getting into arguments about style.
Unfortunately, the participants are human. They get embarrassed when people point out their mistakes. Sometimes they get angry and defensive. Sometimes they pull rank, consciously or unconsciously - a junior can't possibly have found a meaningful bug in a senior's code.
Then there are the managerial incentives. Maybe developers are judged on following the process (their code must be reviewed), but otherwise only on whether they get their own features in on time. Maybe the schedule has been drawn up with insufficient time for the reviewers to do a decent job. Maybe the least experienced developers predictably turn out to be the only ones with time available for reviewing,
Then there are the career incentives. Maybe everyone knows that the code will be "thrown over the wall" to a maintenance team, bugs and all, with no negative consequences to the developers for its bugginess. Maybe getting a reputation for being "difficult" has a really bad impact in this organization, and it requires immense amounts of tact to draw attention to even an extremely blatant bug without getting this reputation. Maybe there are other inter-personal land mines. (The boss' pet will get you if you point out their bugs.)
Implementing code reviews usefully requires that the developers trust that all their peers are committed to improving the code. It requires a culture where the respected response to being shown a bug in your code is "thank you" or "nice catch". This culture needs to be modeled from the top of the technical hierarchy down. Implementing good code reviews also requires that management get out of the way, including simply being absent from review meetings. And it requires that developers have enough relevant skill and experience to find bugs in the particular code being reviewed.
Many organizations choose, consciously or otherwise, to merely check the "code reviews were done" box, thus demonstrating management's commitment to (performative) quality, without making them useful.
I recall my "first code review" experience in two different organizations.
In the first one, new hires were being asked to do reviews of already submitted code, as an "extra quality measure". I found a latent bug, which would only become an actual bug if the compiler changed. The author said "the compiler won't ever change". End of discussion. (Later, the organization switched compiler technology.)
In the second one, the team lead asked me to review some of his code. I found a problem, and reported it. The team lead's response was "the process worked!". He fixed his bug, and we eventually wound up as friends as well as colleagues.
I suspect there are similar things going on with paper presentations in academic organizations. Not the same things. But confused incentives, and perhaps selection of methodologies to avoid interpersonal drama rather than maximize either quality ot learning.
A draft of my dissertation was OK for the four full professors on my committee. But the young assistant prof (put on the committee when another professor went on leave) went through the math and found an error that rendered my estimation process invalid. So the four others had just skimmed the document. (Fortunately, I had a back-up estimation methodology, less elegant but functional when the math error was fixed.)
I believe, or at least hope, Professor Bruce Kobayashi or one of his colleagues continues to host the Law & Economics Colloquium at Antonin Scalia Law School, at George Mason University.
When I took the colloquium as a Law and Economics LL.M. student in 2019, the course was a semester-long seminar that Professor Kobayashi embedded within a law and economics faculty workshop that he hosted for outside professors and scholars who wished to present their papers to the law school faculty in a Chicago style seminar. It was the most intellectually stimulating and rewarding course that I had taken as either a JD or LL.M. student.
The Chicago style seminar should be a part of any academic institution's curriculum, or faculty workshops at a minimum, and it would be a great shame if the tradition eventually dies out.
I have at least three comments to make here. They aren't especially closely related. Given how substack works, we might be better served if I make 4 comments - this one and one for each theme.
1) To this retired software engineer, your Chicago style seminar sounds a lot like the better version of the early code review, in particular likely to have succumbed to some of the same problems.
2) When economics fails to check whether its predictions are satisfied in the real world, it might as well be philosophy or theology. I don't know how non-trivial predictions can be checked without statistics.
3) To this non-economist, more than half of the math I encounter in the writing of economists looks like part of some kind of strange ritual. It isn't real economics unless it has math; relevance and usefulness are optional. I say this as someone originally reasonably talented at some kinds of math (800 on the quantitative section of the GRE) who discovered an intense allergy to theoretical math while still an undergraduate, and so never went beyond standard STEM math and standard social science statistics.
I may elaborate on some of these in additional comments, particularly the bit about code review, where I may have some relevant expertise.
Not quite the same, but at one of the most valuable companies in the world: https://money.com/amazon-meetings-no-powerpoint/
This reminds me of the sort of Bible study popular in the Navigators, a Christian group for college students (and also the military and some other places). In most other Bible studies I've been in, we read the passage together for the first time at the study. But in Navigators, we made a point of having each one of us read and analyze the passage individually before coming to the study.
I think it helped us get into a lot more detail, and if I remember correctly, several of my friends agreed. It's probably true across a lot of other fields of study too.
IIRC, the Institute for Advanced Study at Princeton had a weekly physics seminar which you were only allowed to attend if you were prepared to give the talk. At the start of the meeting, a name would be drawn from a hat to give the talk (on a pre-announced topic). I believe that Feynman said this in one of his books.
I attended many two-hour seminars in the philosophy and government departments for some years at LSE. This invariably involved the speaker reading aloud for at least an hour (zzzzzzz). Only during the questions did it usually become useful and interesting. I would much prefer a 15-minute summary with the option to read the paper in advance.
I think code reviews and this kind of seminar may have similar incentive problems, leading to a proliferation of implementations that are less effective for their stated purpose.
Code reviews are supposed to be about improving the code, on the principle that many pairs of eyes are better than one. They are particularly supposed to be about finding defects, rather than improving the code or getting into arguments about style.
Unfortunately, the participants are human. They get embarrassed when people point out their mistakes. Sometimes they get angry and defensive. Sometimes they pull rank, consciously or unconsciously - a junior can't possibly have found a meaningful bug in a senior's code.
Then there are the managerial incentives. Maybe developers are judged on following the process (their code must be reviewed), but otherwise only on whether they get their own features in on time. Maybe the schedule has been drawn up with insufficient time for the reviewers to do a decent job. Maybe the least experienced developers predictably turn out to be the only ones with time available for reviewing,
Then there are the career incentives. Maybe everyone knows that the code will be "thrown over the wall" to a maintenance team, bugs and all, with no negative consequences to the developers for its bugginess. Maybe getting a reputation for being "difficult" has a really bad impact in this organization, and it requires immense amounts of tact to draw attention to even an extremely blatant bug without getting this reputation. Maybe there are other inter-personal land mines. (The boss' pet will get you if you point out their bugs.)
Implementing code reviews usefully requires that the developers trust that all their peers are committed to improving the code. It requires a culture where the respected response to being shown a bug in your code is "thank you" or "nice catch". This culture needs to be modeled from the top of the technical hierarchy down. Implementing good code reviews also requires that management get out of the way, including simply being absent from review meetings. And it requires that developers have enough relevant skill and experience to find bugs in the particular code being reviewed.
Many organizations choose, consciously or otherwise, to merely check the "code reviews were done" box, thus demonstrating management's commitment to (performative) quality, without making them useful.
I recall my "first code review" experience in two different organizations.
In the first one, new hires were being asked to do reviews of already submitted code, as an "extra quality measure". I found a latent bug, which would only become an actual bug if the compiler changed. The author said "the compiler won't ever change". End of discussion. (Later, the organization switched compiler technology.)
In the second one, the team lead asked me to review some of his code. I found a problem, and reported it. The team lead's response was "the process worked!". He fixed his bug, and we eventually wound up as friends as well as colleagues.
I suspect there are similar things going on with paper presentations in academic organizations. Not the same things. But confused incentives, and perhaps selection of methodologies to avoid interpersonal drama rather than maximize either quality ot learning.
It is done at the economics department of University of Zurich
A draft of my dissertation was OK for the four full professors on my committee. But the young assistant prof (put on the committee when another professor went on leave) went through the math and found an error that rendered my estimation process invalid. So the four others had just skimmed the document. (Fortunately, I had a back-up estimation methodology, less elegant but functional when the math error was fixed.)
I believe, or at least hope, Professor Bruce Kobayashi or one of his colleagues continues to host the Law & Economics Colloquium at Antonin Scalia Law School, at George Mason University.
When I took the colloquium as a Law and Economics LL.M. student in 2019, the course was a semester-long seminar that Professor Kobayashi embedded within a law and economics faculty workshop that he hosted for outside professors and scholars who wished to present their papers to the law school faculty in a Chicago style seminar. It was the most intellectually stimulating and rewarding course that I had taken as either a JD or LL.M. student.
The Chicago style seminar should be a part of any academic institution's curriculum, or faculty workshops at a minimum, and it would be a great shame if the tradition eventually dies out.
I have at least three comments to make here. They aren't especially closely related. Given how substack works, we might be better served if I make 4 comments - this one and one for each theme.
1) To this retired software engineer, your Chicago style seminar sounds a lot like the better version of the early code review, in particular likely to have succumbed to some of the same problems.
2) When economics fails to check whether its predictions are satisfied in the real world, it might as well be philosophy or theology. I don't know how non-trivial predictions can be checked without statistics.
3) To this non-economist, more than half of the math I encounter in the writing of economists looks like part of some kind of strange ritual. It isn't real economics unless it has math; relevance and usefulness are optional. I say this as someone originally reasonably talented at some kinds of math (800 on the quantitative section of the GRE) who discovered an intense allergy to theoretical math while still an undergraduate, and so never went beyond standard STEM math and standard social science statistics.
I may elaborate on some of these in additional comments, particularly the bit about code review, where I may have some relevant expertise.
The Chicago Style Seminar is alive and thriving at the University of Chicago Law School. (So, for that matter, is the study of economics!)