霊犀社2

第2霊犀社(元祖第一は、田中逸平主宰の私塾)@霊際社@P。ネタ帳・メモ書きなど、まあガラクタ。嫌疑76件わざと表示拒否中

“measure of uncertainty” or attenuation

2008-02-29 15:47:28 | LinkRecords
"Grammatical Man" shannon entropy Neumann

http://humanthermodynamics.wetpaint.com/page/Information+theory
Information theory - Encyclopedia of Humanthermodynamics

Why Shannon defined “missing information” in telephone lines as entropy?
“measure of uncertainty” or attenuation attenuation
“My greatest concern was what to call it. I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty’. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, ‘You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.”

5. M. Tribus, E.C. McIrvine. (1971). “Energy and information”, Scientific American, 224 (September).

    -------
According to another source: [7]

"When Shannon discovered this function he was faced with the need to name it, for it occurred quite often in the theory of communication he was developing. He considered naming it "information" but felt that this word had unfortunate popular interpretations that would interfere with his intended uses of it in the new theory. He was inclined towards naming it "uncertainty" and discussed the matter with the late John Von Neumann. Von Neumann suggested that the function ought to be called "entropy" since it was already in use in some treatises on statistical thermodynamics... Von Neumann, Shannon reports, suggested that there were two good reasons for calling the function "entropy". "It is already in use under that name," he is reported to have said, "and besides, it will give you a great edge in debates because nobody really knows what entropy is anyway." Shannon called the function "entropy" and used it as a measure of "uncertainty," interchanging the two words in his writings without discrimination.

7. M. Tribus, "Information theory and thermodynamics", in Harold A. Johnson (ed.), Heat Transfer, Thermodynamics and Education: Boelter Anniversary Volume New York: McGraw-Hill, 1964; page 354.
   ------
According to another source: [8]

At first, Shannon did not intend to use such a highly charged term for his information measure. He thought “uncertainty” would be a safe word. But he changed his mind after a discussion with John von Neumann, the mathematician whose name is stamped upon some of the most important theoretical work of the first half of the twentieth century. Von Neumann told Shannon to call his measure entropy, since “no one really knows what entropy is, so in a debate you will always have the advantage.”

8. Campbell, Jeremy. (1982). Grammatical Man – Information, Entropy, Language, and Life, (pg. 22). New York: Simon and Schuster.
   --------
11. Muller, Ingo. (2007). A History of Thermodynamics - the Doctrine of Energy and Entropy (ch 4: Entropy as S = k ln W, pgs: 123-126). New York: Springer.
コメント (2)   この記事についてブログを書く
« 通信路の経済、パッキングの... | トップ | \"判読限界\" 再生紙 OR エ... »
最近の画像もっと見る

2 コメント

コメント日が  古い順  |   新しい順
Shannon 1982 Interview (tokumaru)
2011-06-30 10:32:07

Thermodynamics and entropy; cryptography
Shannon: Well, let me also throw into this pot, Szilard, the physicist. And von Neumann, and I’m trying to remember the story. Do you know the story I’m trying to remember?
Price: Well, there are a couple of stories. There’s the one that Myron Tribus says that von Neumann gave you the word entropy, saying to use it because nobody, you’d win every time because nobody would understand what it was. Shannon: [laughs]
Price: And furthermore, it fitted p*log(p) perfectly. But that, but then I’ve heard . . .
Shannon: von Neumann told that to me?
Price: That’s what you told Tribus that von Neumann told that to you.
Shannon: [laughs – both talking at once]
Price: Bell Labs too, that entropy could be used. That you already made that identification. And furthermore in your cryptography report in 1945, you actually point out, you say the word entropy exactly once in that report. Now this is 1945, and you liken it to Statistical Mechanics. And I don’t believe you were in contact with von Neumann in 1945, were you? So it doesn’t sound to me as though von Neumann told you entropy. Shannon: No, I don’t think he did.
Price: This is what Tribus quoted.
Shannon: Yeah, I think this conversation, it’s a very odd thing that this same story that you just told me was told to me at Norwich in England. A fellow —
Price: About von Neumann, you mean?
Shannon: Yeah, von Neumann and me, this conversation, this man, a physicist there, and I’ve forgotten his name, but he came and asked me whether von Neumann, just about the thing that you told me, that Tribus just told you, about this fellow. . .
Price: That was Jaynes, I imagine the physicist might have been [Edwin] Jaynes.
Shannon: Yes, I think it was, I think so. Do you know him?
Price: Well, he’s published in the same book as Tribus, you see. This is a book called The Maximum Entropy Formalism. You’ve probably seen that book, but they have chapters in it, and Jaynes, the physicist —
Shannon: Now, I’m not sure where I got that idea, but I think I, somebody had told me that. But anyway, I think I can, I’m quite sure that it didn’t happen between von Neumann and me.
Price: Right. Well, I think that the fact that it’s in your 1945 cryptography report establishes that, well, you didn’t get it from von Neumann, that you had made the p*log(p) identification with entropy by some other means. But you hadn’t been —
Shannon: Well, that’s an old thing anyway, you know.
Price: You knew it from thermodynamics.
Shannon: Oh, yes, from thermodynamics. That goes way back.
Price: That was part of your regular undergraduate and graduate education of thermodynamics and the entropy?
Shannon: Well, not in class, exactly, but I read a lot, you know.
Shannonのcryptograpy report 1945 (raycy)
2011-07-17 14:33:22
このどれかってことかな。
http://www2.research.att.com/~njas/doc/shannonbib.html

``A Mathematical Theory of Cryptography,'' Memorandum MM 45-110-02, Sept. 1, 1945, Bell Laboratories, 114 pp. + 25 figs. Superseded by the following paper. Not included.

``Communication Theory of Secrecy Systems,'' Bell System Technical Journal, Vol. 28 (1949), pp. 656-715. ``The material in this paper appeared originally in a confidential report `A Mathematical Theory of Cryptography', dated Sept. 1, 1945, which has now been declassified.'' Included in Part A.

``Mixed Statistical Determinate Systems,'' Typescript, Sept. 19, 1945, Bell Laboratories, 17 pp. Not included.

コメントを投稿

ブログ作成者から承認されるまでコメントは反映されません。

LinkRecords」カテゴリの最新記事