Information Theory Research Group
Spring 2005

Participants

Purpose of Meetings

There will be two separate themes of the research covered this semester, implying that we will often separate into two groups. One group will be more focused on the statistics of natural imagery. One will work through a set of more general topics.

Possible topics (not ordered) include:

1. Statistics of Natural Imagery
2. Two-Way Communication Channels
   1. R. Blahut, Principles and practice of information theory, Addison-Wesley, 1987, pp. 347-354.
   2. A. Hekstra and F. M. J. Willems, "Dependence balance bounds for single-output two-way channels," IEEE Transactions on Information Theory, vol. 35, pp. 44-53, no. 1, Jan. 1989.
   3. HB Meeuwissen, J. P. M. Schalkwijk, and A. H. A. Bloemen, "An extension of the achievable rate region of Schalkwijk's 1983 coding strategy for thebinary multiplying channel," 1995 IEEE International Symposium on Information Theory, p. 445, Whistler, BC, Canada, Sept. 1995.
   4. C. E. Shannon, "Two-way communication channels," Proc. 4th Berkeley Symposium Math. Stat. Prob. vol. 1, pp. 611-644, reprinted in Claude E. Shannon: Collected Papers, pp. 351-384, 1993.
   5. Z. Zhang, T. Berger, and J. P. M. Schalkwijk "New outer bounds to capacity regions of two-way channels," IEEE Transactions on Information Theory, vol. 32, pp. 383-386, no. 3, May 1986.
3. Other unsolved problems in information theory: distributed source coding, broadcast channel, etc. See T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley-Interscience, 1991, chapter 14.
4. Data embedding (hiding, steganography, digital watermarking, etc.).
   1. P. Moulin and J. A. O'Sullivan, ``Information-Theoretic Analysis of Information Hiding,'' IEEE Transactions on Information Theory, Vol. 49, No. 3, pp. 563-593, March 2003. (pdf version)
   2. B. Chen and G. Wornell papers
   3. Ton Kalker papers
   4. books: Ingemar J. Cox, Matthew L. Miller, and Jeffrey A. Bloom, Digital Watermarking. San Francisco: Morgan Kaufmann, 2002.
   5. Mauro Barni and franco Bartolini, Watermarking Systems Engineering: Enabling Digital Assets Security and Other Applications. New York: Marcel Dekker, 2004.
   6. Joachim Eggers and Bernd Girod, Informed Watermarking. Boston: Kluwer Academic Publishers, 2002.
5. Space-Time Coding and Wireless Communication
   1. A. Paulraj, D. Gore, and R. Nabar, Introduction to Space-Time Wireless Communications Cambridge University Press, 2003.
   2. B. Vucetic and J. Yuan, Space-Time Coding. Wiley and Sons, 2003.
   3. G. L. Stuber, Principles of Mobile Communication, Kluwer Academic Publishers, 2001.
6. Information Theory and Complexity, communication and computation
   1. E. Kushelevitz and N. Nisan, Communication Complexity. Cambridge University Press, 1996.
   2. G. Chaitin books.
   3. Andrew Yao's work including: "Computational information theory," in Complexity in Information Theory, pp. 1-15, Springer-Verlag, 1988.
   4. C. Papamitriou, Information theory and computational complexity: the expanding interface, IEEE Information theory society newletter, special golden jubilee issue, pp. 12-13, 1998.
   5. Stochastic complexity, Rissanen, Schartz, Wallace complexity
   6. Kolmogorov complexity (see Cover and Thomas)
7. Quantum information theory; information theory and physics
   1. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantm Information. Cambridge University Press, 2000.
   2. Information Physics
   3. Chris Adami, especially his work on quantum information and black holes.

Weekly Meetings

• Feb. 8: Norbert Agbeko, statistics of natural imagery. Paper by Srivastava, Lee, Simoncelli, and Zhu. pdf file.
• Feb. 15: Po-Hsiang Lai, sparse coding, statistics of natural imagery, a little neuroscience
• Feb. 22: Lindsey Raddatz, more statistics of natural imagery
• Mar. 1: Geoff San Antonio, MIMO radar
• Mar. 8: Spring Break
• Mar. 15: Steve McLaughlin visits WU, gives seminar in ESE
• Mar. 22: Po-Hsiang, two-way communication channels
• Mar. 29: Dr. O or Brandon W.: other open problems in information theory (distributed source coding, broadcast channel, etc.)
• Apr. 5: Adam Tyburski, data embedding
• Apr. 12: Shalini Priti, more data embedding
• Apr. 19:
• Apr. 26:
• May 3: finals?
• May 10: finals?
• May 17:
• May 24:
• May 31:
• June 7: no meeting
• June 14: Aikaterini, quantum information theory
• June 21: Aikaterini, quantum information theory

Potentially useful links

• Introduction to variational methods. See Michael I. Jordan, Z. Ghahramani, T. S. Jaakola, and L. K. Saul in their paper in Jordan's book. See Jordan's home page (link below) for his on-line publications.
• Shannon's 1948 paper in pdf. Total 4645180 bytes.
  
• Michael I. Jordan's home page. See his book "Learning in Graphical Models" --- (MIT Press, 1999).
  
• Brendan Frey's home page. See his tutorial on probability propagation.
  
• David Mackay's home page
  
• Frank R. Kschischang's home page
  
• "Codes and Decoding on General Graphs," the doctoral thesis of Niclas Wiberg
  (also, the downloaded pdf version, 1.3MB).
  
• Robert J. McEliece's home page
  
• Yair Weiss's home page
  
• IEEE Information Theory Society homepage; see the on-line link to all papers ever published in the IT Transactions
  Viterbi's orignal paper with the Viterbi algorithm in pdf. Total 1.5 MB.

Independent Component Analysis(ICA), Principal (PCA), etc.

• Aapo Hyvärinen Home Page; See his publications
  Links from Hyvärinen Home Page:
  
• Paris Smaragdis' ICA &BSS Page
  
• Jean-Francois Cardoso's ICA Central
  
• Allan Barros' ICA Page
• ICA research at HUT
• Entropy on the World Wide Web Page
• Stanford Information Theory Group Homepage
• Lucent Information Theory
• Bell Labs Shannon Day
• Institute for Quantum Information

Max-Plus, Max-Sum, etc., (for those interested)

J. A. O'Sullivan Home Page