Gambling and Shannon’s Entropy Function Part 2.

In the last post I gave an introduction of Kelly’s paper where he describes optimal gambling strategies based on information received over a potentially noisy channel. Here I’ll talk about the general case, where the channel has several inputs symbols, each with a given probability  of being transmitted, and which represent the outcome of some chance event. First, we need to set up some notation:

p(s) –  probability that the transmitted symbol is s.

p(r | s) – the conditional probability that the received symbols is r given that the transmitted symbol is s.

p(r, s) –  the joint probability that the received symbol is r and the transmitted symbol is s.

q(r) –  probability that the received symbol is r.

q(s | r) – the conditional probability that the transmitted symbol is s given that the received symbols is r.

\alpha_s – the odds paid on the occurrence of s, i.e. the number of dollars returned on a one-dollar bet on s.

a(s/r) – the fraction of capital that the gambler decides to bet on the occurrence of s after receiving r.

Continue reading

Gambling and Shannon’s entropy function.

A little while ago Matt posted an overview of the definition of Shannon’s entropy function and it’s use in assessing interaction between random variables. I’d like  to stay with the 1960’s Bell Labs team and describe some of the results from J. L. Kelly’s wonderful paper, which arises from the definitions and ideas in Shannon’s original work.

Consider at first the trivial scenario where we have a noiseless binary channel that transmits the results of say a baseball game. If the odds are even and the gambler has access to this channel he can grow his capital exponentially by making certain bets (since he knows the outcome before betting). His capital would grow at a rate of 2^N , after N bets. In view of the fact that any such function should be maximum in the above scenario, Kelly defines the exponential rate of growth of capital as

G = lim_{N \rightarrow \infty} \frac{1}{N} log_2(\frac{V_N}{V_0}),

were V_0 is the initial capital and V_N is the capital after N bets.

Now suppose that the channel is noisy, and a given symbol has a probability p of error and q of correct transmission. The question now is, how much should the gambler bet?

Continue reading

HackNY Fundraising event

Tomorrow night there is a fund raising gala for HackNY. The organization brings together students in computer science, data mining, math with the quickly evolving NYC technology sector. They have created a summer fellows program, host regular hackathons, and in all do a wonderful job. Sailthru, the company I currently work for, is one of the sponsors for the event and it should a blast. Tickets are on sale now.

A response…

Yesterday I was happy to receive the following email from Ryan Croke at the Department of Mathematics, Colorado State University. He had read the article I wrote for the AMS notices regarding moving from the academic to the startup world and had similar views on a number of subjects, especially those related to the insularity of academia and its lack of acceptance of other options. I had stayed shy of delving into the full details of how I perceive the situation in the article itself. Below is an excerpt from Ryan’s email, which I thank him for allowing me to post, where he discusses some of these issues.

 I am graduating this year with a PhD in mathematics and my experiences have been eerily similar to yours.  I recently got a job with Raytheon, a defense contractor.  I was applying for postdocs, but once contacted by Raytheon I realized there are other opportunities out there.
What spoke to me about your article is the heretical nature of working in industry prevalent in academia.  I am at Colorado State University, a third tier research university.  Even here the idea of going to industry is blasphemous.  I routinely hear faculty, who have never worked outside of university, pan industry because it is “restrictive” and you “don’t have freedom.”  It’s incredible.

The biggest disservice is done to the grad students.  Every year I see students scratch and claw to go to universities they don’t really want to go without even considering working outside of academia.  Many of them have skill sets that would be very successful in business but they never give themselves the opportunity.    And, as you say in your article, we have a unique skill set that makes (most of us) very malleable and amenable to vastly different industries.

Between you and me, I find the whole system ridiculous.  My colleagues are writing letter after letter, competing with hundreds of other people for positions that pay little and may have to go to places they don’t want to.  The entire process with Raytheon took under four weeks.  It included 4 interviews, a background check, a bit of paperwork, and that’s it!  I was one of 45 that got jobs that began with around 1100 applicants.  According to Raytheon my strongest asset was my advanced degree in mathematics.  Most mathematicians do not realize the gift they have (or learned) to assimilate new information quickly and solve problems and how that is useful in other areas.

I find it very strange that I will be working on projects that put peoples lives on the line and the process was so streamlined and efficient, but, some of my colleagues that will be going to liberal arts universities “just to teach” are going through a hellish process.

Anyway, I found your article refreshing and enlightening.  I think that departments need to nurture and cultivate ties with industry and government.  A corollary to few people considering jobs outside academia is that academia is flooded with possibly under qualified and out of place people.  It’s crazy if you ask me.  But, what do I know?

Thanks for the article,
Ryan