Put four experienced improvisers in a room and you'll have five different theories on how it works. (But they'll never argue.) The two main schools are: Follow the rules! and Rules are bad! Within each school there are a multitude of factions. What rules to follow? How to follow them? How to learn them? If you are to follow no rules, how do you achieve that? How do you learn something with no rules? Or are you learning the rules but only by example the way children pick up language.
The odd thing is, if you put a "Follow the Rules" improviser on stage with a "No Rules!" improviser, you are no more or less likely to get a good scene. The outcome is not so much to do with the underlying ethos, but the interaction between the two performers. Each performer will find the theory that works best to explain the way that they perceive the underlying system. But on stage they will play and support their partners just the same.
Other people's theories are very interesting because they show you a little how they think about performing and give you a new perspective.
"Improvise: Scene from Inside and Out" is the book of the theories of Mick Napier, founder of Annoyance Theatre and Resident Director of The Second City. He belongs to the no-rules school, believing it is best not to teach rules for improv as it gives people things to think about when they should not be thinking. He does, however, offer guidelines of things to avoid but stresses to apply them without thinking about them.
The book is not very thick, but this is mostly due to Mr Napier's succint style. Not for hom the flowing prose and endless examples. He makes his point and moves on. Mr Napier's theory applies more to the teaching of children or for people well able to practice extensively, in my opinion. But his dismantalling of the importance of the rules is very interesting. It tallies with my theory (of course I have one) in that there really is only one rule, and all the rest are guidelines.
One of the most interesting chapters, and actually annoyingly short, is the one on Improvisation and the Second Law of Thermodynamics. I would love to see this expanded to really explore the analogy. There is a whole book in this idea, I think. The other useful thing is a list of exercises to do on your own at home. I found I have already been doing them for years, but it was nice to feel that I wasn't mad. Or at least not alone.
The other book I have been reading on the subject is "Musical Improv Comedy" by Michael Pollock. Mr Pollock is Musical Director of The Second City, LA, so probably knows Mr Napier quite well. Mr Pollock does not come to us with a theory. He comes with a slightly thinner book and a CD full to the brim. Instead of theories, you get a practical guide on everything music improv. Despite the thinness of the book, it really does cover everything. And with the minimum of fuss and nonsense. Where the real content is on the CD. It has over [an hour and a half] of excellent examples and sample music for you to practice song styles and techniques with. In that respect it well is worth the asking price. There are about 10 examples of different musical styles which you could easily use in a show if you don't have a musician. (PS I do not advise doing musical improv without a musician. It's usually the musician that makes the singers look good.)
As a student of improv, I urge you to read a little on the subject, and go to see as much of it as possible. "Musical Improv Comedy" is great for groups who want to get into doing (more) musical improv and "Improvise: Scene from Inside and Out" is interesting for those who want to explore the world of improvised theatre in relation to modern theories of energy or who like to improvise best in the shower.