A model for readership dropoff

Kindle has popular highlights and a pseudo-pager number system. So a 200 page book might have 5000 locations. A book might have 15 popular highlights, maybe 20 people for the first, 10 the second, 2 the 15th.  The highlights are located at various locations, say 1% in, 5% in, and the last is at 20% in.

The quality of a sentence affects the # of highlights.  Also the location of the book affects # of highlights because people stopped reading the book.

readers = a + b x
– where the intercept is 100% , all readers are present on page 1.
– the slope is negative and represents the drop off rate.

highlights = quality of sentence  * readers(x)

The number of highlights depend son the quality of a sentence, which is constant, but unknown. So a good sentence will be highlighted say, 1% of the time.

So someday when I have time, I want to see if I can establish the confidence intervals for the curves. Because there are so many constraints, it seems like we should be able to get good estimates of the drop off rate despite relatively few data points.

My Take Away from Screen Time

TV can either be bad or good or neutral, depending on a lot of factors. 

1) Kids need age appropriate TV.  Signs of age appropriateness: the kid pays attention and shows signs of understanding.

2) Kids need a model for watching TV. If you don’t watch seriously, the kid also will watch with half of their attention.

3) Foreground TV can be okay, but background TV interferes with all sorts of important activities, particularly play, sleep, language learning.

4) Passive TV is lousy for language learning. 

5) Parents watching with children and pausing the TV for commentary is better than kids watching it just by themselves.

6) Dora the Explorer, Sesame Street Post-2002,  Elmo, Mister Roger’s Neighborhoood, Dragon Tales, have positive

7) Age appropriate means age appropriate within one year. This is complicated by the fact that kids can mature faster than others (but not by mutliple years)

8) Educational TV, e.g. science, will be appropriate for a single grade, not a range of grades. 

9) Adult TV, violent TV and TV that shows kids fighting, bullying, or using relationship aggression (even if it is to illustrate how *not* to behave) leads to kids copying that behavior, especially for the youngest viewers.

10) TV may have a role in children that are sick (e.g. with flu) or that have profound issues, like autism.  In the case of kids in pain, the TV acts like anathesia. In the case of autism, shows like Thomas the Engine can help kids recognize facial expressions.  Imho, this doesn’t really change issues 1 – 9, regarding always on & background TV.

11) On demand TV is better than broadcast TV.

12) Kids are lousy at realizing what is advertising. They are lousy and getting the “moral of a story”. They are lousy at dealing with stories that jump around with flash backs and flash fowards. 

Book Review: Pancreatic Oath – A secret quantified self sort of book

I call it secret because you have to read about a third of the book before you realize that the cornerstone of the health advice is using glucometer to figure out what meals you eat are a kick to the pancreas for *you personally*. For diabetics, this is old hat. The difference here is that the advice is aimed at non-diabetics. Also, since the advice is aimed at non-diabetics. In a nutshell, you check your blood sugar levels on wake up, before eating, 2 hours after eating and record everything in a diary. If pie doesn’t spike your blood sugar levels above 100, you can eat it. Otherwise, you have to drop it, shrink the portion or change the recipe until you get your blood sugar levels back between 70 and 100. [n.b. if you are diabetic, the advice is different and diabetics often aim for slightly higher numbers because the consequences of being low are severe, e.g. coma, vs the consequences of being high, e.g. a slow life long decline and early death]

This fits in very nicely with anyone who has been exposed the the Quantified Self movement (fad? geeky hobby?) Unlike the fitbit, my withings scale, my withings blood pressure cuff and my nintendo DS pace counter, glucometers hurt and involve drawing blood– unless you do your research. Old fashioned glucometers and measuring on the finger tip (instead of the side or arm) seem to be correlated with pain, the latest state of the art ones are supposed to be significantly less painful (or not painful at all).

The first third of the book is preachy. I’m mostly already sold, but I understand why the book has to be written this way. Dietary habits are incredibly stable. Normally, threat of imminent death is not enough to change dietary habits. I think it is because humans as a species have *never* lived in an environment with an abundance of food, so even 100-200 years ago, it was always less risky to eat more than less because of the risk of future famine or food shortage. It takes a near religious conversion to get people to change.

The middle third is about the mechanics of blood sugar, insulin and how the spikes that non-diabetics experience in blood sugar and insulin damage our blood vessels, eyes, kidneys… you name it. [This contrasts with diabetics who don't just deal with spikes, but all day highs or wildly fluctuating levels because it isn't just a overworking pancreas but one that is broken] After reading this, I got a distinct impression that the pancreas is a keystone organ. When it is broken, the rest of the body begins to fail, like a big slow motion house of cards. This is a bit different from lung or heart failure, since if you can’t breath or pump blood you die in five minutes. So our body has over engineered those, they put up with a lot of abuse– just look at how much abuse smokers get away with before it kill them. Another way to look at it is that your brain (stroke), lungs (drowning), heart (heart attack) is about life and death health issues. The pancreas is the organ that is relevant to health (that and the liver, but I get the impression that the liver is more durable than the pancreas, although the liver will show signs of distress the same as the pancreas when blood sugar levels spike). But the rest of the organs work as long as they aren’t abused. The author uses the phrase “get your pancreas into idle” It’s a good metaphor. The non-diabetic doesn’t want to make the pancreas run at full tilt most or even some of the time.

Finally, we get to the advice on measuring.

At the end are a bunch of recommended recipes. This book made a valuable observations about the glycemic index approach– which is a closely related diet/heath advice. Glycimic index charts promise to tell you if a given food will cause your blood sugar levels to spike after you eat it. But in practice, the same foods affect different people differently. So the glycemic index needs individual measurements to account for individual differences. In which case you can ditch the glycimic index chart and just use the actual glucometer readings.

The author has a detectable left slant, pro-organic, pro-farmers market, pro-yoga and mediation. The author interestingly had a pragmatic approach to exercise– a lot of people who have let their health go have gotten themselves into a position where exercise isn’t an significant option– the knees are shot, so they have to focus on improved health by look on the energy intake side and not the energy usage side. The author is also pro-vitamins. I like micronutrients, but other than the bit about cinnamon and other blood sugar linked micronutrients, I thought the vitamin advice was unnecessary. I say, a book should fight one battle at a time and the case for using a glucometer among non-diabetics was made, but the case for vitamins wasn’t really well supported.

The author’s website is here with links to get the book. Also available on Kindle, which is how I read it.

Escapist Sci-Fi : Generation Ships

When I was a kid I read the Exiles Trilogy, by Ben Bova. The third book was gripping and fantastic. I can’t remember the first two thirds. 

A year or so ago I saw Pandorum and thought it was a fantastic movie.  I probably had an unusal resposne, I really, really wanted to know what langauge the ship board native spoke.

I was hoping there was a book, but there wasn’t. I tracked down a Heinlein book “Orphans of the Sky” which seems to be one of the earlier (earliest?)  The book is pretty good, but in the last chapters Heinlein got bored with the story and rushed to move the characters to a suitable endpoint. 

On wikipedia I found generation ship stories  is a subgenre of maybe a dozen or more books.

I’m working my way through Greg Bear’s Hull Null Zero. So far it kind of feels like a first person shooter adventure, with characters doing a ship search in a dangerous ship.  Bear’s ship is drowning in gidgits and doodads, so I often have a hard time visualizing what the heck is going on.

Common Themes

Social collapse and primitivization. A reversion to hunter and gatherer times.

Return to superstition and/or Religious zeal.  A sort of reversion to medeival times.

Stratefied societies.  Same idea but rolling back the clock to feudal times.

Cannibalism. Food shortages are okay for the story, but shortages in radiation shielding, air, water, heat would make for a really short story.

Simplification of language. The ship inhabitants start to lose personal experience with a lot of the words in their earthly mother tongue. This leads to “there’s-no-word-for” which is nonsense and “I-don’t-know-what-this-word-is” which makes perfect sense.  Also, since the trips take forever, the language evolve.


Generation ships are a type of hard science fiction, in the sense that the story doesn’t appeal to magic to solve problems that are currently without a solution in the real world, namely faster than light travel. 

Lowtech. After civilization collapses, the survivors are primative. No computers or AI. Sci-fi writers before the 80s completely missed the coming computer revolution and generally omit AI.

High tech. On high tech ships, manufactoring goes on or gadgets just work. There might be some computers or AI.

Organic technology. Evolution, cloning and genetic engineering are common themes. If not, then the travelers tend to arrive at new planets that just happend to be exactly like Earth.

Waking trips, sleeping trips.  In waking trips, civilization collapses. In sleeping trips, the machinery malfunctions.

Invisible destinations.  Most Sci-fi writers missed the recent revolution in detecting planets outside of our own.

Book Review: Hackers and Painters

This is really like a collection of long essays more than a single book.

I liked the chapter on geeks in high school, it’s very accurate. The war stories about running a startup are valuable data points.

The distinction between makers/hackers/painters and engineers is a valuable one. I think a good educator or project manager should take into consideration which of the two skills and personality types they are working with.

The article on economics was a bit too much folk-economics for my taste. I’ve got a masters in Economics and found some of the analysis to be the naive. For example, if some programmers are doing 99% of the productive work, then they should reap 99% of the wages & profits of the programming industry. They don’t, so on one hand the author is saying the free market system is inefficient, on the other hand, the rest of what he says implies he very much a laissez faire economics sort of thinker. So what gives? Also, the bit about how one’s wages are an averaging of all the employees outputs and thus inefficient seems suspect. If one can make another company richer, then the other company will bid for that employee, driving up their wages until an employee’s salary is equal to their marginal contribution.

Also, the bit about suggesting workaholism is a good thing is a bit naive. There are diminishing marginal returns to any input. The rate at which marginal returns diminishes varies, but no matter what the job, your 80th hour of work in a week is less productive than the 40th and is something less productive than 2 people working two weeks. In my personal experience, workaholism leads to catastrophic exhaustion, where eventually you just stop wanting to work. So you gain, say a 25% productivity boost for a few weeks, and then lose a larger absolute value of productivity when your brain and body fail and you can’t get any work done for a few weeks. Just because the machines can work 24×7, it’s reckless advise to try to make the wetware peripherals (ie. the programmer) do the same.

The programmers-are-libertarians bit, I don’t agree with. Online communities suffer from echo chamber effects and founder effects. Place like slash-dot and the like are overrun with likeminded libertarians because they got there first, drove off anyone that thinks differently and now they only hear each other. The now image that libertarians rule the world, that Ayn Rand is a popular author and Ron Paul has a chance. The ironic part is, that if one believes that the libertarians rule the world (even just among programmers) then you aren’t really exposing yourself to a wide variety of ideas, or ideas that are out of fashion– because you won’t be able to see what fashion is anymore! All you can see is that everyone on slashdot is a libertarian.

I am a programmer and I am not a libertarian. I think libertarians are sociopaths, who mostly just want others to believe and adopt libertarian policies to their own benefit to the detriment of others. I think they are as self serving a billionaire who opposes taxes on billionaires and deluded as the penniless, undereducated, unemployed rural conservative, who thinks policies favoring the rich and powerful are good because someday he’s going to be a rich and powerful man, too.

The don’t-trust-anything the-suppressed-ideas-are-right thinking in not automatically true! It is a intellectual tool you want to use, it’s just something to keep in mind. Sometime unpopular ideas are controversial because, well, one side really is wrong and yet the idea has something very attractive to it for dull witted and emotional people. At best, investigating out-of-fashion ideas is a starting point. One will have to do hard work and thought to find out if a currently controversial idea with vocal detractors is non-sense or not. Worse, the more vocal detractors there are, the more likely it is for the arguments to get sophisticated — i.e. have the trappings of science, reason but none of the substance. The gun controversy in the US is a good example– a good sign that the controversy in the US is clouded by things other than reason is that many places elsewhere in the world,

Book Review: Coders at Work

The book is the transcription of a bunch of interviews with a bunch of programmers that in one way or another have become inordinately famous, often by inventing languages or operating systems. It takes decades to become famous, so a lot of the developers interviewed are approaching the world with a 1950 to 1970 mindset. Compilers and OS kernels are a pretty specific variety of programming– the unsung heros writing code that never becomes famous were largely absent, although many of these developers had war stories about working on similar non-compiler non-OS-kernel projects. As an example, most were asked about how they debug and what tools they use– most said “print statements” If all you’ve had for your whole debugging career is a hammer, I guess it isn’t surprising you haven’t switch to nail guns, but it won’t help my debugging skills to put down the nail gun and use the hammer just because that’s how people used to do it.

The book over all is vary ramble-ly and disorganized, like a conversation, but unlike the hyperstructured layout of a novel or textbook that the modern reader has come to enjoy. It could have used some more aggressive editing for size and organization. Some points are raised by multiple developers which is interesting as a poll, but I’m just as happy to read “9 out of 10 master developers find multithreading to be a fiercely difficult problem” I don’t need to read the exposition from each, which in anycase was hard to follow if you haven’t done multithreaded programming much before.

Ironically, the concept of “Literate Programming” was revisited many times, but this book had none of the polish that one would find in a document subjected to draft, rewrite, rewrite, final draft process that English language writer subject their own documents to.

Speaking of Literate Programming– many concepts like this are discussed, but the reader is assumed to kind of know what it is about already. Proving the correctness of programs, compiler design, OS kernel design, low level computer function and and programming (e.g. assembly), were discussed, but the audience was really someone who’s already mastered these things and they are evaluating how effective they are in practice.

As a plus, this story had very little of the typical popular-press IT stories, such as death marches and project management.

I recommend reading 6 out of 10 of the pages of this book. To be fair to the developers, I recommend selecting one developer interview to read at random, subtracting the pages read from total pages then repeating until you’ve read 6 out of 10 of the pages in the book. Please post an more efficient algorithm in the comments if you can think of one, with the formal proof that will eventually stop and has no invariants violated.

A Science Fiction reader discovers the internet

The last time I tried to get hooked into Sci-fi websites, I subscribed to mailing lists that spammed my mailbox with untargeted garbage, mailing lists full of strangers I will never meet, I subscribed to podcasts of irregular quality and RSS feeds that hit tripple digit un-read numbers overnight.  So now I try again.  Has web 2.0 and the social web saved the sci-fi cyberuniverse?

GoodReads. It has membership and is growing as fast as facebook.  The book club section is innovative, but not feature complete, i.e. could have better search aimed at in person book groups.

Shelfari. Beautiful site, Shelfari’s group section is useless for organizing in person book groups.

Library Thing. Groups and events are aimed at big organizations and big spectator events, like book stores, libraries holding book signings or author readings.  The other group features don’t seem to have any in-person features.

Just about all publishers sites offer catalogs, calendars (usually not iCal or otherwise subscribable), and newsletters.

Tor. Very impressive site with some social features, some free stories.

Baen. Free books, some free online content.

Random House. RH’s approach is to create a site as a bookstore. No obvious social features.

Del Rey. Part of RH. No obvious social features.

Amazon. Weak on science fiction specific features, but still the best place to buy books that aren’t available used.  The best social feature is the shared wishlists.

Pyr. Small publisher, uses FaceBook.  Better than nothing, suffers all the same problems that facebook does.

Book Review: Wicked

I highly recommend this book, but it is a long and slow read, chock-full of words that are barely English, like “shirtily” (meaning ill-tempered) It is a origin story and a tragedy. At one chapter I was ready to put down the book and stop reading if he killed a particular character.

L Frank Baum wrote stories about humans and not Humans. In Macguire’s book a distinction is made between animals and Animals, the later being fully sentient, the former were stupid automatons. Gregory Maguire wrote out all characters as full Humans. The good and evil characters of L Frank Baum’s world were humans with a small “h”. In Macguire’s version , each character had real motives, instead of being motivated by “Good” or “Evil” as if they were automatons. This is important because a theme of the book was, “What is evil? Was the witch wicked?” Like in real life, the distinction isn’t simple with people with good intentions and goals being cruel and vicious and people who seem cruel and vicious are just trying to get by in a cruel and vicious world, doing only what could be expected of them being in their position.

Book Reviews: Broken Angels, Woken Furies, Altered Carbon

I read all of Richard K Morgan’s books way too fast.  Woken Furies was no different that the first two in the series.  Kovacs has gone from detective, to mercenary, to criminal drifter.

The first book, Broken Angels is a detective story, but also an action story. Kovacs solves the murder mysteries using his futuristic special forces skills.  The book does dabble in morality.  It isn’t said, as much as illustrated the moral implications of a technology that could record the state of your brain and put it onto a device, which could then be moved to another body.  People’s cortical stacks move from body to body, giving the look of immortality.

The next book, Altered Carbon, is a heist of ancient Martian artifacts.  The action story is capped by a statement on technological progress in warfare.

The third Kovacs book, still is fresh and energetic as the first two.  The Kovacs story and character stay fresh on account of the rules of magic, which allow Kovacs to move from one body to another (one sleeve to another), and one planet to another.  This story has a subtle message about land mines.  Kovacs by fate but not choice, ends up on a war machine decommissioning team. 

The novels are superbly written.  They are also violent, full of sex, rough language.  It’s a guys’ book.  The R-rated material is hardly gratuitous.  Kovacs lives in a world where the moral consequences of murder and death are blunted by sleeving technology, which brings back to live many people, especially soldiers, and the middle to upperclass. 

Kovacs a superbly developed character and each novel does a reasonable job of developing secondary characters.  Many characters failed to become distinctive in my mind, which in the second and third novels was a problem when they were reintroduced later.

Kovacs doesn’t suffer as much from the “hero syndrome” of most character centric stories.  Kovacs remains for all novels, fallable, imperfect and not always likable, much like a real person.  Through resleeving and stints in computer generated virtual worlds, Kovacs can meet significant setbacks, yet not be eliminated from the story all together.

The stories all have endings that work as contrasted with William Gibsons novels, which if nothing else, have taught me the importance of a novel that ends well.

Someday, the Kovacs novels will be made into movies and the movies are not going to be nearly as good as the novels.  I fully predict that the Takashi Kovacs movies will be standard action hero fare to be forgotten the day after viewing.  They will make Richard Morgan a well deserved pile of cash that the books sales probably can’t match, but the novels, I forecast, will still be regularly checked out at libraries a decade from now.


I’m reading “Why Darwin Matters” for a book club. While reading it, I’ve decided it really is about apologetics. The fact that it spends it’s time on evolution and intelligent design is just a completely worked out example.

Apologetics is the defense of faith using the tools of logic and debate. The word probably will be extended to other ongoing debates, mostly those that live the the realm of the fuzzy sciences, like sociology, political science, criminology, sometime biology, less so in the fields of math and physics.

Ideas show up and are extremely attractive–the idea posits a view of the world that is attractive—and people begin accumulating defenses. The is an asymetrical battle of wits. Creating a large pile of attractive sounding soundbites is effortless and makes and arguement to a casual observer look unbeatable. Especially in the field of social sciences, refuting a claim is intellectually and sometimes economically expensive.

An example is the Intelligent Design vs Evolution debate. It only takes a flippant remark to claim eyes are unexplained by evolution. The painstaking tracing of the evolution of the eye from single cell light detecting tissues to modern complex eyes is enough work to keep post docs busy for years. Upon the publication of the hundredth tome of irrefutable proof, an evolution doubter could say then say, “Well, but you have explained how the nose evolved.”

I think on many of these debates, we could save ourselves a lot of bickering by skipping the the heart of the matter: the techniques.

Universal qualifiers. Statements with universal qualifiers are probably wrong. Note I say probably, because there are some fields of thought were universal qualifiers are just fine, like algebra and mathematical physics. Example of inappropriate use of universal qualifiers would be say, “All public policies need to be extensible to all similar cases” or “Not all gun control laws work all the time, so no gun control laws are better” The abuse of universal qualifiers is so common it isn’t surprising that Aristotle first line of business in the field of logic was setting down a theory of valid and invalid syllogisms.

Confusing perferences with truth. Confusing value judgements with truth. I’d prefer we pull out of Iraq–I prefer the risk of not finishing the war to the risk of finishing a war, I judge the war there to be a collosal mistake. On these grounds, I can have good name calling debate with anyone in favor of the war. To move out of statements of preference, I and proponent of the other policies would have to engage in the hard work of collecting data, defining methodologies and establishing standards for truth in a world where events are often random, data collection is expensive and analysis even more so.

And there are many more.

On the other hand, being apathetic or apolitical is something of similar logical lapse. Not attempting the answer the pressing questions of the day is an appeal to authority. Someone is going to be making these policy decisions.

And on the third hand, accumulating viral ideas–attractive but not supported by hard data– from the internet or floating around in society, is another form of appeal to authority–the authority of the mob.