maryrosecook

The end of the world, Lee Marvin, shooting police officers, Bruce Sterling and Godspeed You! Black Emperor

This evening, I talked to my girl and then got into bed. Because I’m in hospital, each night, I fall asleep hooked up to Monty the heart monitor and my iPhone. I listened to a heart-breaking song by Des Ark, then a beautiful, cyclical, looking-across-the-misty-hills-early-in-the-morning type song by my friend, Sherry, that then segued into talks from Webstock 2009. I heard a bit of Tom Coates on tracking data about aspects of our lives and then segued my own way into sleep.

I woke up a while later having had a restless dream where I was on some sort of mission to sabotage or subvert the government. I’d been through a sort of army-like training programme with some fellow activist friends and now we were running through darkened offices and municipal buildings at night. I think I shot a policeman. I woke just after I had stuffed myself into a dumb waiter to find that Tom Coates had finished talking and Bruce Sterling had been going for a while.

He alternated between a monotone that sounded like Lee Marvin describing the end of the world in Godspeed You! Black Emperor’s song, The Dead Flag Blues, and a more robust sing-song delivery that was like the accompanying music. In my half-sleep state, my revolutionary dream merged with the end of the world he was describing into a shaking, tremulous feeling filled with possibility.

You can listen to the audio of his talk [relevant section starts around 1h and 31m in], or you can watch the video [relevant section is the last seven or eight minutes], or you can read the transcript [relevant section excerpted below].

---

Bruce Sterling at Webstock 2009, excerpt:

“Are there some non-financial structures that are less predatory and unstable than this radically out-of-kilter invisible hand? The invisible hand is gonna strangle us! Everybody’s got a hand out — how about offering people some visible hands?

“Not every Internet address was a dotcom. In fact, dotcoms showed up pretty late in the day, and they were not exactly welcome. There were dot-orgs, dot edus, dot nets, dot govs, and dot localities.

“Once upon a time there were lots of social enterprises that lived outside the market; social movements, political parties, mutual aid societies, philanthropies. Churches, criminal organizations — you’re bound to see plenty of both of those in a transition… Labor unions… not little ones, but big ones like Solidarity in Poland; dissident organizations, not hobby activists, big dissent, like Charter 77 in Czechoslovakia.

“Armies, national guards. Rescue operations. Global non-governmental organizations. Davos Forums, Bilderberg guys.

“Retired people. The old people can’t hold down jobs in the market. Man, there’s a lot of ‘em. Billions. What are our old people supposed to do with themselves? Websurf, I’m thinking. They’re wise, they’re knowledgeable, they’re generous by nature; the 21st century is destined to be an old people’s century. Even the Chinese, Mexicans, Brazilians will be old. Can’t the web make some use of them, all that wisdom and talent, outside the market?

“Market failures have blown holes in civil society. The Greenhouse Effect is a market failure. The American health system is a market failure — and most other people’s health systems don’t make much commercial sense. Education is a loss leader and the university thing is a mess.

“Income disparities are insane. The banker aristocracy is in hysterical depression. Housing is in wreckage; the market has given us white-collar homeless and a million empty buildings.

“The energy market is completely freakish. If you have no fossil fuels, you shiver in the dark. If you do have them, your economy is completely unstable, your government is corrupted and people kill you for oil.

“The human trafficking situation is crazy. In globalization people just evaporate over borders. They emigrate illegally and grab whatever cash they can find. If you don’t export you go broke from trade imbalances. If you do export, you go broke because your trading partners can’t pay you…

“Kinda hard to face up to all this, especially when it’s laid out in this very bald fashion.

“But you know, I’m not scared by any of this. I regret the suffering, I know it’s big trouble — but it promises massive change and a massive change was inevitable. The way we ran the world was wrong.

“I’ve never seen so much panic around me, but panic is the last thing on my mind. My mood is eager impatience. I want to see our best, most creative, best-intentioned people in world society directly attacking our worst problems. I’m bored with the deceit. I’m tired of obscurantism and cover-ups. I’m disgusted with cynical spin and the culture war for profit. I’m up to here with phony baloney market fundamentalism. I despise a prostituted society where we put a dollar sign in front of our eyes so we could run straight into the ditch.

“The cure for panic is action. Coherent action is great; for a scatterbrained web society, that may be a bit much to ask. Well, any action is better than whining. We can do better.

“I’m not gonna tell you what to do. I’m an artist, I’m not running for office and I don’t want any of your money. Just talk among yourselves. Grow up to the size of your challenges. Bang out some code, build some platforms you don’t have to duct-tape any more, make more opportunities than you can grab for your little selves, and let’s get after living real lives.

“The future is unwritten. Thank you very much.”

Plosiv

Plosiv helps you live with the pizazz, the panache, you always wanted.

Now, you can follow a depression of the mind with a depression of a button that precipitates a depression of the air. A sense of drama will pervade your actions. A feeling of significance, or tension.

Some have an impact that others lack: movie stars set hearts thumping, video game characters have combustible accessories and musicians are supported by the drop.

Imagine if you could walk into a room and have everyone’s gaze drawn to you. Imagine seeing a pretty young lady’s eyes widen as you deliver a telling insight. Imagine your father clapping you on the back with the approval and trust you always wanted.

With Plosiv, all this is possible.

It’s so simple. Keep it in your pocket. Finger its hand-crafted, noir-sheen plastic. And, when you need that extra emphasis, press the button.

Plosiv. Individual emphasis.

Saw Oxes live at The Luminaire in London on Tuesday. They were pretty fucking great:

SCJP

Giles Bowkett is rocking my world. Not just for Archaeopteryx, not just for his rant about VC and paternal patronage (a tautology) but because of the way he just hacks away at stuff because it’s fun. I do that, too, but I don’t have his admirable ability to not care if no one else cares.

Creating a music recommendation engine

I have spent the last six weeks writing a music recommendation engine, theperceptron.com It was fun. From the user’s perspective:

1. Enter a band you like.
2. Get recommendations for other bands you might also like.
3. Test out the artists recommended by visiting their Myspaces and websites, reading their Wikipedia summaries and listening to sample tracks.
4. Say whether you like or dislike your recommended bands.
5. Add promising bands to your playlist so you can listen later.
6. Suggest an artist or two that the site didn’t recommended.
7. Get on with your life.

From the code’s perspective: Recommendations are made based on connections between artists. These connections are found in data taken from the internet:

• Recommendations made by actual humans: tinymixtapes.com and epitonic.com and users of the perceptron.
• Artist admiration: artists’ top friends on Myspace.
• Artists on the same mixtape: muxtape.com
• Artists on the same record label: wikipedia.org
• Artists posted to the same mp3 blog: hypem.com
• Artists who have played gigs together: myspace.com

Each rating action that a user can perform on a recommended artist – liking or disliking, visiting websites, listening to songs or adding them to the playlist – is associated with a certain number of points. These points are used in two ways. First, each source has a running total of points given to the recommendations made by the source. Second, each artist connection has a running total of the number of points it has accrued. Recommendations are given a score based upon these point totals. Ignoring the weightings of the source and connection score, a recommendation’s score is calculated thus: score = (source_points + connection_points) / num_source_connections the perceptron’s algorithm is pretty obvious. What makes the site good is the choice of data sources. However, the algorithm does allow experimentation with adding data sources. If I add a bad one, the scores given to its recommendations drop very rapidly. It only took about 200 user rating actions to get the site’s data source weights pretty good. Here is the current table (higher numbers are better):

Source	Score
Epitonic similar artists	0.439
Tiny Mix Tapes similar artists	0.316
Myspace top friends	0.128
Mixtapes	0.075
Record labels	0.020
Epitonic other artists	0.016
MP3 blogs	0.003
Gigs	Score hasn’t settled, yet.
the perceptron user recommendations	Score hasn’t settled, yet.

Why am I giving away my secrets?

1. I would love to see people take my ideas and make them better, or apply them to something else.
2. I’m a fucking genius. I’ll have some even better ideas tomorrow.

the perceptron

the perceptron is not a music recommendation website. It is a giant, bmx-riding, AIWA-headphoned, sunglassed, could-totally-get-an-A-in-maths-brained robot that processes the musical internet, notes your every move, and then plays you music you will fall in love with.

- the perceptron blog

He is my hero

Easy evaluation of data mining approaches

[This article runs through the approach and provides illustrative Ruby/Rails code snippets. Download all the code here.]

I have been working on a dating web site for a while. I have developed a structure for evaluating different methods of picking pairs of users to be introduced. This evaluation structure could be used in any app.

Some precepts. First, methods of picking pairs of users to introduce are called systems. Second, for every match, each system can produce a grade between 0 (worst) and 1 (best). Third, each pair of users that is graded is called a system match. Fourth, there is some “objective” system of measuring the real compatibility of two users that can be used as a benchmark. In the case of my site, this objective system gives a grade of 1 to a pair of users where one has said they like the look of the other, and 0 otherwise.

Onto the code.

First, a class to embody the idea of a system. This class can be subclassed to provide an implementation of a specific system. All this implementation needs is a method, calculate_raw_system_grade(), that takes two users and returns a grade. generate_system_matches() goes through each pair of users and calculates its grade using each system. Note how the SYSTEMS array contains some constants, each of which refer to a System subclass. The method then calls calculate_raw_system_grade() on each System subclass for each pair of users. evaluate_system_matches() gets a score for each system by finding the average difference between the system’s grades and the objective system’s grades.

class System “type = ‘#{system}’”)
  end
end

Second, the class for a system grade. The only notable code here is the Rails data model.

class SystemGrade
Third, the SystemMatch class. Objects from this class represent a pair of users. Each user match has many system grades attached to it, one for each system. get_objective_system_grade() pulls out the system grade for the “objective” system and get_non_objective_system_grades() pulls out all the others.

class SystemMatch “User”,
             :foreign_key => “first_user_id”

  belongs_to :second_user,
             :class_name => “User”,
             :foreign_key => “second_user_id”
  
  has_many :system_grades, :dependent => :destroy
  
  def get_objective_system_grade()
    # boring code
  end
  
  def get_non_objective_system_grades()
    # boring code
  end
end

Fourth, the controller. This just has a method, evaluate_systems(), to run all the lovely system evaluation code.

class AdminController
Fifth, sixth and finally, there are the classes for specific systems, ObjectiveSystem and BlahSystem. The former calculates a benchmark grade for every pair of users. The latter is a system we actually want to test. The model for these classes is:

class ObjectiveSystem
Now, if you have a brilliant idea in the middle of the night for a new system of matching things, you can just implement a class like the one above, add it as a constant to System and add that constant to the SYSTEMS array and then run your code.

Hacking and transience

Some people think that programmers are like artists¹ and others think they are like designers and can, at their best, be like craftsmen².

Sense and Sensibility, published in 1811, is still widely read and praised. A beautiful chair carved a hundred years ago can still be admired and sat upon today. The tin can, invented in 1810³, remains the best method of cheap, long-term food storage. Thus, there are examples of pieces of art, craftsmanship and design that have longevity.

However, do the fruits of hacking have the same longevity? I am confident that, however good it is, no one will be using the current version of my dating web site in ten years’ time. In fact, the same could be said of any of today’s programs and web sites.

Hacking, then, is closest to design because both methods are iterative, whereas artistry and craftsmanship are not. Novels are written, revised and then published. Chairs are carved, adjusted and then sold. In contrast, successful programs and web sites do not remain static, they are improved. The tin cans we use now are significantly different from the ones made in 1810. Thus, the difference between hacking and design, and art and craftsmanship, is that, with the former, stasis means death.

Will this change as hacking matures? It hasn’t changed for design. Design meets a specific, current need. Art and craftsmanship soothe more general, perennial desires: feeling vicarious sadness, or taking the weight off one’s feet.

Why are designers and hackers happy making things that will not outlive them? Perhaps the pleasure in the work is enough. Or perhaps the urge for contemporary approval is greater than the urge for longevity. But it still makes me sad that I am pouring my creativity onto the ground so that it will nourish the grass for a little while longer.

¹ Paul Graham’s essay, Hackers And Painters.
² Joel Spolsky’s essay, Craftsmanship
³ Wikipedia article on the tin can

When you come to the end, you go back to the beginning

[In this post, there are lots of spoilers for The Wire. If you haven’t seen all five series, go and watch them.]

Lester Freamon in The Wire: “We’re building something here, Detective. We’re building it from scratch. All the pieces matter.”

The writers on The Wire occasionally let their characters say what it is the programme is driving at.

David Simon, the creator, has said that The Wire is about the way institutions affect people’s lives¹. He has also said the programme is modeled upon Greek tregedies², and as he noted, Grecian tragedy is obsessed with fate. If you combine these three insights, you come up with something magical: a piece of art that simultaneously champions two opposing views. In the case of The Wire: that every piece matters, and that the characters are fated.

For the characters, they are fated. Their personalities dictate what they will do when placed in circumstance. For the viewer, every piece matters. They do not have a complete picture of the characters’ personalities. Thus, they cannot predict what the characters will do in each situation until every personality facet has been revealed.

This contradiction is illustrated by Jimmy McNulty’s progression through Series 1. McNulty takes pride in his work and refuses to let police politics or his home life get in the way of his work. This means that when his superiors become concerned that his indictments will start riling their superiors, he continues his investigation. At the end of the series, this leads to him being exiled to a dead-end detail in the Baltimore port.

However, there is an extra subtlety: people can be beholden to zero or more institutions. Can the characters in The Wire choose the institutions to which they belong, and thus have a choice in their fate?

By the end of Series 3, McNulty has had enough of detective work and become a beat officer. This allows him to join the institution of marriage by moving in with Beatie Russell.

Thus, at the end of Series 3, McNulty withdraws from one institution and joins another. Did he choose this, and thus his fate? In a sense. He willfully chose to leave the first institution because it could not meet his terms.

By Series 5, McNulty is a detective again and, consequently, he goes back to drinking and philandering. However, he was most definitely ripped out of his newly adopted institution by the pull of his old one. Thus, in that sense, he had no choice.

¹ “[The show is] really about the American city, and about how we live together. It’s about how institutions have an effect on individuals, and how…whether you’re a cop, a longshoreman, a drug dealer, a politician, a judge [or] lawyer, you are ultimately compromised and must contend with whatever institution you’ve committed to.” Source: David Simon “The Target” commentary track [DVD], 2005.

² “Much of our modern theater seems rooted in the Shakespearean discovery of the modern mind. We’re stealing instead from an earlier, less-traveled construct—the Greeks—lifting our thematic stance wholesale from Aeschylus, Sophocles, Euripides to create doomed and fated protagonists who confront a rigged game and their own mortality.” Source: David Simon interview in The Believer, 2007.

If Twitter has invented a new mode of communication, and Digg has invented a new mode of content filtering, does that mean that there is lots of room for Twitter clones that cater to different communities, in the same way that there are lots of Digg clones that lean towards different topics and styles of content?

Listen! Listen!

Just found an amazing American public radio interview programme, The Sound Of Young America. The interviewer asks interesting questions and gets out of the way. I have listened, in a prostration of amazement, to the following:

Ariel Schrag
Steve Albini

Don't talk to the cops

An American lawyer enumerates the reasons you should never talk to the police and, more importantly, hammers home the fact that there aren’t any reasons to talk. Part 2, a police officer confirms: