The questions came from the guys at wildfireapp.com and you can read them below, followed by our response.

How exactly are humans involved in your content analysis process? You say a human touch is a pretty key diversifier for you, and I’m hoping you can elaborate on that a bit more and how you anticipate scaling the human component.

How does your system deal with, for the lack of a better word, non-linear communication: sarcasm, double-entendres, etc? I know this has been a tricky issue in the past, and I’m wondering what, if anything, you have done to deal with that type of communication.

I guess the best way to answer these questions is for you to take a look at the image below.

It represents an analysis of Twitter content containing keywords related to P&G’s Max Factor cosmetic brand. If you look to the far left of the image (mid way up) you’ll see a thematic cluster that’s been annotated with an ‘exemplary’ text (coloured green), which reads:

“RT @katieweasel: Cher Lloyd spotted throwing a brick through the window of Superdrug Peckham and running off with two trolleys full of Max Factor foundation”

I’ll try and translate that for you…

Cher Lloyd (pictured above) is an X-Factor UK finalist and therefore a C-list celebrity, who’s also well known for wearing a lot of makeup. Back in August you may have heard the UK was rocked by a spate of civil disturbances or flash-mob riots that resulted in a lot of looting – a very broad swath of English society / social classes were involved in the trouble.

This tweet is making a joke within that context – it’s saying that an X Factor finalist (i.e. a paragon of our new ‘classless’ / ‘lacking class’ society) was spotted taking part in a riot and stealing two shopping carts full of Max Factor foundation crème from a popular store called Superdrug in riot-torn Peckham (South London).

The story,of course, isn’t true – it’s merely a sarcastic comment on Cher Lloyd’s excessive make-up wearing habits etc.

The virtue of our content analysis system is that we would never expect a computer to algorithmically understand all of that. Instead, we use the machine to break down the unstructured text content into key themes and topics (a ‘reporting’ process that is fast and highly scalable), in order to present to the human user a clear, qualitative summary of what’s being said by the community.

The user can then apply their judgement to what this information actually means, which is something we can’t rely on a computer to do for us.

Click on the image below (and then click again) to enlarge or click here to download a PDF.

We’ve built a simple browser-based interface that allows a user to bung in a small number of keywords.

Our system then pulls off the latest Twitter messages that contain the keywords, analyses the content and breaks it into colour coded cluster-themes.

This image is just a simple test using the keywords , & . After playing with it we’ve also found you can plug-in Twitter @IDs and get an analysis of what single individuals or small groups of people are currently talking about.

There’s probably a bunch of other stuff you could do with this kit that we haven’t thought of yet. If we made it more widely available, how would you use it?

Tell us what you think.

Click on the image below to enlarge (then click on it again) or click here to download a PDF.

Web monitoring can often yield hundreds, if not thousands, of brand-related web articles and social media posts every day.

This deals with the acute FOMO (Fear Of Missing Out) issue that all companies have with the social web – “if someone is talking about our brand or business, we’d really like to know about it!”

However, web listening platforms have created a new problem. They tip more user generated content onto our screens than any normally-overworked exec can possibly wade through.

This leads to questions.

What is all that content saying about us? Who are the key people generating this material that we really need to keep an eye on? Why do we bother to collect more social web data than we can usefully analyse?

This is a problem the Dollywagon team are working hard to resolve. We’re developing content analysis algorithms that try to see the wood despite the trees.

The image below is a simple demonstration using Twitter data. Over the space of a few days we collected more than 100,000 tweets that contained the keywords , and .

The algorithms have broken this huge volume of material down into key themes, which have been automatically clustered and colour-coded. The system then provides ‘exemplar’ texts to illustrate what each theme is about.

The key insight from the image is that a simple keyword filter strategy can give rise to a wide range of content themes. In this example we have everything from ‘ethical consumerism’ to ‘lewd teenage humour’.

With this information it’s possible to quantify the ‘size’ of each theme and calculate its ‘share of discussion’. This can be a cool thing to do if you want to monitor the rise and fall of images or perceptions associated with your latest ad campaign.

You could also use this information to refine ongoing keyword strategies and avoid burning through your meagre Radian6 ration of 10,000 tweets per month. Get in touch if you’d like to know more.

Click on the image to magnify or click here download the PDF file.

This post takes the transcript material for the preceding interview with Rupert and James Murdoch (19th July 2011). Our computer analysis seems to have done a good job of identifying the main themes and topics within the transcript (roughly 25,000 words!) and presenting it as an attractive and easy-to-consume infographic.

The Influence Engine deploys a specialised computational language analysis process that we call ‘N-gram analysis’ for short. N-gram analysis enables us to process the huge volume of content generated by the Social Web (such as personal status updates, Twitter messages, blog posts and even Parliamentary Select Committee transcripts) and reveal the general thrust of ideas, opinions and debate within any body of material.

In many ways we’ve found this approach to analysing what people say on the web to be more accurate and insightful than common types of ‘sentiment analysis’. We’ve been using the technology to rapidly break-down and analyse the avalanche of material that social networks generate about brands etc. and then supply timely market intelligence back to our clients.

It seems to offer an effective synthesis between a computer’s ability to mechanically process huge quantities of information with the ability of human beings to apply qualitative interpretation and judgement.

Click on the image below to enlarge the n-gram infographic, or click here to view a PDF version.

The story came to a head on Tuesday this week (19th July 2011) when Rupert and James Murdoch and Rebekah Brooks (pictured) were called to appear before the Commons culture, media and sport Select Committee of the UK Parliament.

For a little bit of fun we have just downloaded the transcripts for these sessions and subjected them to our proprietary computational (n-gram) language analysis.

The transcripts are quite long – 25,000 words for the Murdochs and 15,000 for Brooks. But the analysis has generated some cool images that reveal key themes within the evidence presented at the select committee hearing.

Below is the image of our analysis of the Rebekah Brooks transcript (we’re still working on the Murdochs’). Click on the image to enlarge it or click on the link below to view a PDF version. We’ve added a few excerpts from the transcripts to help interpret the image and bring the analysis to life a little. We hope you like it – feel free to leave comments.

Rebekah Brooks Phone Hacking Analysis – Dollywagon n-gram image

It’s becoming widely recognised that massive-passive Social Web data generated by humdrum, everyday occurrences has a profound ability to reflect, in real-time, hidden patterns of relationship and behaviour in our society. Perhaps even more surprising is the related, and growing, body of evidence that suggests the same online ecologies can accurately predict (or at least pre-figure) the future of brands and markets.

For instance, recent papers by HP’s famous Palo Alto Labs demonstrate how Twitter data can be used to predict box office success for new movies. Other examples (such as this and this) show how social media data can be used to predict the future of stocks. This isn’t science fiction or some kind of digital voodoo, but a natural outcome of the latest Network Analysis technology.

Dollywagon is intensely interested in this field. Our Influence Engine’s capabilities have developed rapidly over recent months and are now extracting exciting predictions about future market movements from a range of web-based data sources. For instance, the image below is derived from a recent analysis of around 10,000 news articles about the Market Research industry (we chose MR just because we’re familiar with it).

We used a computational language analysis (or N-gram) process to analyse this huge volume of web page content. The image is annotated and represents the final frame in a time-series animation (not shown) that reveals the underlying relationships between various ideas, knowledge and trends that underpin the behaviour and development of the Market Research industry.

A key feature of the analysis was an early indication of the eclipse of traditional full-service business models in the MR industry. This trend co-occurred with spectacular growth in the adoption of online research panels, which itself is being out-shone by the current rise of Social Web Analytics.

Early results suggest this approach to analysing both Twitter and other sources of web content show great promise for making good predictions about future market developments.

They reveal that on average the brand campaigns tested so far in South Africa outperform the creative impact benchmarks for 400+ radio campaigns already tested in the UK.

Jason Brownlee from Dollywagon Media Sciences which formulated the original UK study and is now providing ongoing analysis of RadioGAUGE: South Africa research results says:

“The RadioGAUGE Ad Credibility Benchmark data so far suggests that consumers in South Africa are more likely than their counterparts in the UK to perceive radio as an advertising medium that is good at conveying a brand’s personality and values.”

Building on this, RAB South Africa General Manager Norman Gibson adds that the beauty of the South African study is its capacity to provide focused findings that allow South African brands to better understand how their radio advertising campaigns perform.

RadioGAUGE has been able to provide broader learnings about the medium that can show any advertiser how to get better results from radio.

Gibson comments, “it’s not just a case of saying we feel this is how you should approach the medium, we now know what’s best for a radio campaign. This is because hundreds of consumers have told us during our in-depth research interviews just what makes a radio message resonate with them allowing us to provide proper insight, based on solid effectiveness research.”

Solid evidence of the depth of insight RadioGAUGE provides is drawn from the second campaign tested by RadioGAUGE for a broadcast brand.

This test revealed not only that radio successfully reached a brand’s target market that was highly receptive to information about its services, but that radio advertising was also able to reach efficiently the key decision makers within households for this particular brand category.

However, RadioGAUGE also strives to be as balanced as possible and reveal all aspects of a campaign’s performance.

The same brand campaign study revealed that a poor standard of creative execution in the brand’s radio campaign led to a negative effect on listeners’ perceptions of the brand, resulting in an reduced level of campaign effectiveness.

Uniquely, RadioGAUGE was also able to recommend what actions could be taken to improve the creative’s performance for the brand’s next radio campaign.

This adds credence to the view that pre-production for radio advertising needs more attention with careful thought given to the use of voice over artists. Using too many different voices within the individual ad executions that made up this campaign was perceived by listeners as confusing and unnecessary.

A key learning here is the need for consistency of message and creative delivery throughout every radio campaign.

Another interesting finding, this time from the first wave of research on a cellular mobile network brand, revealed how Radio’s ‘virtual TV’ effect made a very cost-efficient contribution towards maintaining a cellular network brand’s market-leading awareness and ‘share of mind’ scores.

This case study provided evidence of radio’s unique ability to establish strong emotional bonds with an audience, and how this media effect can rub-off onto an advertiser’s brand.

The study demonstrated Radio’s unique ability to imbue brands with perceptions of greater emotional ‘warmth’ i.e. people exposed to the brand’s radio advertising tended to feel ‘warmer’ about the brand.

Gibson says it is precisely these kinds of invaluable findings that show just how worthwhile the RadioGAUGE study is.

“Finally we have concrete evidence backed by solid research data which not only proves that the medium works but also provides key learnings on how advertisers can increase their return on their radio spend. Our aim now is to take those lessons and guide advertisers as to what they should be doing as part of our mandate of championing radio in South Africa”.

RadioGAUGE key findings at a glance:

• Radio is able to reach a target market that is more receptive to a particular brand message
• Radio can target customers with a message right at the time they are considering a purchase
• Radio is able to reach the key decision makers within households for a particular brand category
• Poor creative has a negative effect on listener perception/views leading to poor results and the reduced effectiveness of a campaign
• There needs to be consistency of message & creative delivery
• Radio’s ‘virtual TV’ effect makes a highly cost-efficient contribution towards maintaining brand awareness and ‘share of mind’ scores
• Radio has a unique ability to establish strong emotional bonds with an audience and imbue them with perceptions of greater emotional ‘warmth’

For more information about Radio Gauge: South Africa, contact

Norman Gibson on (011) 325 4935 or email norman@rab.co.za

“So what?” I hear you ask.

Well, content networks might just play an ever more important role in how we analyse what people do on the Internet and social web.

When analysts study network relationships on the web or within specific social media environments, they usually hunt for real links between people or web pages. These links generally take the form of hyperlinks, comments or inter-personal connections between ‘friends’ and ‘followers’ etc.

This is a really useful thing to do, but sometimes it’s a good idea to see how people, websites or organisations are linked by ‘ideas’, ‘interests’ or ‘activities’.

This means trying to understand what specific qualities individual members of a community have in common, which can be quite a hard thing to do.

One of the easiest ways to build a true ‘content network’ is to use Twitter hash tags like #fail or #oscar etc.

Hash tags are great because computers can use them to understand what a Tweet is ‘about’. This is thanks to millions of normal people doing all the hard ‘interpretive’ work by flagging what Tweets actually mean by applying a descriptive hash tag to their Twitter messages.

If you use hash tag data creatively it’s possible to map the ‘mind’ of any market or field of interest that has a life on Twitter. My last post offers some good pictures of what this can look like.

However, the problem with using hash tags to create content networks is this – perhaps no more than a third of all tweets will contain a hash tag. This means that hash tag analysis may not be fully representative of everything people have to say about a topic on Twitter.

A more comprehensive method of creating and analysing content networks would therefore find a way to understand what web articles or social media posts were talking about and then create links between them and other similar content on the web.

This is one of the things we’re working on at Dollywagon. We’ve been mapping content relationships in the Market Research industry to see how it organises itself into defined sub-sectors and what the current hot topics are in each.

To do this we let loose our Influence Engine system onto a database of 10,000 Market Research news articles that we took from the web. The image below represents some of our early results – if you have an insider’s knowledge of this industry (which I do) it makes for interesting reading.

Click on the thumbnail to view the picture (and then click on it again to fully enlarge it), or download a PNG file by clicking here.

This post moves the debate on significantly by examining how social web data can predict the future of markets and brands.

Our world is increasingly shaped by a vast army of bloggers, website authors and social media users that have created a massively complex ‘online ecology’ of opinion, content and debate.

There is growing recognition that some aspects of this online ecology, like social web activity, have a sincere ability to reflect hidden patterns of relationship and behaviour within human activity.

But perhaps even more surprising is the growing body of evidence that suggests the same online ecology can accurately predict (or at least pre-figure) the future of brands and markets.

This isn’t science fiction or some kind of digital voodoo, but a natural outcome of the latest complex systems science and network analysis technology.

Dollywagon has been studying this phenomena and has used the Influence Engine to map structures and patterns of web activity within many different markets, communities and fields of interest.

This case study focuses on the Smartphone market sector and presents our recent R&D efforts to investigate the ability of social web data to predict market and brand outcomes. The data driving this image was collected from Twitter back in April 2010 over a four week period.

It reveals the underlying strength of the iPhone’s consumer proposition. It provides a clear diagnosis (and prognosis) of the problems facing Nokia and yields an early indication or prediction of Android’s current market success.

Click here to view our case study.

Here are some other recent examples from academic (rather than commercial) teams that demonstrate current interest in this topic:

Hedge Fund Will Track Twitter to Predict Stock Moves

Researchers successfully predict stock market by analysing 10m Tweets

HP Labs: Predicting movie box office success with social media

As always, we love to hear your thoughts and feedback. Please feel free to use the comments box at the foot of the page.

Despite this the Web has miraculously self-organised itself into the vast source of text, imagery, data and video that we know and love today.

The numbers and complexity associated with the Web are mind-boggling – it consists of billions of pages stitched together by literally trillions of hyper-links. In fact, the web recently overtook the human brain to become our galaxy’s ‘Most Complex Entity’ (unless of course some distant aliens we don’t know about have got a better MCE…)

We already know that when a network becomes as complex as something like the human brain pretty amazing things start to happen. Mega-complex networks tend to give rise to stuff like perception, consciousness and personality etc.

So perhaps we shouldn’t be surprised when a network as complex as the World Wide Web begins to develop special properties too.

For instance, it’s becoming more widely recognised that some aspects of the Web, like social media activity, have an uncanny ability to reflect hidden patterns of relationship in human society and even predict (or at least pre-figure) the future.

This isn’t science fiction or some kind of digital voodoo, but a natural product of the latest complex systems science and network analysis technology.

Dollywagon has been studying this phenomena and has used its Influence Engine to map the structure and patterns of interaction within many different markets, communities and fields of interest on the Web.

Take a look at two of our latest market sector maps below. You’ll see they reveal how one community (in this case the Architectural and Construction professions in the United States) has self-organised into over-lapping sub-groups of interest and concern. The images clearly indicate who the go-to guys and influential ‘weather makers’ are in this specific field of interest.

The data was derived from a specialised web-crawl technique that we developed which picks out the connections between web properties based on their relationship with a common theme or topic.

Our next post will look at Twitter’s uncanny ability to predict the future – enjoy!

Image 1: a Network Analysis map of the Architecture and Construction sector as found on the web (click on the image to enlarge, then click again on the next image)

Image 2: focussing in on the Architecture community (click on the image to enlarge, then click again on the next image)