I have recently had reason to focus on the area of monitoring of social media which has involved looking once more at the whole range of black box monitoring solutions that are out there.  This has caused me deep feelings of confusion and uncertainty.

The reason is this: when I do monitoring for a client, or advise a client on how to do monitoring, this is what I do.  I work out what issues the client needs to track.  I devise from this relevant search queries.  I punch these queries into a range of search tools, since no one tool covers all the bases.  I have a quick look at the results and tweak the search queries if necessary and also have a quick look at the general shape of the conversation (i.e. any especially relevant blogs, the balance of conversation between twitter, blogs, social networks etc.)  I then pull the feeds from these searches into an aggregator (Netvibes seems to be the best for this) and segment things by creating relevant tabs.  I then start to watch what is going on and chase down the threads that seem to be interesting.  Over time I add to and tweak the model as I start to get a better feel for where the conversation is happening.

With such a tool I seem to have everything I need to stay tuned in to the conversation.  I can write a report or analysis of this for a client (or the client can access the information by tapping into the tool direct) as well as having all the information the client needs  to actually “do” social media – i.e. respond to the conversation, create relevant content, engage with relevant communities.

However, out there is a huge industry selling incredibly impressive black boxes that reel of reams of charts and data and figures and tracking, with sentiment analysis and conversation mining (conversation mining?) and all sorts of other wizardry.  This is very intimidating – surely my little homespun costs virtually nothing solution that relies primarily on the intelligence and analysis of a real person cannot be anywhere near as impressive as these mighty emperors with all their fancy clothes?

But the problem is – when I look at all these impressive reports I can’t work out how they help me design and run a social media strategy.  They could help me craft a one-to-many message (but that’s called advertising not social media) and the sentiment / volume metrics might help in measurement – we did x and the volume / sentiment needle moved x per cent in this direction.  But that’s about it.  And in any case all of this intelligence I would be getting through using my homemade tool anyway – albeit the intelligence would be in my head, not in a chart.

The analogy that springs to mind is this.  Suppose there was a large room and inside it were all the key stakeholders of your organisation.  Let’s say these people were at a drinks reception that you had organised – gathered around in groups chatting about you, subjects relevant to what you/ they do or maybe just about sport and the weather.   As hosts of this party what do you do?  The logical thing would be to go into the room circulate around the groups, listen to what people are saying, have a chat, tell people what you were doing.  What you wouldn’t do is send someone else in to listen-in on the conversations or set up a form of remote surveillance and then sit in another room and wait for someone to prepare a presentation and report back.  It may well be that this report is very detailed and gives you more information than you could obtain just by circulating round the room.  It would tell you exactly what topics were being discussed, what volume and sentiment of conversation was attached to each, who was speaking the most, correlate the people with the most to say with the colour of their shoes, determine that those with black socks are marginally more positive about you than those with brown socks – or any other way you would want to “mine the conversation”.  But exactly how useful would this information be?  If all you were going to do was to walk into the room, stand on a platform and deliver a speech, this type of stuff might help.  You could, for example drop in a subtle reference to the fashion credentials of black footwear.  But that’s advertising.  That’s good old-fashioned one-to many communication.  Its not social media.

But there again – it appears to be big business.  People are buying these remote sensing and analysis products.  Is it just snake oil and are the purveyors of such trading (albeit unwittingly) on ignorance? Or have a missed something?

Update: Clearly I am not the only one – see this from Asi Sharabi