Subjectively Social Data: The Friction Between Users and the Data They Seek

In Motek’s last two blog posts, he has discussed the differences between objectively and subjectively Social Data, and identified two key challenges to crafting a workable business plan for subjectively Social Data: PERCEPTION and CONNECTION. Today he’s going to discuss a third challenge, FRICTION between users and the data they seek.

There are few frustrations in the realm of data sharing and curation that can compare to discovering a magnificent dataset, speaking directly to the problem you’re trying to solve, provided with good intentions by a data collector/creator who genuinely wants you to solve your problem – but it’s in PDF form. Why is it in PDF form? Too often, because someone, somewhere in the chain of information transmission had some notion that this “protects” the data. A concrete example of this happened to me recently: I was provided a 20-row, 5-column data table – less than a page of data – in PDF format. I asked for a CSV-formatted version, but was told that “the data has not been presented to project funders, so before I allow information to be accessed openly I would like them to be able to see it and comment on it.” So I made the obvious point: if you don’t want me to share the data with other people, that’s fine – just tell me so, and I won’t. But how can you think that pdf-versus-csv helps this process? If I’m hellbent to breach your trust and indiscriminately share your data, I can just post the pdf to the web; or, better, I can type your data into a spreadsheet; or, better still, my OCR software can do that for me! And meanwhile, you have failed to actualize the potential of your data, by creating an artificial impediment to my playing with it, visualizing it and reconfiguring it and combining it with other data in new and novel ways. In effect, you have tried to prevent me from sharing the data with myself, preventing any of us from unlocking its true potential.

This is the obstacle to subjectively Social Data that I label FRICTION, and it is remarkable how often it is the primary bottleneck to data distribution – how often we finally discover valuable data, whose creator wants us to have it, only to learn that it’s inaccessible for irrational reasons. There’s data that is not machine-readable. There’s data that is poorly formatted. There’s data that people think is confidential or sensitive that is neither, but no one has checked. There is data that’s being held hostage by one junior clerk’s nervousness about spreadsheets, their care and maintenance.

Why do I hasten to point out that this notion of FRICTION is only an obstacle to subjectively social data? Because objectively social data uses the notion of “friction” to describe something completely different. When Mark Zuckerberg demands (and is in the process of achieving!) “frictionless sharing”, the friction he refers to has nothing to do with data formats or obsolete usage policies. Those impediments are all gone for objectively social data, because products like Facebook and Twitter (brilliantly, shockingly) imposed their own set of norms and protocols on all of their users. Facebook decided data format, data sensitivity, data dissemination, and data security policies unilaterally, and by this act they made all of the bottlenecks I’ve pointed out disappear, leaving just one, crucial point of friction: your privacy. Your privacy is the only point of friction for objectively social data, and because eroding it achieves the aims of corporations and governments alike, even this last point of friction may yet cease to be a relevant obstacle.

The situation is completely different for subjectively social data. Here, the logistical sources of friction are much more problematic, because the model is focused on servicing the users’ unique data needs. In the past, obstacles like this have been overcome by consortiums of service providers banding together to create and/or agree upon industry-wide standards (like the W3C), and the recent push behind the “Green Button” and “Blue Button” initiatives are proof of how powerful such initiatives can be. But so far, these initiatives have largely been government-initiated and focused on specific verticals (energy, healthcare); a true universal standard for cleaning, formatting, and licensing data remains a distant goal. Until it is achieved, any business hoping to profit in the subjectively social data must provide some kind of credible answer to the question, “how do you plan to reduce the friction between your users and the data they want?”

Adding the above observations to my previous posts, here is a recap of the key obstacles a successful subjectively social data strategy must overcome:
(1) Users are unable to PERCEIVE that there is a difference between the actual and potential value of their data, and that they can only obtain optimal value by effectively sharing their data with just the right people.
(2) Users find it difficult to CONNECT with data suppliers, and data suppliers have equal difficulty connecting with users.
(3) Users must overcome FRICTION that prevents them from successfully harnessing the data that they are able to find.

In my final blog post, I will present a blueprint for a business model that manages to overcome all of these obstacles by way of a metaphor as old as civilization itself: the metaphor of data-as-story. Using this metaphor as a framework, I will show how it is possible to have a deep and profitable impact on the subjectively social data space.

 — Motek Sherman, BuzzData investor

Subjectively Social Data: The Connection of Supply with Demand

In Motek’s last blog post, he discussed the differences between objectively and subjectively Social Data, and identified the first challenge to crafting a workable business plan for subjectively Social Data, the challenge of PERCEPTION. Today he’s going to discuss a different challenge: CONNECTION – of supply with demand.

When someone needs data, they won’t usually need abstract data, or generic data. They’ll need specific data that answers a specific question, or speaks to a specific desire or need. At first glance, this seems like a trite observation, but it leads to a profound (and stark!) conclusion: any subjectively Social Data strategy that can’t explain how to connect specific data needs (demand) with correspondingly specific data content (supply) will probably fail.

Of course, every successful business must solve this challenge, bridging the gap between their product or service and a market. But there are certain obstacles to this process that are unique to the realm of subjectively Social Data. For example, since subjectively Social Data is all about providing users with the specific data that they want – and since each user potentially has wildly different needs – an effective product will necessarily be broadly configurable by the user. Specifically, for a subjectively Social Data strategy, it will be important to let the user determine things like (1) the specific subject of the data they need, (2) the degree of fineness or coarseness they require, and (3) the specific fields or datapoints they’ll need to collect.

Note how different this is from the objectively Social Data strategy, which is all about rolling out products with the absolute minimum number of user-configurable options. Since objectively Social Data is all about extracting monetizable data from users, this makes perfect sense: the same data points need to be extracted from each user, in exactly the same way, in order to maximize the value of the aggregate data. Put more simply: Facebook can’t exploit data about which brand of car people might prefer, or which pop star they’re most interested in chatting about, unless they carefully control the way in which their users provide and absorb data via the Facebook platform.

So, on the “demand” side of the equation, CONNECTION requires a very different approach to creating a viable business model if you’ve chosen to take the subjective approach to Social Data. What about on the “supply” side? Here, the obvious first strategy for any prospective Social Data company is to try and create a virtual marketplace where data sharers and sellers will be able to find data users who need their specific datasets. Assuming that a Social Data company (1) has created, or has access to, a marketplace that is stable and efficient, and (2) has figured out how to entice a critical mass of data suppliers to participate in the marketplace, it stands a good chance of making the connection between supply and demand. The 500-pound gorillas of the objectively Social Data space have all employed this strategy. But it has been much more difficult for companies big (like Microsoft’s Azure) and small (like InfoChimps) to follow suit in the subjectively Social Data space. Why might this be?

Part of the answer lies in the fact that if you’re not monetizing users’ data, then it’s far less clear how to pay for the infrastructure of the marketplace (Brokerage fees? Advertising? Flat user fees, or seller fees, or both?). But surely if this was the only obstacle, or even the major obstacle, then Microsoft or Google would be able to overcome it, if only by the sheer gravitational force of their cash supply. The fact that Azure and Fusion Tables remain more-or-less niche products suggests that there is something deeper at work here. In my opinion, this deeper issue is the third critical obstacle that I’ve identified: FRICTION. I believe that, whereas Mark Zuckerberg’s dream of “frictionless sharing” of (users’) data is close to being realized in the objectively Social Data space, the problem of friction remains difficult and pervasive for subjectively Social Data, and this friction makes the problems of Perception and Connection even more challenging. I’ll explain why in my next blog post…

 — Motek Sherman, BuzzData investor

Next post: Motek tackles the challenge of Friction
in the subjectively Social Data space

Good times at GovCamp 2011

On Wednesday BuzzData attended Canada’s GovCamp 2011. The bling factor was definitely jacked up for the event this year, which was held at the gorgeous MaRS Innovation Centre in Toronto.

Photo from MaRS

We demo-ed our platform to attendees (as well as a number of employees at MaRS who couldn’t help but check out the brightly coloured interface) and were lucky enough to get a chance to speak on a panel called “Making Data Social,” which focused on how to enliven, expand and sustain open-data communities.

You can barely see me sitting onstage to the very right. In this photo GovCamp organizer Mark Kuznicki (@remarkk) speaks about helping found the OD community in Toronto.

GovCamp was a great opportunity to meet progressive policy and government representatives who gave us really interesting insider insights about how to coax government entities into adopting BuzzData. Apparently it won’t be as easy as casually saying, “um, fyi, our platform solves all your open-data problems” and signing them up.

After the social data panel I listened to Ontario Ministry of Research and Innovation’s Alex Sirota talk about how to consolidate grant distribution processes across the province and it was really eye-opening to hear him describe how backward many internal tech processes are in government. He’s been working on this problem for five years and it sounds like one hell of an uphill battle.

After the Canadian election David Eaves wrote a great blog post warning young, newly-elected MPs about getting sucked into a technologically complacent work environment:

The real first test will come when you set up your office. The test will happen so quickly and so subtly you won’t even notice it. It will go something simple like this: you want to use a Mac or an Android phone as an MP. A friendly parliamentary IT staff will deferentially but sternly tell you that for reasons of security and compatibility this isn’t possible. There’ll be a moment of awkwardness and, the next thing you know, you’ll be sitting in your office, using a HP desktop computer with Windows 7, thinking “I’m so lucky to have been set up this quickly…”

As chilling a forecast as this is, Sirota’s presentation implied the situation is even more dire; he said much of the data-processing protocols used by the Canadian government are “mid-20th century” (let alone Windows 7), as hard as this is to imagine. We’re talking about opting for manila envelopes over cloud-computing services. Yikers.

Our CTO Pete Forde and project manager Sarah Facini talk to a MaRS employee who works with data about where BuzzData fits in the value chain. Dunno about you guys, but I dig Forde’s passion for loud paisley …

What’s frustrating about this looming obstacle is the fact that we at BuzzData want to embrace a legitimate authority model as much as possible. We would much prefer to have, say, the City of Toronto be present on BuzzData, publishing their open data there and curating the community that grows around it, over having other people download Toronto’s data and then upload it to our platform. Not because we’re against independent efforts, clearly we support them, but because it’s always better to get things straight from the horse’s mouth, whether it’s gossip, executive orders or datasets.

So it’s a bit sobering to hear that getting government on board with BuzzData is going to take strategic “jujitsu” over plain old solution-oriented common sense. That said, we’ve been incredibly successful getting some fantastic organizations with serious clout on board with our upcoming beta, so perhaps it’ll just be a matter of demonstrating by example, rather than “jujitsu” per se.

Oh, and just to end off, here’s my favourite slide from my presentation about social data and open-data hubs at GovCamp. I had a little fun with it :)

-Momoko Price