In what's likely just the beginning of a long-term story, job listings indexed by employment search engine Indeed.com indicate that market demand for data scientists and people capable of working with "big data" took a huge leap over the last year. David Smith of Revolution Analytics performed several related queries and posted the results today on his company's blog.
The most common definition of "big data" is datasets that grow so large that they become awkward to work with using on-hand database management tools, such as Excel. It's a soft term and is super trendy right now - but that doesn't mean the trend's not big and real.
Thursday, July 21, 2011
Jobs for Data Scientists Explode Across The Market
Wednesday, January 5, 2011
“Madness of Crowds” or “Wisdom of Groups”? - Improvisations - MIT Sloan Management Review
Co-authored by MIT’s Thomas W. Malone, Alexander Pentland, and Nada Hashmi, along with Anita Williams Woolley of Carnegie-Mellon and Christopher F. Chabris of Union College, the Science story details two studies about how groups perform.
The researchers concluded that “group intelligence” correlates less with the intelligence of the individuals and more with the social sensitivity of group members, an equality in how conversation is handled, and even the proportion of females in the group.
A Boston Globe story last month, “Group IQ,” picked up on the Science report and added a few interesting details. “People have been studying group dynamics for decades, seeing crowds variously as sources of madness and wisdom,” writes the Globe’s Carolyn Y. Johnson. “[Senior author of the study] Malone and colleagues could not find an example in which people had asked the relatively simple question of whether groups had intelligence, the same way individual people do.”
Wednesday, December 22, 2010
5 Predictions for Online Data In 2011
Summary
In the midst of all the data-driven innovation we are seeing, this will be also the year of separating the non-trivial from the trivial.
It’s one thing to acquire terabytes of data, and it’s quite another to cleanse, disambiguate and mobilize that data in service of real-time insights into markets young and old.
The intellectual and experiential barrier to entry in social media, I think it’s fair to argue, is relatively low. It’s therefore harder to distinguish oneself, but certainly easy to get started. That’s been the beauty of the experiment all along.
Until now, data science and data marketing have been relegated to the realm of a self-selecting and highly motivated few, but as new tools democratize access, we’ll start to see a different dynamic.
But with great power comes… well, the Mark Twain quote I used last time still applies: “There are three kinds of lies — lies, damned lies, and statistics.”
Data can create new insights and open new opportunities, but it can also be twisted to serve an agenda or simply tell us what we want to hear.
It’s all in there, though — there in the data somewhere, if you know what you’re doing and how to do it well. Data knows everything we know, everything we don’t know, and, as it turns out, even a few things we don’t know we don’t know.