Your Next Tech Career: Big Data
By Melissa J. Anderson (New York City)
“There are a couple of big trends that happen every ten years or so in technology,” said Nancy Cooper, member of the Board of Trustees of the Anita Borg Institute for Women and Technology and retired Executive Vice President and Chief Financial Officer of CA Technologies.
“It was the internet, then the cloud, and now it’s big data,” she said. “Career paths are going to change because of big data.”
In fact, the push for technologists in big data roles is not years away. Job growth in this field is not even right around the corner – it’s right now. In fact, according to Deloitte’s 2012 Technology, Media, and Telecommunications Predictions report, by the end of this year, 90% of the Fortune 500 will have initiatives aimed at the big data field – and revenues will “likely be in the range of $1-1.5 billion,” the firm projects.
Dr. Francine Berman, Vice President for Research and Professor of Computer Science at Rensselaer Polytechnic Institute, also member of the Board of Trustees for ABI, pointed out the scale of information that’s available right now. She referred to an IDC report indicating that there are currently 1.8 zettabytes of data – put another way, she explained, “that’s 1.8 trillion trillion – sextillion – bytes of data out there in the world, and people are using this data in dramatic ways.”
She added, “Data-focused careers are hot now and becoming more important all the time.”
Tomorrow’s Careers
Big data jobs are expected to flourish in many industries, particularly those that rely on information or analysis of behavior or trends. Cooper said, “Going forward we’re going to be able to measure human behavior. We’ll be able to know if something works or doesn’t work.”
Big data is big business. The Deloitte report continues:
“From being the sort of toll that was only needed for meteorology or physics simulations, big data has recently moved into the mainstream: individual big data conferences are drawing thousands, BD companies are attracting funding rounds of over $50 million, BD venture funds are being created, and large existing software players are validating the markets by partnering with or acquiring outright early stage leaders in the space.”
Fields like social networking or retail seem like a natural fit for data careers – but so is the financial services industry. Dr. Berman said, “There’s a huge focus on information. When you look at competitive advantage in the 21st century world, it’s all about the data.”
Cooper continued, “All of the sudden there is a new tool we can use to gain insights into trends than we ever had in the past. This is transforming all information based industries, of which financial services is one.”
Berman added, “It’s a great area, and it would be great to see more women involved in it.”
Critical Questions
As with any new field, big data also presents challenges and uncertainties, and individuals into the industry need an understanding of the issues they may be dealing with. A recent article by danah boyd and Kate Crawford, both of Microsoft Research, published in Information, Communication, & Society, provides an overview:
“Will large-scale search data help us create better tools, services, and public goods? Or will it usher in a new wave of privacy incursions and invasive marketing? Will data analytics help us understand online communities and political movements? Or will analytics be used to track protesters and suppress speech? Will large quantities of data transform how we study human communication and culture, or narrow the palette of research options and alter what ‘research’ means?”
“In the business of data, privacy has always been an issue – we live in a world of huge transparency. When we are talking about business transaction, we are talking about handling massive amounts of data in aggregate, looking at patterns and trends, rather than individuals,” Cooper said.
Berman continued, “There are two Achilles heels when it comes to big data. The first is privacy – access to and control of information have tremendous potential to both help and hinder an information-driven society. The internet was not really created to incorporate the privacy controls we need.”
Coming up with the middle ground is the challenge for those in the field, she said.
“The second challenge is the economics of data,” she continued, pointing out that the physical infrastructure and workforce needed to store, transmit, and analyze data, as well as the effort of evolving information forward for future access, creates a “data bill” that must be paid for information to persist.
Electronic data has been collected for decades, and formats go obsolete quickly. “It’s like music – it’s hard to find a record player these days. Like music, data must persist through generations of media, and managed to accurately transition forward. Without that stewardship, digital content can cease to exist.”