By Decade’s End, Big Data Will Be Bigger Than Ever

Editor’s Note:  But will we have the needed security professional discussed in the article below?  If not, what are the consequences?  See here for more on this crucual issue.

From: US News & World Report

Specialized tech knowledge will be necessary in our data-driven society

By John Sandman

Information technology will be one of the fastest-growing fields this  decade as commerce has gone digital in industries across the board.  While the world may not see the turn-of-the century staffing crunch that  was feared during the transition to Y2K and euro conversion, the  proliferation of data could outstrip the number of people being trained  to manage it.

Teri Morisi, economist and  branch chief at the Bureau of Labor Statistics, identifies computer  occupations the BLS is tracking for growth. Included are computer and  information research scientists; computer systems analysts; computer  programmers; software applications developers; systems software  developers; database administrators; network and computer systems  administrators; computer support specialists; and information security  analysts, Web developers, and computer network architects.

“All  computer occupations are expected to grow faster than the average for  all occupations at a rate of 14.3 percent for the 2010 to 2020 period,”  Morisi says. An exception is computer programming. Although it has long  been a target for outsourcing, it is still expected to grow at a rate of  12 percent.

The occupation with the most job  openings, software applications developers, is expected to gain 143.8  million jobs by 2020, a 27.6 percent increase, followed by systems  software developers, with approximately 127,000 jobs at around a 32  percent increase. Database administrator jobs are expected to spike 30.6  percent and gain about 34,000 new positions. Computer hardware  engineers, which are not classified with computer occupations, will  climb 9 percent, from approximately 70,000 to 76,300 jobs. Computer and  information systems managers, also not in the computer field, will grow  from about 308,000 positions to approximately 364,000, an 18 percent  increase.

A Ph.D. in computer science or a  related field is required for most computer and information research  scientist jobs. In the federal government, a bachelor’s degree may be  sufficient for some jobs. According to the BLS, an associate’s degree  could suffice for network and computer systems administrators, as well  as for computer support specialists.

The term  “Big Data” is generally used to describe the exponential growth of  structured and unstructured data. Volume shows no sign of abating. The  velocity with which it moves is approaching light speed on stock  exchanges in the securities industry. There’s also a much greater  variety than in the past, including text, video, and audio. By some  estimates, 80 percent of an organization’s data is not numeric.

This  data-driven sea change in the workplace has been going on at least  since 1964, when IBM invented the mainframe computer, a milestone event  in the conversion from manual processing of data to automation. In the  mid-1960s, the New York Stock Exchange closed every Wednesday just to  match buy and sell orders so trades could be settled. Trade volume  spiked from 5 million shares in 1965 to 15 million in 1968. Today, in an  era of high-frequency trading and 100 gig data centers, the  NYSE-Euronext, as it is now known, averages in excess of 1 billion  shares per day, reaching nearly 2.5 billion on June 22. Mega-volume of  this kind could not have been supported by traders waving paper tickets  at each other on the trading floor.

As  data proliferates, one solution could be to put the ability to manage  it—and the tools to manage it with—in the hands of more people who  aren’t IT pros.

“The fundamental assumption of  Big Data is the amount of that data is only going to grow,” said John  Ginder, the lead of the big data analytics and the environmental science  teams at the Ford Motor Company, during a July interview with ZDNet. If  there is a shortage of database scientists, Ginder favors giving  non-technical people tools to solve big-data problems themselves.

“That’s  the endpoint I’d love to see us move toward,” he said, “but there  aren’t enough tools out there to enable us to do that yet.”

But  the data industry is well into the build-it-and-they-will-come phase.  The data is being built and the users are coming. The tools to use it,  whether designed for the non-tech user or IT engineers, are likely to  follow.

“It’s all about data,” says Adam  Honoré, research director at the Aite Group, a Boston-based consultancy  specializing in information technology research in the financial  services industry. “We need people who can move it, transform it, and  correlate it. There will be large volumes of global market data, coupled  with a myriad of additional data sources like news, social media, and  by 2020, audio and video. We also need people who can apply semantics to  it and make it all machine-readable for both risk management and alpha  generation,” a solution used by algorithmic trading to develop  quantitative trading models.

Honoré also cites  the need for visualization specialists. They typically work as graphic  designers, but their job functions often bleed into analytics and  research as they look to improve the way data is represented. “The  securities industry, as one example, can’t keep adding screens to a  traders’ desktop in response to the data torrent they get,” says Honoré,  “although tablets are changing the way people react to data.”

And  integration specialists have an important role. “Managing complex  integrations between global venues and market participants is going to  require a higher skill set than currently supported,” he says. “Further,  as more unstructured information becomes part of the trading  mainstream, multilingual skills will be important for semantic  analysis.”

The demand for hard computer skills  is trickling down to non-technical job descriptions that cross  industries—including financial services, pharmaceuticals, and  manufacturing.

[See The New Concerns of an Evolving Workforce.]

“I’m  constantly asked to recruit people who understand both marketing and  IT,” says Kathleen Shelby, director of recruiting at Montvale,  N.J.-based FlexTime Solutions, a management search firm specializing in  marketing, communications, and design.

“We deal  a lot with the pharmaceutical industry now and training materials are  primarily interactive,” she says. “The relationship between content  developers—people writing the words and producing graphics—and Web  design, even coding up a website is becoming tighter.” She adds,  “Everybody wants something in the way of an app for the iPad. We can’t  find enough developers.”

In general, there have  been gaps in the move to automation. Insurance carriers, including  industry leader Allstate, have been able to estimate which of their  customers are likely to make a claim using predictive analytics, a  variety of statistical techniques from computer modeling, data mining,  and game theory that analyze current events and historical facts to make  predictions about the future. Claims adjusters may bring handheld  devices to the scene of an accident, but that doesn’t prevent  information from being transferred to a paper claim form.

But  one thing is clear: The hybrid of half-automated, half-manual  processing will go the way of the Pontiac and eight-track tapes. Data is  the medium of exchange and for it to flow most efficiently, it is being  digitized.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Leave a Reply

Your email address will not be published.

Please Answer: *