Federal regulators—especially those who oversee the financial markets—are always looking for ways to use analytics to uncover suspicious behaviors and ensure regulatory compliance. That’s not an easy mission, and the data specialists who watch the markets only expect things to become more complex. As traders increase their use of tools such as algorithmic trading—which leverages computers to rapidly execute trades according to a specific set of instructions—and rely on increasingly sophisticated analysis to develop their strategies, the data that markets generate is growing in terms of both volume and speed. In fact, suggests Saman Michael Far, the New York-based senior vice president of technology for the Financial Industry Regulatory Authority (FINRA), the trading of equities and options his organization follows can reach 75 billion transactions in a single day. FINRA is a not-for-profit organization set up by Congress to oversee the securities industry. It’s one of several regulators that keep an eye on the markets, including the Securities and Exchange Commission, the Consumer Finance Protection Bureau and the Commodities Futures Trading Commission. Bear in mind that when we talk about “markets,” we mean more than the stock market you see covered every day in the business media. The commodities market, for example, is where traders deal in things such as petroleum, crops and livestock. Whatever the market, the upward trend of data and speed is a common challenge for officials charged with keeping track of activity and ensuring each transaction has been lawfully conducted. To do that, they increasingly rely on Big Data and analytics, which allow them to look for unusual patterns of activity and then drill down for indications of what Far calls “bad behavior.” Although there are a few exceptions, “we operate in stream mode, looking at data as it comes in,” Far said. He describes the investigation process as being “exactly like a funnel,” where analysts look first at a wide pattern, then focus on individual transactions. “Eventually, a person has to look at the issue,” he explained. “At some point, a decision has to be made on whether to keep investigating, take enforcement action, or move on.”
Under the Hood
Obviously a lot of technology is required to make this happen. This month, FINRA completed a three-year effort to re-implement its architecture so its systems can dynamically adjust to the market on any given day, using tools such as Hadoop, JBoss and Redshift. Given its intense software needs, it’s no surprise that, on the coding front, FINRA is active in the open-source community. What makes the work interesting, and finding the right talent so important, is what Far terms the markets’ “constant state of evolution” and regulators’ need to keep up with it. “It's all about data analysis,” he emphasized. “Lots of innovation will be happening in open source big data, and we need to ride that wave. We need to keep advancing and building systems that let regulatory analysts get a grasp of what’s going on in the market, so they can be ahead of the curve, looking for emerging problems.”
What the Team Looks Like
What kind of people does Far need to make that happen? First, FINRA employs “a lot” of software engineers to work on the Big Data machinery, using Spark, Hadoop and similar frameworks and tools. Data scientists are in place to put the systems to work; their backgrounds vary, Far notes, with some coming from statistics, others from quantitative analysis, and still others with expertise in algorithm and data structures. Also, the organization has teams of usability and visualization experts, as well as systems engineering professionals. With regard to the tech-labor market, some of these skills are particularly hot right now. How does a quasi-government agency compete with brand-name companies who may offer options and a wider range of perks, to say nothing of more upside in terms of compensation? “We pay competitively and have a good culture,” Far said. But just as important—in fact, more important to many tech pros—is the fact that “people want to work for exciting technology in an interesting area.” FINRA also offers a commitment to career development. “Developing people is a big part of our thinking,” Far said. “It’s not like we develop a system and we’re done. So we need people with us for the long term. We’re not looking to hire someone to fix our problems today and then leave.” When looking for candidates, Far casts a wide net, hiring people right out of school, the startup scene and big tech companies. In most cases, candidates don’t need a financial background. “Our volume is very much like an Internet of Things operation or telecommunications,” Far explained. “We also hire people who may not have Big Data experience but do have very strong technical fundamentals. If they know the fundamentals under the hood, we’ll talk to them.” Finally, FINRA is among the growing number of tech employers who are creating career tracks for tech pros who want to rise through the organization without having to become a manager. For example, it has tracks for subject matter experts, product managers and project managers. Though FINRA is just one organization, conversations with agencies including the Federal Reserve, the CFPB and the FDA suggest that data will continue to grow in importance to federal regulatory organizations. If you’re up to the challenge of working with ever-expanding data sets being tapped at warp speed, it’s an area worth considering as you look for a new position.
