Coding Test Cheaters: How They Get Caught by Test Providers

If you’re trying to get a technology job in an investment bank or pretty much any other large organization nowadays, chances are that you’ll be expected to take a coding test. These tests can be challenging, and you’re unlikely to get much further in your application if you fail. Predictably, therefore, there’s a temptation to cheat. 

Beware, though, the test providers are onto you.

Grzegorz Jakacki, founder and chief scientist at Codility, a coding challenge provider used by American Express, Barclays, Deutsche Bank, and PayPal, among others, says his company’s research indicates that around 10-11 percent of test-takers cheat across all industries. In sectors like media and entertainment it’s much lower, at around 3 percent; in societies where wealth and incomes are highly polarized, the proportion of cheaters is far higher. 

Cheating on coding tests comes in various forms. Students applying for entry-level technology jobs at investment banks tell us it’s not uncommon for everyone taking a test to sit together in a room and share their solutions. Jakacki says the most common cheat involves bringing a friend along: “You bring someone with you, and they solve the task.”

In the less sophisticated version of this cheat, Jakacki says the person sitting next to you solves the problem; in the more sophisticated version, you allow someone to remotely login to your computer to achieve the same aim.

Other cheats involve simply looking up the solution online (sometimes the same problems come up again and again) and copying and pasting-in the answer. Alternatively, students who’ve already taken the same test share their solutions with friends, who all submit the same thing. 

If you do any of this, chances are good you will get caught. Jakacki says Codility has a tool that flags when someone switches their IP address during a test. (“It usually means something strange is going on,” he says.) Codility also checks each candidate’s responses against tests that have already been submitted, and uses an engine that looks for similarities between a candidates’ submission and solutions posted online. “There are thresholds,” Jakacki adds, “But we have technology that notices when people are committing fraud.” 

Codility isn’t alone in rooting out cheating. Vivek Ravisankar, co-founder and CEO of Hackerrank, which is used by Goldman Sachs, Morgan Stanley and others, says they’ve built a web scraper that scrapes some of the popular sites where questions are leaked.

Ravisankar says Hackerrank also has a machine learning model that looks for code plagiarism based on several telltale signs, including pasting the code in, the number of edits made, and the number of differences between the answer submitted and known public responses. The model throws out a score which indicates the likelihood of cheating. “We have tested this extensively to reduce false positives,” Ravisankar says. “For example, there are certain questions like Fizzbuzz where you can only have so many variations of code snippets, and we don’t flag them unnecessarily.”  

Hackerrank also has a digital interviewing platform. When this is used, Ravisankar said candidates are also asked to provide a photograph to verify their identity and confirm the right person is taking the test. 

While these measures may not be impenetrable, they make cheating on coding tests a far more risky proposition than most candidates probably presume. “You can never be 100 percent sure that someone isn’t cheating,” Jakacki says. “The technology sector offers very lucrative jobs. There was a strong motivation to get a technology job before the pandemic, and that motivation is even stronger now.”

A modified version of this article originally appeared in eFinancialCareers.

One Response to “Coding Test Cheaters: How They Get Caught by Test Providers”

  1. Bill Johnson

    Obviously if companies are doing all this testing instead of training then there is not a technology shortage just a we want cheap labor shortage.