What is the impact when artificial intelligence transitions from assisting job applicants to evaluating them? HackerRank, a platform widely used by companies to assess and recruit developers, offers a preview of this future with Chakra. This AI agent conducts interviews, observes candidates’ work processes, and evaluates not only their final answers but also the methodology they employed to reach those solutions.
After approximately six months in its beta phase, HackerRank is now making Chakra broadly available to its clientele as of Monday. The startup reports that this AI interviewer has already conducted over 500,000 interviews during its testing period, with companies such as Snowflake, Snorkel, and Capgemini participating, while HackerRank also utilized the product internally.
AI has long been an integral part of job interviews for some time, with businesses leveraging voice agents and other automated tools to streamline candidate screening and enhance the efficiency of the hiring process. Conversely, job seekers have increasingly adopted their own AI-powered tools to navigate interviews, sometimes unbeknownst to potential employers.
With Chakra, HackerRank is making a bold prediction that AI can revolutionize not just the execution of interviews, but also the criteria employers can measure. Beyond simply checking for correct answers, the startup aims to gauge more elusive indicators like critical thinking and judgment, along with what it terms “AI fluency”—a candidate’s proficiency in framing problems for AI, assessing its output, and guiding it toward a resolution.
“The traditional method of evaluation focused on the output,” stated Vivek Ravisankar, HackerRank co-founder and CEO, during an interview. “Now, thanks to AI, anyone can generate an artifact.” He explained that the crucial question for employers becomes whether they can comprehend the underlying thought process and discernment that led to its creation.
In practice, a Chakra interview is designed to simulate actual job tasks rather than a conventional coding test. Candidates receive an assignment involving a real-world code repository and complete it within an interface that includes an AI assistant. As the candidate progresses, Chakra can utilize the context of their actions to pose follow-up questions, such as inquiring about their chosen approach or how their solution would adapt to a new constraint.
Ravisankar informed TechCrunch that Chakra is fundamentally altering the very structure of the hiring process. What previously involved three distinct stages—a recruiter screen, a take-home assessment, and a subsequent interview with an engineer—is now consolidated into a single Chakra interview, he explained.
Allowing candidates to use AI during an interview might suggest an increased risk of cheating. However, HackerRank reported the opposite effect. Ravisankar noted that flags for suspicious activity were 70% to 80% lower in Chakra interviews compared to similar traditional HackerRank assessments, though this rate varied based on factors like geography and seniority.
Ravisankar informed TechCrunch that providing candidates with AI access diminishes their motivation to covertly employ external tools that could provide them with answers during an interview.
Launched at TechCrunch Disrupt in 2012, HackerRank built its foundation on coding challenges, ultimately assisting companies in evaluating and hiring developers based on their technical skills. The Y Combinator-backed startup now boasts over 3,000 business clients, including Amazon, Nvidia, Clay, and Replit, and a global community of more than 30 million developers.
Chakra signifies a strategic divergence from the very technical assessment business that HackerRank cultivated for years. Its previous core product largely focused on whether developers could correctly solve coding problems. Nevertheless, Ravisankar contends that AI has rendered HackerRank’s earlier model less effective for gauging engineering capability.
Ravisankar internally likened this shift to Apple’s transition from the iPod to the iPhone, noting that while the older product still holds value, the new offering represents the direction the startup believes the market is heading. “Chakra will be the flagship,” he asserted. “It will be our path forward.”
However, assigning AI a more profound role in candidate evaluation raises a distinct set of concerns, particularly regarding the extent to which companies should entrust hiring decisions to an algorithm. Ravisankar informed TechCrunch that Chakra is designed to score candidates, with the ultimate hiring decision remaining in human hands.
He explained that AI can manage the more structured components of an interview by consistently applying employer-defined criteria, allowing human interviewers to dedicate more time to assessing cultural fit and addressing candidate inquiries about the company, team, and specific role.
“AI exhibits considerably less bias than humans, provided it is tuned correctly,” Ravisankar claimed, suggesting that an AI system can be programmed to adhere to the identical rubric for every candidate, thereby avoiding influence from factors such as a candidate’s background or education.
Nevertheless, consistent application of criteria does not inherently ensure an AI system is free of bias. Automated hiring tools can inadvertently absorb or amplify biases present in the data, models, and criteria used to develop them, prompting regulatory bodies to closely examine their application in employment decisions.
The deployment of AI in hiring is already attracting regulatory scrutiny. For example, New York City mandates that employers utilizing certain automated employment decision tools conduct an independent bias audit and inform candidates before their use. Ravisankar acknowledged that hiring is a regulated domain and confirmed that compliance with such requirements has been a development priority for HackerRank.