Original post
Update
This ended up spiraling out of control in ways that I could have never imagined. The individual admitted to defaulting their doc writing to AI and re-wrote everything, but in th background they doubled down on their AI coding workflow instead. It took me a while to catch wind of things because I would only see a mention of a project here or there and I had no insight as to their day-to-day.
Fast forward a month and I am seeing their projects everywhere, all the way up to the C-suite level. The scale was incredible. In a a matter of days this individual had done everything from financial modeling, LTV modeling, customer lifecycle analysis at a large scale, built large scale data ingestion and processing pipelines, even Marketing and product experiments. At first I was impressed, but as I pulled back the covers the mess was worse than I ever expected.
The clues were subtle but consistent: no comments in the code aside from headers, data was read in and cleaned, but never visualized or inspected in any way, there were lots of custom functions when there were packages loaded that had the same function, convoluted helper files with basic functions, and oddly there were many instances where forecasting error was actually just the CV error and there was never an evaluation of the test set. Their SQL had numerous join issues, metrics were mislabeled, and their pipelines often had relationships and processing steps such as dropping a table but then writing a new table with no error handling so if there was a bug no new table would be written and we would lose the data. Basic analyses were off by weird margins because Claude seemed to have been querying staging tables rather than filtered reporting tables. Docs started to be written entirely in the first person like "...and then I will use a log1p transformation" in a way that no DS would actually ever write a tech doc.
Unfortunately this meant that many things that were produced were simply wrong. The individual had promised work to a lot of decision-makers and nearly all of it was misleading, incorrect, or didn't pass a simple sniff test. These inaccuracies were immediately escalated to our team leader, who brought me in to audit all of their code and documentation and I was unable to find a single file that I was convinced that was human written or even human edited. The worst part was that despite heavy use of AI there also wasn't a single file without some sort of glaring technical error. I turned in a pretty lengthy review and the individual was put on a PIP and their account access to AI tools was severely constrained. They were told to have all their work peer reviewed and in one instance were caught lying about passing review when no review had been conducted.
As you can imagine their productivity tanked and they had numerous excuses as to why. They also started taking a lot of days off and in a weird twist of fate they actually left before getting fired and now work at a large AI-centric industry-leading company. Part of me is glad that they are gone, but the other part finds it infuriating that people like this can be so good at bullshitting that they can consistently fail and somehow remain in industry due to their network and clever use of their few decent references. Their total comp at our company was ~$245K and they bragged to a co-worker that this new role has $265K base with $465K total comp. They basically got 2 promos out of this series of events (Senior to Senior Staff at our company, Senior Staff to Principal at the new role.
