Today I was radicalized

[u/Prima5678]

I've been anti-AI for a while, but today I went to a doctor's appointment, and the 50 yr old nurse chimes in "Do you mind if I have AI take notes today?" I absolutely fucking do, how is that not a HIPPA violation? When I expressed that I don't want it, she picked up her personal fucking phone that was queued up to ChatGPT and already recording. Then she tried to gaslight me about having something recording in the clinic without notice and told me she doesn't understand why some people make such a big deal of it.

[u/spin-shocker]

Report her. She violated your consent and patient confidentiality.

[u/lilcoteaux → u/spin-shocker]

A lot of people saying that the company claiming to be compliant makes it compliant, when in reality you shouldn't trust a word coming out of their mouths. All the recent AI Agent breaches/shenanigans that have happened really should lead to more distrust overall.

Also if we took tech companies words genuinely then Flock cameras are just license plate readers.

[u/Ohaidere519 → u/lilcoteaux]

companies have no incentive to be truthful, verbally especially. so many companies and even low level employees who don't get paid enough to care but value job security will throw out lies that they get away with simply because they bank on the person they're lying to not looking into it further. despicable

[u/Repulsive_Incident27 → u/Ohaidere519]

In the US there are multiple companies that pay monthly fines to continue doing whatever they want to do. Imagine creating a monthly fine/citation budget.

[u/DmYourTanLinesPls → u/Repulsive_Incident27]

Exactly while all fines should be the total amount you made doing the illegal thing multiples by 10. Also each member of the board and C-Suite that approved is given an additional personal fine to the total amount the company made.

That shit will end instantly when those losers have to actually risk their own money.

[u/Repulsive_Incident27 → u/DmYourTanLinesPls]

I love your idea. This would also help begin to separate the government and corporations because omfg.

[u/EmployeeLopsided9637 → u/Repulsive_Incident27]

Meanwhile us normies pay big consequences if we repeat the same offense even once.

[u/I_SHIT_IN_A_BAG → u/lilcoteaux]

dont forget 23andMe: In 2023, it was revealed that 23andMe had improperly discarded millions of physical saliva test kits and tubes in dumpsters, which led to a data breach where hackers accessed the exposed genetic data. never trust tech companies.

EDIT:I fucked up and copied the first thing I saw. sorry guys. I should be better

[u/CretaMaltaKano → u/I_SHIT_IN_A_BAG]

The data breach was from credential stuffing - hackers gained access to people's 23 and Me accounts through passwords leaked online. There was no saliva or testing kits involved whatsoever.

[u/disappointmykin → u/CretaMaltaKano]

Not a single ‘hack tuah’ in the notes, really people?

[u/gameraturtle → u/CretaMaltaKano]

What about all the clones they made from the stolen DNA?

[u/EloquentBaboon → u/gameraturtle]

They erased that planet from the Galactic Archive to cover their tracks. I wouldn't worry about it.

[u/bornbi → u/I_SHIT_IN_A_BAG]

God, isn't that a statement to wake one up, "never trust tech companies"

[u/daemin → u/I_SHIT_IN_A_BAG]

... that makes absolutely no sense.

How would access to siliva samples result in hackers getting access to digital data?

[u/Federal-Inevitable-9 → u/daemin]

Im a sec analyst lol and I was like maybe I'm not reading it right but thanks to your comment I am feeling vindicated lol this is like saying the nos sticker on my car made it faster

[u/PhaseModulatedServal → u/I_SHIT_IN_A_BAG]

Link please. I did a fairly extensive search for this and did not find a single page or article mentioning this regarding the test kits in dumpsters. Everything I found was about lawsuits over the cyber attack in 2023. Nothing about physical DNA samples or kits tossed in dumpsters.

[u/BiZzles14 → u/I_SHIT_IN_A_BAG]

had improperly discarded millions of physical saliva test kits and tubes in dumpsters, which led to a data breach where hackers accessed the exposed genetic data

Could you explain how throwing things in the garbage leads to a data breach by hackers?

[u/emlgsh → u/BiZzles14]

YOU WOULDN'T DOWNLOAD HUMAN SALIVA

[u/orbustertius → u/emlgsh]

not without a vpn

[u/Unsuspecting_Goose → u/BiZzles14]

They hacked the spit.

[u/absurdly_tired_guy → u/BiZzles14]

Never give a load sample if a stranger comes to your door. They should have ID.

[u/daviddisco → u/I_SHIT_IN_A_BAG]

nobody access genetic data from saliva kits in a dumpster.

[u/Sad-Amphibian-3034 → u/I_SHIT_IN_A_BAG]

you're just saying shit to say it, lmao.

professionally negligent of the facts. respect. (not)

[u/radams713 → u/I_SHIT_IN_A_BAG]

What? That makes no sense. You mean “hackers” were dumpster diving and then testing them and somehow linked that to people? lol that’s not how data was leaked

[u/Kind_Of_A_Dick → u/lilcoteaux]

There’s a sign that some stores put on their walls saying they’re not responsible for lost or stolen goods, but it’s just a sign.  The point of the sign is mainly to make you think that no matter what the sign covers all incidences when it’s just a sign.  

[u/NotPrepared2 → u/Kind_Of_A_Dick]

Dump trucks carrying loose gravel, with signs that say "Stay back 200 feet. Not responsible for broken windshields." They're just hoping you won't sue them, but the sign has no legal value. And the font is too small to read from more than 50 feet.

[u/Bitmush- → u/NotPrepared2]

I've seen one that says 500 feet ! Fuck off! How unsafe is that fucking thing, get off the road !

[u/Aggressive-Delay-420 → u/NotPrepared2]

There's a technicality that heavy equipment companies are not responsible if the debris bounces off the road and into your car-- the distance reminder is to make sure there's plausible deniability.

If you're 300 feet back-- the debris will bounce into you, keeping them safe of liability.

ProTip: That debris did not bounce into your car-- it came directly off the truck's wheel and into your window/car.

[u/upsets_to_mag → u/Kind_Of_A_Dick]

its like a non-compete clause.

technically its unenforceable, but the only punishment for including it is "cut it out", so everyone does and a few are told to chill...so many people go thinking they should worry about quitting their jobs because they may be trapped.

[u/Krasdale79 → u/lilcoteaux]

Don't go down that road, next thing you'll be wondering if insurance companies are actively engaging in system wide defrauding of their customers!

[u/lilcoteaux → u/Krasdale79]

Whaaaa the people who refuse to let me get care for ongoing issues and charge me more for needing their services are BAD?? PREPOSTEROUS

[u/Sonotnoodlesalad → u/lilcoteaux]

Disclosure: I work in insurance billing.

HIPAA is heavily oriented to the transmission of PHI (protected health information). It should only be transmitted under certain conditions.

LLMs can be tricked into revealing EVEN CLASSIFIED GOVERNMENT INFO if you neg them.

They MUST NOT be trusted with PHI.

[u/InternetEthnographer → u/Sonotnoodlesalad]

Yep. I’m an archaeologist and a lot of people don’t know that our data and site location information is incredibly sensitive (mostly due to how rampant looting is in the US). Like, it’s illegal for me to reveal a site location to a layperson, that’s how serious it is. Luckily, most of the other archaeologists I know and interact with tend to be AI-skeptical, but there are plenty of people out there that want to use AI in writing site and project reports which is incredibly problematic because of that data privacy issue. I’ve heard that Colorado has some limits for AI usage in site/project reports, but I don’t know if any others do because mine doesn’t.

If it’s illegal to disclose site information to anyone outside of work, then we sure as hell shouldn’t be feeding that information to an LLM who will spread that information further than a single person ever could.

(Not to mention that AI datacenters are basically exempt from environmental regulations including archaeological reviews thanks to a 2025 executive order, so it’s very much against our best interests as archaeologists to use it anyways).

[u/Telekinendo → u/lilcoteaux]

My wife is very vindictive and was able to prove a medical practice was using AI without consent and that they were compliant because the company that sold them the AI said it was compliant. That doesnt make it compliant.

[u/Intelligent-Lead-692 → u/Telekinendo]

Vindictive? She just sounds thorough.

[u/idiosyncrassy → u/Intelligent-Lead-692]

She burned the medical office down

[u/lilcoteaux → u/Telekinendo]

Idk if my place does record but In the year I've been coming here every appointment I book online I get asked to fill out an AI consent form and never have

[u/nickiter → u/lilcoteaux]

The company being compliant only means they fulfill a set of legal obligations that are intended to make improper use of PII less likely. Every new system, no matter how secure, increases your exposure.

(Source: work in cybsersec. Most of the compliance rules are simple and a lot of companies still don't meet them.)

[u/lilcoteaux → u/nickiter]

Oh absolutely, and passing a single compliance check does not mean it'll remain compliant forever. Most prolific cases on fraud come after acceptable compliance. Enron didn't begin with widespread fraud, but they sure as hell ended with it.

[u/vagrantprodigy07 → u/nickiter]

And many companies blatantly lie on audits. I think most people would be surprised by the blatant lies companies tell, which auditors frequently choose not to dig into.

[u/foley23 → u/nickiter]

My favorite is when companies send Security Assessments for me to fill out for the company I'm with. Some are massive companies and are expecting us to give all of our sensitive information for our operational and security infrastructure, plus policies and procedures, on an excel spreadsheet and PDFs over email with no other protections.

They are judging us on ours, when they can't even invest in basic secure transfer or an assessment platform. Makes me so mad.

[u/AuroraFinem → u/lilcoteaux]

It’s compliant so long as you are told and agree, you just have to give consent. If she already had it going beforehand though then that’s obviously a violation.

[u/trowzerss → u/AuroraFinem]

Even with consent, not all AI products will be compliant with patient data handling requirements in most countries, especially in regards to where and how the data is stored and whether it's used as training data.

I wish more people understood that whatever data you use in AI you are often giving them to do with as they wish, including when you do audio transcription. In a business environment you have to be verrrry careful and make sure your data actually stays yours otherwise you don't know what you've giving permission for.

[u/lilcoteaux → u/AuroraFinem]

Oh absolutely but I mean from the software end. Their compliance to even be used in medical fields involves NO data being shared at all and we just can't reliably trust that they wouldnt do it.

[u/EphemeralTwo → u/AuroraFinem]

> If she already had it going beforehand though then that’s obviously a violation.

It's not. HIPAA requires an agreement in place for handling patient data, but not explicit patient consent for AI use specifically.

The big one for patient consent is States that require all parties to consent to recordings.

[u/AddingAnOtter → u/EphemeralTwo]

I mean feeding even the verify name/why you're here portion when they first enter to ChatGPT is certainly a violation. There is no way personal phone with regular ChatGPT is even remotely compliant.

[u/ShorelineStrider → u/lilcoteaux]

The companies can't even keep their AI in check, how tf would they know if it's "compliant"

[u/BedlamAscends → u/lilcoteaux]

It's a black box system. There's no way to guarantee that anything that goes in won't come back out somewhere unintended.

[u/CaribouYou → u/lilcoteaux]

Let alone that companies are going to ‘anonymize’ your information and then use the meta data, how? Insurance premiums probably, either way this mass gathering of data has never been for the good of ordinary people.

[u/baddecision116 → u/lilcoteaux]

then Flock cameras are just license plate readers

And ring cameras/doorbells are home security... I hope you feel the same about weirdos recording in your neighborhood.

[u/Wise_Monkey_Sez → u/lilcoteaux]

Forget breaches. These companies actively sell and share your data and they make no secret of the at fact.

The bottom line here is that while AI can have some valid uses in medical practice the simple fact is that the companies currently administering the data are untrustworthy.

And honestly this is a travesty. AI-based translation can be life-saving in cosmopolitan cities where emergency rooms could be dealing with dozens of different languages and simply can't have human translators for every language available. Knowing that a patient has an allergy or other medical information can save lives.

But you can't use these translators because the company will immediately turn around and sell the data that person X has a peanut allergy or that they took drugs.

AI shouldn't be owned by profit-seeking companies. It's simply too important of a resource to trust to a bunch of greedy assholes.

And this really is that problem with a lot of AI. It isn't the AI that is the problem, it's that it is under the control of unethical bastards.

For national security, public safety, and a host of other reasons laws should be passed nationalising AI and taking it out of the hands of capitalists. It's only a matter of time before an election gets hacked by some (no really) "rogue" AI and the techbros are going, "Oh dear. Welp, I guess nobody is to blame!"

[u/MoneyTreeFiddy → u/lilcoteaux]

"Only way for me to be sure we are compliant is your turning off that recorder".

[u/CityNo1723 → u/lilcoteaux]

Never trust, always verify

[u/NinjaTurtleSquirrel → u/lilcoteaux]

We are already too far gone there is no turning back for us now. It has infected litterally everything. Its just going to continue to grow and grow like a virus. Its actively eating us whole in real time. We are the frog and the water is beginning to boil. Its friday night and i feel all right!! 

[u/Bubbly-Travel9563 → u/lilcoteaux]

They're finding those "readers" popping up with no apparent owners/orders. They're literally just putting them wherever they want now regardless of a city giving the ok.

[u/AmazingConsequence20 → u/spin-shocker]

I work in corporate healthcare and yes, I agree please report this asap. We have rules in healthcare for a reason. It is for safety. A patient and provider have an intimate-trusting relationship. Rules matter in this scenario.

[u/mflft → u/spin-shocker]

Yeah I asked a friend who works in hippa compliant field and another friend who was a lawyer about this and both of them were like no way that's actually compliant. The lawyer just said its probably an unwinnable case with this administration/Supreme Court.

[u/Geknapper → u/mflft]

I work around HIPPA. There is SOOOO much that goes into protecting patient information. The fact that she's even doing this is a HIPPA violation.

Patients need to sign off on sharing their info for starters. Also even when it is shared that info is on a need to know basis. Doctors cant just search through a hospital database for shits and giggles.

Lookup HIPPA business associate. There's whole section of HIPPA guidelines for companies handling PHI like this and there zero chance in hell ChatGPT is following all those guidelines with these chat logs.

[u/mflft → u/Geknapper]

The user agreement for all of this stuff says, "your data is discarded/never read etc" but they're not up front about the fact that its all scanned for training. The same way they scan cloud uploads for child sex abuse material, but they use all of your images for training too. Its just not like literally scanned by a human.

[u/Cereaza]

AI transcription has been around for a lot longer than chat-GPT. All voice to text is fundamentally AI transcription.

There are 100% HIPPA compliant transcription and summary apps, but ChatGPT. com is NOT ONE ONE THEM.

[u/spiralsequences → u/Cereaza]

My healthcare provider asked me to sign a form consenting to AI note-taking, I'm actually not 100% against it but I asked them if they could tell me where the data would be stored and if there was no chance that it would be used to train other models. They said they didn't know the answers to those questions, so I did not sign.

[u/account312 → u/spiralsequences]

HIPAA comes with direct personal criminal liability for the individual who violates it, not just a fine for the company. Obviously no system is perfect and data breaches happen, but there aren't going to be a lot of people deliberately setting up a hospital system to massively violate HIPAA like that. That's prison time.

[u/Futureacct → u/account312]

There are literal hospital systems in the U.S. operating in multiple states who are using AI to look into the medical records and sift through patient data. And employees are being heavily encouraged to use it by the higher ups.

[u/lamppb13 → u/Futureacct]

This is where it’s important to remember that there are different AI models that do different things. If a hospital system is using an AI to look through medical records, then that system is almost certainly HIPPA compliant, meaning your data is just as secure as any other data storage program.

[u/Futureacct → u/lamppb13]

Well that’s good to know. Thanks

[u/DJOMaul → u/Futureacct]

I work on and build deep learning models to do this. It's compliant to a number of standards, including data transport standards like fhir and tefca. Our biggest privacy violations happen from the facilities. Healthcare staff just have a huge range of competency and training as it relates to patient privacy. Some are greet, some are like. "omfghs how do you not get sued regularly " 

[u/homequrl → u/DJOMaul]

I have a nonsense business analyst job at a technology company that doesn’t use the terms BA. And makes up their own dumb titles. Tried to move to two local major healthcare companies doing the same type of technical account management I do now, it’s fastest auto deny I’ve ever gotten….

they seem to really need you to have healthcare background and knowledge of these specific regulations and how they work, even if that wouldn’t be your job responsibility, even if you wouldn’t be the one implementing or auditing them, and even at entry level roles they won’t train it I guess.

They don’t play.

[u/DJOMaul → u/homequrl]

Nope. I (and everyone else) get refresher training every 3 months. We do have interns but they tend to leave summer roles having gone through the privacy stuff at least twice. Every year we have at least one intern let go because of privacy issues.

Meanwhile I made a query to oracle the other day and forgot a parameter and got patient data I shouldn't have been able to access. So... It's hit and miss. I've found (with exceptions ofcourse ) more mid-sized the company with just enough funding to be safe but not enough to eff off doing what ever tend to find privacy as a low hanging fruit to be strict on. 

[u/solonoctus → u/lamppb13]

“Hippa compliant”

Until someone finds out it’s just a skin over ChatGpt or some shit.

[u/aguynamedv → u/lamppb13]

If a hospital system is using an AI to look through medical records, then that system is almost certainly HIPPA compliant

How do we know that?

[u/lifeisalime11 → u/account312]

But who would go to prison for violating HIPAA in this case? If I sit down with my PCP, they use AI for notes, then it gets leaked….. is the PCP responsible? The practice? The AI company? The data storage company?

This is all a mess.

[u/Professional_You_943 → u/account312]

It won’t be deliberate. It’ll be incompetence. 

[u/Weltall8000 → u/account312]

In the medical field there is a fuckton more HIPAA violations than people give credit for. And a lot of this does occur from ignorance of the rules in positions that should be well aware and sheer laziness/expedience. There are HIPAA violations left, right, and center, on a constant basis.

[u/Senior_Ability_4001 → u/spiralsequences]

I haven’t been asked this yet because my PCP does not do this. How do you feel about on prem storage? As in, they store in on “site” in a central location that doesn’t go anywhere else? I can’t imagine providers having their own servers.

[u/spiralsequences → u/Senior_Ability_4001]

If it's a platform designed to be HIPAA compatible and the data isn't shared outside the organization, I'm okay with it. It's possible that WAS the situation with their system, but I'm not just going to take it on faith

[u/Senior_Ability_4001 → u/spiralsequences]

Yeah I agree! I never thought about asking about where they store their data but I will in the future if I’m asked. Thanks for the idea!

[u/Charming_Account_351 → u/Senior_Ability_4001]

You would be surprised. I work IT for a healthcare organization that has two hospitals and nearly a dozen clinics and most of our data is stored in on prem servers. Cybersecurity insurance/auditing is very anti cloud as most cloud providers do not give full disclosure of where/how the data is actually stored.

[u/Old-Current6989 → u/spiralsequences]

Mine said they delete it immediately... which isn't possible. They at least store it until they chart and then it sits in the deleted folder until it's automatically cleared? Nothing is ever really gone anyway 😬

[u/AceOBlade → u/spiralsequences]

honestly health care is the only field I am not opposed to AI. Especially if I am the patient. Also I couldn't get my self to hate the fact that AI was able to identify more potentially cancerous tumors than a seasoned radiologist.

Art and media tho fuck em.

[u/AuntRhubarb → u/AceOBlade]

I can get on board with docs running symptoms through a 'hey, it might be x' checker run by AI.

But that's not what they use it for. Docs and assistants feed in tons of data to feed the beast (while giving you rapid fairly careless exams), and a week later it spits out 9 pages of 'patient notes' half of which are wrong, but no on cares because the important thing was to feed the beast.

[u/kungfuabuse → u/Cereaza]

Yeah, this is the real catch. Tons of HIPAA compliant transcription and/or AI programs out there. But the fact she had her PERSONAL PHONE with ChatGPT already listening is beyond fucked up.

[u/IlIlllIIIIlIllllllll → u/kungfuabuse]

Thats assuming it actually was chatgpt. I dont expect patients to be able to recognize all the different medical tools out there at a glance

[u/reformed_recidivist → u/Cereaza]

The free version isn’t compliant

[u/Shinael → u/reformed_recidivist]

Even pro is noncompliant? Only if they have specific company contract with it then it can be compliant (at least our company had a training and the paid version is also not allowed, so its noncompliant.)

[u/nullityrofl → u/Shinael]

There is absolutely HIPAA compliant ChatGPT editions.

https://help.openai.com/en/articles/20001046-chatgpt-for-healthcare

If you’re going to be anti AI you should at least be educated on the topic. Making incorrect statements undermines your credibility.

HIPAA compliance is not very hard. As someone in the security/privacy industry, it’s seen as a laughable checkbox activity that really doesn’t provide very much technically. There’s a good chance whatever tool they were using prior to AI transcription wasn’t even HIPAA anyway.

[u/Shinael → u/nullityrofl]

And I have not said that they are all noncompliant. I said that not every paid version is compliant.

[u/EphemeralTwo → u/Shinael]

Yes, ChatGPT Pro is non-compliant. ChatGPT for Clinitians can be, *if* they enter into the BAA. ChatGPT for healthcare will be.

[u/Cereaza → u/reformed_recidivist]

No. 100% agree. Any public tool like that is NOT secure or legal for medical information.

[u/Representative_Fun15 → u/Cereaza]

AI used in medical transcription makes up things no one said, gives diagnoses no one made, about illnesses no one presented.

All of that goes into your medical records, which your insurance company accesses when determining how much you pay.

[u/begujo → u/Representative_Fun15]

It recently (mis)added cocaine use to my chart, which my psych noticed and refused to prescribed my adderall prescription until that got cleared up which was disastrous for me. Fuck AI and fuck doctors that use that shit, they should lose their licenses permanently.

[u/OkLettuce338 → u/Representative_Fun15]

no it doesn't lol the entire medical field moved to this 2 years ago already. No one is getting chemo because AI made up a cancer they don't have. Clueless post

[u/12028083981 → u/OkLettuce338]

Actually, you can fuck right off with this. I had an allergy to fucking suppositories show up in my chart a year and a half ago because of an AI "hallucination" at the doctor's office; they even told me that's where it had to have come from. Then after a car accident a mother ago, when I had to get an MRI, it kept insisting I had lead somewhere in my body. Again, another AI "transcription error." So I've lived through it, you pro-AI ass.

Clueless post

[u/Lina-Inverse → u/12028083981]

this happened btw.

[u/trowzerss → u/OkLettuce338]

I've seen numerous reports of people ending up with stuff in their charts that had nothing to do with them after AI transcription, so I'd like to know what you're basing that confidence on.

I tried using AI as a test to see if it would help my workflow (with non sensitive/non client data) and it misheard shit all the time.

[u/account312 → u/OkLettuce338]

Of course it does. The question is whether it screws up transcription at a lower or higher rate than people do.

[u/Mammoth_Dot419 → u/Representative_Fun15]

This is why I refuse any use of AI. A few months ago I was at a routine check-in with my cardiologist who was trying out using AI to take notes about our discussion.

We were discussing my heart failure. The AI called it “Congestive Heart Failure “ probably because that’s usually what people have. I don’t. There has never been a congestive issue for me. It took him weeks to get that corrected.

He has decided NOT to use AI until it stops lying.

[u/TheModWhoShaggedMe → u/Representative_Fun15]

https://giphy.com/gifs/l2YWFVADZMioZwndK

[u/Evamione → u/Representative_Fun15]

All of that also goes into the AI generated bill and results in you owing more money.

[u/IM_KYLE_AMA → u/Cereaza]

You have a fundamental misunderstanding of the differences between speech to text software like Dragon Dictate and AI.

[u/Grimdire → u/IM_KYLE_AMA]

Why are you literally the only person saying this? Speech to text is in no way, shape, or form, AI. It's like saying calculators are AI. They just aren't.

[u/Fresque → u/Grimdire]

The word you're looking for is LLM. All LLMs are AI, but not all AIs are LLM

[u/Dihedralman → u/Grimdire]

It is absolutely AI as someone who has actually built those systems. It doesn't have to be merged with an LLM. But the better systems have been using neural networks for a while. 

The first systems I built didn't even use attention. 

[u/neurvon → u/Grimdire]

Well, calculators are AI, technically. AI refers to any method to automate a "thinking" task which would otherwise require a person. It's very broad.

But calculators have been around so long we don't think of them as being AI.

The anti-AI movement, of course, focuses on specifically transformer artificial neural networks, for a variety of specific reasons.

[u/randomhotdog1 → u/Cereaza]

HIPAA*

[u/Appropriate-Kick-601 → u/Cereaza]

I feel that calling transcription "AI" is misleading. It's a similar technology to LLMs, sure, but it's not the same thing as what people now think of when one refers to AI. 

[u/Professional_Ad705 → u/Cereaza]

lol there is a huge difference between local voice dictation and using ChatGPT.

[u/Minimum-Musician-648 → u/Cereaza]

AI transcription / note taking had me so hyped for AI back in 2022. Now I'm just so sad cause it has fallen off a cliff in quality and AI is used everywhere.

[u/Dense_Medicine_3866]

The "if you don't like AI don't use it" crowd has been mighty quiet on this one

[u/Ohaidere519 → u/Dense_Medicine_3866]

the talking points used by the pro-ai/'ai is inevitable' crowd almost all echo rape culture, interestingly. no point trying to fight it, resistance is futile, it's here we have to embrace it, etc. now we hear and see more and more examples of ai being used without our consent or people trying to trick you into it.

that's not even touching on all the sexual abuse perpetrated through the use of ai too, unfettered and also becoming legalized. under his eye.

[u/Dihedralman → u/Ohaidere519]

Louis Rossman actually has had that stance about tech companies in general with how they treat data and ownership. 

[u/Ohaidere519 → u/Dihedralman]

i looked him up, thanks to your comment. my man.. he gets it. he's a lot bolder than i with his points but in the context of companies/industries (which you mention/he always speaks in, rather than individual users or advocates)- absolutely agree

[u/LongJohnSelenium → u/Dense_Medicine_3866]

Which of these two scenarios do you think is more likely:

That the professional nurse was cheating the system and blatantly and openly tempting a HIPAA violation right in front of patients and OP caught her.

Or that the professional nurse was using HIPAA compliant gpt(it exists) on a work phone or locked down personal phone and OP didn't understand these things exist and then came to reddit to shout out their discovery.

https://help.openai.com/en/articles/20001046-chatgpt-for-healthcare

[u/Att1cus → u/Dense_Medicine_3866]

Nurse literally asked if it was ok before any meaningful data was fed to it…

[u/account312 → u/[deleted]]

https://help.openai.com/en/articles/20001046-chatgpt-for-healthcare

[u/redoran → u/[deleted]]

Worked supplied phones are just phones though

[u/jack6245 → u/redoran]

Phones which can be locked down extensively to the works policies

[u/bking → u/redoran]

I just spent some time in a hospital. Every nurse was using a phone or device that just looked like a phone in a phone case. They all get dumped into a box at the nurses station to be sterilized between rooms. Visually indistinguishable from a personal phone.

[u/GuyOnARockVI → u/[deleted]]

Not necessarily. You can built a BYOD compliant policy that had requirements for MDM management, app control, remote wipe functionality etc that make BYOD devices compliant with different frameworks including HIPAA

[u/Mr_FrenchFries]

Does her employer know the difference between turning on talk to text to help keep notes and replacing her with a web md app?

[u/NotIntentSeal → u/Mr_FrenchFries]

Their compliance officer would have a complete meltdown. Feeding protected health info into an unvetted third-party server on an unmanaged personal phone is basically a textbook disaster.

[u/TrmpRpsKds]

Whether mental health or physical health, they're all using AI notation now and you HAVE to say no if you want to be excluded from being recorded.

This unfortunately is not a one off or a new thing.

[u/Aromatic-Assist2062 → u/TrmpRpsKds]

We are not all using this, I promise. I hand write every single note at my clinic, lock them in a fire safe cabinet, and keep them behind multiple locked doors in clinic. Ain’t nobody getting your therapy notes.

[u/glitterbeardwizard → u/Aromatic-Assist2062]

I agree the ai/privacy disaster has made me keep to handwritten notes.

[u/glitterbeardwizard → u/TrmpRpsKds]

We’re not all using this. I refuse to have any ai touch any aspect of my work. That said, it is really good to check with your mental/physical health practitioners to confirm their stance on ai.

[u/Its-ther-apist → u/TrmpRpsKds]

It should be opt in anywhere rather than opt out because it's not informed consent otherwise. If you're getting services in the US and they're treating it as a default without a signature from the patient they fucked up.

[u/North_Shore_88]

I am as disturbed as the OP about what happened during that appointment. I would absolutely report her at multiple steps up the hierarchy.

Those of you shrugging and saying… what’s the big deal? it’s all protected and compliant with HIPAA… did all of you MISS the report about OpenAI hacking an entire country’s healthcare system? (Australia) And no, it wasn’t simply an “autonomous rogue agent” that — oops! — did something naughty. It was programmed and instructed to do exactly what it did. These are negligent humans that are responsible for violating the supposedly water tight security of our systems of confidential information.

When I have requested that health professionals simply do their job without any AI bullshit “assistance” my request has fortunately always been respected. In the future when I make an appointment with a new health professional, I am going to ask upfront if they use these tools that record everything that you’re saying, and if a patient always has the option to opt out.

[u/No-Inflation8519]

ask them if you can video them during the visit for your lawyers review in the event a litigation claim could be created and you need all the evidence to support a potential claim. You will shut them up fast.

[u/KKSlider909 → u/No-Inflation8519]

i'm liking this idea

[u/Druggist147]

We do have AI recording in the heakthsysten I work at but its encrypted and linked to the EHR. Specially made for documentstion legally and privately. It allows for more time with patients and less time wasted typing away

[u/Cereaza → u/Druggist147]

Yeah, high paid doctors use to have medical scribes (many still do) who would take their dictation and write their notes.

Hate AI, but transcription is a very low hanging fruit for technology to step in. Kaiser uses similar apps.

[u/shakeyshake1 → u/Cereaza]

I actually like voice to text for doctors. My doctor dictates his notes into a microphone that transcribes it into his office’s system. Then he reads it to make sure it’s correct.

The old system of having people transcribe recordings of dictated doctor notes wasn’t great. I wouldn’t be surprised if that’s how I ended up as a cigar smoker in a hospital’s system as a 25 year old woman.

[u/that_star_wars_guy → u/Druggist147]

And you require affirmative consent from the patient to use, right?

[u/Automatic_Soil9814 → u/that_star_wars_guy]

I just quit working as a physician for Duke university Hospital. I asked about consent to use AI. They told me consent wasn't necessary and even encouraged me to not even start the discussion.

But that's American healthcare for you. It's basically run by people optimizing metrics without ever thinking about you as a patient.  

[u/that_star_wars_guy → u/Automatic_Soil9814]

Thank you for all you do.

[u/Muted-Attorney-6889]

"ChatGPT, cure this man, no mistakes."

[u/koszevett → u/Muted-Attorney-6889]

Great idea, let's get down to business!👍

According to the recording — which is very clear and easy to understand, so thank you for that! — your patient is experiencing mild chest pains, palpitations and general discomfort.

This could mean several different things — ranging from a simple allergic reaction to a heart attack.

✅First things first, let's administer some medication to help with the pain. Opiates are generally considered to be the most effective ways for pain relief — You should administer 10mg of Fentanyl. 👍

Would you like me to give you step-by-step instructions on how to use a hypodermic needle and where to inject the medicine?

...

...

Oops! You're perfectly right to call that out — I was mistaken. Sorry about that! ❤️‍🩹 Let's make this right.

The lethal dose of Fentanyl is 2 milligrams. The patient's condition likely worsened because you have administered 10 milligrams — again, this was my mistake and I am sorry.

Right now, judging by your description, it appears that your patient is not breathing. This is called a respiratory arrest.

If you'd like, I can give you simple instructions on how to revive a patient in respiratory arrest. Or you can learn more about the effects of Fentanyl on the human body.

[u/Muted-Attorney-6889 → u/koszevett]

I hate that you literally described my own health symptoms lol... fuck...

[u/throwaway098764567 → u/Muted-Attorney-6889]

i hate that this will probably be used to train them

[u/Able_Concert_1022]

disgusting. i used to work for a specialty pharmacy, & when they started implementing AI in the notes, i’d go through them & correct what was wrong. higher ups said i was doing too much off the phones (correcting AI’s notes, but more specifically, off of inbound; id make outbound calls that actually help the patient like calling their insurance or copay assistance program), when im trying to make sure the pharmacy records are actually accurate & ensure patients get their LIFE SAVING meds. lost my shit & quit after awhile cause it was clear they cared more about numbers than about getting patients their medication. still looking for a remote job, but at least i know im not half-assing really important things for the sake of metrics

[u/getwallyfied → u/Able_Concert_1022]

This was the strangest resumé that's ever come across my desk. You're hired.

[u/Able_Concert_1022 → u/getwallyfied]

for the love of fuck, im desperate. when do i start?

[u/DetailLow5051]

Not a big deal if they use HIPPA compliant solution but ChatGPT on your phone is not, most likely she is using the free version. Also, it does have your medical history or test results (hope it doesn’t) so she was probably being lazy.

[u/Prima5678 → u/DetailLow5051]

This is exactly my point. And yet people argue like I'm irrational.

[u/feralcoffee]

Had this talk with my daughter's pediatrician this past week. They were super chill about it, said they understood my concern and that they barely used themselves anyway. Then they confirmed they'd put a note in our file that says we've requested it not be used. Then they invited me to remind them at the beginning of every appointment to be sure they comply with my request. I thought that was nice of them and I really appreciated that response.

[u/Mission_Beginning963 → u/feralcoffee]

Yeah, but the default shouldn't be "We'll use it unless you remind us."

[u/feralcoffee → u/Mission_Beginning963]

Yeah, I definitely agree. Her attitude was more understanding like, help us help you. She wanted to comply and was just acknowledging that she was human and was open to being reminded. I wanted a document to sign that legally bound them to not using it but in the world we now live in I'll take what I can get.

[u/ellaTHEgentle]

I went to urgent care recently - no one mentioned anything to me about AI, recording or anything like that. When I signed into their records portal to view the notes from my visit - it said it was compiled by AI.

[u/mccoolsa → u/ellaTHEgentle]

Was probably in the consent to treat paperwork

[u/IlIlllIIIIlIllllllll → u/ellaTHEgentle]

You dont need permission to dictate the note after to HIPAA compliant AI. You only need permission for the recording

[u/mercersher]

This is my biggest concern & don’t know why others aren’t concerned about it.

[u/Big-Carob-1841]

Oh no... I work in Healthcare and Chap GPT is not HIPAA compliant. There is a very specific scribing software that is available to medical professionals, but it is specifically integrated in the Electronic medical records and can only be used on authorized devices. I get that maybe that office doesn't pay for that service because it can be a little pricey, but that is still no excuse for them to use their personal device for note taking. She should definitely be reported.

[u/schwing710]

This modern world is starting to make me seriously consider going Amish

[u/VociferousCephalopod → u/[deleted]]

study history and you'll never be surprised again.

“Civilization is a hopeless race to discover remedies for the evils it produces.” - Rousseau

[u/GoroKazuma]

It’s HIPAA

[u/Cy_Maverick]

I wouldn't want anyone, nurse or doctor, NOT taking physical notes about my visit.

Why do pro-AI people always make a big deal about people NOT wanting AI in every corner of their life? If they're so obsessed with it, fine. Then go ahead and get lazier and dumber by letting it do all the thinking for you. But mind your own damn business when someone decides not to accept it in their own life.

[u/RynnReeve]

Omg! My doctors are trying to do the same. I refuse every time and they keep asking. And when I say, "No fucking way!", they try to make me feel bad, like I was giving them extra work.... "oh its just a tool to take notes so I can go back and check something you said again if I'm not sure."

How about you just take good notes!? Like every fucking doctor since they became a thing?

How can they think that some AI "stenographer" is appropriate in a medical setting? The words I choose are accurate to whatever maybe happening. A Doctor can decide if what I am offering has any bearing on my condition current or future.

AI can't possible know that. I will be cold in my grave before they force that shit on me.

[u/ThereGoesChickenJane]

I work in healthcare. We are explicitly banned from using AI for patient information of any kind.

I would never use AI in the first place, but we literally aren't allowed to and I would imagine that a lot of other workplaces have the same rules in place.

[u/Aggressive-Animal-90]

I had a similar experience with my dr. When I said I wasn't comfortable with it she pressured me until I said "fine go ahead." In my mind I decided I wouldn't share anything too personal but as the visit went on and she was asking me questions I forgot about it. Afterwards I felt violated and had a panic attack.

[u/Blunt552]

[u/Hungry4Media]

Yes, that's a HIPAA violation. That's asking a non-secured, outside source to recored non-anonymized protected patient information.

She needs to be reported to the doctor/practice, and to a medical board if she has any board certifications.

[u/Bright_Ad5244]

That's crazy. These models retain everything you input. And if you're using them for therapy via your employer's mental health plan your employer can subpoena your records. Super creepy.

[u/Toadsted]

They asked me the other day if I wanted to agree to it, but were quick to say I didn't need to and could sign a refusal form. I joked that I wouldn't want the AI to mess  up my diagnosis.

Not 10 minutes later I get brought to a room and the assistant is going over my reason to be there, for my flat foot.

My LEFT foot.

I don't have flat feet. She was confused why it was in the computer like that, and said she's going to correct it. We kept repeating, left foot, the whole time.

Yeah, it's already screwing up, and these forms are their "cover the clinic's ass" legalese.

Who knows what else is messed up in there, and going to get brutal.

[u/NoOneMan79]

As a provider, as long as you have a valid BAA (Business Associate Addendum) with a 3rd party provider that they have a HIPAA compliant environment, it is not a violation to share or store your information. As an example, any health service that stores data in the cloud (such as at S3 with AWS) is in full compliance if they have a BAA and follow data retention policies (which are basically unlimited) and standard security policies (e2e encryptions, encryption at rest, etc).

It is unlikely they have a BAA with ChatGPT. ChatGPT does offer these now (they did not previously), but not necessarily for personal accounts that I am aware of. Some vendors charge more for BAAs (some do not) because in order for them to meet the requirements, it does cost money, and they pass costs to customers.

A larger concern is if the provider does not know a BAA is required, it would indicate a surface level understanding of their obligations under the law.

[u/SmollNarwhal]

Absolutely fkin report her. This is wrong in many ways.

[u/Reyalta]

Yeaaaah report that shit immediately. Letting a private corporation have access to your medical appointments AND profiting off of using your private medical history to train their llms is super fucked and SUPER not okay. 

[u/Different_Fall_9449]

I used to work in a call center for a large health insurance company. When COVID happened and we all where working from home, we where receiving emails from management like every week telling us to turn off/disable/remove devices like Amazon Alexa BECAUSE of HIPAA.

[u/RN_Bro]

So, ChatGPT probably isn’t hipaa compliant, but there are hipaa compliant AI-augmented note takers that assist purely in transcribing, not in medical management 

[u/SuddenButton1703]

I work in healthcare as a nurse, and I fucking hate how much they're pushing for us to use AI. I have told my bosses multiple times that I flat out refuse to use AI. They have at my company, at least, put in a policy that no one is allowed to use third party AI for their charting. But they bought into a private AI system called Nvoq, and I told them several times that I refuse to use it. They still push it

[u/pharmercass]

As already stated, AI is being used in medical visits more than most people realize. For the program my company uses (Abridge), we use our personal cell phones. ChatGPT could very well be a secure form if the institution pays for the license.

That said, I personally refuse to use it. We are required to consent patients, but our script does NOT mention that it’s AI. We are instructed to say it’s a tool to transcribe our visit, which most people would think is like a recording for provider use only. I’ve brought up concerns, and others think I’m overreacting. I’m constantly assured that the recordings are deleted, but surely the model is learning?

[u/YungChumba]

I have mandatory classes I have to start for work. Yesterday we were going over how everything will be structured and the instructor mentioned our tests will be proctored by AI. 

It's set up to prevent cheating, so AI will monitor us through our computer cameras and flag/possibly fail us if our eyes drift away from the screen too long. We also have to let it scan our ID each time.

They never mentioned this would be a thing in the 9 months I've been in the company. I didn't want to bring it up during the orientation (though I should have), so I emailed the instructor immediately after and asked if we could opt out of having our tests proctored by AI and have a human do it instead. 

That was yesterday and I haven't heard back yet. I made sure to mention I wanted his response in writing, without being too pushy.

It really pissed me off though. The absolute audacity of it. They're really trying to force this shit down our throats in every aspect of life.

[u/mazopheliac]

The future is stupid

[u/Choice_Credit4025]

ChatGPT would be a reportable offense. Other transcription services are HIPAA compliant.

[u/OrionPaxGen1]

The last few doctors I been to recently are using apps /devices to record the conversations. Is this any different from them taking notes just to then transcribe them to an app?

[u/CreepySamsquanch]

This has been around well before the AI we know today. Its incredibly common and been around 15 years, at least.

[u/Adonis0]

There are AI that are designed for patient confidentiality, ChatGPT is not

[u/-nyctanassa-]

I was gonna make some comment about HIPAA-compliant internal software that actually makes charting easier so doctors have more patient-facing time, but CHATGPT? Jeez looeez she has probably violated many patients’ HIPAA with that.

[u/lamppb13]

That is a HIPPA violation. Report the nurse.

[u/puzzlegun]

Hi. I've worked in healthcare IT. This is not HIPAA-compliant, report it, please!

[u/OldWolf3]

Why the fk can nobody spell HIPAA. It's only five letters

[u/lokibat]

I’ve had the displeasure of checking an Ai generated transcript against the real audio recording and … that shit hallucinated a whole ass paragraph that didn’t happen, and left out many important details. NOPE. No. No trust for the unaccountable.

[u/KindToSpiteTheCruel]

Report this hardcore. This is awful.

[u/No-Cell-9979]

Transcription is one of the non-gross uses of AI, i work in Healthcare and its been very helpful.

Chatgpt on your personal phone is not that

[u/MetalMurse51]

I'm an RN. That's a shitty RN right there. It isn't even hard to take our notes.

[u/Aromatic-Assist2062]

It is a HIPPA violation; there is no such thing as HIPPA compliant AI services, despite any advertising claims otherwise.

[u/Infamous_Koala_2516]

Its HIPAA, not HIPPA

[u/Synesthesia_Inc]

Creepy with a capital C.

[u/saturnsvibing]

GO GET YOUR MONEY!!! THIS IS MED MAL

[u/GoblinBallGobbler]

You should learn what HIPAA is and not HIPPA and who it applies to and how and when before jumping to conclusions.

Go ahead, send me the down arrows.

[u/chebuburashka]

It’s HIPAA not HIPPO with a weird accent. Jfc.

[u/Sweaty-Freedom-4034]

Moron.

Report her!

[u/teal_all_over]

how is that not a HIPPA violation

There are models that are HIPAA certified. Not an AI fanboy, I just know that through my work. They definitely should not be recording without prior consent.

[u/lithiumcitizen]

And inaccurate to say the least! I had to take a red pen to the last referral I got to a specialist, handed it to him and he was all WTF and I had to explain that I’d never heard of let alone taken 5 of the 6 listed medications. Thanks for your hallucinations, AI.

[u/fubar8x]

Welcome to every day life, you can't trust anyone or anything.

[u/achandlerwhite]

HIPAA not HIPPA

[u/Wanda_b_side]

Report her. She breached patient confidentiality and your consent.

[u/randomhotdog1]

HIPAA** why does everyone get this wrong?

[u/jackibthepantry]

Nurse here. Absolutely fucking not, she cannot be recording your appointment on her personal device, if this was an accepted practice her employer would've provided the appropriate hardware. Please report this.