What shouldn't you tell ChatGPT?

The Silent Data Leak: Why Your ChatGPT Chats Are Riskier Than You Think

The most dangerous data leak in your company probably isn't a hacker — it's a well-meaning colleague pasting a spreadsheet into ChatGPT at 4:45 PM on a Friday.

The scale of casual disclosure is staggering. According to recent enterprise research, 77% of employees paste data into generative AI prompts, and 82% of those pastes come from personal accounts that sit entirely outside company oversight. On a typical day, employees paste roughly 14 times into non-corporate accounts — at least three of them containing sensitive material.

The trend line is worse. Sensitive corporate data entering AI tools has climbed from 10.7% of all data movements two years ago to 27.4% last year, and projections put it at 34.8% heading toward 39.7% by 2026. ChatGPT alone accounts for 92% of enterprise GenAI usage, making it the single largest corporate AI exposure channel in existence.

Here's what makes this a security problem rather than a hygiene problem: leakage happens through normal workflows, not attacks. As Cyberhaven's analysis of AI data leakage notes, the most common vector is simply employees pasting work documents into a chat box. That's why sensitive information disclosure now ranks #2 on the OWASP Top 10 for LLM Applications, up from #6 in 2023 — the industry's own security experts now treat everyday chatting as a top-tier vulnerability class.

And the data being pasted isn't trivial:

• Customer records, PII, and payment card data — 40% of files uploaded to GenAI contain PII or PCI
• Source code and trade secrets — the exact categories behind Samsung's infamous semiconductor leak
• Financial projections, legal documents, and strategic plans
• Medical records and HR decision inputs, both flagged heavily in DLP violation data

The consequences are not theoretical. Samsung engineers leaked confidential source code while debugging; a Redis bug briefly exposed other users' conversation titles; and a 2024 infostealer campaign put over 225,000 OpenAI credentials on dark web markets, handing criminals full chat histories. Anyone whose chat log contains a customer list or an unreleased product spec just handed over a roadmap.

There's a lesson here for anyone building AI workflows, too: the same casual-trust dynamic applies to the agent skills and connectors you attach to ChatGPT and other models. SkillzBench — the free, independent directory and MCP tool SkillzBench is building — exists precisely because published research found roughly 26% of agent skills carry at least one vulnerability. The prompt box and the tool behind it both deserve scrutiny.

The fix isn't a ban — it's knowing exactly what never goes in the box.

What ChatGPT Actually Does With Your Data: Retention, Training, and the ZDR Gap

Most people who click "opt out of model training" assume their chats vanish into thin air. They don't — and the gap between what users believe and what actually happens is where serious data exposure lives.

Opting out of training is not the same as opting out of storage. According to Teleskope's analysis, OpenAI retains inputs and outputs for a default 30-day window — even after your conversation disappears from the interface. Excluding data from training is a separate setting entirely, and conflating the two leaves what security experts call a real risk gap.

Zero Data Retention (ZDR) sounds like the fix, but it's out of reach for almost everyone. As Teleskope's Amy Ryu explains, ZDR isn't a setting you switch on — it's approval-gated, limited to eligible API endpoints, and available only on the enterprise API tier. Conversations typed into the ChatGPT browser interface are simply not covered.

There's a human element, too. In non-ZDR deployments, OpenAI personnel may review customer content, and the company notes that serious safety risks "often only become clear when multiple interactions are viewed together" — implying cross-interaction analysis of what you type. Even under ZDR, exceptions exist for flagged content and abuse-monitoring metadata, per OpenAI's own policy statements.

The stakes are not theoretical. Consider what has already happened:

• Samsung engineers pasted confidential semiconductor source code into ChatGPT while debugging — and 65% of Samsung employees later expressed serious GenAI security concerns, per incident reporting.
• A bug in an open-source Redis library briefly let ChatGPT users see titles and first messages of other people's conversations.
• A 2024 breach exposed over 225,000 OpenAI credentials on dark web markets via infostealer malware — giving criminals access to full chat histories.

That last point deserves emphasis: your credentials are keys to your chat log. Anyone who obtains them can read everything you've ever pasted into a conversation. A retention policy only governs what happens after data reaches the model; it does nothing to stop sensitive information from being sent there in the first place.

The practical takeaway is straightforward. Treat anything you type into a consumer ChatGPT chat as potentially stored for at least 30 days, possibly reviewed by humans, and readable by anyone who compromises your account. If you're building AI workflows that handle sensitive inputs, the same discipline applies to the tools you connect — SkillzBench is building a free directory and MCP connector that security-scans and benchmarks agent skills, so the components you attach to your agent don't become the weak link.

The Never-Share List: 7 Categories of Data to Keep Out of ChatGPT

Every time you paste something into ChatGPT, you're handing it to a third-party system that stores it for at least 30 days by default — and 77% of employees are doing exactly that, often from personal accounts their employers can't see. The scale is staggering: Zscaler ThreatLabz research has flagged 410 million data-loss-prevention violations for ChatGPT alone, including Social Security numbers, source code, and medical records.

So what should never make it into that chat box? Here is the practical list.

1. Customer data, PII, and payment card information. Roughly 40% of files uploaded to generative AI tools contain PII or PCI data, making this the single most common leak vector. A "quick cleanup" of a customer list is a reportable data breach waiting to happen.

2. Source code and trade secrets. Samsung engineers famously pasted confidential semiconductor code into ChatGPT for debugging help — and 65% of Samsung employees later reported serious concerns about GenAI security. Code pasted into a consumer chat is code you've shared with a vendor you didn't negotiate with.

3. Financial projections and strategic documents. Unreleased forecasts, M&A plans, and board decks are the crown jewels DLP systems look for. Once in a chat, they sit on OpenAI's servers for at least 30 days and may be reviewed by OpenAI personnel, since Zero Data Retention only covers eligible enterprise API customers — never the browser interface most people use.

4. Medical records. These appear repeatedly in flagged DLP violations and carry steep regulatory exposure under HIPAA and GDPR, where cumulative fines have already exceeded EUR 4 billion.

5. Credentials and API keys. A 2024 infostealer campaign put more than 225,000 OpenAI credentials on dark web markets — and anyone holding your account credentials can read your entire chat history. Treat every secret in a chat as a public secret.

6. Inputs for critical HR, legal, or finance decisions. Using consumer ChatGPT for firing decisions, contract analysis, or audit work creates both exposure and liability, with EU AI Act penalties reaching 7% of worldwide turnover.

The common thread is simple: retention policies govern what happens after data reaches the model — they do nothing to stop it from getting there. As Teleskope's Amy Ryu puts it, an engineer who copies a production table into a prompt has created an exposure event regardless of how long it's stored. Anonymize or fictionalize before you share, and if you're building AI workflows that handle sensitive steps, agent skills can automate the sanitization — SkillzBench is building a free directory and MCP connector for exactly that kind of vetted tooling.

How to Keep Using ChatGPT Safely: Anonymize, Fictionalize, Govern

The good news is you don't have to stop using ChatGPT — you just have to change how you use it. With 77% of employees pasting data into generative AI prompts and 82% of those pastes coming from personal accounts outside company control, the risk lies in habits, not the tool itself.

The first habit to build is anonymizing or fictionalizing before you share. Instead of pasting a real customer record, swap in invented names, rounded numbers, and placeholder details — the underlying problem you're asking about usually survives intact. This aligns with consumer sentiment, too: only 54% of people are comfortable sharing anonymized personal data to improve AI products, which tells you anonymization is the accepted standard, not a nice-to-have.

The second habit is understanding that "don't train on my data" and "don't store my data" are two different settings. Excluding your chats from model training does not stop retention — OpenAI's default window keeps inputs and outputs for at least 30 days, even after they disappear from your interface. Zero Data Retention exists, but it's approval-gated and limited to eligible enterprise API customers, so conversations typed into the ChatGPT browser interface aren't covered by it.

Third, treat your account credentials as keys to your entire chat history. A 2024 breach exposed over 225,000 OpenAI credentials on dark web markets via infostealer malware, meaning criminals could read victims' full conversations. Enable strong authentication, use a password manager, and never put secrets in a chat you wouldn't want read aloud.

For businesses, the evidence points clearly toward governance, not bans. Shadow AI breaches already hit 20% of organizations in 2025, rising to a projected 43% in 2026, and 97% of AI-breached organizations lacked AI access controls. A blanket ban just pushes usage underground — 88% of employees with enterprise AI access already use personal AI tools for work anyway. What works instead:

• A clear acceptable-use policy naming which data categories may never enter AI tools
• Data classification so employees know what counts as sensitive before they paste
• AI-aware data loss prevention that flags SSNs, source code, and customer records in prompts
• Enterprise-tier AI accounts, since personal accounts carry weaker protections that most employees don't distinguish

The regulatory stakes make this urgent rather than optional. EU AI Act penalties reach up to €35 million or 7% of worldwide annual turnover, with full compliance by August 2026, and GDPR fines have already exceeded €4 billion since 2018. Governance now costs a fraction of a breach later, where the average data breach runs $4.88 million.

The same discipline extends to the broader agent ecosystem. As teams connect AI agents to tools and skills, verifying what those skills actually do matters as much as vetting what you type into a chat — which is exactly why SkillzBench is building free security scanning and benchmarking for agent skills, with a waitlist now open for teams that want proven, not assumed, AI capabilities.

Beyond ChatGPT: Extending Safe AI Habits to Agents and Skills

The same instincts that keep you safe in a chat window matter even more in the next frontier: AI agents. Agents don't just answer questions — they connect to your files, tools, and workflows, which means the blast radius of a bad habit grows from one conversation to an entire system.

The data explains why this discipline is urgent. Sensitive corporate data flowing into AI tools has climbed from 10.7% two years ago to 34.8% today, with 39.7% projected by 2026, according to industry research. And 97% of organizations hit by AI-related breaches lacked proper AI access controls — meaning most teams are bolting powerful tools onto their stack without vetting what those tools can touch.

Now layer on agent skills. Published research found that roughly 26% of published agent skills carry at least one vulnerability. A skill that reads your inbox, queries your database, or writes code can quietly become a new exposure channel — one that operates with your permissions, at machine speed.

The habits from this article translate directly:

• Never grant a skill access to data you wouldn't paste into ChatGPT — customer PII, source code, financials, or strategic documents stay out of reach by default.
• Verify before you install: check what a skill actually does, what it accesses, and who maintains it.
• Pin versions. A skill that was safe last month can change upstream without you noticing.
• Run new skills in a sandbox first, so a vulnerability never meets your production data.

This is exactly the problem SkillzBench is being built to solve. It's a free, independent directory of agent skills plus a free MCP connector that brings only the skills you choose to any agent. Every skill version is pinned to a commit SHA, security-scanned, license-checked, and benchmarked in a sandbox — with and without the skill — so you can see the real Lift before anything touches your data. A 0–100 trust score draws on security, quality, license, maintenance, and community signals, and money never touches scores.

The principle is the same one that runs through this entire article: govern what reaches your AI, don't just hope it behaves. As Telescope's Amy Ryu puts it, a retention policy "does nothing to stop sensitive information from being sent there in the first place" — and the same is true for agent permissions. Vet the tools, then let them work.

SkillzBench is currently in pre-launch, and the free directory, MCP connector, benchmarks, and alerts will all be free when it ships. Join the waitlist today and browse the free skills directory — because agents should be equipped only with what you've vetted.