Protecting Patient Data in the Age of AI-Generated Code

As hospitals, digital health platforms, and health insurers turn to AI coding assistants to build software faster, security and compliance teams face a new kind of risk.

By Eran Kinsbruner, Vice President of Product Marketing at Checkmarx

Artificial intelligence is rapidly transforming how software is built across healthcare. From patient portals and telehealth platforms to claims processing systems and care coordination tools, hospitals, digital health companies, and health insurers are increasingly using AI coding assistants to write, review, and improve software faster than ever.

The benefits are clear. Healthcare organizations face growing demands to modernize digital services, support rising patient volumes, and manage increasingly complex data environments. AI-powered development tools can accelerate software delivery and help teams respond more quickly to changing business and regulatory requirements.

However, as AI becomes embedded in development workflows, it also introduces new security and governance challenges. For organizations responsible for protecting protected health information (PHI), faster software development must be matched by stronger approaches to securing the code that powers critical healthcare systems.

The rise of AI coding assistants in healthcare

AI coding assistants powered by large language models are quickly becoming part of modern developer toolchains. These systems can generate code, identify bugs, suggest fixes, and assist with software design decisions in seconds.

Across healthcare, these capabilities are being applied to systems such as electronic health records, patient portals, telehealth applications, remote monitoring platforms, claims processing systems, health information exchanges, and care coordination tools.

A second use case is emerging alongside new development: migrating the legacy code much of healthcare software still runs on to more modern architectures, moving from older programming languages to current ones. Not every application gets migrated, but the ones that do are often core systems touching PHI directly, which makes the security profile of this work just as important as building something new. This is exactly where AI-generated code and legacy systems intersect, and where scanning needs to keep pace regardless of what language the code started in or what tool refactored it. The most relevant approaches for this kind of work pair a deterministic security foundation with AI reasoning that isn’t limited to a single model or programming language, and increasingly run that analysis inside a healthcare organization’s own cloud environment so source code never has to leave its infrastructure boundary, a distinction that matters directly for HIPAA-driven data residency requirements.

For hospitals, providers, and payers, faster development cycles can translate into meaningful improvements, including quicker deployment of patient-facing services, more responsive digital experiences, and faster updates to complex operational systems.

But the speed and accessibility of AI-generated code also introduce new risks. Code produced by AI models may contain vulnerabilities, insecure configurations, exposed secrets, or vulnerable open-source components that require careful review before reaching production environments.

This is not a theoretical concern. Researchers at Georgia Tech’s Systems Software and Security Lab have tracked vulnerabilities linked to AI-generated code through the Vibe Security Radar project. The number of disclosed vulnerabilities associated with AI-generated code has increased rapidly, with six new CVEs identified in January 2026, fifteen in February, and thirty-five in March. While this research is not healthcare-specific, any vulnerable code introduced into systems that process PHI creates a healthcare security concern.

Why protecting PHI is becoming more complex

Healthcare organizations already operate under strict privacy and security requirements, including HIPAA regulations in the United States. Health insurers also face additional obligations around protecting claims data, member information, and sensitive records shared across healthcare networks.

AI-assisted development adds another layer of complexity. Independent testing highlights a significant gap between AI-generated code that works and code that is secure. The Weather Report, a peer-reviewed study conducted by researcher Ilya Kabanov and commissioned by Checkmarx found that AI models generate functional code 83–95% of the time, but only 24–36% is both functional and secure. Even with AI-assisted security techniques, that figure improved to just 47–56%.

In most industries, insecure AI-generated code is a software quality issue. In healthcare, it can become a compliance, privacy, and patient trust risk when vulnerabilities exist within systems that manage sensitive health information. AI coding assistants also introduce potential data exposure concerns if developers inadvertently include sensitive details such as database structures, authentication logic, or patient-related system information in prompts processed by external cloud-based models. Organizations should understand how AI providers retain, store, and use prompt data, while recognizing that the speed of AI-assisted development can outpace traditional end-of-cycle security reviews, making continuous security validation throughout development increasingly important

These challenges add to an already significant cybersecurity burden. IBM’s 2025 Cost of a Data Breach Report found that healthcare breaches average $7.42 million, the highest average cost of any industry for the 14th consecutive year, and organizations take an average of 279 days to identify and contain breaches. Vulnerabilities introduced through AI-assisted development can extend that risk if security practices fail to keep pace with development speed.

The future of healthcare software security

The rise of AI-assisted development is forcing many healthcare organizations to rethink where and how security fits into the software lifecycle. Historically, security testing often occurred later in development, during staging environments or final release checks. That approach is increasingly inadequate in a world where developers may generate and iterate on code continuously using AI tools.

Many organizations are responding by adopting what is increasingly being called an agentic development lifecycle (ADLC), a model in which AI agents perform security tasks directly rather than simply flagging issues for humans to review later.

In practice, this begins with agentic prevention that identifies vulnerabilities as code is written or refactored, whether inside AI-native IDEs such as Cursor and Kiro or through coding assistants like Claude Code. It also includes scanning that detects both newly introduced vulnerabilities and pre-existing issues hidden in the legacy code being modernized. Beyond detection, agentic triage and remediation can prioritize findings based on real-world exploitability and recommend fixes before code is merged.

Another important, and often overlooked, component is an AI inventory or AI-BOM: a record of the models, agents, and AI components involved in building and securing software. This provides compliance and governance teams with the audit trail needed to demonstrate that AI usage itself is being governed and secured, not just the code AI helps produce.

The urgency behind this shift continues to grow. Anthropic’s Claude Mythos, a research model previewed in 2026, demonstrated the ability to discover and exploit vulnerabilities at a scale and speed that surpassed earlier AI systems. It offered a glimpse of how quickly AI-driven vulnerability discovery could accelerate across the industry.

That kind of capability is precisely why a legacy migration cannot be secured with a single scan at a single stage. Because modernization efforts touch nearly every part of an application, security coverage must span the entire lifecycle: from AI IDEs and coding assistants as they refactor legacy code, to CI/CD pipelines as changes move toward deployment, to runtime environments after modernized applications go live, and finally across the software supply chain that supports them.

Adopting AI development tools while maintaining compliance

AI coding assistants can deliver significant value in healthcare, but organizations need clear governance and safeguards to use them responsibly. Several practices can help balance innovation with security:

Establish clear AI usage policies. Organizations should define how developers can use AI coding assistants, what information can be shared with AI systems, and which tools meet security and compliance requirements.

Prevent sensitive data exposure. Developers should avoid including PHI, proprietary system details, or sensitive architecture information in external AI prompts. Organizations can also implement automated controls to detect and prevent sensitive information from being shared.

Integrate security into development workflows. Security tools should analyze applications, open-source dependencies, infrastructure configurations, and secrets throughout the development process rather than relying only on final-stage reviews.

Maintain transparency and auditability. Healthcare organizations need visibility into how software is developed, including how AI tools are used and what AI components are involved. An AI-BOM, maintained alongside traditional audit trails, can support compliance efforts and improve incident response.

Innovation and security must move together

AI coding assistants are becoming a permanent part of modern software development. For hospitals, digital health companies, and insurers, the question is no longer whether these tools will be adopted, but how they can be integrated safely, whether that means building new capabilities or modernizing what’s already running.

AI-assisted development has the potential to accelerate healthcare innovation, from improving digital patient experiences to supporting more efficient healthcare operations. But organizations must ensure that faster development does not come at the expense of security and privacy.

The healthcare organizations that succeed will be those that treat security as a fundamental part of AI adoption. By embedding protection earlier in the software lifecycle, establishing strong governance, and maintaining visibility into AI-assisted development practices, organizations can take advantage of AI’s benefits while preserving the trust of patients, members, and regulators.

About the Author

Eran Kinsbruner is Vice President of Product Marketing at Checkmarx, where he leads product marketing strategy for the company’s application security portfolio. A seasoned enterprise SaaS marketing executive, Eran is a recognized thought leader, board advisor to stealth technology companies, researcher, inventor, and best-selling author of four books.

He brings deep expertise in B2B SaaS, AI, observability, DevOps, and software quality, with a proven track record of developing and executing go-to-market strategies that strengthen product positioning, elevate thought leadership, increase brand awareness, and drive business growth.

Throughout his career, Eran has successfully led cross-functional teams to deliver competitive, high-value products that accelerate sales and market adoption.