Our plan for a more secure npm supply chain

[leobalter]

We recently published a post about our plan to improve security on npm as a supply chain supplier and I'd love to gather everyone's feedback.

Please use this thread for general feedback. Other threads might be pinned as well as they get popularity as I want to keep free formatting for all.

Thanks!

[ljharb]

• on killing TOTP: https://github.com/orgs/community/discussions/174505
• on forced expiry: https://github.com/orgs/community/discussions/174506

[wesleytodd]

Copying some feedback here from our Slack discussion:

Ok, this is overall a good step, specifically disallowing non-2FA local setups and setting better defaults. It is still missing the workflow for 2FA in CI, it seems to limit ci publish to only trusted publishing (unless you want to deal with the toil of manually minting a token every week) which means we loose one existing option for a secure setup, and also need to request a fix for the missing identity on publishes.

So while it is moving in the right direction, my first optimistic read was anchored on the defaults and missed the whole bit that they are not actually addressing the ask for native 2FA workflows and even making one secure (albeit complicated to setup) workflow even harder since they dont have a proper api for minting tokens.

I think mainly these changes are good, they are just not complete. Adding first party support for the CI workflows to require 2fa as well is the main missing piece for me.

Additionally, see #174508 for some impact from the expiry and automation token removal.

[IchordeDionysos]

it seems to limit ci publish to only trusted publishing

As a package consumer, I would want to have every package on npm use trusted publishing, period. Not even allow, local publishing with 2FA or publishing with an expiring token.

Trusted publishing allows you to look into the exact commit and build pipeline that was used to build a package.

[wesleytodd]

Trusted publishing allows you to look into the exact commit and build pipeline that was used to build a package.

You have been able to do this for a long time without trusted publishing. The packument contains the commit it was released from. That is how things like npx reproduce work. Unfortunately it is not common that you can reproduce a build. Additionally, the provenance data is only kept as long as GHA keeps past builds (right now I think 90 days max). So you are not really getting much more from TP, and you are loosing forced 2FA because those are mutually exclusive today.

So while I think your goals are spot on, the implementation today of TP is drastically lacking for achieving said goals. So until those gaps are closed, a responsible maintainer with 2FA turned on is more secure than TP. Once the gaps are closed, I think we can recommend it but that doesn't mean there is anything to gain from removing local publish.

Instead we should focus our attention on increasing the reproducibility of builds and adding 2FA support in all the right places.

[voxpelli]

There’s also some still valid and unaddressed feedback in the discussion around the OIDC / Trusted Publishing launch, such as eg my comment there: https://github.com/orgs/community/discussions/161015#discussioncomment-14419367

[dasa]

From https://docs.npmjs.com/trusted-publishers

Self-hosted runners are not currently supported but are planned for future releases.

Is this support planned to be released before deprecating legacy classic tokens?

We use GHES with self-hosted runners in our publishing workflow, and would like to switch OIDC / Trusted Publishing. It would be a breaking change for us to no longer be able to use legacy classic tokens with no ability to use OIDC instead.

[voxpelli]

Feel free to join in on ossf/wg-securing-software-repos#90 to discuss the general principles and guidance for adding new platforms to Trusted Publishing enabled registries (goes beyond npm)

[saquibkhan]

@voxpelli it is not related but i want to share with you and others for visibility on npm oidc APIs (currently supports github/gitlab) - https://api-docs.npmjs.com/

[MarshallOfSound]

There's a lot of good feedback elsewhere in these discussions but I want to make my take public outside of the slack as well.

I'm going to go through the bullet points in the github blog post and then provide my own additional requests at the bottom.

Current Planned GitHub / NPM Action Items

Deprecate legacy classic tokens.

✅ This is good, get rid of them. Scoped and granular tokens are intrinsically more secure and the developer pain of "ah darn I gotta run npm login again" is worth the security win IMO.

Deprecate time-based one-time password (TOTP) 2FA, migrating users to FIDO-based 2FA.

✅ Again, this is great. I had this as an item in an unpublished blog post that NPM should do. It effectively moves NPM from "2fa" to "phishing resistant" 2fa, which if in place 2 months ago would have prevented several of the initial compromises we saw hit the npm registry due to phishing attacks against maintainers (some of whom had 2FA and got their TOTP mitm'ed).

To be clear, this is going to be quite annoying for so many people. Not in the least me, Electron publishes ~50+ packages all using TOTP codes (securely sent via http://github.com/continuousauth to github actions) that all need to be migrated to trusted publishing. But this is work we were going to do anyway because we'd already made our own conclusions about trusted publishing being the more secure future.

Limit granular tokens with publishing permissions to a shorter expiration.

❓ This one is rough, there is currently an unsolved "Enterprise" usecase here that I will outline in "Missing Pieces" below.

Set publishing access to disallow tokens by default, encouraging usage of trusted publishers or 2FA enforced local publishing.

✅ Finally, yes, thank you. Guiding developers towards the more secure option by default is a quick and easy win here that doesn't impact any existing package.

Remove the option to bypass 2FA for local package publishing.

✅ Yessss. Thank you. Forcing 2FA for everyone is the single biggest win out of all of these, if you run npm publish locally it shouldn't matter what token you have, there should be a final form of "step up" auth in this case 2FA. Will it results in some developers having to open a browser every time they publish? Yes. Is that the end of the world? No

Make everyone have secure phishing resistant 2FA, and then force everyone to use it 💯

Expand eligible providers for trusted publishing.

❓ This one is also rough, but not because I'm against the idea, I'm wary of the implementation. See the "other OIDC providers" bit in "Missing Pieces" below

Missing Pieces

Let's pretend that everything in the above list shipped, what are we still missing to have a secure and simple package publishing strategy that works for everyone who currently uses npm

Trusted Publishing should be One Way

Once you opt in to Trusted Publishing and publish your first release with it, that should be a one way choice only changeable in extreme cases with intervention from npm support. Currently just having access to a maintainers npm account effectively let's you perform a "downgrade" attack on them by removing trusted publishing and/or changing publishing settings. Once a package is published the Super Safe way, it should only ever be publishable that way.

Trusted Publishing should use repository ID not name

This is a common issue I see with github apps and other github integrations, I'm disappointed to see it from npm. By validating the owner name and repository name you are intrinsically creating the following issues:

• Typos / typo-squatting
• Race conditions when renaming
• Race conditions when transferring repos between orgs / to an org

The current UI in npmjs.org needs to be an actual dropdown of repositories using github oauth to provide a list and then internally it should validate the repository_id claim of the OIDC token not the repo slug as that is unsafe.

Trusted Publishing should set up and enforce github environment usage

Current GitHub environment usage for trusted publishing is a manual and optional step, in addition to the above once you have an oauth token for the users github account, set up the environment for them. Enforce the environment can only be used on the {DEFAULT_BRANCH} by default (user can change later if they want) and even give them a $CI_PROVIDER template file that consumes that environment in a safe way.

This should be a series of clicks in the UI, not a custom yaml file, a hand crafted environment and manually typing in repo org and name.

Trusted Publishing should disable all other legacy forms of publishing when active

Currently even with trusted publishing enabled, a compromised maintainer account with a TOTP mitm can still publish these packages. Having Trusted Publishing active should just make all other forms of publishing disabled, regardless of how much auth you provide. See Trusted Publishing should be One Way again for more reasoning.

The Enterprise Usecase and Other OIDC Providers

These two I don't have an answer for, but calling them out anyway. Enterprises use self hosted runners, even to publish open source packages, they also use CI systems that aren't github actions or gitlab. What's the story going to be for them, enterprises with self hosted CI (Jenkins, BuildKite, CircleCI, etc.) should still be capable of publishing packages safely without having to manually update a token every 7 days 🤔 I would like to see a plan for that.

[twesterhuys]

The Enterprise Usecase and Other OIDC Providers

Not just third parties like Jenkins are missing at the moment also first party GitHub Enterprise Server which can only be used with self-hosted GitHub Action Runners. This is a problem and so far (I might have missed it) there is little mention of a roadmap for this. As mentioned below very happy to contribute/help to making this work.

Update: Adding this for visibility ossf/wg-securing-software-repos#90

[pimterry]

Once you opt in to Trusted Publishing and publish your first release with it, that should be a one way choice only changeable in extreme cases with intervention from npm support.

Trusted Publishing should disable all other legacy forms of publishing when active

While generally tightening this up is good, imo exclusive-use + one-way-only always is too much.

It's hard to know how people will use this and want to change their setups in future (especially as the ecosystem develops and new solutions to these issues appear), or will get stuck with broken setups, and all that will create extra support load for npm en route. HPKP is a good example of how this goes wrong. Mainly I worry hard limits like this will discourage people from enabling trusted publishing in the first place, since it makes it a much higher risk more complicated thing to enable.

Trusted publishing needs to work somehow for people maintaining just one package as a tiny sideproject, and people just testing it out and exploring, not just for large mature orgs who know what they're doing. Otherwise most packages will never use it, and we all know how many tiny packages maintained by individual hobbyists are powering the ecosystem.

There's plenty of good middle ground though: e.g. adding a delay for any changes (e.g. doesn't disable for 48 hours, maybe a week) and notifications allowing any maintainer of the package to block the change. That would significantly increase the challenge of a downgrade attack, which still making it possible for normal users to manage trusted publishing configs in a reasonable way.

Obviously any change like this should require 2FA auth anyway though... And if you can do a downgrade attack providing the password & 2FA, you could also just change the account's email, password & 2FA setup directly, in which case you can just do whatever you want (including legitimately talking to NPM support as the account owner)... Maybe the delay is the only solution for this extreme case, since at least that way the real owners have some warning & time to object before anything can happen.

It might also be interesting to explore a 'modification lock' approach (like https://en.wikipedia.org/wiki/Registrar-Lock for DNS) independent of just setting up Trusted Publishing - once all is good and you're happy, then you tell npm to make it extra hard to change these settings later, and enforce a 7 day delay/support request/visit to the NPM office/notarized letter from your head of state/whatever.

[3nprob]

What you are proposing results in that publishing can only practically be done from GitHub and GitLab architecture. There must be a compelling, proven, and established flow for self-hosted/third-party Trusted Publishing on npmjs before we should consider enforcing it across the NPM platform and community.

Ideally with a realistic timeline between announcement and enforcement (years or months, not weeks or days).

[MarshallOfSound]

What you are proposing results in that publishing can only practically be done from GitHub and GitLab architecture. There must be a compelling, proven, and established flow for self-hosted/third-party Trusted Publishing on npmjs before we should consider enforcing it across the NPM platform and community.

That is not what I am proposing in the slightest.

There is no reason why local publishes can't still be supported in a safe way in a world where Trusted Publisher is the *default, easy to use and bullet proof (which is what most of my feedback covers).

A developer should be able to run npm publish and provide username & password & webauthn based authentication, potentially additionally verify by additional validation via email verification in "suspicious" cases (country changes, lots of rapid publishes, etc.). But they should have to do that for every publish, not have a magic token that let's you publish from anywhere at any time which is the current world we live in.

The same system could theoretically be used by other CI providers, the npm publish job currently outputs a URL that the developer must authenticate the publish with. Your CI can just "pause" on that URL and you can provide the auth on your machine. Automated publishing requires a level of trust in the infrastructure that we can't just assign to everyone but I agree there should be a clear set of guidelines and requirements for onboarding new trusted OIDC roots beyond just GitLab and GitHub. I just don't see a world where self-hosted ever meets that bar, third party folks like CircleCI/BuildKit and such though I definitely want to see on this list. (Note: I called this out in "The Enterprise Usecase and Other OIDC Providers")

[MarshallOfSound]

It might also be interesting to explore a 'modification lock' approach (like https://en.wikipedia.org/wiki/Registrar-Lock for DNS) independent of just setting up Trusted Publishing - once all is good and you're happy, then you tell npm to make it extra hard to change these settings later, and enforce a 7 day delay/support request/visit to the NPM office/notarized letter from your head of state/whatever.

Anything a developer has to opt in to will not be used. See: 2FA

If people wanted to be more secure at scale everyone on the registry would voluntarily have secure 2FA right now, but they don't.

At a certain point we need to make the trade-off as an ecosystem: Security of the ecosystem must be the priority and developer UX can take a back seat in some areas. Baseline registry security should never be optional or opt-in

[leobalter]

Here is a new post with changes we are applying first as part of this program. Please take a look and thanks for the continued feedback, as it is helping us doing some fine adjustments to the new measures we are implementing.

[voxpelli]

why not publish the underlying technology pieces and processes so that teams could adopt this themselves

@twesterhuys The Trusted Publishing concept was pioneered by PyPI and described and documented by the OpenSSF Working Group on Securing Software Repositories in eg the Trusted Publishers for All Package Repositories document.

Making oneself technically able to become a Trusted Publishing platform is the fairly easy part – getting oneself onboarded as a valid platform by npm is the harder part.

Ultimately there should be guidance on this, I opened ossf/wg-securing-software-repos#90 to track it

[twesterhuys]

@voxpelli that's great news and many thanks for the background - would be nice if GH more explicitly mentiones this in their docs rather then to referring this much more vaguely. Good to hear that what I assumed was the problem (onboarding rather then tooling/standards) to be the bottleneck and many thanks for crystilizing this in the issue.

[ljharb]

@IchordeDionysos trusted publishing provides a stealable short-lived token too, so no, it wouldn't.

[MarshallOfSound]

trusted publishing provides a stealable short-lived token too, so no, it wouldn't.

This doesn't cover the full picture, trusted publishing provides significantly shorter lived tokens and they aren't on end-user machines. Stealing them would require compromising the entire privileged build environment, not just phishing a user which is what happened multiple times last month. You can't phish an OIDC token, and you can't steal access from a human or a compromised developer machine if they didn't have it to begin with.

[ljharb]

I believe the shai-hulud attack exfiltrated a token from a workflow, which would work the same way for OIDC.

[DNin01]

I think npm accounts should be as important as... I don't know, Apple developer accounts. Great to see that security updates are being made!

[ericcornelissen]

If you're going to force people to generate new tokens by forcing (short) expiry windows, at least make it easy for them to regenerate tokens using the previous configuration. As someone who has been using short-ish expiry windows already, this has been by far my biggest frustration with it - this feature has been long overdue...

If a token of mine expires, I probably need one with the same granularity to replace it, so give me that option. If not, I think many developers will just generate overly permissive tokens because it's easier than specifying one package per token (again, I have considered this but luckily I don't publish that often). I believe this would contribute significantly to preventing attacks as a leaked token for one project wouldn't allow publishing all the package maintainer's other tokens.

See also my earlier piece of feedback: npm/feedback#1088

[grochoge]

Please clarify the scope for this. It's linked from the main GitHub page; is it only for NPM or is it for everyone?

I believe there are some things that you can do with classic access tokens that you cannot do with fine-grained tokens

[voxpelli]

The specifics in the post is for the npm registry in particular (which is owned by GitHub)

The concept of Trusted Publishing comes from OpenSSF / PyPi and GitHub has supported such publishing to such platforms for years.

See eg. https://repos.openssf.org/trusted-publishers-for-all-package-repositories for an overview and description of the general principles and reasoning behind the “Trusted Publishing” concept

[43081j]

One thing we seem to be skipping over:

GitHub environments should have an option to require 2fa

All of these npm changes are fine if there's 2FA in the trusted workflow.

Currently, you can already bind a trusted workflow to an environment in that it must be run through that to be considered trusted.

In reality the only reason anyone is complaining about the changes so far is because such a flow doesn't exist.

The flow should basically go like this:

• create a release (however you prefer, e.g. tags)
• workflow is triggered against an environment
• environment waits for approval (already possible)
• in order to approve, maintainer must complete 2FA
  • similarly, changing the workflow settings (like removing approvals constraint) should need 2FA

Admittedly this isn't an npm change but given how we're all being pushed to use GitHub for publishing, it's a very important feature to make the whole picture work.

[voxpelli]

Totally agree, I mentioned this here: https://github.com/orgs/community/discussions/161015#discussioncomment-14419367

What I would want:

Multiple reviewers are possible for branch protection rules, feels like it should even more so be possible for deployment environments.

[saquibkhan]

@43081j @voxpelli can you help us with a scenario where a workflow without 2fa challenge on approval can be compromised/hacked and insecure?

[43081j]

if someone compromises an action right now, they may be able to capture a github token.

lets say they manage to do that, and the token has the permissions necessary to commit and to trigger a release (whether it be by a tag or by a GitHub release, or whatever git-based trigger).

if they got this far, they can probably use the same token to either disable the approval constraints or to approve the workflow (assuming you bound the workflow to an env that requires approvals).

if we could require 2FA when approving workflows, that means even a stolen token can't approval a release at the end of it. they'd need to pass through 2FA to approve it, or to disable the approval constraint.

right now, someone could get this far with a compromised token and successfully publish a package as far as i understand

(some of the recent incidents were compromised, overly permissive GitHub tokens acquired through workflows. so we have to assume that will happen again)

as far as i can see, most other concerns are moot if this gets solved. people who are unhappy about losing TOTP, are only unhappy because they're still publishing manually right now - and the only reason they're publishing manually is because they can't enforce 2FA on publish approvals.

[shajz]

Hi, thanks for this post. Some questions about npm login and its use of legacy tokens:

To be able to run npm ci on projects that use private scoped packages on the npm registry, we have to run npm login to be authenticated and gain access to those packages (otherwise we get a 404).

Right now, the npm login command generates legacy tokens.

When you run npm login, the CLI automatically generates a legacy token of publish type. For more information, see About legacy tokens.

How will this be impacted by these changes? The github blog post does not address this issue.

Thanks!

[kategengler]

I am also wondering about this. I use npm login in order to change a dist-tag (to promote a stable release to our lts tag).

[leobalter]

You're correct that npm login currently generates legacy tokens. We've planned for this - when we complete the classic token revocation, npm login will automatically switch to receiving compatible short-lived session tokens that expire after a reasonable session duration. Your existing workflows for accessing private scoped packages with npm ci will continue to work without any changes required on your end.

The transition will be handled transparently by the CLI, so you can continue using npm login as you do today. We'll share full implementation details in our upcoming release announcements over the next few weeks.

Let us know if you have specific concerns about your CI/CD setup that we should consider.

[kategengler]

I'm happy to hear that -- those tokens make me nervous!

[arcanis]

@leobalter will that work with older versions of the npm cli ? Does that change how 3rd party package managers should interact with the registry endpoints ? Yarn implements yarn npm login as well - do we need to change our implementation ?

[jacobtb23]

Thanks for the post. Related specifically to the Shai Hulud attacks I saw mentioned in the article that there are controls in place to prevent packages with Shai Hulud IoC's from being published. Can you detail that for me? We are trying to assess if NPM/Github has this under control or if we need to build out additional checks for Shai Hulud effected packages (Search for IoC's ourselves).

[std4453]

Our company has its own CI system and we do some npm package publishing using long-live tokens. We would like to migrate to Trusted Publishing, yet documentation says that it does not have wide support, nor is there any clear directions on how to implement Trusted Publishing for custom CI systems.

With long-live tokens going to be revoked in mid-November, does that mean our only option is to npm login every time before CI tasks? How does that suppose to work along with 2FA, and do we have examples on that?

[fmal]

Will long-lived classic tokens with read-only scope (not able to use for publishing) will also be revoked? They're useful for granting read-only access to packages in a private scope and don't pose security risk like publish tokens.

[boutell]

For those who are just now catching up: you can create an all-purpose read-only token for one or more orgs as a "granular" token, and you don't need to respond to 2FA each time you "npm install." These tokens are expressly designed for CI/CD situations, e.g. automated installs. You will however have to rotate them every 90 days.

[Gerrit0]

I am lucky that (currently) all of my automated publishing happens through GitHub actions, so this doesn't affect me that much today as mostly switching to trusted publishing only took half a day or so (though a forced breaking change like this with such a short timeline before npm will break all of my automation is not appreciated!)

I have two main issues with this today:

1. Custom CI systems are effectively dead in the water. This includes stuff like Azure pipelines used by TypeScript
2. Even if using GitHub Actions for publishing, there can only ever be one workflow for publishing. That might be fine for a simple project, but many projects with publish multiple tags. TypeDoc is an example of this which has 3 release pipelines for latest/beta/lts.

My recommendation for this would be:

1. Don't require anything to do with trusted publishing until May 2026.
   I chose this date because OIDC requires at least npm@11, which isn't installed by default for Node 20, and that's when Node 20 goes out of support.
2. Don't deprecate any functionality until the more secure replacement is fully operational. A custom CI implementation needs to be able to publish without requiring manual intervention every 7 days to make nightly releases continue working. Not being able to do this is absolutely insane.

[jakebailey]

TS/DT publishing is "okay" (though, via manual token rotating 🫠), but everything else here I agree with. Any futher changes to restrict things will only make things even harder.

[kategengler]

Reusable workflows can be the workflow file specified, so you can have a reusable workflow to do the publish step used by beta/release/lts.

Of course, I didn't discover this until after I had refactored ember.js publish to only be in one workflow...

[ckohen]

As far as I can tell, reusable workflows cannot be the workflow file specified if they are not in the exact supported configuration, and seemingly aren't supported at all. Possibly if you generate the token in the reuseable file this could work though?

We (discord.js) use a composite action to do our publishing because releases have to be sequenced appropriately, and we have two top level actions that do different types of releases. One for stable publishes and one for dev/pr-tag publishes because actions doesn't give us enough control to do this all in one workflow.

A further thing we do that still requires a token - and has no other option - is deprecating old dev/pr-tag publishes and occassionally old stable versions. Until this is supported either by allowing us to programatically regenerate a token in a more secure context to make available for the less secure context, or maybe use oidc to deprecate.

Manual token rotation is not exactly practical at these frequencies, and I strongly suspect we will end up with lots of failed actions runs due to this (we publish nightlies).

[kategengler]

Yes, the id-token: write needs to be in the reusable workflow.

ember.js also needs a token for updating lts tags. We only promote a release to lts once it has baked for 6 weeks.

[johnnyreilly]

I generally agree with the points here. My view is that moving to trusted publishing generally is a good thing that will improve security in the npm ecosystem.

The timeline for action is very ambitious. Effectively, you've got a month to get migrated. That minimal period of time is likely to cause more security issues as people struggle to get things in place before their release workflows start to break due to the changes

Not all tooling appears to be ready for trusted publishing; consider:

JoshuaKGoldberg/release-it-action#672

It's also worth thinking about the time each package takes to migrate. It's not simple and straightforward. There is no automated migration path. There is work to do.

I fear this timeline will have a short term negative affect on the state of npm security. I would request a longer timeline and more automation to help migration.

[sokcuri]

Our company currently uses Hosted GitHub Runners and GoCD. I appreciate npm’s determination to address supply-chain attacks, but I feel the measures miss the core issue.

For small-scale development, hobbies, or testing, requiring 2FA to publish or forcing token renewal every time seems excessive. And OIDC feels like it merely shifts token management to another platform.

There are many disruptive changes that risk breaking a lot. How about splitting publishing into "secure" and "unsecure" modes? In the unsecure mode, you could delay promotion to the latest tag and add a few constraints (e.g., disallow script execution). I believe that for most use cases, even a less secure publish would be acceptable without major problems.

[SleeplessByte]

I maintain over 45 packages spread over 12 organisations, none of which are using GitHub.com or GitLab.com, and none of which who have ever been even near a supply-chain attack. The current tokens have a lifetime of 01-01-2100, but restricted to a single scope (or package). Forcing token renewal like this basically forces me to start publishing every package manually.

I have no issue with the forced 2fa for local publishing (in fact, I've had it on for years), but the cost to me and my company "waiting" for trusted publishing to come to self-hosted runners and then having to upgrade the CI again isn't just excessive, it will likely cause us to either lose the clients or lose money. If this trusted publishing system was available today, I would be sad because of the forced work both as a business owner and open source maintainer, but I can live with that. Right now there is no path forward other than NPM hopefully supporting it Soon™.

Our workaround it to drop NPM completely, which we can, but surely that's not the intention?

[electrovir]

I replaced my OTP authenticator with a passkey but the CLI is still asking me for a OTP when I publish with "Require two-factor authentication for write actions" turned on. What OTP is the CLI expecting?

[saquibkhan]

I replaced my OTP authenticator with a passkey but the CLI is still asking me for a OTP when I publish with "Require two-factor authentication for write actions" turned on. What OTP is the CLI expecting?

@electrovir can you plz submit a npm support ticket for this?