GitHub really needs something finer-grain then just pretending the repo never existed during these incidents. [1]
The bad package version has also just disappeared from crates.io [2] with no indication its been yanked. There's no security advisory there either [3] "No advisories found for this crate."
I feel crates.io was unprepared for a security incident like this since they're managing the response [4]
This is exactly what PMG is designed for ie. install/build time process level sandboxing. It currently doesn't support cargo, but I believe the challenges are same.
Here is my learning building PMG:
Sandboxing is good when the workload is predictable, and the goal of sandbox is to guard against exploitation of vulnerabilities, like sandbox protecting chrome tabs (renderers). But unfortunately build scripts are not predictable, at least not in npm/pypi world and I have seen build scripts doing weirdest of the things which is no different from malware. When popular packages do weird things, build breaks and users end up turning off the sandbox. This is a perpetual problem to deal with while building sandbox (or any least privilege solution) to protect unbounded workloads.
build.rs by design can run absolutely anything. There tons of build.rs scripts that invoke a whole-ass C compiler toolchain to build and link C dependencies...
It isn't so much a question of sandboxing build.rs, as fundamentally changing the way that foreign dependencies are integrated into the rust toolchain (i.e. moving from a rust-centric system like Cargo to something more general like buck2)
The vast and overwhelming majority of build scripts are building C code, so the other solution is to move away from integrating with C dependencies to native Rust dependencies.
Sandboxing just build.rs would be
be a minor inconvenience for the attacker but a big pain for many that are in the unfortunate position to really need build scripts.
At the very least, it wouldn't be overly onerous when adding a dependency that requires a build script to require an opt-in via Cargo.toml, e.g. `build-script = true`. You'd make it viral so that any transitive dependency that requires a build script would affect its parent, then add the key as defaulting to `true` so as to not break backwards-compatibility, then switch the default to be more restrictive over a new edition.
I imagine it would be sandboxed by default with an escape hatch to run build scripts outside of the sandbox with user verification. It makes people stop and think about what’s happening. Not perfect, but it does help. When working on JS ecosystem projects I manually approve build scripts and spend some time researching dependencies with build scripts to see if I can avoid running the build script. Some people will ignore it and run everything, but it’s a huge step in the right direction to make it operator-decided.
It would be more than a minor inconvenience. I can handle sandboxing my tests and production infra, but I can't handle sandboxing build scripts because I don't own that code in any sense.
Sandboxing for build scripts can't work properly. If you sandbox too much, some necessary stuff can't be done. If you sandbox too little, it has no practical value.
So let each build script define its own level of sandboxing and then users can determine whether they are okay with that level or not, e.g. `cargo build --sandbox-level=...`
There's no good reason a proc macro can't run in a no-IO sandbox by default. None. Doesn't require a language change. Doesn't require some microvmcapabilityeffect BS. It requires looking people straight in the eye and saying "no" when they complain about needing to prompt for privileges.
I mean sure, but anything the build script could do, the build artifact could also do, That is to say, if you don't trust your source why do you trust the thing it compiles into?
cargo add + rust-analyzer instantly executes build.rs before you have a chance to audit the code.
Cargo, please PLEASE give me a way to disable third-party build.rs and whitelist the ones I need. And please loudly mark any update that adds a build.rs where there was none before.
cargo-deny can audit build scripts, but unfortunately not prevent execution of malicious build scripts exactly for the reason you gave. It could still help if you only ever use cargo add and update in a sandbox.
I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
This was actually the main thing that put me off of Rust. I get the argument for a small std lib. I just also don’t agree it’s worth it. Go seems to handle having a batteries included standard lib just fine.
Doing software development outside of strict containerization, at the very least, looks increasingly prone to disaster.
Yes, we can argue about the culture of package management (as some of us have with especially npm from day one), but it's done, and your colleagues or AI sidekicks cannot be trusted not to download whatever and try to build and run it. All you can do is limit the effective blast radius.
We need effect based languages now. It's the only way to guarantee policies like no network, no file access, no unsafe code or FFI for a library before it even compiles. If anyone from epic is reading this please give us a timeline for open sorucing the Verse compiler.
In the mean time I think it's possible to hack Cargo and run all build scripts in a microVM. The blast radius will be limited to malicious code in the binary instead of uploading all your CI secrets and deleting the whole hard drive.
> It's the only way to guarantee policies like no network, no file access, no unsafe code or FFI for a library before it even compiles.
It's not the only way. You can sandbox processes. I think sandboxing is the much more reasonable approach, because in the end of the day, there are still closed-source software products where you can't demand that the manufacturers use certain language safety features.
Sandboxing the process only works well when the malware requires more capabilities than the software itself.
So if your software needs to make HTTP requests and read the filesystem then the malware will be able to make HTTP requests and read the filesystem, which is enough for a ton of malware. Sure, maybe you can limit the directories it can access a bit and possibly some sort of network filtering, but it isn't a silver bullet.
Capability security in-language would make a huge difference, because the more granular you "sandbox" the less likely it is that the compromised component has the access it wants. For example if the HTTP client library is compromised maybe it can't access the filesystem so can't steal your cookies. Or maybe like in this case no permissions were needed at all and despite the process having enough capabilities for malware this malware can at worst return bad values and try to chain this to an exploit which is far more difficult than just running it itself.
I don’t even understand what a capability based language is, but presumably closed source software wouldn’t apply here? Is it common for closed source software to give the source code to customers to build?
> Is it common for closed source software to give the source code to customers to build?
Yes, somewhat common. I work in embedded, and where a supplier gives us the ability to build their code it is a lot easier for us to deal with the next time we need a new build from them.
This is why I'm building this language. It's capabilities based. Build scripts can't just do whatever the hell they want to, everything is auditable, and it layers its sandboxing.
It's for fun and anyone looking should understand that this is AI driven building with human driven design, but the goal is to demonstrate.
I think the only language that's used in production where malicious dependencies can't do arbitrary effects is https://roc-lang.org - but it's a pre-0.1.0 language and as such its production usage is extremely minimal for now.
Why does every build have the ability to run arbitrary code by default in the first place? That seems like something you should opt into only under very specific scenarios and even then it should probably be built around a WASM sandbox or microVM (as the parent suggested).
Because many many builds end up doing something just a little weird that the build system cannot handle by default. Ideally the build system would be fully property/dependency based and so it wouldn't have to run arbitrary code. In the real world everybody has something weird about their build that the build system cannot make work.
This is partially because for nearly every project the build system is something they need and don't care about. When they need something weird they hack just enough to make it work and never ask "how should the build system change so that this was a property instead of running code", and thus build systems are slow to improve. In a number of cases the build system did have a way to do that thing, but the person didn't know about it.
There are also a lot of code generators out there. I have yet to see a large project which didn't have their own code generator for something specific to their project (protobuf is an example from Google that has escaped and become useful elsewhere, but there are many others that are specific to one project. Yacc is from the 1970s, and stands for "yet another compiler compiler" - implying the idea was already common 50 years ago). You cannot have/use these useful tools without running arbitrary code.
I don't mind code execution as even make has shell scripting embedded, same thing with meson. My issue is with network access to download random stuff without it being declared somewhere and signed.
I'm using OpenBSD and the ports system set up two users `_pbuild` and `_pfetch` for building packages. The first one is for building and the sample `pf.conf` (firewall) forbid it from accessing the network. The second does fetch the files , but it's declarative with every files listed and signed. Even for languages like go and rust.
Fetching files while building is the bad idea there.
Good point. Many build systems also want to be a package manager, but the two are and should be separate. If you have a simple problem it makes things easy to combine them into one. However they need to be separate anyway, both for security and also to make other weird situations easier.
First we need capability-based OSes, like we should have had decades ago if worse-is-better hadn't stuck us with Unix. I don't need to care whether or not a program was written in a capability-aware language if the OS fundamentally takes care of that for me.
MicroVM this, effects that --- can we discuss security without needing to invoke bay area buzzwords?
The idea is "least privilege", and we desperately need it in computing. The precise technical mechanism we use to achieve it is less important than committing to the idea that a dependency doesn't run with full privilege of its host program and an install script doesn't run with full privilege of the programmer.
The original Linux seccomp is old enough to drink. It's always been possible to do things like expand macros in a no-IO environment. Nobody's bothered to do it over the past two decades. Why would anyone bother in the next two decades?
The technical mechanism is exactly the issue to figure out, there's a reason why projects don't have this and it's because different implementations have different tradeoffs.
No, the reason people haven't been doing this -- and we've had technologies for ages -- is that it's a huge pain in the ass, a "tax", that it's hard to get developers inside a company to pay, much less participants in open source ecosystems.
Look at how snap, flatpak, etc. provoke people to just turn off security rather than deal with breakages.
A new language won't help because the problem is social, not technical.
FFS, you can't even get people to use filesystems via intermediate objects in today's languages. People think your language is broken if you don't have an ambient open(). You don't need a new language to enforce capability discipline. You need to whack people repeatedly with a cluebat until their laziness and brain damage abate.
You can't simultaneously say that we've had the tech for years and also say that it doesn't work. We have not had the tech for years. x-plat sandboxing is extremely difficult, especially in a way that's performant.
Sandboxing almost always involves having a dedicated, privileged service, due to how operating systems have designed things. It requires platform specific code.
This is a technical problem and a social problem, there's zero reason to believe it's just one.
The technology works fine if you use it in a disciplined way. The problem is that people don't! You're necessarily going to break programs if you put them in restricted environments. That's not the same as the restriction technology not working properly.
> Sandboxing almost always involves having a dedicated, privileged service, due to how operating systems have designed things.
Or just invoking bwrap. Or sandbox-execute. There's no performance overhead. Complexity is minimal. Just read current Codex source code. It's not so bad. People have been making these sandbox tools for AI agents for years. Just need to apply sandboxing to all domains.
You want some kind of silver bullet that makes code "safe" without anyone having to change anything? To prompt for nothing? To use no IPC portal? That's not happening.
The people saying we need a new language or something are half right. We don't need a new language. A pure library solution is fine! But people do need to do work to make an ecosystem based on capabilities and least privilege to work. And we need to tell people who refuse to do this work to go to hell.
A new language is neither necessary nor sufficient. What we need is new set of balls.
> The technology works fine if you use it in a disciplined way.
Then obviously we have to discuss the implementation!
> Or just invoking bwrap. Or sandbox-execute. There's no performance overhead. Complexity is minimal. Just read current Codex source code. It's not so bad. People have been making these sandbox tools for AI agents for years. Just need to apply sandboxing to all domains.
Tools like bwrap literally could not have existed until very recently without also requiring suid/ privileges, and even today unprivileged user namespaces are not universally enabled. This is why the implementation matters. No, unprivileged sandboxing has not been around for years, especially not x-plat. bwrap is a perfect example of what I'm talking about, great reference - it either requires root or it requires unprivileged user namespaces and it's not x-plat. Great.
> You want some kind of silver bullet that makes code "safe" without anyone having to change anything? To prompt for nothing? To use no IPC portal? That's not happening.
I have no idea why you think I've said this, I'm pointing out that the implementation and technology matters deeply.
> The people saying we need a new language or something are half right. We don't need a new language. A pure library solution is fine! But people do need to do work to make an ecosystem based on capabilities and least privilege to work. And we need to tell people who refuse to do this work to go to hell.
Again, you can't say "this technology has existed for years! You can do it in a library!" and then say "but they don't!". You misunderstand the complexity and assert that it's merely a matter of will. It is both.
Browsers have been sandboxing their tabs since 2008. Don't sit there and tell me that bwrap, sandboxing, whatever is some kind of recent innovation with uneven distro support.
And yes, bwrap needs root or namespaces. So what? So does sudo. Setuid binaries are as old as time.
You know what's also been around and doesn't require root? Landlock. Have you heard of it? Has anyone? There's a huge PR and visibility gap in this space. We don't have to invent some new thing. We have to get people to learn about what we already have and use it.
"Oh, the technology works, but it need setup! Oh, my distro doesn't enable it! Oh, it doesn't work out of the box with my code!"
This is a social problem. It's not a technology problem. It's people not wanting to do the work, not understanding what already exists, dismissing solutions based on non-problems (like bwrap needing privileges on some systems), and in general adopting an attitude of "no", not a can-do problem solving stance.
And we're supposed to solve this problem with let another precious little effects language or another fucking MicroVM environment? FFS.
Rust suffers from the same faults as the JS ecosystem. Any significant crate imports hundreds if not thousands of dependencies. The probability that one of the authors gets targeted by AI-assisted attacks is just too high.
Also most of these dependencies provide a breadth of features that the end package does probably not need.
My experience has been that it has a major advantage, in that freeze + offline actually work properly. You can collect the dependencies you need once, put them in version control and never ever talk to remote registry again
I mean, yes, update your dependencies. But probably after a week or so after they've been release and tested by the first penguins willing to jump into the ocean.
Holding periods for new updates are a good idea, but I would also prefer having a tool that could provide diffs of the entire project state pre- and post-update; including transitive dependencies being added or removed. Git diffs won't show you that information, by design.
Are there any plans to more seriously develop the standard library in Rust? Or is the plan to remain in this status quo where users of Rust import nonsense and the dependency tree explodes (or users are forced to invent their own wheel?).
Are there any comparisons between the state of the stdlib in C++ vs. Rust? I’d think that would serve as an excellent jumping off point to start chipping away.
The idea that Rust has a small standard library is a weird meme. Rust has an enormous standard library. Look at any Rust release and you'll see dozens of new library APIs added, and consider that Rust has about nine releases a year, meaning that Rust adds over a hundred APIs per year, and has consistently for the past ten years. Rust frequently adds things that obviate popular crates, most recently the cfg_if crate was made redundant by the new cfg_select macro. And half of every dependency graph that people wrong their hands over are actually first-party external crates provided by either the Rust organization itself or by known contributors to the project. When people say that Rust has a small standard library, mostly they seem to just mean that it doesn't include a webserver.
Summary: it's cultural. Rust likely inherited the practice from Nodejs, who inherited it from Ruby. I think in Rust online spaces in particular there is also this undercurrent of "you're not smart enough to use certain parts of the language, so download libraries that handle that stuff for you."
The language also takes backwards compatibility very seriously, anything that goes into std "must be maintained forever". It is also argued that a large stdlib means the maintainers have less time to work on the language itself.
I understand not every language can have Go's amazing stdlib, but I would much prefer Pyhton's approach where every now and then some package/function from the stdlib gets deprecated/removed. Rust's 3rd party ecosystem is the worst thing from the language, worse than the compile times.
i'd like to welcome you to the hell that is c/c++ dependency management. Make? cmake? qmake? conf? autoconf? configure? autotools? submodules??? AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
I'd just select a tool, vendor the libraries I need to the codebase and call it a day. Did these for Eigen, liboption++, Catch2, Easylogging++ (now archived and not maintained anymore, sadly).
Build a simple makefile, and you're off to the races.
None of those (other than submodules which is just vendoring) does dependency management. They are more configuration management than anything. C, Python, Perl (and maybe ruby?) relies on flag switches and environment variables to find all the necessary files and modules for compiling/running a script. Cmake and autoconf just configure those.
With NPM and Rust's focus on project's level dependencies, there's no longer emphasis on API stability. Instead we have breakage every months, forcing everyone on the upgrade treadmill. It's easier to audit C library because they focus mostly on security updates instead of redesigning the API for the nth time.
The "npm-ness" of Cargo (centralized and standardized dependency management used at every opportunity) is generally pitched as one of the primary developer experience advantages over C++.
Rust as a language is mostly alright in my opinion. The problems I have with it are similar to the problems I have with C++, and it makes up for them in other really compelling ways.
Where it loses me is Cargo and everything surrounding it. I'm essentially forced into an extreme where I just never use anything in the Rust ecosystem, or I have to deal with insane dependency graphs that have the density and microstate complexity of a neutron star.
As a person who doesn't like Rust very much, no, it doesn't suck. Some of its features make some folks very excited for very right reasons, and evokes "Silver Bullet Syndrome" in others for all the wrong reasons.
People weaponizing Rust rewrites with permissive licenses is another problem, but it's not about the programming language itself.
Rust is actively incorporating more functionality into the stdlib. The functionality of this crate has been in std since 2024. The ecosystem is just slow to update (not everything is maintained, etc).
There are two reasons you might want to use a crate, The first being that you want to use a good solid implementation that you know someone has spent more time doing and works better than almost any solution you could integrate. The second is you don't want to spend time implementing that.
Writing macros in rust is a pretty horrible experience but it's not difficult
The languages that have a poor standard library support have this issue and other languages encourage you to import tons of libraries to fix the problem.
This is why Javascript and Typescript suffer from this the most and has little to nothing to do with "popularity" and likely 9/10 of these npm packages import an external library.
Golang on the other-hand is just as popular and has a stronger standard library which people build against and it is encouraged to use its standard library rather than rolling your own or importing another package to solve the problem.
Rich official standard library vs "import tons of libraries" are not the only two options.
Java's standard library had arguably also been poor for a very long time and "import tons of libraries" just had not been practical for most of that time because the tooling and ecosystem for that did not exist yet.
The solution was apache-commons and guava. Two large libraries with everything the developers heart desired and well maintained by large organizations.
For Rust be probably will never have anything exactly like that because requirements from no-std development to fully fledged backend service are too diverse, but there is still room for a small number of well
maintained backed by reputable developers convenience libraries in my opinion.
Go has a stronger standard library for certain use cases like basic CRUD web applications, but a lot of the Go standard library is also extremely low quality (flag, container, image, json, log, math, path, regexp, sync, time). Many of these aren't usable outside of toy use cases and have weird edge cases all over the place. Over time many will probably get new incompatible versions just like json. The container package in Go is the worst collection library in any mainstream programming language by a huge margin. It's actually astounding how bad it is.
I don't want to see a large standard library for Rust. If something is added to the standard library, then it is very difficult to change it afterwards because backwards compatibility.
It would be better to have blessed crates in crates.io. The Rust core team would release or audit them. If the blessed crates need breaking changes, it can be done by increasing their major semantic version number. That can't be done to the standard library.
Actually, there could be a "trust" level for crates: 1. blessed crates by the Rust core team, 2. trusted developers, 3. untrusted developers. Or something like that..
I've never seen a non trivial Python or Go project without external dependencies. The dependency tree of comparable Go and Rust projects seem comparable, IMO.
I don't think it's just that, though I agree they are clearly correlated.
The reason I don't think it's a sufficient explanation is that there is a clear history of large, 3rd party libraries being created exactly to supplement poor standard libraries. C++ has Boost, Java has Apache Commons (though Java also has a pretty huge standard library), arguably we could even say C has Posix/Win32/Cocoa.
I believe there is some deeper cultural reason why certain language ecosystems coalesce large utility libraries, while others prefer myriad tiny dependencies.
The important difference IMO is in the small vs big libraries culture.
Lots of languages have a bad stdlib but don’t fall into the trap of having thousands of micro libraries.
The reason people do it is because it brings clout and money. Just look for articles defending micro libs: the popular ones are by people who make a living on donations, due to maintaining 1000+ packages.
And collaborating in larger libs/stdlib is hard. Plus: Rust, Node, all have a lot of visibility.
You need a good stdlib culture to avoid it (like Go did).
The reason is simple: the package manager works well. Helps if the package manager is standardized and there is a "default" registry for open source projects.
I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.
I don't think this is true. People like to scapegoat the JS standard library but in reality most JS implementations have pretty featureful standard libraries, especially browsers. I've never felt the need to use any third-party libraries in the javascript projects I've worked on (except for maybe `ws`).
Oh, so it's not only "JavaScript bad and npm bad". Apparently, if your language uses third party dependency registry, you are prone to malicious code, regardless if it's Javascript or not.
Use containers for development. And reduce the amount of third party deps you import into your projects. This is only going to get worse.
Pulling in lots of dependencies creates this kind of risk regardless of the ecosystem. That being said in the JS/NPM world you tend to have a LOT more dependencies (especially indirect ones) than other languages. I saw someone do a cursory analysis and JS/Node projects tend to have 5x the number rust or ruby projects.
This is really a cultural problem not a technical one.
If it weren't a registry it would be ./configure scripts and makefiles. The issue is that sandboxing technology is kinda shit (especially x-plat) and languages don't build it in by default.
Just wondering, whenever things go wrong with Rust - why do you all (Rust devs) point the finger at JavaScript?
It’s the weirdest form of projection. More than 1 person is doing it here. It’s like the second Rust hits a failure mode you immediately go “well JavaScript is worse!”
Like that has anything to do with your failure (even if it was true)
> Just wondering, whenever things go wrong with Rust - why do you all (Rust devs) point the finger at JavaScript?
I know it is almost a sport to point fingers at the "evil rust evangelists" on HN at this point, but a quick look at the fnoef's comment history would show you that they are not a rust person.
Your account, on the other hand, is a sock puppet created specifically to bitch about rust. Pot, meet kettle?
You're literally replying to someone saying npm is unfairly called out.
Rust developers aren't the ones who have problems with it. Otherwise they likely wouldn't be using Rust in the first place. I don't see anyone saying JavaScript is worse about it either, more that they're the same.
Cargo (and PyPI) is undeniably better than NPM, which is just shockingly bad for cultural reasons. Yet it's not safe, and it's subject to the same class of exploit, as we're seeing.
Indeed, the solution is to get away from the wild soup of author-managed dependencies and go with something with an audited collection of software that is maintained by separate human beings from the known-vulnerable hackers writing the software.
And indeed Go and .NET and Java all qualify. But the gold standard here is Debian and all its downstreams.
> the solution is to get away from the wild soup of author-managed dependencies and go with something with an audited collection of software that is maintained by separate human beings from the known-vulnerable hackers writing the software.
I think we might be able to crowdsource audits. At least in the Rust ecosystem I'm confident that this is feasible with the right tooling.
It usually takes some time for an updated dependency to actually get shipped to users in a release, by which time there's a good chance the attack has been noticed.
How do we know they don't? Who is going to read the source for all these micro packages?
You might assume we are able to detect all malicious behavior at runtime. But "stuxnet" and more recently, the xz compromise say otherwise. What if a not-so-popular crate deep in the dependency chain subtly introduced a LPE in it's code?
> Why do none of these hijacks embed runtime attacks? It seems like worming the build machines is the goal, rather than compromising downstream users.
Developer machines are quite juicy targets. They tend to have all sorts of credentials lying around, so it often isn't too difficult to escalate from that to compromising AWS/GCP/etc.
On top of that there's very little preventing a compromise. Developers are inherently expected to run untrusted code, and the usual Linux / MacOS laptop probably isn't even running any kind of anti-virus protection. Want to compromise the downstream app? Now you also need to pass Play Protect & friends.
> It seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
Yes, we really should. It is frankly a miracle that it has taken so long for fetch-time / install-time code execution to start blowing up in our faces. If we are spending so much effort on run-time isolation, why are we completely ignoring all those practices during development?
Why are developers "inherently expected to run untrusted code"? Running untrusted code on developer machines seems like a terrible idea, given that it will compromise everything they build or deploy and (as you mentioned) all their credentials.
Unfortunately Cargo doesn’t have security controls in place to prevent these kinds of attacks. For example pnpm has controls to allowlist install scripts for dependencies and will warn about new install scripts (without executing them).
Compromising the code that is then most likely run in a test instead of compromising a build script is just a very slight inconvenience for the attacker.
I share the dislike for arbitrary build scripts but restricting them will not help the supply chain issue in a significant way.
Also there are several ways to control build.rs execution in the Cargo ecosystem as well, for example with cargo-deny.
Disallow lists are ineffective - better to disallow by default and require opt in.
Also, crates.io can defer serving up newly uploaded scripts that have a new build.rs / proc-macro dependency and warn publicly that a version introduces it.
Restricting build scripts 100% will help mitigate the impact, just not if you only deny it once. And they can develop other things like sandboxing for build scripts by default and escaping that to be the exception that has to be explicitly allowed.
You’re right about attackers being able to change runtime code.
pnpm does have some other features to prevent supply chain attacks, so there is still something to learn from other ecosystems. For example pnpm has a cooldown period for new dependencies and can prevent trust policy downgrades (eg new version published without build provenance where older versions did have it). See https://pnpm.io/supply-chain-security
Jumping off the title and ignoring contents of post, as is customary:
look at this graph and guess which languages use each number of dependencies for their website: https://www.youtube.com/watch?v=E82ly38YEEQ&t=325s
I won't spoil the claim in the video about what the dependency number is correlated with, and I'm not even sure how true it is in general, but it's very interesting.
These very small dependencies that are then later causing issues either due to malicious nature or incompetence, have become pervasive in computing (for some reason). I think that these should be less of an issue now than ever. Outside of the largest, most critical dependencies, you really shouldn't be pulling in small libraries anymore. Just generate the code via AI. AI is not great at large scale programming I think, but its amazing at snippets of code. Something I ran into recently, I needed to use FFT2 on some matrices, and what I was using didn't have an existing solution. Converting some numpy fft2 tests to my target language, and having a full native implementation of fft2, and an accompanying test suite so it will behave exactly like numpy fft2. A few minutes and a few thousand lines of code later, I have a trusted implementation. Saves me an external dependency, some weird glue code, and an attack vector.
oof, not rust too. this has been brutal for the node ecosystem, hopefully cargo also gets the proper controls to help avoid this, like the in-flight min-publish-age
I’m disappointed crates.io doesn’t have a stricter bar for serving a crate that has newly acquired a proc macro or build.rs. That seems like a trivial mitigation.
Compromising the code that is then most likely run in a test instead of compromising a build script is just a very slight inconvenience for the attacker.
I'm not particularly fond of arbitrary build scripts either, but restricting them will not help the supply chain issue in a significant way.
Also there are several ways to control build.rs execution in the Cargo ecosystem, for example with cargo-deny.
Defaults matter. It’s nice you can set this up using a plugin to protect yourself, but that doesn’t protect the ecosystem, most of which doesn’t use cargo deny.
I also disagree a build scripts is a mild convenience. A build script always runs for anyone it’s a dependency for with full access and context and often has access to secrets in CI. A compromised runtime has more limited access and requires actual invocation of code paths (if you’re lying as a dependency that’s never executed, no exploit).
Of course Rust should have language-level support for capabilities so that just invoking a function doesn’t grant it access to arbitrary disk access. But that’s a much more difficult change than tweaking the defaults for cargo.
So? Runtime code requires actually executing the malicious code path which isn’t an immediate 100% hit rate for everyone that includes it in the dependency chain. For build.rs it’s a 100% compromise of everyone it’s in the dependency chain for. Additionally, at runtime you may not have access to secrets whereas at build time you most certainly do.
Why this still happens? Why after many previous supply-chain attacks maintainers of package repositories still allow anyone uploading packages and pushing updates without security audit?
Who is funding this security audit? Are folks supposed to volunteer their free time? It's a difficult coordination problem. The best folks have come up is to delay adopting new releases by a few days and hope your dependency is popular enough that a security firm audits it for you in that timespan. If you have enough money I suppose you can start employing llms to audit things for yourself.
Languages like Rust have sources of income to be able to finance such audit.
You don't need to audit all the crap is being uploaded right now. Only really necessary and widely-used packages should be managed in a centralized way, so, auditing all of them isn't that huge task.
Well, it depends on the language: language which "capabilities" (both for the source code and for the building* ) could in theory really reduce a lot of the burden to identify supply chain attacks.
*: some research language have/had capabilities which would make supply chain attack "obvious" but for build systems I don't know if this exist.
> Who is funding this security audit? Are folks supposed to volunteer their free time?
Same people who keep the whole rust project going, a lot of those are volunteers aren't they? Not mad to think they could do the same for core packages at least
> Same people who keep the whole rust project going, a lot of those are volunteers aren't they?
Sure, but from my understanding the Rust project is generally "bottom-up" in that volunteers generally work on what they want to rather than submit their time into a pool for some kind of higher-level management to direct.
It’s absolutely mad and extremely entitled to expect that a volunteer group of developers do an order of magnitude or more additional work for no additional pay or benefits to themselves.
Not really, if you're putting out something like programming languages and tooling, people expect them to work.
Especially because Rust devs brag so much about how it's soo superior to everything else, but then these amateur mishaps happen.
Rust isn't getting the exposure it deserves, I think, partly due to arrogance within the Rust community and a mental complex about "being better than everyone else" - that mentality never works
Mozilla, Google and a couple of others are publishing their audits through cargo vet. There are also additional audits done by individuals you can use through cargo crev. Overall the number of audited crates is in the thousands and you will find audits for most of the popular crates.
In the end it is your decision to use unaudited or refuse unaudited crates.
> In the end it is your decision to use unaudited or refuse unaudited crates.
It should be the default behavior of the package manager to allow downloading only audited/trusted packages. Forcing end-users of the language to be responsible for audit of all dependencies is impractical.
The author proposes to reinvent the wheel and depend on stale dependencies, which are EOL. Neither of that is an acceptable solution with LLM-based Agents being able to produce exploit(ExploitGym) chains in minutes from known bugs. The other issue is that Rust's forces the user to provide more information and APIs are usually kept generic for systems programmer, so standardizing things is not as straight-forward compared to Go where you can assume a memory-management, a virtual thread runtime and mostly ignore dynamic dispatching.
The bad package version has also just disappeared from crates.io [2] with no indication its been yanked. There's no security advisory there either [3] "No advisories found for this crate."
I feel crates.io was unprepared for a security incident like this since they're managing the response [4]
[1]: https://web.archive.org/web/20260820145918/https://github.co...
[2]: https://crates.io/crates/arrayref/versions
[3]: https://crates.io/crates/arrayref/security (I'd give an Wayback link but that's also broken https://web.archive.org/web/20260820150747/https://crates.io...)
[4]: https://github.com/rustsec/advisory-db/issues/3161#issuecomm...
Google should read this too. They simply remove Android apps and Chrome extensions without a single word. No page explaining why they removed it.
¹ https://rust-lang.github.io/goals/2024h2/sandboxed-build-scr...
Here is my learning building PMG:
Sandboxing is good when the workload is predictable, and the goal of sandbox is to guard against exploitation of vulnerabilities, like sandbox protecting chrome tabs (renderers). But unfortunately build scripts are not predictable, at least not in npm/pypi world and I have seen build scripts doing weirdest of the things which is no different from malware. When popular packages do weird things, build breaks and users end up turning off the sandbox. This is a perpetual problem to deal with while building sandbox (or any least privilege solution) to protect unbounded workloads.
https://github.com/safedep/pmg
It isn't so much a question of sandboxing build.rs, as fundamentally changing the way that foreign dependencies are integrated into the rust toolchain (i.e. moving from a rust-centric system like Cargo to something more general like buck2)
(Oh and btw, proc macros also run arbitrary code.)
Cargo, please PLEASE give me a way to disable third-party build.rs and whitelist the ones I need. And please loudly mark any update that adds a build.rs where there was none before.
See https://embarkstudios.github.io/cargo-deny/checks/bans/cfg.h...
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
Direct post link: https://blog.rust-lang.org/2026/08/20/supply-chain-attack-on...
Initial report: https://github.com/rustsec/advisory-db/issues/3161
Other vendor posts:
* https://www.stepsecurity.io/blog/arrayref-rust-crate-supply-...
* https://research.jfrog.com/post/arrayref-proc-macro1-crates-...
* https://www.aikido.dev/blog/two-popular-rust-crates-arrayref...
Yes, we can argue about the culture of package management (as some of us have with especially npm from day one), but it's done, and your colleagues or AI sidekicks cannot be trusted not to download whatever and try to build and run it. All you can do is limit the effective blast radius.
Many things I run I want to limit to r/w a single dir, and to have to request permission to make network calls.
In the mean time I think it's possible to hack Cargo and run all build scripts in a microVM. The blast radius will be limited to malicious code in the binary instead of uploading all your CI secrets and deleting the whole hard drive.
It's not the only way. You can sandbox processes. I think sandboxing is the much more reasonable approach, because in the end of the day, there are still closed-source software products where you can't demand that the manufacturers use certain language safety features.
So if your software needs to make HTTP requests and read the filesystem then the malware will be able to make HTTP requests and read the filesystem, which is enough for a ton of malware. Sure, maybe you can limit the directories it can access a bit and possibly some sort of network filtering, but it isn't a silver bullet.
Capability security in-language would make a huge difference, because the more granular you "sandbox" the less likely it is that the compromised component has the access it wants. For example if the HTTP client library is compromised maybe it can't access the filesystem so can't steal your cookies. Or maybe like in this case no permissions were needed at all and despite the process having enough capabilities for malware this malware can at worst return bad values and try to chain this to an exploit which is far more difficult than just running it itself.
Yes, somewhat common. I work in embedded, and where a supplier gives us the ability to build their code it is a lot easier for us to deal with the next time we need a new build from them.
https://hale-lang.org/proof/
This is why I'm building this language. It's capabilities based. Build scripts can't just do whatever the hell they want to, everything is auditable, and it layers its sandboxing.
It's for fun and anyone looking should understand that this is AI driven building with human driven design, but the goal is to demonstrate.
https://github.com/brianv0/formwork
This is partially because for nearly every project the build system is something they need and don't care about. When they need something weird they hack just enough to make it work and never ask "how should the build system change so that this was a property instead of running code", and thus build systems are slow to improve. In a number of cases the build system did have a way to do that thing, but the person didn't know about it.
There are also a lot of code generators out there. I have yet to see a large project which didn't have their own code generator for something specific to their project (protobuf is an example from Google that has escaped and become useful elsewhere, but there are many others that are specific to one project. Yacc is from the 1970s, and stands for "yet another compiler compiler" - implying the idea was already common 50 years ago). You cannot have/use these useful tools without running arbitrary code.
I'm using OpenBSD and the ports system set up two users `_pbuild` and `_pfetch` for building packages. The first one is for building and the sample `pf.conf` (firewall) forbid it from accessing the network. The second does fetch the files , but it's declarative with every files listed and signed. Even for languages like go and rust.
Fetching files while building is the bad idea there.
The idea is "least privilege", and we desperately need it in computing. The precise technical mechanism we use to achieve it is less important than committing to the idea that a dependency doesn't run with full privilege of its host program and an install script doesn't run with full privilege of the programmer.
The original Linux seccomp is old enough to drink. It's always been possible to do things like expand macros in a no-IO environment. Nobody's bothered to do it over the past two decades. Why would anyone bother in the next two decades?
Look at how snap, flatpak, etc. provoke people to just turn off security rather than deal with breakages.
A new language won't help because the problem is social, not technical.
FFS, you can't even get people to use filesystems via intermediate objects in today's languages. People think your language is broken if you don't have an ambient open(). You don't need a new language to enforce capability discipline. You need to whack people repeatedly with a cluebat until their laziness and brain damage abate.
Sandboxing almost always involves having a dedicated, privileged service, due to how operating systems have designed things. It requires platform specific code.
This is a technical problem and a social problem, there's zero reason to believe it's just one.
> Sandboxing almost always involves having a dedicated, privileged service, due to how operating systems have designed things.
Or just invoking bwrap. Or sandbox-execute. There's no performance overhead. Complexity is minimal. Just read current Codex source code. It's not so bad. People have been making these sandbox tools for AI agents for years. Just need to apply sandboxing to all domains.
You want some kind of silver bullet that makes code "safe" without anyone having to change anything? To prompt for nothing? To use no IPC portal? That's not happening.
The people saying we need a new language or something are half right. We don't need a new language. A pure library solution is fine! But people do need to do work to make an ecosystem based on capabilities and least privilege to work. And we need to tell people who refuse to do this work to go to hell.
A new language is neither necessary nor sufficient. What we need is new set of balls.
Then obviously we have to discuss the implementation!
> Or just invoking bwrap. Or sandbox-execute. There's no performance overhead. Complexity is minimal. Just read current Codex source code. It's not so bad. People have been making these sandbox tools for AI agents for years. Just need to apply sandboxing to all domains.
Tools like bwrap literally could not have existed until very recently without also requiring suid/ privileges, and even today unprivileged user namespaces are not universally enabled. This is why the implementation matters. No, unprivileged sandboxing has not been around for years, especially not x-plat. bwrap is a perfect example of what I'm talking about, great reference - it either requires root or it requires unprivileged user namespaces and it's not x-plat. Great.
> You want some kind of silver bullet that makes code "safe" without anyone having to change anything? To prompt for nothing? To use no IPC portal? That's not happening.
I have no idea why you think I've said this, I'm pointing out that the implementation and technology matters deeply.
> The people saying we need a new language or something are half right. We don't need a new language. A pure library solution is fine! But people do need to do work to make an ecosystem based on capabilities and least privilege to work. And we need to tell people who refuse to do this work to go to hell.
Again, you can't say "this technology has existed for years! You can do it in a library!" and then say "but they don't!". You misunderstand the complexity and assert that it's merely a matter of will. It is both.
And yes, bwrap needs root or namespaces. So what? So does sudo. Setuid binaries are as old as time.
You know what's also been around and doesn't require root? Landlock. Have you heard of it? Has anyone? There's a huge PR and visibility gap in this space. We don't have to invent some new thing. We have to get people to learn about what we already have and use it.
"Oh, the technology works, but it need setup! Oh, my distro doesn't enable it! Oh, it doesn't work out of the box with my code!"
This is a social problem. It's not a technology problem. It's people not wanting to do the work, not understanding what already exists, dismissing solutions based on non-problems (like bwrap needing privileges on some systems), and in general adopting an attitude of "no", not a can-do problem solving stance.
And we're supposed to solve this problem with let another precious little effects language or another fucking MicroVM environment? FFS.
You can't solve a social problem with technology.
Also most of these dependencies provide a breadth of features that the end package does probably not need.
Are there any comparisons between the state of the stdlib in C++ vs. Rust? I’d think that would serve as an excellent jumping off point to start chipping away.
I won't mention lack of date/time lib because that's complex and changes often.
That's why projects end up with 100s of crates, sometimes 1000s.
Discussion: https://news.ycombinator.com/item?id=49372853
Why do so many languages fall into this horrible practice?
https://www.youtube.com/watch?v=E82ly38YEEQ
Summary: it's cultural. Rust likely inherited the practice from Nodejs, who inherited it from Ruby. I think in Rust online spaces in particular there is also this undercurrent of "you're not smart enough to use certain parts of the language, so download libraries that handle that stuff for you."
I understand not every language can have Go's amazing stdlib, but I would much prefer Pyhton's approach where every now and then some package/function from the stdlib gets deprecated/removed. Rust's 3rd party ecosystem is the worst thing from the language, worse than the compile times.
Build a simple makefile, and you're off to the races.
At least, for my cases, that is.
It never ends .·°՞(っ-ᯅ-ς)՞°·.
With NPM and Rust's focus on project's level dependencies, there's no longer emphasis on API stability. Instead we have breakage every months, forcing everyone on the upgrade treadmill. It's easier to audit C library because they focus mostly on security updates instead of redesigning the API for the nth time.
Where it loses me is Cargo and everything surrounding it. I'm essentially forced into an extreme where I just never use anything in the Rust ecosystem, or I have to deal with insane dependency graphs that have the density and microstate complexity of a neutron star.
> Where it loses me is Cargo and everything surrounding it.
High praise!
What? Its just a programming language. Go dependency free if you want. Or vendor everything. Nobody is forcing you to pull in 3rd party dependencies.
People weaponizing Rust rewrites with permissive licenses is another problem, but it's not about the programming language itself.
Writing macros in rust is a pretty horrible experience but it's not difficult
This is why Javascript and Typescript suffer from this the most and has little to nothing to do with "popularity" and likely 9/10 of these npm packages import an external library.
Golang on the other-hand is just as popular and has a stronger standard library which people build against and it is encouraged to use its standard library rather than rolling your own or importing another package to solve the problem.
Java's standard library had arguably also been poor for a very long time and "import tons of libraries" just had not been practical for most of that time because the tooling and ecosystem for that did not exist yet.
The solution was apache-commons and guava. Two large libraries with everything the developers heart desired and well maintained by large organizations.
For Rust be probably will never have anything exactly like that because requirements from no-std development to fully fledged backend service are too diverse, but there is still room for a small number of well maintained backed by reputable developers convenience libraries in my opinion.
It would be better to have blessed crates in crates.io. The Rust core team would release or audit them. If the blessed crates need breaking changes, it can be done by increasing their major semantic version number. That can't be done to the standard library.
Actually, there could be a "trust" level for crates: 1. blessed crates by the Rust core team, 2. trusted developers, 3. untrusted developers. Or something like that..
The reason I don't think it's a sufficient explanation is that there is a clear history of large, 3rd party libraries being created exactly to supplement poor standard libraries. C++ has Boost, Java has Apache Commons (though Java also has a pretty huge standard library), arguably we could even say C has Posix/Win32/Cocoa.
I believe there is some deeper cultural reason why certain language ecosystems coalesce large utility libraries, while others prefer myriad tiny dependencies.
Lots of languages have a bad stdlib but don’t fall into the trap of having thousands of micro libraries.
The reason people do it is because it brings clout and money. Just look for articles defending micro libs: the popular ones are by people who make a living on donations, due to maintaining 1000+ packages.
And collaborating in larger libs/stdlib is hard. Plus: Rust, Node, all have a lot of visibility.
You need a good stdlib culture to avoid it (like Go did).
I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.
Use containers for development. And reduce the amount of third party deps you import into your projects. This is only going to get worse.
This is really a cultural problem not a technical one.
Just wondering, whenever things go wrong with Rust - why do you all (Rust devs) point the finger at JavaScript?
It’s the weirdest form of projection. More than 1 person is doing it here. It’s like the second Rust hits a failure mode you immediately go “well JavaScript is worse!”
Like that has anything to do with your failure (even if it was true)
Edit: 7 times and counting
I know it is almost a sport to point fingers at the "evil rust evangelists" on HN at this point, but a quick look at the fnoef's comment history would show you that they are not a rust person.
Your account, on the other hand, is a sock puppet created specifically to bitch about rust. Pot, meet kettle?
Rust developers aren't the ones who have problems with it. Otherwise they likely wouldn't be using Rust in the first place. I don't see anyone saying JavaScript is worse about it either, more that they're the same.
Why are you getting so defensive about it?
Indeed, the solution is to get away from the wild soup of author-managed dependencies and go with something with an audited collection of software that is maintained by separate human beings from the known-vulnerable hackers writing the software.
And indeed Go and .NET and Java all qualify. But the gold standard here is Debian and all its downstreams.
I think we might be able to crowdsource audits. At least in the Rust ecosystem I'm confident that this is feasible with the right tooling.
It seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
You might assume we are able to detect all malicious behavior at runtime. But "stuxnet" and more recently, the xz compromise say otherwise. What if a not-so-popular crate deep in the dependency chain subtly introduced a LPE in it's code?
Developer machines are quite juicy targets. They tend to have all sorts of credentials lying around, so it often isn't too difficult to escalate from that to compromising AWS/GCP/etc.
On top of that there's very little preventing a compromise. Developers are inherently expected to run untrusted code, and the usual Linux / MacOS laptop probably isn't even running any kind of anti-virus protection. Want to compromise the downstream app? Now you also need to pass Play Protect & friends.
> It seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
Yes, we really should. It is frankly a miracle that it has taken so long for fetch-time / install-time code execution to start blowing up in our faces. If we are spending so much effort on run-time isolation, why are we completely ignoring all those practices during development?
There is an open issue for this: https://github.com/rust-lang/cargo/issues/13681
I share the dislike for arbitrary build scripts but restricting them will not help the supply chain issue in a significant way.
Also there are several ways to control build.rs execution in the Cargo ecosystem as well, for example with cargo-deny.
Also, crates.io can defer serving up newly uploaded scripts that have a new build.rs / proc-macro dependency and warn publicly that a version introduces it.
Restricting build scripts 100% will help mitigate the impact, just not if you only deny it once. And they can develop other things like sandboxing for build scripts by default and escaping that to be the exception that has to be explicitly allowed.
pnpm does have some other features to prevent supply chain attacks, so there is still something to learn from other ecosystems. For example pnpm has a cooldown period for new dependencies and can prevent trust policy downgrades (eg new version published without build provenance where older versions did have it). See https://pnpm.io/supply-chain-security
Cargo has `min-publish-age` in nightly, and it's currently heading towards stabilization: https://github.com/rust-lang/cargo/pull/17335
`cargo add` is sufficient to compromise you, before you have a chance to even vet the code.
I won't spoil the claim in the video about what the dependency number is correlated with, and I'm not even sure how true it is in general, but it's very interesting.
we put together a Scroll on how this manifested with node, if anyone finds it helpful to understand that attack vectors & mitigation steps: https://endash.us/toolkit/items/mini-shai-halud
Compromising the code that is then most likely run in a test instead of compromising a build script is just a very slight inconvenience for the attacker.
I'm not particularly fond of arbitrary build scripts either, but restricting them will not help the supply chain issue in a significant way.
Also there are several ways to control build.rs execution in the Cargo ecosystem, for example with cargo-deny.
I also disagree a build scripts is a mild convenience. A build script always runs for anyone it’s a dependency for with full access and context and often has access to secrets in CI. A compromised runtime has more limited access and requires actual invocation of code paths (if you’re lying as a dependency that’s never executed, no exploit).
Of course Rust should have language-level support for capabilities so that just invoking a function doesn’t grant it access to arbitrary disk access. But that’s a much more difficult change than tweaking the defaults for cargo.
You don't need to audit all the crap is being uploaded right now. Only really necessary and widely-used packages should be managed in a centralized way, so, auditing all of them isn't that huge task.
Well, it depends on the language: language which "capabilities" (both for the source code and for the building* ) could in theory really reduce a lot of the burden to identify supply chain attacks.
*: some research language have/had capabilities which would make supply chain attack "obvious" but for build systems I don't know if this exist.
Same people who keep the whole rust project going, a lot of those are volunteers aren't they? Not mad to think they could do the same for core packages at least
Sure, but from my understanding the Rust project is generally "bottom-up" in that volunteers generally work on what they want to rather than submit their time into a pool for some kind of higher-level management to direct.
This crate isn't one of them.
still caught in hours though, so just as a general rule: never install anything newer than 7 days old packages
cargo feature for this is still unstable infuriatingly:
https://github.com/rust-lang/cargo/issues/17009
Especially because Rust devs brag so much about how it's soo superior to everything else, but then these amateur mishaps happen.
Rust isn't getting the exposure it deserves, I think, partly due to arrogance within the Rust community and a mental complex about "being better than everyone else" - that mentality never works
In the end it is your decision to use unaudited or refuse unaudited crates.
It should be the default behavior of the package manager to allow downloading only audited/trusted packages. Forcing end-users of the language to be responsible for audit of all dependencies is impractical.
Just because you don't develop a package manager for your language, doesn't mean someone else won't. See NPM.