Flatten Your Npm Dependencies With Dedupe
I use npm for a while now and it is heavily integrated in my daily workflow. One thing that always bothers me, is that a fresh
npm install on your project will probably download the same packages over and over again. Let us imagine your project has multiple dependencies that all have one package defined as another dependency.
1 2 3 4 5
As you see above package
c will be downloaded two times, although the dependencies
d could use the same package. You might think now “So? Not a big deal…”, but depending on dependencies of your dependencies you end up with a much slower
npm install process, because packages got downloaded multiple times. And if you deal with a big project with a lot of dependencies I am not talking about a few seconds. ;)
Additionally the installation process can become less stable. In my case I included the following dependencies in my
1 2 3 4 5
The two grunt plugins either directly or one of their dependencies depend on
phantomjs. This leads to the following output during the installation process.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Two times the same package is downloaded and unfortunately the
phantomjs package downloads
phantomjs via npm script functionality. It downloads and unzipps data itself. In my case I had to fight against an unstable network, which led to a bunch of failing intallation processes. Sometimes the first
phantomjs download did not work. Sometimes the second. And all the time
npm install failed. Especially
phantomjs is a really popular dependency and I am sure a lot of projects download it much more often than only two times. ;)
After this experience I started digging around a bit and discovered
npm dedupe – docs are here. It will check your installed dependencies and finally flatten the dependency structure by moving shared packages higher in the tree. Less duplicated packages – yay!!!
To show the difference – the old structure was:
1 2 3 4 5 6
And after using
dedupe I ended up with:
1 2 3 4 5 6 7 8 9
To make sure all your colleagues receive the same flattened dependency structure use
npm shrinkwrap to really lock all packages including versions and hierarchy. In case you have not used
shrinkwrap before I also wrote a post about it.
To sum up, my workflow was the following:
1 2 3 4 5
dedupe is currently marked as experimental and unfortunately I made the experience, that it is not 100% stable at the moment. I filed already an issue here. But we should really start using it, because it will make the installation of npm packages just quicker and safer. I am really looking forward to this feature and I am sure, that it will become stable soon.
Thanks for reading and all comments on this topic are as usual more than welcome. ;)