tl;dr: Let your users handle it. See https://github.com/BuilderIO/this-package-uses-fetch/
I’ve maintained a handful of JavaScript SDKs over the years. And for each one, the question of how exactly to provide support for fetch in a Node environment inevitably comes up. In fact, we just recently had to resolve some issues concerning this in our Builder.io SDKs.
Given the ever-changing landscape of JavaScript environments (seriously: I’ve been a web dev since 2017…do things ever slow down?), I figured it’s worth sharing my perspective on the matter.
You’re putting together an npm library that involves fetching data. There are many popular packages that can help you (axios, superagent, etc.) but in the spirit of not shipping bulky/redundant JS code to the browser, and the progress that browsers and JavaScript have made over the years, you try to use the platform whenever possible.
And when it comes to fetching data, there are 2 standards provided by browsers:
- XMLHttpRequest (a.k.a. Ajax)
- Fetch
We won’t dive too deep into the differences between these two, as this post is focused on how to use fetch(), not how to decide between fetch() and XMLHttpRequest! So we’ll assume you went with fetch() here.
So you’ve chosen and started using fetch() in your code, making calls like fetch('https://example.com'). Great! This works in most modern browsers:
But as the above chart shows, around ~4% of internet users today might still be relying on browsers that do not support fetch. What can you do for them?
Before jumping to conclusions, make sure that you actually have users in non-fetch environments (e.g. by importing your analytics into caniuse and looking at the browser usage of your users). It might be the case that you can actually get away with not doing anything at all!
If it turns out that you do have users on older browsers, then the way to fix this is to provide a fetch polyfill to support them:
A polyfill is a piece of code (usually JavaScript on the Web) used to provide modern functionality on older environments that do not natively support it.
In this case, Github offers a great fetch polyfill for browsers: https://github.com/github/fetch
Adding a browser polyfill is great, but JavaScript code also runs on the server, and your new npm library is no different! Node.js is, by far, the most popular JavaScript server runtime, but it did not have fetch support until v18 (released in April 2022).
So for all versions of Node.js ≤17, you’ll need a Node fetch polyfill, the most popular of which is node-fetch: https://github.com/node-fetch/node-fetch.
A somewhat recent development is the explosion of new JavaScript server runtimes. Here are some of them, along with their Node support:
- Deno: supports Fetch out-of-the-box
- Bun: support Fetch out-of-the-box
- Cloudflare Workers: supports Fetch out-of-the-box
- Netlify Edge Functions: built on top of Deno
As you can tell, many of the newer runtimes provide Fetch support from the get-go.
What would happen if you, the author, added these polyfills to your code?
You’d start by installing node-fetch as a package in dependencies, maybe include the GitHub polyfill for browsers too, and finally include some logic that toggles between server and browser polyfills. It might look something like the following:
export async function getFetch() {
const globalFetch = globalThis.fetch;
const isNodeEnv = typeof global !== 'undefined';
// do nothing if `fetch` exists
if (globalFetch) {
return globalFetch;
} else if (isNodeEnv) {
// get node polyfill
return import('node-fetch').then((d) => d.default);
} else {
// get browser polyfill
return import('whatwg-fetch').then((d) => d.fetch);
}
}
// use it like this
const fetch = await getFetch()
fetch('https://example.com')The nice thing about implementing this toggle is that, if implemented just right, your users won’t have to worry about a thing: they can npm install your-cool-library and start using it with no extra steps.
However, this approach runs into a host of issues & dilemmas:
node-fetchis now a hard dependency in your app. Since it depends on certain Node.js APIs, it might cause issues if it (accidentally or otherwise) gets imported in other server runtimes like Cloudflare Workers (which do not support these).- Both you & your users will have to pay very close attention to how dynamic imports are bundled, to make sure your polyfill imports don’t end up in the wrong environment.
- You will have to come back to this logic and tweak it as different JS runtimes show up, requiring different methods of checking which environment you’re in.
- if your library’s users already installed another version of these polyfills (for example,
node-fetch) for another purpose, you’d be forcing them to needlessly have 2 versions of the same package. - (Bonus): the delightful conflict between ESM and CommonJS module standards has a big consequence here:
node-fetchv3 is ESM-only and does not work in CommonJS projects. You need to either stay with v2 to support both or choose for your library to be ESM-only as well.
You can get around some of the above issues using a library called isomorphic-fetch, but not all of them. It also hasn’t been updated since 2015, and is locked on v2.x of node-fetch (whereas v3 has been out for some time now).
As much as we’d prefer the “we got you covered” approach of implementing polyfills on behalf of our users, it is too painful and brittle to do it this way.
Instead, as a general rule, I argue that you (a proud npm package author) should not include polyfills in your published code or dependency tree. You should leave it up to your users to do as they see fit, based on their needs & environments. This guarantees that:
- no one receives unnecessary code
- no future JS runtime has conflicts with how your code handles
fetch(). - no one ends up with duplicate versions of a given polyfill
- less code for you to maintain
- since most apps and packages do data fetching of some kind, end-users probably don’t need to do any extra work (and can reuse whatever polyfill they’ve already set up)
- you can go back to using the global
fetch()instead of importing a wrappedgetFetch()everywhere.
With Node v16 reaching end-of-life in Fall 2023, we are quickly approaching a time when polyfilling fetch() is far less important. But we are probably still years away from most Node users updating to v18, so we still need to provide clear documentation on how to polyfill fetch. This is why we created this github README: https://github.com/BuilderIO/this-package-uses-fetch
It includes a summary of the above, with clear actionable steps for each environment. The goal is to avoid the need for all package authors to explain the same thing over and over again. In fact, we just started using it ourselves in our SDK code: https://github.com/BuilderIO/builder/pull/1661
We invite other folks who use fetch to link to this document in their READMEs, and contribute any ideas they have!
The drag & drop headless CMS
Builder.io is a headless CMS that lets you drag and drop with your components right within your existing site.
// Dynamically render your components
export function MyPage({ json }) {
return <BuilderComponent content={json} />
}
registerComponents([MyHero, MyProducts])Visually build with your components
Builder.io is a headless CMS that lets you drag and drop with your components right within your existing site.
// Dynamically render your components
export function MyPage({ json }) {
return <BuilderComponent content={json} />
}
registerComponents([MyHero, MyProducts])
