Support loading external data file

[gavincarr]

Hey @lupomontero,

I'm enjoying using psl - thanks for your work on it!

One extra I'd like is the ability to load the rules from an external data file rather than using the internal one. This might just be to track the upstream MPSL more closely, or it would allow you to fork the MPSL and make custom changes for particular applications. Would you be open to this kind of change?

If so, what kind of interface would you like? I've got a prototype in my fork:

https://github.com/gavincarr/psl

that allows you to do e.g. var psl = require('psl').loadData('effective_tld_names.dat'); to synchronously load a (MPSL format, not json) data file at require time.

Thoughts?

[lupomontero]

Hi @gavincarr,

Thanks for the suggestion 👍

At the moment this library works both on the server and the browser, so adding Node's fs as a dependency can be a bit a bit tricky. It probably makes sense to keep the file system stuff outside of psl. We could maybe provide a function to set or override the rules, so that userland code could use their own custom rules without psl having to know about the where the rules came from (ie: the file system).

Alternatively, we could tweak the build process to allow you to chose what dat file to use...

Thoughts?

L

[justanotheranonymoususer]

Nice library, and I have the same feature request. I actually use it in a browser environment, so I don't have fs. My goal is to download a fresh version of the list and cache it in local storage, and re-download it once it expires (say, a month).

Perhaps the library can can allow to receive the content of effective_tld_names.dat as a string, and it would be up to the user to fetch the file contents. Then it can be done via fetch in the browser.